In reality it depends on what you mean by building trails.
Clearly the land os owned, and you should ask permission before doing anything.
unofficial jumps, berms, obstacles and north shore is a complete no no – not only will they likely get destroyed, but you’ll also bring a lot of hassle, threats of bans and prosecutions, and general opprobrium upon the mountain biking community as a whole, every one if us needs to act as an ambassador.
if however you’re talking about acting in a community minded manner with trail maintenance and improvement of existing tracks there, filling in puddles, reinforcing bits, cutting back vegetation… then more often than not nobody will be bothered.