It works both ways, my friends and I are buzzing along training/ or just riding XC and you get dozy 40yr olds stopped in the middle of the track who doesn't get out of your way.
having said that, some kids dont get out of our way either, which is always annoying when you a) maintain/build the trails they are riding b) are always very curtious to walkers horseriders and other cyclists.