Generally you want to set the 'B tension' screw so the upper jockey wheel is close (but not touching) the cogs on the rear cassette. This gives you the best shifting (that way, when the mech moves to change gear, it makes a sharper angle) and also wraps the chain as far around the cassette as possible (to reduce risk of skipping).
Essentially the screw adjusts the balance between the spring that keeps the chain pulled taut (the 'A Tension') and the spring that pulls the mech back around the mounting pivot (the 'B tension').
If, with a properly set B tension, the mech chatters in the highest gear but not in the 2nd highest, check the H limit screw is set properly.