It's like anything else - generally, if you pay peanuts you get monkeys.
Cheap gegs are OK if you don't mind having to replace them every six months or so when the lenses get scratched, frames snap or whatever.
I love Oakleys, but they are definitely one of those investments that may seem like an enormous expense at the time but it really is worth it in the long run IMHO. The optical clarity of the lenses is far superior and they are more scratch/breakage resistant (except the Iridium lenses, you have to look after these like your life depends on it). Spares are easily available too.
As for them only being made for midgets, not true. Depends what model you get - if you have a big head, go for Radars. I have a pair of Radars for on the bike and a pair of Oil Drums for casual. Both are great and well worth the wonga.