For apps you'll have to go through the app store, there's over 200 thousand on there so have a look, start with the top 50 free ones to get you going.
Where have you got your music at the mo, any you've got on your pc stick it into itunes and sync to the pod
If you're flexable on the legal side of things then there's torrents of everything out there