Does anybody know if I have to buy one of the rather expensive bearing kits for an Enduro '04 or can I order bearings from a large bearing manufactuer?

There are kits on eBay for around £20, so about a tenner cheaper than Specialized. Buying good quality bearings is not cheap. It'll cost you between £6 and £8 per bearing for something like FAG or SKF. You can also get (reportedly) decent Chinese imports for around half that.

I used SKF bearings for the BB and link pivots, then the Chinese ones for the Horst pivots on my '07 SX Trail. The total was somewhere around £70. I switched all but the Horst bearings myself; they were removed with the proper tool by a friend at a shop. Typically, you'll pay £30 for a full replacement.

Check out Force Seven on the Team Valley. If they don't have what you want in stock they'll usually get it by 11am next day. Lovely bunch of people, too.

Any idea of what sizes I would need to order?

OEM bearing kits are invariably complete rip off, you can easily save something like 75% if you source bearings from a regular bearing supplier instead. Any bearing the number of which starts with 6 (most of the bearings used on bikes) can be got from pretty much any bearing shop; some of the suspension bearings are angular contact (7xxxx), and those can be harder to source.

I normally use http://www.smbbearings.com/, they have decent stock, take paypal payments, and the service has always been great. (I also buy the cheap Chinese bearings, I don't think the more expensive bearings life is proportionately longer to their price, and I reckon that is what is in most of the OEMs kits anyway.)

Here's the Specialized PDF for the '04 Enduro FSR manual. On pages 12 and 13 you will find a schematic for your frame and a list of all all bearings required.

I used AK bearings. They were spectacularly good. Talked me through all the options and it was cheap and very fast. I also was stupid and ordered the wrong thing (after them telling me what I needed) and they phoned me to double check that I knew what I wanted! Saved me a lot of hassle.

AK Bearings

Would I just recycle the spacers?

Yes, recycle the spacers. When I did the wife's fsr recently, I got the enduro bearing kit for it. Not much price difference between that and bearing suppliers. Plus, the max bearings suit pivots better than conventional deep groove ball bearings. Greater load capacity and frame pivots don't spin, they just oscillate.