Software changes can help to reduce load, which in turn can stop servers going bang.

Exactly. You’re the largest client on BBPress which is not a nice place to be!

Be a small guy on some of the software designed for the big guys!

The Toms have the data if it was software I’m sure they’d look at something else.

They may have the data. The data may say a software change is needed. The thing is tweaking the existing code is far far less time consuming/cheaper than migrating to a new software platform, so I imagine, and rightly so, they are going to try everything they can with the current setup.

What Jamie said 🙂

Thanks for the update Tom(NondB).

here’s now a number of solutions I’m looking at implementing forthwith – one involves “archiving” some of the posts in the forum

This makes a lot of sense and would avoid the occasional 2 year old zombie thread that sometime pops up. Maybe archive them as “read only” after 6 months? I guess then you could have them as static pages and get the off the DB or at least just move them to different tables.

On the rare occasion that someone wants to discuss some ancient thread they can always start a new thread with a link to the archived one.

Tom, I’ve said it on these threads before. If you have an i/o problem due to high load on the database you really should investigate the use of memcache. This is how Facebook handle huge loads. It does need some software tinkeration to ensure you only cache certain queries, but the benefits are huge.

Thanks DavidB, and as ever to GrahamS. We do get some good advice from these threads, occasionally. I hear Facebook now have over 60,000 servers, which probably also helps spread the load a little 🙂

the amazon issue seems to have been fixed. Haven’t had it happen today

If only memcache was as simple as plugging in a sever and installing the software, I/O bottlenecks have been the bain of my life over the years so I commisurate with you and understand what you are going through.

Thanks DavidB, and as ever to GrahamS. We do get some good advice from these threads, occasionally. I hear Facebook now have over 60,000 servers, which probably also helps spread the load a little

Facebook do, but PinkBike are running on just the one (at least not counting static content) with a LOT of traffic going through.

Interesting article about that over here: http://radek.pinkbike.com/blog/pinkbike-speed.html

To update. I’ll be making some adjustments over the weekend. I’ll have to take the forum offline for a short time, but hopefully when it’s quiet.

Look, I just found this page http://www.singletrackworld.com/forum/statistics.php which tells you the most popular forum topics of all time. And there are 2.2+ million posts at the moment.

f only memcache was as simple as plugging in a sever and installing the software,

It’s not, but it is one of the best investments you’ll ever make. They key is isolating the read traffic and memcaching that. Our DB servers sat on their arse doing nowt once this had been done. It was a damned clever bit of code that took a week for two of us to write, probably saved £50-100k in hardware long term.

Update.
It’s probably inappropriate to say the site will never collapse under load ever again, but we have made some suitable adjustments to lots of relevant things and it’s behaving much better. We also have some more stuff in the pipeline. We’ll continue to monitor the situation closely.

I’ll unsticky this thread now. Thanks for your patience.

Fingers crossed.

delusional – that PB article about their server was very interesting, thanks 🙂