Outside of the odd timed test in a group test, I rarely see objective/measured information about the performance of reviewed bikes and components. Understandable, given the time and money required to conduct such tests. The subjective opinion of reviewers is probably more valuable for most items, but for some, I’d be really interested in some accompanying scientific tests. Would you, have you seen any of interest?

Recently I found the Pinkbike impact rig tyre insert test interesting, and previously found this Italian lab test of knee pads useful. Not so much on their own, but with accompanying pieces/video with the reviewer’s thoughts.

Car magazines do tyre tests on many characteristics from straight dry braking, to wet cornering grip, to rolling resistance. I’ve never seen anything remotely close to that for bike tyres, just individual reviewers describing the tyre they’ve been riding recently, with the rare reference to how it compares to one of the well-known tyres. And people on internet forums endlessly discussing what has the most grip, clears mud better, or rolls fastest.

Brakes – power, fade resistance, modulation, always described in few words. I’d much rather see a little graph that shows finger force vs. braking power for each model, than “these have modulation, those don’t so much”. Enduro mag did a lab test on power and fade (and explained their methodology) which I thought was pretty good, despite some odd numbers. Plenty of us on here endlessly contradicting each other sharing experiences of different brakes when someone asks for experiences.

Same with chain lube, plenty of 4/5 star reviewed ones out there, and people on forums defending their latest personal choice and having wildly different experiences to what other people had. I did come across this test on YouTube which is pretty in-depth.

Vast amounts of information about what tyres for almost anything.

https://www.bicyclerollingresistance.com/

Understandable, given the time and money required to conduct such tests.

This.

I always see tests that are 100% scientific as almost completely pointless as parts designed 100% in a lab. Bikes are tactile interactive objects, a human is needed to get them to work, and a combination of science and emotive tests are far more useful.

There was a road bike designed in a lab in Bristol about 10yrs ago that was supposed to be the fastest, stiffest, most efficient ever – had duel crown bladed forks iirc and was around £15 a bike, which, at the time was around double of any other super bike. But it rode like dog poop, was just far too stiff – no one who rode it liked it and it disappeared into obscurity. Can’t remember the name of the company.

Enduro mag did a lab test on power and fade (and explained their methodology) which I thought was pretty good.

But even they admit that 1. most brakes have more then enough power to break the traction of a tyre, 2. in the lab it mostly came down to pad construction, and this was supposed to be a test about brakes, and 3. admits that lab testing doesn’t amount to much if you can replicate outside conditions of rain, sun, puddles, fatigue of the rider, concentration and so on, and 4 Isn’t really a test of reliability.

We all use bikes in such variable conditions that I think lab tests are a bit of non starter really, look at chain lube, depends entirely on what kind of soil you have as to how long your lube lasts…That’s why I think folk are still happy to rely on experience.

I always see tests that are 100% scientific as almost completely pointless as parts designed 100% in a lab.

They don’t have to be lab tests, but real-world testing is incredibly expensive. Imagine you want to scientifically compare two tyres. You need to compare pedaling efficiency with cornering and braking grip on different surfaces in different weather conditions. Some of this can use instrumentation to quantify things like rolling resistance, but ultimately you need to put human riders on and time them. You need riders of different ability and with different skill profiles. A pro-XC racer will probably prioritize climbing speed but have the descending skill to deal with a lack of grip. An amateur trail rider will probably want a tyre that is predictable in mixed conditions. So you need to get a whole bunch of different riders to do multiple runs on different trails in different weather conditions, but you also need to replace the tyres very frequently so you aren’t comparing new tyres with worn tyres (unless that’s part of your experimental design). Plus, you might want to compare them on different width rims and on hardtails and suspension bikes, and so on. Scientifically testing a single product in real-world conditions would be a massive undertaking, doing it on dozens or hundreds of products every year is utterly impractical.

I think it depends what you’re talking about, as above some assessment of bikes, components, etc is subjective, even if the person doing the scoring feels they’re being objective.

As an example, very simple; the trend for lls. using very basic measurement you can say what the various measurements are – length of TT, stack height, wheelbase, and so on; but whether a rider likes that one vs that one in the end can only be decided by the rider.

But when you get to more complex measurement – eg: friction losses in a particular drivetrain component? Is a derailleur more lossey than a hub geared bike? Or how does this chain lube compare to putoline? Or how much more aero are these wheels in a 25kph crosswind when travelling at 46.8kph…….etc. then there is already a surprising amount of proper science being used.

Then we decide what we like best or even what looks best.

Some work my colleagues in Force metrology helped with

https://www.notion.so/NPL-Evolve-sprocket-efficiency-test-report-6a3fddcb7b1449d3aaf2764604543de8

Post of the stuff I’ve seen is up there with L’Oréal adverts and purely designed to get roadies to cough up 2 grand on jockey wheels

An exception being suspension but even then by the time it gets to the user they’ll go off “feel” anyway so it’s plush or poppy not 26.47 plushermeters with 64.9% popposity

You could get in touch with the Project Farm guy on YouTube. He seems to go to nerdy levels to test to test all sorts of reason things

 26.47 plushermeters with 64.9% popposity

🤣

Grit and Flow, anyone?

How is your German? Bike Magazine always seems to be lab testing various bikes and components in the waqy you would expect a German magazine to do. For example E MTB brakes….

https://www.bike-magazin.de/emtb/test/test-2020-scheibenbremsen-fuer-e-mountainbikes

If you are able to sprechen the lingo there is a whole world of lab based test results out there

Enduro mag seem like they at least try.

Similar disc brake (2018) test: https://enduro-mtb.com/en/best-mtb-disc-brake-can-buy/
Winter shoes: https://enduro-mtb.com/en/best-flat-pedal-mtb-winter-shoes/
There are probably more

Car magazines do tyre tests on many characteristics from straight dry braking, to wet cornering grip, to rolling resistance.

I get what you’re saying, but car tyres are much more tightly specified to start with, ie, there is strictly specified rim width, tyre pressure, etc.

When it comes to mtb tyres I’d be happy if we could come up with a standardised way of describing the tyre. I want to know what are the dimensions (both width and height) for a given rim width, what is range of pressures and are these realistic, and then let’s start looking at durometer values.

I’ve got 2.25 tyres that are wider than 2.5 tyres. Don’t get me started on the height.

It’s expensive and pointless.

All people care is if it looks cool and new.

The bike industry is a marketing industry and we are all suckers.

I think an issue is that when comparing two similar products (say XC tyres) the differences will be so small as to be impossible to measure reliably. Even if you managed to remove all confounding variables, the differences in performance might be of the same order as your measurement error. Much easier to compare things which are very easy to measure, like weight and cost.

the differences will be so small as to be impossible to measure reliably.

Reliability is a function of sample size. You can measure very small differences if you have the resources to get enough measurements. What you are really getting at is that the effect size is not substantively meaningful. Thing is, if you do 10 back-to-back DH runs on two different tyres and the difference is within the measurement error, you’ll still choose the faster tyre because it’s more likely to be faster even if you aren’t highly confident in the measurement.

The bigger issue is that terrain and weather conditions vary so much between different places or times that pretty much any decision about what’s best has to be qualified with a huge list of contextual qualifications.

I can’t really see much use for it, as others have said it’s far more subjective and personal preference.

Take tyres as possiby the item with the biggest impact but least repeatable testing.

A car tyre you can very repeatably measure it’s aquaplaning resistance, wet braking, wet cornering, etc etc. You can’t even begin to do that with a bike tyre.

How do you measure mud shedding? Might be fairly easy actually. Untill you have to decide on the mud, mineral particle sizes, organic content, the wheels speed, tyre pressure, static loading, dynamic loading (frequency, amplitude and directions).

And even simple things like frame stiffness, or putting a shock on a Dyno. You still have to figure out what you consider optimum. Is there a certain number of Pascalls that’s considered the optimum spring in a chainstay? What about for the torsion between BB and headtube? Lateral deflection of the front hub during cornering Vs stiffness when upright? Or you just get a test ride, and think “yea this is what I like”.

IMO reviews are mostly a symbiotic relationship between brands and journalists. It’s advertising by the back door and job creation. Both sides have a vested interest in convincing their buyers that the process is usefull to sell parts and magazines.

The last thing I actually bought based on a grouptest review was a set of roadie handlebars. But even then I kinda knew what shape I wanted so the choice was already narrowed down. But it was usefull to have some more info on their relative stiffness.

Vast amounts of information about what tyres for almost anything.

https://www.bicyclerollingresistance.com/

Yeah – that was the one I first thought of. Even that (which is really good imo) is just a starting point – I think they use one drum ‘surface’ which even for the road isn’t representative of everything obvs.

Scientifically testing a single product in real-world conditions would be a massive undertaking, doing it on dozens or hundreds of products every year is utterly impractical.

You also need to devise tests that are directly relevant to real life use – and that’s before you’ve factored in user subjectivity / variability. Waterproof fabrics are routinely lab tested to MVT – moisture vapour transfer – which mostly gets abbreviated to ‘breathability’, but the way the lab tests work using fabric samples stretched over containers of heated water for example, doesn’t actually relate very well to real life use / lived experience or whatever you want to call it, because breathable waterproof fabrics don’t work in a simple or uniform way.

I’ve actually been present at a Gore-Tex launch where they produced a chart showing that Gore-Tex Active was ‘more breathable’ than a cotton tee-shirt.

At which point well-informed, subjected experience seems rather more useful.

Really interesting points here.

How is your German? Bike Magazine always seems to be lab testing various bikes and components in the waqy you would expect a German magazine to do.

Nice. They’ve done knee pads, drivetrain longevity, helmets, saddles (how?), tyres many times, sealant, and disc brakes multiple times.

And this about the lab process.

Scientific testing is what is done when assessing the product against the requirement, real world is just the same, all you’re doing in lab tests is simulating those over a shorter period, such as many reaction rates that double over a 10 degrees increase in temperature, same with shake, rattle and roll on items to simulate real world inputs/insults/etc.

With that you at least get a baseline that you can work from to see if it performs as it should, also remember quality varies batch to batch as well, testing should cover that as well, how many ‘test bikes’ or ‘test components’ have been put together with more care and attention than the ones in the shop?

Outside of the odd timed test in a group test, I rarely see objective/measured information about the performance of reviewed bikes and components.

Partly because it’s expensive and time-consuming to actually conduct such tests and partly because most people lack the knowledge and/or patience to read and understand such articles.

Good reviewers can describe the differences in kit without resorting to complicated figures at one end or trite cliches (“rides like it’s on rails” / “laterally stiff but vertically compliant” / “buttery smooth”) at the other.

Plus as mentioned above, there are far too many real world variables to be bothered with exact figures. Where testing is useful is confirming the manufacturers claims – especially around weight / stiffness – or in comparing (say) a dozen different bars where you can hang a weight off the end and measure deflection and put a figure on which is the stiffest. That still doesn’t necessarily translate into “this is the best” though – too dependent on shape, width, riding style, what bike it’s bolted to and so on.

I bought a very expensive tyre on the basis of a review on that tyre website and it really is very good. I was only interested in weight, rolling resistance and puncture resistance and so far it has lived to its review.

There seems to be a certain amount of straw man arguments in this thread.

most people lack the knowledge and/or patience to read and understand such articles

Having just paid to download the brake tests and ran it through an online PDF translator, I agree. Many MTBers, outside of Germany at least, are just going to go into just look at the pictures mode. I just had to look up what a “hysteresis loop” is to fully understand charts of lever force vs. braking force.

I’d highly recommend it for anyone buying brakes, but alongside more traditional reviews. They’ve done controlled tests both on the trail and in the lab, but the focus is more on the data – missing that other important factor as a few posters above have explained. And as @nickc pointed out about the Enduro mag test, they didn’t standardise the pads in this test either.

In the last few years, it seems the pro racers have recently got into the vaguely scientific testing methods, across all disciplines. Whether that’s off season testing or practise data acquisition.

Hearing the feedback though, and it’s still an uphill struggle. Timed results saying it is faster vs long held beliefs on what should be better, or worse, what ‘feels faster’.

Pro racers have the constraints of: faster, lasts the duration of the race; and are limited to the sponsors components. The recreational rider has a much tougher challenge.

To add, regarding brakes.

My criteria:
1. Are they powerful enough to lock a wheel with minimal physical effort.
2. Do they feel nice (anecdotal modulation).
3. Are they simple to bleed and maintain, including changing pads without the need to bleed.
4. Will they last many years.
5. Have I got a vague hope of buying pads for them in a bike shop if I’m caught short.

Everything in the market achieves number one, the others will not be solved or ranked in a lab test.