Our company has multiple offices of different sizes. Every year each office puts forward a random number of its staff for promotion. Within each office colleagues vote on their respective nominations on a scale of 1-5 and the average score for each nominee then calculated. The nominees for all offices are then ranked in order across the whole company, and the highest scorers put forward for consideration.

Sounds fair enough, but surely the differences in office sizes skew the average scores? For example office A is 10 people and has 1 candidate. Nine rank him a 5 and one a 1. His average score for the company wide ranking is 4.6. Office A is 40 people and has 1 candidate. Thirty-nine rank him a 5 and one a 1. His average score for the company wide ranking is 4.9 and he will appear higher up the list. Is there a test that can account for different sample sizes or in practice is this not even a problem?

Thanks – feeling I should have paid more attention in maths class!

yes, but can’t think of the right word to google on that

T-Test? Seems to be Google’s suggestion but I’m only skim reading at the moment whilst pretending to be doing real work.

I don’t see how sample sizes are a problem in this instance – they don’t affect fairness (directly).