Why do so many reviewers use only the top half of their rating scale? Has the Metacritic age led to score inflation? And why do some reviewers refuse to give out perfect scores? Answers to all of these questions, plus why Metacritic no longer lists scores from X-Play.
It’s no secret that most review magazines and websites today only use the top half of their rating scale when issuing a review score. Are reviewers being paid off by videogame publishers? Do magazines and websites give 7/10s to mediocre games out of a fear of losing advertising dollars? Other than few isolated instances, the reality of the situation isn’t nearly so scandalous.
For those who are unaware of Metacritic, it’s a review aggregator that collects review scores from various sources, and displays an overall average score for the game, movie, or album. However, in order to make things easier to average, Metacritic converts every review score into a scale out of 100. So 9/10 would become 90/100, 4/5 would become 80/100, etc.
On the surface this seems simple enough, but things become trickier when you figure in letter grades. Traditionally, 90-100 = A, 80-89 = B, etc. The problem is, one person might give a mediocre game a 50/100, while another will give it a C, which would be a 75/100. Metacritic tries to compensate for this by changing the values that letter grades are given so that a C = 50/100, but in doing so they’re overlooking a larger problem: any time a scoring system is based on a scale of 1-100, there is a natural tendency for many people to view the score as an equivalent of a letter grade. Hence why so many mediocre games are given a 75/100.
What makes videogames different than movies or music is that the videogame industry actually pays attention to Metacritic scores. Indeed, there are even reports of publishers denying developers a bonus if the game receives too low of a Metacritic score. The website has become so embedded in the culture of the industry that it has resulted in reviewers adjusting their own scores to reflect how they want it to show up out of 100.
A “Perfect” Score
I’ve got a funny story for you. In college, I had this teacher who was like a female version of John Malkovich’s character from Art School Confidential. One day she had us participate in an exercise where we were to mix white and black paint to create a sequence of ten rectangles illustrating a perfect scale of shades of gray between white and black. The middle rectangle needed to be a perfect mix of the two, a “true” gray, and each of the other rectangles needed to be as close to a 10% difference in shade as humans are capable of.
We’d each show her what we had, and she’d say “ah, but see, this gray is closer to the one before it than to the one after it,” and things of that sort. We worked on this for two hours, constantly revising to try and achieve perfection, until she revealed to us a little factoid: never in all her years of teaching, has she ever seen a gradient scale she thought was perfect. In fact, even the ones printed in textbooks she never entirely “agreed with.”
Thing is, she wasn’t saying this as some sort of lesson. In fact, she still expected us to spend the next few hours trying, and was still going to grade us on the results! She was merely telling us this anecdotally, as if we would find it fascinating, when in reality we were all looking at each other like “we are so screwed…”
There’s a belief among some gamers and reviewers that there’s no such thing as a perfect game, and thus a perfect score should be given only on rare occasion, if at all. But if a teacher revealed to you that they’d only given 100/100 on six papers over the course of a decade, because there’s no such thing as a “perfect” paper, you’d probably think they were senile or something. You never see a music critic refusing to give any album five stars because there’s no such thing as the perfect album, and Siskel & Ebert never refused to give a movie two thumbs up just because there’s no such thing as a perfect movie. So why should games be any different?
Metacritic Vs. X-Play
Imagine you’ve just been given 50/100—a failing grade—on a paper you’ve written. When you go to the teacher to ask for more clarification on what you did wrong, they tell you that the paper wasn’t actually terrible; it was merely average, and the teacher doesn’t believe in using only the upper half of the grading scale. If that has a negative impact on your grades, well, there’s always going to be some casualties when trying to enact change.
That teacher wouldn’t be X-Play. G4′s X-Play rates games on a five-star scale, which would look kind of silly to only use the upper half of. When a developer complained to Adam Sessler about how a grade of three stars or less from X-Play significantly drags down a game’s average (due in part to scores from reviewers of higher stature being given more weight), Sessler actually contacted Metacritic to see if they could work something out.
Unfortunately, Metacritic has no interest in adapting game scores (rather than converting literally) just because their own 100 point scale can easily be interpreted like an academic grading scale. And X-Play doesn’t want to switch from their long established five star scale just because of the way some website recently started affecting videogame culture. Ultimately, they came to an agreement to simply have X-Play scores no longer listed on the site.
It might seem like I’m defending review scores that only use the upper half of the scale. In a perfect world, everyone would go back to using a 1-10 or five star scale, rather than a 5-10 scale. Unfortunately, I don’t see that ever happening without Metacritic either falling out of popularity, or just changing their 100 point scale to a 10 point scale (to distance itself from the similarity to a academic grading system).
In the meantime, I don’t see any point in punishing developers out of pure stubbornness, the way a few major review sites continue to do. Taking that approach isn’t going to change anything. On the other hand, perhaps if enough major sites were to ask to be delisted from Metacritic the way X-Play has, they’d start to get the message…
Did you enjoy this article? Would you like to see more like this in a similar vein? Please let us know by commenting below.