The Theory
Pitchfork is a popular-as-hell indie music blog. It's got a hipster-snobby reputation and the reviews are best summarised as "When Adjectives Attack," but their recommendations tend to be on the money and I've found a lot of good music thanks to their Best New Music category.
Pitchfork's also got a reputation of being a major tastemaker, anointing albums & artists to the big leagues. But is this backed up by the data?
The Practice
I used Python to save a copy of every Pitchfork review I could, and parse out the data. I also pulled in data from Last.fm, MusicMetric, and The Echnoest. The data set is 12488 reviews, with 391 receiving the Best New Music award and 110 receiving Best New Reissue.
The Results
How evenly distributed are Pitchfork's reviews?
If Pitchfork reviewed a random cross-section of new albums, you'd expect to see an even bell-curve centered around the 5.0 mark.
As you can see, these skew slightly high, with the median score being 7.2. I was a little disappointed not to see an uptick at the start – eviscerating reviews are fun to read.
Have standards changed over time?
This is a random sampling of reviews, both to keep it comprehensible and below the Google Charts limit for data size. Over 10 years Pitchfork's been consistent with the scores for their best new music, although there's been some high-scoring albums that didn't merit an award.
The scale used to judge an album seems consistent. Maybe Pitchfork have been giving out more awards each year?
Nothing too remarkable; we can see that the Best New Music award wasn't introduced until 2003, Best New Reissue wasn't introduced until 2009, and the number of awards has remained roughly proportional to the number of reviews. 2006 was a poor year for music.
How does a Pitchfork review affect an album's popularity?
This is the heart of the matter. Does a good Pitchfork review result in a listener spike? And does it stick around?
I tried really, really hard to find data on this at the album level. But alas it's not feasible. So let's pick some artists and see what happens following their first Pitchfork review.
This is a work in progress; these graphs may be inaccurate.
Jessie J
Gorillaz
Drake
Wild Beasts
Phoenix
Girls
St. Vincent
Atlas Sound
Animal Collective
The Conclusion
Nothing can be concluded yet, but my research and analysis continues. I hope to have more details in the next couple of weeks.