Media bias ratings at AllSides are not set in stone. We like to think of them as alive, constantly growing and changing -- often driven by you, the reader. They are updated on a regular basis, using blind bias quizzes, community ratings, independent research, and editorial reviews.

Recently, we conducted our first blind editorial review, where we examined five sources that are highly contested by the AllSides community. The team collected a series of articles from each news outlet, but removed any attribution or reference to that particular publication. Survey participants read three full articles from each source, and without knowing the origin, assigned a bias rating from Left to Right. Our entire staff took part, as did several employees from Living Room Conversations to ensure the group was balanced across the political spectrum. The team’s biases were as follows:

  • Left: Four participants

  • Center: Two participants

  • Right: Four participants

There are advantages and disadvantages to a blind review. On one hand, it allows readers to evaluate a source without letting any previous biases impact their judgement of that publication. However, participants are only provided with three articles. These pieces often give a good picture of a publication’s political slant, but such isolated examples sometimes fall short of providing the full picture.

With this in mind, we also conducted a transparent review, where two AllSides staff members (one Lean Left, one Lean Right) evaluated the full body of research for the same five sources. While we considered data from the blind editorial review, we also analyzed AllSides community feedback, blind bias quiz results, and previous research. 

The five new outlets are NPR*, CNN*, Fox News*, Daily Beast and Politico. We will reveal the media bias results for each publication in a series of forthcoming blogs. Stay tuned, as all of them will evolve in some way.

Next time -- NPR’s bias rating (currently Center) and how it will change.

*AllSides only rates online journalism, not broadcast or radio news.