Through out AllSides.com, on almost every page, you will find references to the bias ratings of sources, including online news sources, authors and more. We believe it is essential to reveal bias as much as possible in order to better inform all of us so we can better evaluate what we are reading and better understand the topics and each other.
You can see a full list of our bias ratings on our AllSides Bias Ratings page. These bias ratings are designed using a patented process to identify and display the average judgement of Americans. They are fluid and updated as new information is assessed.
On a media source’s page, you’ll see how we assigned the media bias rating, which methods influenced our bias rating, and how confident we are in our rating of that source’s bias. To see the media source page, just click on the name of the media source (publication, author, etc) with the bias rating that we have throughout the site. It looks like this:
What does it mean to be rated right, left, or anything else? These are subjective judgements made by people across the country. Our rough approximation for what that means is here: bias ratings for left - lean left - center - lean right - right.
The Basis of the Media Bias Rating
The rating you see on a source’s AllSides Bias Rating page is based on the methods listed with a check mark next to them.
- Blind Survey: If this is checked, the bias rating is driven by the results of our blind surveys. AllSides collects and analyzes data from blind bias surveys of media sources. The survey asks readers to rate the bias of news without knowing the news source or author they are rating. AllSides normalizes this data, making sure that the audience accurately reflects the social and political diversity of the nation. This prevents “gaming” (when someone or a group tries to unfairly impact the ratings) or giving too much influence to any segment of the population. This provides the most robust evaluation of the political leanings or bias available, and is at the core of the patented AllSides Bias Ratings™ methodology.
- Third Party Data: If this is checked, the bias rating has been influenced by third-party research or data that AllSides has determined to be credible. Examples of this include academic research by UCLA and well-sourced articles in Wikipedia. AllSides notes which studies or research it uses in the bias rating for each source.
Community Feedback: If this is checked, the bias rating has been influenced by votes submitted by AllSides readers. This feedback is not blind. When people rate sources in this way they can see which source they are rating and generally see how AllSides has previously rated the source and how other people have voted. Since it is not blind, it is subject to bias itself as users are influenced by their preconceived notions or other people’s opinions.
The community feedback is also not normalized. In other words, the raw data has not been balanced to fairly represent the various segments and groups in America. For example, if 100% of the people who rated a source were themselves liberal, the bias ratings they would give would be very biased toward their own point of view and not America overall. A news outlet that America might on average consider “left” might instead be rated by this left group as “center” because the source more closely reflects their own views. This open-to-anyone voting system can also be “gamed” by a group of people who want to influence the outcome.
Because of these inherent biases, AllSides uses community bias ratings and feedback as a “warning system” that our current bias ratings may be off. This feedback sometimes triggers more robust blind surveys, analysis or other kinds of more robust research to sharpen the bias rating accuracy.
AllSides shows the raw votes by our users, with the first number representing the people who agree with our bias rating, and the second number represent those who disagree. We color code these for easy reference:
It is not unusual for more than half of people to disagree with the AllSides Bias Rating, but the average rating often agrees with the AllSides Bias Rating. For example, AllSides may have rated a source as “lean left” and the majority of people disagree with that rating. But more than half will rate that source as “left” and the rest will rate it as “center” or “lean right”, and the average will equal “lean left”, thus supporting the original AllSides rating. See notes for each source for more details.
Sometimes you will see a bias rating to the right of this community feedback indicator. This is the average rating of people who disagreed with the AllSides bias ratings and who bothered to also submit what they think the rating is (many disagree but do not rate themselves). It is not the average rating of the community at large since it does not include those who agree with the AllSides rating, those who disagree but did not rate, and is not normalized to reflect all of America (so one group can easily be over-represented, skewing the results.)
- Editorial Review: If this is checked, the AllSides editorial staff has reviewed the works of this source and their consensus on the bias for this source has influenced this rating. These reviews always include a diversity of individuals covering the range of bias from left to right.
- Independent Research: If this is checked, the bias rating is influenced by online research performed by AllSides. Staff may review sample content and investigate what the media and other sources, both partisan and nonpartisan, report about the political leanings of this source. This method is frequently used for initial bias ratings before stronger methods can be applied.
The Confidence Level in our Bias Rating
The Confidence Level on an AllSides Bias Rating page may show as Low or Initial, Medium or High, depending on how confident we are about the bias rating of the media source. We base our Confidence Level on two things:
- How many methods have been applied and which methods have been applied. The more reliable methods applied, the higher our Confidence Level. For example, we usually give more weight to our Blind Surveys and the less weight to what we call Secondary Research, which is an AllSides staff member’s initial assessment of bias based on online research.
- The strength and consistency of the data. The more strength and consistency in the bias information, the higher our Confidence Level.For example, with Secondary Research, if a journalist is self-described as “conservative” and Wikipedia labels her that way too, that shows both strength and consistency. If the Community Feedback includes thousands of users, that has more impact than feedback from fewer votes. If we take academic Third Party Data into consideration in a bias rating, but the research is 20 years old, we wouldn’t give it as much weight as more recent data. Or, if two of our assessment methods indicate the source is Right, but one of our methods indicates it’s Left, our Confidence Level in the bias rating is lower.
We update our AllSides Bias Ratings and Confidence Levels as more data is gathered and assessed.
Unless otherwise noted, a bias rating refers only to a source’s online news articles, not on the source’s opinion pieces, TV, or radio. The opinion writers from the same media source may have different bias ratings, so individual writers often are rated separately.
Learn more about AllSides Bias Ratings and how to rate your own bias.