How AllSides Calculates Media Bias

The AllSides patented media bias detection and display technology drives arguably the world's most effective and up-to-date media bias detection engine. It's powered by a combination of wisdom-of-the-crowd technology and the best statistical research and methodologies.

You drive the media bias ratings.

What you do at AllSides affects our media bias ratings. That includes how you rate your own bias and how you rate the bias of news sites on our Media Bias Ratings page, in our blind bias surveys, and throughout our site. All of this is added to our crowd data, which is statistically normalized to represent a balance of the American public.

Multiple methods for calculating media bias ratings.

Our blind bias survey, described in the graphic below, represents our most robust media bias rating method. It is not the only method we use, and we usually don't need anything as robust as that. This is because the source itself might openly share its bias, third party research may have already determined the bias, an independent review might be decisive, or a broad consensus could be sufficient. Take a look at the multiple methods we use to measure media bias.

AllSides Media Bias rating - how it works

Our media bias detection engine gets smarter as time goes on.

We are constantly evolving the media bias rating engine. And, the more you participate, the better our media bias ratings will be and the more sources we can rate.

We also ask you to rate your own bias.

We're continuing to improve ways to help you get the most accurate bias self-rating so you can participate on AllSides and in life with transparency and self-awareness. Make the world a better place by understanding and sharing your own bias openly! 

Media Bias Rating Methods

Throughout, you will find references to the media bias ratings of sources, including online news sources, authors and more. Since there is no such thing as unbiased news, it is essential to reveal media bias as much as possible to inform all of us so we can better evaluate what we are reading and understand the topics — and each other.

You can see a full list of our media bias ratings on the AllSides Media Bias Ratings page. Media bias is rated using a patented process to identify and display the average judgement of Americans. The ratings are fluid and updated as new information is assessed.

On a media source’s page, you’ll see how we assigned the media bias rating, which methods influenced our rating, and how confident we are in our rating of that source’s bias. To see the media source page, just click on the name of the media source (publication, author, etc) with its rating — we show this throughout the site. It looks like this:


CBS    Leaning Left


We update the AllSides Media Bias Ratings and Confidence Levels as more data is gathered and assessed. That means the AllSides Media Bias Ratings are fluid and subject to change over time.

Unless otherwise noted, a media bias rating refers only to a source’s online news articles, not on the source’s opinion pieces, TV, or radio. The opinion writers from the same media source may have different bias ratings, so individual writers often are rated separately.

The Basis of the Media Bias Rating

The bias rating you see on a source’s AllSides Media Bias Rating page is based on the methods listed with a check mark next to them.

  • Blind Survey: If this is checked, the media bias rating is driven by the results of our blind surveys. AllSides collects and analyzes data from blind bias surveys of media sources. The survey asks readers to rate the bias of news without knowing the news source or author they are rating. AllSides normalizes this data, making sure that the audience accurately reflects the social and political diversity of the nation. This prevents “gaming” (when someone or a group tries to unfairly impact the ratings) or giving too much influence to any segment of the population. This provides the most robust evaluation of the political leanings or bias available, and is at the core of the patented AllSides Media Bias Ratings™ methodology.
  • Third Party Data: If this is checked, the bias rating has been influenced by third-party research or data that AllSides has determined to be credible. Examples of this include academic research by UCLA and well-sourced articles in Wikipedia. AllSides notes which studies or research it uses in the bias rating for each source.
  • Community Feedback: If this is checked, the media bias rating has been influenced by votes submitted by AllSides readers. This feedback is not blind. When people rate sources in this way they can see which source they are rating and generally see how AllSides has previously rated the source and how other people have voted. Since it is not blind, it is subject to bias itself as users are influenced by their preconceived notions or other people’s opinions.

    The community feedback is also not normalized. In other words, the raw data has not been balanced to fairly represent the various segments and groups in America. For example, if 100% of the people who rated a source were themselves liberal, the bias ratings they would give would be very biased toward their own point of view and not America overall. A news outlet that America might on average consider “left” might instead be rated by this left group as “center” because the source more closely reflects their own views. This open-to-anyone voting system can also be “gamed” by a group of people who want to influence the outcome.

    Because of these inherent biases, AllSides uses community bias ratings and feedback as a “warning system” that our current bias ratings may be off. This feedback sometimes triggers more robust blind surveys, analysis or other kinds of more robust research to sharpen the bias rating accuracy.

    AllSides shows the raw votes by our users, with the first number representing the people who agree with our bias rating, and the second number represent those who disagree. We color code these for easy reference:

     Agree or Disagree

    It is not unusual for more than half of people to disagree with the AllSides Media Bias Rating, but the average rating often agrees with the AllSides Media Bias Rating. For example, AllSides may have rated a source as “lean left” and the majority of people disagree with that rating. But more than half will rate that source as “left” and the rest will rate it as “center” or “lean right”, and the average will equal “lean left”, thus supporting the original AllSides media bias rating. See notes for each source for more details.

    Sometimes you will see a media bias rating to the right of this community feedback indicator. This is the average rating of people who disagreed with the AllSides bias ratings and who bothered to also submit what they think the rating is (many disagree but do not rate themselves). It is not the average rating of the community at large since it does not include those who agree with the AllSides rating, those who disagree but did not rate, and is not normalized to reflect all of America (so one group can easily be over-represented, skewing the results.)

  • Editorial Review: If this is checked, the AllSides editorial staff has reviewed the works of this source and their consensus on the bias for this source has influenced this rating. These reviews always include a diversity of individuals covering the range of bias from left to right. Sometimes we also do "blind" editorial reviews when our editorial staff does not know the source of the content they are rating.
  • Independent Research: If this is checked, the bias rating is influenced by online research performed by AllSides. Staff may review sample content and investigate what the media and other sources, both partisan and nonpartisan, report about the political leanings of this source. This method is frequently used for initial bias ratings before stronger methods can be applied.

The Confidence Level in our Media Bias Rating

The Confidence Level on an AllSides Media Bias Rating page may show as Low or Initial, Medium or High, depending on how confident we are about the bias rating of the media source. We base our Confidence Level on two things:

  1. How many methods have been applied and which methods have been applied. The more reliable methods applied, the higher our Confidence Level. For example, we usually give more weight to our Blind Surveys and the less weight to what we call Secondary Research, which is an AllSides staff member’s initial assessment of bias based on online research.
  2. The strength and consistency of the data. The more strength and consistency in the bias information, the higher our Confidence Level. For example, with Secondary Research, if a journalist is self-described as “conservative” and Wikipedia labels her that way too, that shows both strength and consistency. If the Community Feedback includes thousands of users, that has more impact than feedback from fewer votes. If we take academic Third Party Data into consideration in a bias rating, but the research is 20 years old, we wouldn’t give it as much weight as more recent data. Or, if two of our assessment methods indicate the source is Right, but one of our methods indicates it’s Left, our Confidence Level in the bias rating is lower.

What does it mean for a media source to be rated Right, Left, or anything else?

These are subjective judgements made by people across the country. Here is our rough approximation for what the media bias ratings mean: bias ratings for Left - Lean Left - Center - Lean Right - Right.

Learn more about AllSides Media Bias Ratings and how to rate your own bias.