6 Ways Fact Checkers are Biased

Last updated May 19, 2023

When conducted properly, fact-checking can be used by the public to cut through the polarization around an issue and get right to the facts. But if you look more closely, many fact checks are biased, imbalanced, or incomplete. Why?

Fact checks often exhibit as much bias as regular news articles and analyses. Fact checker bias is revealed by the claims they choose to check; the sources they choose to use; and the opinionated judgements they present as fact in the process of checking a claim.

Related: AllSides Fact Check Bias Chart™

Media bias in fact checks takes many other forms, and is often hidden underneath lengthy prose and a slanted focus on experts. As revealed by the AllSides Fact Check Bias Chart™, most fact checkers are on the political left or center.

Here are six distinct ways fact-checkers can be biased, misleading readers and gaming the system against perspectives they disagree with:

Just fact check the other guys, leave “their side” mostly unchallenged

This is apparent in the case of President Joe Biden and President Donald Trump — when Biden falsely said during a meeting with Russian President Vladimir Putin that Trump supporters at the Jan. 6 Capitol riot “killed a police officer,” few fact-checkers paid any attention to it. Many of the same outlets that didn’t cover Biden’s claim published inaccurate stories in the riot’s immediate aftermath about the officer’s death. Conversely, few right-rated sources spend any time fact-checking Trump's claims that election fraud cost him the 2020 election.

Fact checkers may generally trust or mistrust the military, government agencies, leaders or officials from one party or the other, teachers vs parents, workers vs bosses, etc., which could lead them to give one side more clout and attention than others.

Fact check extreme or narrow statements to label them as false, implying that a more general idea is also false

PolitiFact (Lean Left bias) recently analyzed the statement that “critical race theory has moved into all our schools in Virginia” and found it to be false. The claim is in essence true, though highly exaggerated. If PolitiFact instead had fact checked whether “themes of CRT are being taught in schools today,” that answer, based on their own description, would be true.

Fact checkers on both sides issue true/false binaries on claims that are largely subjective. In an article about a Florida bill that would prohibit discussion of sexual orientation or gender identity in schools, Breitbart Fact Check rated the claim that “Conservative Florida legislators are targeting vulnerable gay, lesbian, bisexual, transgender, and intersex children” as “FALSE”, even though the claim itself is highly subjective. Breitbart’s presentation of the claim minimizes more specific criticisms of the bill about freedom of expression and general concerns about the LGBTQ+ community.

Subjectively judge claims, often beyond the hard facts, to give a final “fact check” rating 

One example of this is in this controversial fact check by The Washington Post of Carly Fiorina’s statement, “I started as a secretary, typing and filing for a nine-person real estate firm. It’s only in this country that you can go from being a secretary to chief executive of the largest tech company in the world, and run for president of the United States. It’s only possible here.” 

The Washington Post Fact Check (Lean Left) gave her statement a “3 Pinocchio” rating (which means “mostly false” or has “significant factual error and/or obvious contradictions”), saying that "In telling her only-in-America story, she conveniently glosses over the only-for-Fiorina opportunities and options beyond what the proverbial mailroom worker has," while also affirming with detail that she had indeed started as a secretary.

Trust only a few government experts or others that are likely biased toward one conclusion

Fact checkers often show bias via which authority they appeal to. They may trust claims and data given by institutions or authorities such as government agencies or military branches that are themselves being charged with falsehood or controversial behavior.

For instance, years ago, PolitiFact fact checked the GOP accusation that many would not be able to hold onto their current doctor with Obamacare. PolitiFact appealed to statements from Obama’s administration to state as fact that you could keep your doctor, and called accusations to the contrary as false, not covering the arguments and data showing that you would lose your doctor. (To their credit, after Obamacare was passed and many people were forced to give up their current doctors, exposing the lie, PolitiFact called Obama’s claim that you could keep your doctor their “Lie of the Year”.) 

Another example are recent fact checks by the Associated Press Fact Check (Lean Left) and Reuters Fact Check (Center) that argued “mass formation psychosis” is an unfounded theory. It is "unfounded" by definition, but AP and Reuters only appealed to a few experts who agreed that the theory is unfounded and left out voices who argue it has merit.

We see this type of bias — the appeal to specific authorities — outside of the fact check sections as well.

While it doesn’t label any of its content as “fact checking,” The Epoch Times (Lean Right bias) sometimes appeals to authorities that directly contradict mainstream media and political narratives, especially about COVID-19. Many articles list Harvard professor Martin Kulldorff as a source on why vaccine requirements, mask mandates and other pandemic regulations are ineffective or oppressive. Kulldorff and other sources frequently cited by The Epoch Times in COVID-19 coverage are outspoken in opposition to mandates, and The Epoch Times rarely includes additional arguments that go against them. 

Only provide or emphasize data that supports one conclusion

On issues such as gun control, racism in policing, and abortion, there is a range of data and research that support and oppose the various arguments about the issue. A fact checker might emphasize one set of data and facts while completely ignoring or greatly downplaying another set of data and facts that support an alternative conclusion. 

They might emphasize or just use data that is not well supported and contrary to the greater body of evidence (this can go to the extreme of providing “false equivalency”), or ignore any contrary data that opposes the official or popular conclusion (especially when given by a source or party that is more trusted by the fact checker).

One example of this is when Snopes (Lean Left) rated the claim “Do Police Kill More White People Than Black People?” as “Mixed.” Snopes acknowledges that more white people are in fact killed by police than black people, but the outlet emphasizes proportionality to conclude that black people are “several times more likely to be killed in police shootings” than white people. In that analysis, it also omitted that you would need data like "how often black and white civilians encounter police officers" to make any sound conclusion.

Lack of thought diversity within the fact checker team

Journalists tend to lean left. This may explain why Politifact doubled down on a fact check debunking the theory that Kyle Rittenhouse's possession of a rifle on the night he killed two men and wounded one during Black Lives Matter unrest was "perfectly legal." A judge tossed out a charge of a minor carrying a weapon, reportedly due to unclear laws regarding the alleged violation, but Politifact wrote a lengthy explanation about why they believe their fact check conclusion still had merit.

Conclusion

“Fact-checkers” can be just as biased as regular journalists, and this display of media bias can be very harmful, since these sources present themselves as especially trustworthy and truth-seeking.

Reference our Fact Checking curation hub to compare facts and fact checks from across the political spectrum so you can know more and find the truth. You can also submit a claim of false or misleading content to AllSides for our balanced editorial team to review, and we’ll get back to you with a personal email response.

This piece was originally written by AllSides CEO John Gable and Managing Editor Henry A. Brechter (Center bias), and was reviewed and revised by Research Intern Aidanne DePoy (Left), Director of Marketing Julie Mastrine (Lean Right) and Data Journalist Andrew Weinzierl (Lean Left).

Image: Randall Munroe, CC BY 2.5