Global Disinformation Index's Risk Assessment Shows Bias Against the Right

From the Center

The Global Disinformation Index (rated as Left bias by AllSides) recently published its 2022 “Disinformation Risk Assessment” on U.S. news media sources. The assessment ranked 69 sources on the likelihood that they’ll publish disinformation, by the group’s definition (explained later).

It’s good to want to be aware of misinformation and disinformation. But GDI’s methodology is flawed. It lacks transparency about the reviews and the biases of individual reviewers. It also evaluates media bias and information quality together in a confusing fashion.

In addition, GDI’s ranks showed bias against right-rated sources by potentially downplaying the risks of false content from other outlets. AllSides has previously documented high-profile instances of several of “low risk” sources publishing false or misleading claims.

Jump to:

GDI Rates Right-Rated Sources as High Risk; Left-Rated as Low Risk
Breaking Down GDI’s Disinformation Criteria
Past Examples of Misinformation from “Low Risk” and “High Risk” Sources
What is the Global Disinformation Index?
Be Wary of Bias in Credibility Rankings

GDI Rates Right-Rated Sources as High Risk; Left-Rated as Low Risk

The GDI report highlights what it calls the 10 most and least “risky” online U.S. news outlets.

GDI lists the 10 “least risky” news sources as:

The first four on the list are rated as “Minimum” risk. The following six are rated as “Low” risk. In its report, GDI also states that these “ten lowest-risk sites published content that was generally free of Bias, Negative targeting and Sensational language.”

GDI’s 10 “riskiest” are listed as:

The first seven on this list are rated as “High” risk. The final three are rated as “Maximum” risk.

So, does this truly mean the “right” side of the media is much more at risk of peddling disinformation? Should you avoid these sources altogether?

It’s far from that simple, but the short answer is no. GDI’s “risk” assessment completely depends on its definition of disinformation, which is quite broad. Cutting sources completely out of your news diet can do much more harm than good when it comes to getting diverse perspectives and finding the truth.

Breaking Down GDI’s Disinformation Criteria

When zooming in on GDI’s definition of “disinformation,” it looks a lot more like a definition of “media bias.” And its risk assessment methodology explains how numerous people participate in a review process that ensures “neutrality,” but doesn’t mention whether their political biases and other leanings are considered. That added level of transparency is part of what makes AllSides the most trusted evaluator of media bias. Read more on our Methodology page.

GDI defines disinformation as:

“Adversarial narratives, which are intentionally misleading; financially or ideologically motivated; and/or, aimed at fostering long-term social, political or economic conflict; and which create a risk of harm by undermining trust in science or targeting at-risk individuals or institutions.”

“Intentionally misleading” is usually disinformation’s dead giveaway, but this definition goes much further. By the AllSides definition, everything else mentioned would be classified instead as media bias. When a news source publishes content that advances an “financially or ideologically motivated adversarial narrative,” that’s biased content – not necessarily “disinformation.”

In addition, judging this sort of intention from a news source is purely subjective. How can we know that a news site is intentionally trying to mislead? Who decides which views foster conflict and which are sincerely held beliefs? Who decides which narratives “create a risk of harm by undermining trust in science or targeting at-risk individuals or institutions”?

Explaining its definition, GDI says it “was developed to transcend many of the semantic arguments and other challenges facing the anti-disinformation space.” Furthermore, it says, “Not all false statements are disinformation,” and “a statement that is technically true can be presented out of context in a misleading and harmful way.”

GDI collected data for the study in two “pillars”: the Content Pillar, which is “based on a sample of content published on the site including news and opinion articles,” and the Operations Pillar, “which reflects the operational policies, practices and past behaviour of the media outlet.”

Within those, GDI looks to see things like whether an outlet is transparent about funding and ownership, whether it publishes some form of editorial guidelines or policies, whether bias is evident in its articles, and whether it links to sources, to name a few.

GDI also includes its own definition of media bias in its methodology, saying, “biased writing misrepresents facts, is based on faulty logic, and/or fails to include or unfairly engages with different views on the story.”

GDI’s definition of disinformation covers a lot of ground, and expands significantly on the common definition, at least according to Merriam-Webster: “false information deliberately and often covertly spread (as by the planting of rumors) in order to influence public opinion or obscure the truth.”

If any news outlet that shows bias or advances “ideologically motivated adversarial narratives” is therefore at risk of pushing disinformation, the list of “risky sources” should be much longer – and would include several from the “low risk” list.

GDI lists the 69 sources studied, but doesn’t list risk assessments for any other than the 20 highest- and lowest-risk – a questionable lack of transparency. The most detail offered to that end is as follows: “A third of the domains in the study (26 out of 69) were assessed as low- or minimum- risk; Twenty sites fell into the medium-risk category, while another 20 sites were high-risk; Three sites were rated as maximum-risk.”

GDI explains how it “ensures neutrality” through a content review process involving analysts, “independent researchers,” and “locally based media experts” who review “anonymised, plain-text content, from which the name of the media outlet and associated journalists have been removed.”

What the neutrality explanation doesn’t mention is whether GDI considers the political biases and ideologies of its reviewers. So how can we be sure it’s neutral? While GDI’s risk assessment focuses especially on how transparent a source is about its policies and staff, it seems less concerned about transparency on its own methodologies.

Being open with readers about your methods and the biases and ideologies of your staff is a key element of trust. Balancing those biases to measure and represent the judgment of average newsreaders – not just “experts” and journalists – is a critical component of rating and ranking news sources on bias, credibility, and the like.

The GDI methodology also states that “a rating of minimum risk in this study does not imply that disinformation will never appear on a given site. All newsrooms are vulnerable to disinformation risks, ranging from everyday human error to more nefarious tactics. Conversely, a maximum-risk rating does not imply that specific pieces of disinformation have been identified on a given site.”

That’s an important caveat, especially the bit about everyone being vulnerable. And as AllSides has documented (see below) sources on both the “low risk” and “high risk” sites have at times spread misinformation.

Past Examples of Misinformation from “Low Risk” and “High Risk” Sources

AllSides defines misinformation as “false or misleading information shared without explicit intent to deceive,” while disinformation is typically intentional. When a news source commits either infraction, it rightly raises credibility questions.

AllSides has documented several examples of misinformation from some of the “high risk” sites. For example:

Newsmax recently published a misleading claim from Sen. Rick Scott about cuts to Medicare benefits without adding context.
In July 2021, The Daily Wire incorrectly reported that California Gov. Gavin Newsom had implemented new COVID-19 restrictions. It later added a correction.
In July 2016, Breitbart linked to a fake ABC News article to provide evidence that pro-Trump protesters said they had been paid to protest.

We’ve also documented several examples of misinformation from some of the “low” and “minimum” risk sites. These include:

In January 2021, The New York Times falsely reported that a Capitol Police officer was beaten to death with a fire extinguisher at the Jan. 6, 2021 Capitol riot. It later issued a correction and update.
In April 2021, NPR reported that the Hunter Biden laptop story had been “discredited by U.S. intelligence.” It later made a correction.
In 2020, the Associated Press dubbed the COVID-19 lab leak theory a debunked conspiracy theory.

In explaining what each of the 10 high-risk and 10 low-risk sources performed well or not well, GDI states that readers “can rely on the AP for neutral, fair and well-developed reporting.” It says The Daily Wire’s content “runs the risk of manipulating readers’ emotional responses and disseminating biased interpretations of events.” It gives NPR credit “based on its neutral, factbased content.” It called out Newsmax for “significant frequency and degree of misleading arguments.”

Framing these outlets as being at opposite ends of the “misleading information” spectrum when they’ve committed similar falsehoods could itself be considered an example of misinformation.

The AP has undoubtedly done lots of good reporting. It’s also published an extensive amount of biased content in recent years – even by GDI’s definition of a biased outlet (one that fails to “include or unfairly engages with different views on the story”). For instance, AP’s Stylebook guidelines for journalists instruct them not to include both sides of transgender issues.

After a close and lengthy review process, AP’s leftward slant in politics and issues reporting led AllSides to shift AP’s bias rating from Center to Lean Left in 2022.

And as mentioned in the very first sentence of this article, AllSides assigned the Global Disinformation Index a rating of Left after our team conducted a review in 2021.

What is the Global Disinformation Index?

The Global Disinformation Index was founded in 2018 “as a not-for-profit entity built on the three pillars of neutrality, independence and transparency,” according to its About page.

In addition to its disinformation rankings, GDI also offers a Dynamic Exclusion List of “worst offending websites” it rates at “high risk for disinformation,” so companies can “defund and downrank these worst offenders, thus disrupting the ad-funded disinformation business model.”

The Washington Examiner (Lean Right bias) reported that, according to a “source close to ad-buying operations in right-leaning media,” it is one of the sources on the Dynamic Exclusion List.

It also responded to GDI’s 2022 disinformation risk rankings by highlighting U.S. government funds going to GDI, and framed it as an attempt by the left to blacklist conservative sources. In particular, the Examiner focused on how two State Department-funded groups have donated hundreds of thousands of dollars to GDI in recent years.

Other than The Hill’s (Center bias) Rising TV program, no left- or center-rated sources have covered the GDI report, which was published on December 16, 2022.

The Washington Times (Lean Right bias) and other right-rated sources highlighted how George Soros’s Open Society Foundation is also listed among GDI’s funders.

Be Wary of Bias in Credibility Rankings

GDI suggests that just because an outlet might display bias on some issues, it's at risk of pushing disinformation. That's misleading. By expanding the common definition of “disinformation” to include elements of media bias, GDI’s risk assessment does more harm than good.

Labeling some on the right as “high risk” and others on the left as “low risk” when they’ve committed similar infractions of bias and false information can discourage critical thinking.

This discrepancy could be attributed to semantics. But media bias and disinformation should be separate, rather than lumped together in a single “disinformation risk assessment.” Every news source is biased, but that doesn’t necessarily mean they publish disinformation.

Like other attempts to rate credibility, the GDI rankings appear biased in favor of mainstream, left-rated sources and against right-rated sources.

Rating credibility is well-intentioned, but very hard to do. Instead of relying on rankings that encourage you to ignore some news sources, it’s important to read content from across the political spectrum, ask questions, learn to spot bias, and think for yourself. That way, you can balance the claims and information you’re consuming from each side.

This is not a call for you to read right-rated sources and forgo left-rated sources. This is a call for balance.

People can build resilience to misinformation. We need to. That way, when it comes up, we can easily spot it.

RELATED: How to Spot Misinformation

A 1984-style “Ministry of Truth” is not the solution. Nor is informational anarchy, where no one can be trusted. The path to a more quality information experience begins with balance and consuming different perspectives.

Henry A. Brechter is the Managing Editor of AllSides. He has a Center bias.

This piece was reviewed by Daily News Editor Joseph Ratliff (Lean Left bias) and Julie Mastrine, Director of Marketing and Media Bias Ratings (Lean Right bias).

Misinformation Watch: 'Disinformation Risk Assessment' Lacks Transparency, Shows Bias Against the Right