Last week, Twitter released a 27-page, in-depth analysis that relied significantly on AllSides Media Bias Ratings™ to explore whether Twitter’s recommendation algorithms amplify political content.

The report found media outlets AllSides rates as being on the right are amplified slightly more — 4% more — than outlets AllSides rates as on the left.

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

A post shared by AllSides (@allsidesnow)

Among other things, the analysis sought to examine whether Twitter’s algorithm amplifies certain types of political groups more than others, and whether some news outlets are amplified more than others. Twitter also looked at whether news media algorithmic amplification favors one side of the political spectrum more than the other.

“To study algorithmic amplification of news outlets, we analyzed hundreds of millions of Tweets containing links to articles shared by people on Twitter during the same time period,” Twitter writes on its blog. “The outlets were categorized based on media bias ratings from two independent organizations, AllSides and Ad Fontes Media.”

Overall, Twitter found that tweets posted by accounts from the political right received slightly more algorithmic amplification than those from the political left. News outlets that AllSides and Ad Fontes rate as right-leaning also see slightly greater algorithmic amplification on Twitter compared to left-leaning news outlets.

https://www.allsides.com/sites/default/files/Screen%20Shot%202021-10-27%20at%2011.32.57%20AM.png

Twitter found content from outlets AllSides rates as having a Left media bias is amplified 12% by its algorithms. “The most significant negative outlier in this group is BuzzFeed, with an amplification of -2% compared to the chronological baseline,” Twitter wrote. “By contrast, Vox is amplified 16%.”

Content from outlets AllSides rates as being on the Right are amplified 16% by Twitter’s algorithms. The more significant negative outlier in this group is Breitbart, with an amplification of 12%, while New York Post is amplified by 24%.

About Twitter’s Algorithm Bias Analysis

The analysis examined tweets from elected officials in seven countries (Canada, France, Germany, Japan, Spain, the United Kingdom, and the United States), as well as political content from news outlets.

Twitter looked at its algorithmically ranked Home timeline (the option to see tweets that the algorithm feeds you based on who you follow and interact with; this option also shows you content Twitter thinks you may be interested in) as well as its chronological timeline (the option users have to view the most recent tweets in reverse chronological order).

The company found that tweets from elected officials about political content are amplified by the algorithm compared to the chronological timeline, regardless of party or whether the party is in power. When it comes to right-leaning news outlets being more amplified, Twitter notes that AllSides and Ad Fontes media bias ratings are not exactly the same, so “the results of analysis may vary depending on which source is used,” Twitter noted.

“Algorithmic amplification is not problematic by default – all algorithms amplify,” Twitter researchers wrote. “Algorithmic amplification is problematic if there is preferential treatment as a function of how the algorithm is constructed versus the interactions people have with it.” The content that a Twitter user sees on their Home timeline is “a function of how they interact with the algorithmic system, as well as how the system is designed.”

Twitter Methodology and Difference Between Ad Fontes and AllSides Media Bias Ratings™

Twitter researchers identified tweets containing links to articles shared by anyone between April 1, 2020 and August 15, 2020. They excluded tweets containing non-political content such as recipes or sports. They also separated editorial content from general news coverage. 

The resulting dataset contained AllSides annotations for 100,575,284 unique tweets pointing to 6,258,032 articles, and Ad Fontes annotations for 88,818,544 unique tweets pointing to 5,100,381 articles.

Twitter researchers write:

“When using AllSides bias ratings (Fig. 2A), two general trends emerge: The personalization algorithms amplify more partisan sources compared to ones rated as Center. Secondly, the partisan Right is amplified marginally more compared to the partisan Left. The results based on Ad Fontes bias ratings (Fig. 2B) differ in some key ways. Most notable is the relatively low, 10.5%, amplification of the partisan Left compared to other categories. Among the remaining categories, the differences are not substantial, although the Neutral category is amplified significantly less than other categories.”

“This analysis identified BuzzFeed News, LA Times and Breitbart (based on both AllSides and Ad Fontes ratings) as negative outliers in their respective categories, meaning the amplification of their content was less than the aggregate amplification of the bias category they belong to. Meanwhile, Fox News and New York Post were identified as positive outliers. These outliers also illustrate that, just as we saw in the case of legislators, there is significant variation among news outlets in each bias category. 

The fact that our findings differ depending on the media bias dataset used underlines the critical reliance of this type of analysis on political labels. We do not endorse either AllSides or Ad Fontes as objectively better ratings, and leave it to the reader to interpret the findings according to their own assessment. To aid this interpretation, we looked at how AllSides and Ad Fontes ratings dier, where both ratings are available. We found that while the two rating schemes largely agree on rating the political right, they differ most in their assessment of publications on the political left, with a tendency for Ad Fontes to rate publications as being more neutral compared to their corresponding AllSides rating. Details are shown in Supplementary Figs. S3, S4 and Table S1.”

As mentioned above, Twitter noted that Ad Fontes has a tendency “to rate publications as being more neutral compared to their corresponding AllSides rating.” In other words, Ad Fontes often rates media outlets that AllSides marks as left-leaning as being neutral. For example, Ad Fontes rates The New York Times and Washington Post as neutral, while AllSides rates both of those outlets as Lean Left (AllSides also rates The New York Times’ Opinion section separately as Left). 

This explains why, when Twitter uses the AllSides bias ratings, it finds the right is amplified only “marginally more” than the left, while it sees a stronger difference using the Ad Fontes’ ratings.

(Notably, AllSides doesn’t designate any outlet as “neutral” like Ad Fontes does — our Center rating doesn’t mean an outlet is neutral, has no bias, or is perfectly balanced and fair. It simply means its bias isn’t clearly leaning one way or the other. Center sources aren't necessarily better, more reliable or more balanced — they can still omit important perspectives that you may only get from a left or right outlet. It should also be noted that some AllSides Media Bias Ratings have changed since Twitter used them for its analysis.)

The AllSides system is unique in that it uses a patented methodological system, including Blind Bias Surveys, to evaluate media bias. A mixed, balanced sample of people across the country take the AllSides Blind Bias Survey. That means our ratings reflect the average judgement of Americans, not one elite group. On the flip side, Ad Fontes uses a team of analysts to rate bias and uses a rubric in the analysis designed by Ad Fontes themselves. Our ratings reflect the average judgement of all Americans, while Ad Fontes primarily reflects the judgement of its internal group of experts.

 

 

Conclusion

We’re pleased to see Twitter exploring potential bias in its algorithm using AllSides Media Bias Ratings™, and hope to see more research of this type in the future. As social media sites continue to drive political polarization, we hope AllSides’ work can shine a light on their impact.

Julie Mastrine is the Director of Marketing and Media Bias Ratings at AllSides. She has a Lean Right bias.

This piece was reviewed and edited by Managing Editor Henry A. Brechter (Center bias), CEO John Gable (Lean Right), and Research Assistant & Data Journalist Andrew Weinzierl (Lean Left).