The Economist’s Analysis of Facebook Media Bias Misses the Mark

A recent examination of media bias on Facebook caught our eye here at AllSides. It is an important topic and we found the report to be interesting, but we also found some problems with it that lead to invalid conclusions.

We’ve identified a number of flaws in the methodology used in a report from The Economist, "Facebook offers a distorted view of American news," which says Facebook may “amplify extreme views” even if it isn’t designed to. The graph and explanatory article suggest that “the most prominent news sources on Facebook are significantly more biased than those found elsewhere on the web.”

This insinuation may or may not be accurate, and we support the effort to investigate the effects of social media on news consumption. But we identified several significant flaws with The Economist’s report that make the chart and accompanying article’s conclusion incomplete — and potentially false.

A Limited Approach, A Flawed Conclusion

The first paragraph of the report states that, based on Facebook data, “the two most popular American media outlets on the site last month — as measured by shares, views, comments and other forms of “engagement” — were Fox News and Breitbart, two right-wing news sites.”

“Engagement” could mean people are disagreeing or debating the content, or that they’re agreeing with it and promoting it. The Economist’s report doesn't account for that possible difference. More biased content is typically going to be engaged with on social media more than neutral content because it takes a stance, therefore provoking conversation. The report frames this as "promoting extremism”, but the evidence cited doesn’t really support that. Not all bias represents extremism.

A type of media bias called flawed logic is evident when comparing the report’s headline to its conclusion. The evidence cited doesn't suggest that “Facebook offers a distorted view of American news," as The Economist’s headline says, but rather that Facebook users might engage more with biased news. The fact that certain sources are more engaged with than others doesn’t necessarily mean users, or Facebook, are agreeing with them or supporting their perspective.

In addition, by only looking at 35 of the thousands of news sources on Facebook, The Economist’s analysis begins down a narrow path. Our analysis has found the online news media sphere overall skews left, so looking at 35 highly-trafficked sources may not be expansive enough to return a realistic projection.

Using a very small sample to draw conclusions about how Facebook may be biased on the whole is comparable to someone clicking on an op-ed and — before reading beyond the first few paragraphs — concluding that the author’s argument is imbalanced.

As we said, many efforts to examine bias are commendable. But The Economist also solely uses the ratings from Ad Fontes to determine political bias, another limiting feature of the methodology. Ad Fontes does good work, but rates roughly 125 sources and does not separate news sources’ opinion content from their news content.

By comparison, AllSides rates over 800 sources, and is the only media bias rating system to separate the news and opinion content for sources where bias differs between the news and opinion sections. That is an important distinction when assigning bias ratings to content.

Other data points — like how much content a source posts to its Facebook page overall, or which sources have more Facebook followers — could have also added more depth to the report.

Different Sources, Mediums have Different Audiences

It is important to recognize that the audiences using different media — television, radio, newspapers, social media — vary demographically and in their average political leanings.

Pew Research Center has found that audiences of left-leaning news outlets tend to get their news from a wider range of sources than right-leaning audiences, who tend to rely heavily on Fox News and a handful of other outlets. This consideration was not mentioned in The Economist's report, and likely ties back to the disproportionate number of “legacy” news outlets (NBC News, CNN, New York Times, Washington Post) that lean left versus right.

Some studies of media markets in certain cities have shown that more Democrats use TV for news, while more Republicans use the radio. In addition, certain social media platforms also attract different audiences —Pew has found Facebook to be popular across all demographics, while concluding that Twitter users are more likely to lean left.

In short, The Economist’s conclusion is premature. It is not necessarily wrong, just not sufficiently supported.

The report suggests that “whatever Facebook’s intentions, the social-networking site has more of a political slant than Mr Zuckerberg lets on.” Among other things mentioned above and below, this fails to consider the possibility that people who Facebook users may lean more left or right than the general population. In theory, this would mean audience demographics, not Facebook itself, are skewed.

People are Focusing on the Wrong Thing About Facebook

Facebook is, in many ways, analogous to AT&T.

If we had a way of viewing conversations between average people over the telephone in the late 1970s or early 1980s, we may have seen a very right-leaning conversation overall, preceding Reagan’s election and amid his landslide re-election. In the early 1990s, when Clinton won, perhaps there would have been more left-leaning phone conversations happening. But this wouldn’t have served as evidence that AT&T “amplifies extreme views” or that it institutionally leaned right in the early '80s and left in the early '90s.

The real danger with Facebook is the filter bubbles it can create for people on the left and on the right. When we are only exposed to information and ideas we already agree with, it makes us more extreme in our beliefs, less tolerant, and vulnerable to manipulation.

A study from the University of Colorado noted that “people's attitudes become more extreme after they speak with like-minded others.” That is an unintentional problem with how Facebook drives us to communicate with each other and learn about the world.

Like many modern media tools (YouTube, Twitter, search engines), Facebook just seems to enable people to go into more extreme bubbles. It can also be further manipulated by publishers in various ways to encourage clicks and engagement, who disregard the consequences of profit-focused news content.

A Better Way to View Media Bias

We encourage the analysis of media bias, and consider it vitally important. It is also important for all of us to understand the validity, strengths, weaknesses and inherent biases of those analyses as well. Many come from good intentions, but may have their own limitations or biases — others may have their own hidden agendas.

The Economist’s recent approach has good intentions and does some good work, but there are some problems and limitations we should consider. At AllSides, we stress that the measure of bias is largely subjective, and therefore must be done carefully and thoroughly in order to represent reality. Other media outlets, even the most established and trusted ones, aren’t always as mindful.

More important than defining any bias rating, we must constantly fight these filter bubbles. They are what drives much of the polarization and dysfunction we see in society today. At AllSides, we use technology to ensure that people see left, center and right views of the same story to give them a more complete and balanced view.

We hope our Balanced News, Balanced Search and other features help you out of your own filter bubbles and reveal perspectives you may have missed. Email us with feedback or suggestions.

This piece was written by AllSides CEO John Gable (Lean Right bias) and Managing Editor Henry A. Brechter (Center bias). It was reviewed by Director of Marketing Julie Mastrine (Lean Right bias) and Daily News Specialist Joseph Ratliff (Lean Left bias).

Image Credit: shopcatalog.com/Flickr