The Problem

The Civil Media Company was founded to address the twin crises of trust and sustainability in news. The problem is enormous: more than 1 in 5 US newspapers have closed, leaving many communities so-called “news deserts.” And people often don’t know what information to trust: according to a study by the Reuters Institute, 55 percent of internet users are concerned about their ability to separate what is real and fake on the internet.

Our initial generative research, in which I and one other designer conducted over 50 interviews with journalists and paying news consumers, suggested that trust and sustainability are intertwined: people tend to choose to support large, well-known outlets financially, because they see them as trustworthy. And they’re often unsure about how to evaluate a news story’s trustworthiness without the context provided by a famous brand.

Key learnings from our generative research

News subscribers lack good heuristics for evaluating the credibility of a news story.

In general, paying readers are more likely to subscribe to a small number of well-established publications that they perceive as trustworthy based on their legacies — even though those aren’t the only places for quality reporting. This further shrinks the financial possibilities available to newsrooms that are local or simply newly established.

“When it comes to news unrelated to my field of work, sources I usually read are reputable, like The Atlantic, or New York Times. That helps me to understand credibility.” — Reader

We also learned that many people, even avid news readers who subscribe to multiple publications, often lack the tools to evaluate an article for themselves. When dealing with an unfamiliar source, readers often rely on factors like the appearance of the site or even the length of a given article to gauge credibility.

“I wouldn't click on something from a source I didn't know unless I started seeing that source a lot or if I knew the person.” — Reader

Finally, readers have a hard time differentiating between a well-reported story that uncovers new information and an article that’s an opinion piece or just a summary of others’ work.

Journalists lack a way to differentiate high-quality reporting, causing it to be undervalued.

Everything looks the same on the internet: news articles and misinformation look identical on Facebook. Journalists lack a way to differentiate high-quality, thoroughly reported work that actually brings new information to light from the work of those who just summarize others’ reporting or deliberately create misinformation.

“Everything looks the same on the internet.” — Journalist

Many people who create content online are overstretched or simply aren’t adhering to high journalistic standards. Few readers can tell the difference anyway, and this means misinformation gets repeated over and over and not challenged.

It also means the labor of journalism gets devalued: reporting that uncovers new information from sources takes time and learned skill and money, and it's harder to do if your work is valued similarly to work that simply rehashes others’ reporting and doesn’t adhere to the same high professional standards. The kind of reporting that people need in order to function as citizens is expensive, and when readers don’t realize that, it’s hard to fund it.

“We can’t treat journalism as a loss leader.” — Journalist

What I did

Our findings suggested to me that newsroom revenue and news literacy were connected, and that addressing news literacy was one place to start.

I created high-level wireframes to help the other stakeholders understand my idea, which was to build a tool that would not only serve as a sort of “nutrition label” for news, but would also empower readers to better understand how to evaluate a news story for themselves.

After that, working with one other designer, I created low-fidelity clickable prototypes, recruited people who fit the personas we had created, and conducted usability tests.

Of the two versions we presented, one was a clear winner, and we iterated on this one to create our final version. After consulting with journalists once again to write the final descriptions, we added this feature to the WordPress theme we were designing for the newsrooms that Civil was helping launch.

“I feel like these are things I could verify for myself if I wanted to.” — Reader

“To see that there is a reporter on the ground actually doing their job is great.” — Reader

“I would keep these criteria in mind — It's got me thinking about these criteria in general.” — Reader

How did we know which aspects of a news story were most important for readers to take note of to gauge the story’s reliability?

I dug into our interviews with journalists in order to better understand the most important ways that journalists believed themselves to be creating value and what the traits of the most valuable pieces were. We iterated through a few variations in our tests before settling on the phrasing that seemed most clear to our users. The following worked best: Original Reporting, Subject Specialist, On the Ground Reporting, and Sources Cited.

Outcomes and lessons learned

Civil’s Credibility Indicators

• give transparency into the reporting process so that people can learn about how the info in the story was gathered
• teach readers about what kind of fact-finding they should expect in a piece of journalism
• give readers the confidence to act on what they read (by sharing it with their friends and family or changing their vote based on what it says, for instance), because they can feel confident that an article is credible

“I was just on Block Club Chicago and saw the Civil credibility indicators. What a great thing! Somehow I missed that these were being rolled out. I am very impressed and think that this could be a huge step forward in certifying quality in our newsrooms. Well done!” — Reader

“Of all of the design elements created by Civil, the Credibility Indicators are by far the one that receives the most positive feedback. Not a week goes by that someone doesn’t tell us how they appreciate that innovation. It lets them know that a reporter was actually at the scene of the news, that it’s not an aggregated story, that real sources were quoted and that the reporter is a specialist on the subject. These are some of the most crucial aspects of reporting that newsrooms take for granted — but readers need to know! In this day and age of aggregation, ripped-off stories and yes, fake news, sometimes just telling readers that something is original and real goes a long way towards earning their trust.” — Shamus, Block Club Chicago

The Credibility Indicators we released weren’t right for every newsroom. Some newsrooms that Civil worked with focused on cultural criticism, personal essays, humor, or opinion pieces, and this news literacy tool didn’t apply to their work.

However, newsrooms that focus on hard news have found the Credibility Indicators useful and have consistently received positive feedback on them from readers.

I would have loved an opportunity to update the Credibility Indicators by improving the icons. I would also have liked to be able to extend their use to newsrooms that are not using the Civil WordPress theme. It would also be great to do some followup research to see how we could improve the feature.