Learning Objectives

After reading this chapter, you will be able to:

• Appreciate the need to vigorously evaluate the credibility of information.
• Differentiate between primary and secondary sources.
• Trace the primary source of information.
• Identify clickbait articles and content farms.

What You Know About Evaluating Information Isn’t Enough

You have been taught how to evaluate online information. This may have started all the way back in elementary or middle school, and it definitely happened in high school and college.

English or social studies teachers, or a librarian, cautioned you to be careful about what websites your research came from. You learned to confirm facts using two or more sources. You learned to consider who was the author of the information you were quoting, about checking what sources this author used, and whether there was bias in the author’s writing. Somewhere along the way, you also learned that Wikipedia is a terrible, horrible, not good, very bad source of information.

So far, these evaluation standards have served you well. But now that you’re in journalism school, you need to raise your information evaluation game.

Psychologists say that there are two ways in which we process information. The first way we process information is quickly and with little thinking, using tried-and-true heuristics. They call this type of processing System 1. A heuristic here means a simplistic signal that we use to form an opinion, or to make a decision. The second way we process information, System 2, is by being deliberate, carefully considering the varying dimensions of the matter that’s in front of us.

To understand this distinction better, think about the information processing we do while shopping. Sometimes a simple sign, like a banner screaming “20% off!” or a cute salesperson saying that some garment looks good on us, convinces us to make a purchase. That’s what System 1-based shopping decisions are like.

At other times we rack our brains about whether it’s the right time or circumstance to spend our money, whether the quality or price is quite right, what others will think of us, and whether we will regret our decision after we walk out of the store. Such deliberations suggest System 2 decision-making.

When it comes to evaluating the credibility of information, we have found that most of our students start this class using System 1 processing. This means that they pick up on simplistic signals in the information they are evaluating — heuristics — and use these to make their credibility judgments.

Some of our students’ favorite heuristics are a website’s top-level domain (e.g., Does the information come from a .com website?), an author’s credentials (e.g., Does the author have a Ph.D.?), and whether an article shows its sources (e.g., Does it have a bibliography or a Works Cited page?). Our students also often invoke the “I checked the information in two sources” heuristic. And all of our students assert that Wikipedia is not a good source of information, even though all of them regularly use Wikipedia as a source of information.

As a result of using the heuristics they have relied on for years, and System 1 processing, our students tend to start this class with credibility evaluations that are pretty shallow and not well thought out.

Your credibility is too valuable to rest on heuristics. Before using a source in your writing or in other media you produce, you need to consider carefully whether the source will add to or diminish your credibility.

In this and in the next three chapters, we present and practice evaluation tools that will help you consider the credibility of information at the systematic, deliberate, System 2 processing level. These strategies are organized in order of intensity, from least to most intense. We start in this chapter by discussing the CRAAP model and its shortcomings, and the importance of distinguishing between primary and secondary sources, including how to rabbit-hole toward primary sources. We end this chapter with a discussion of clickbait and content farms. We hope that you will become proficient in all of these strategies, and use elements of all of them whenever you have to consider the credibility of an information source.

Oh, CRAAP

CRAAP stands for currency, relevance, authority, accuracy, and purpose. It is a method for evaluating information that has been taught to many students, in many schools, for many years. You may have been taught CRAAP, or elements of it, in your prior schooling.

In the past, CRAAP may have been a good place to start with credibility evaluations, but we believe that this method leads you to rely too much on simple evaluation heuristics like the ones we mentioned above.

As librarian Kevin Seeber has argued, a simple checklist for evaluating information does not make sense in the face of the misinformation and disinformation that infiltrates our society today. Seeber suggests that we spend more time mulling over the concept of authority and accept that evaluating sources is a complicated process.

If you have used CRAAP before, the evaluation tools we present in this book are like CRAAP on steroids. Our goal is to help you think much harder about evaluating information.

Primary and Secondary Sources

A key question that information experts must ask when coming across a new piece of information is: Is this a primary source, or not?

The attribution chapter introduces and provides examples of primary and secondary sources. Here, we consider what primary and secondary sources have to do with credibility.

A primary source generally is produced at the time of the event, according to a Society of American Archivists (SAA) definition. Primary source documents contain first-hand and the most authoritative evidence of something being the case. Examples of primary sources include diaries, interviews, photographs, letters, newspaper advertisements, news or audio footage, official records and some government documents.

Secondary sources cite information from primary sources, or provide interpretation or analysis of primary sources, according to the SAA definition. Examples include most news articles, books and editorials.

(Note: Historical news articles are usually considered primary sources. By “historical,” we mean, like, 100 years old. For more on this, read the Archives chapter.)

Journalists and strategic communicators strive to be the secondary sources of the information they present to their audiences. They rarely are the originators of information, that is, the primary sources. So being the secondary source of information is the most credible position for journalists and strategic communicators to hold.

It’s like the game of telephone we played as kids: the closer we sat to the origin of the telephone message, the more accurately we communicated the message to the next person in the chain.

Establishing credibility by using credible sources lies in the ability to track down the primary sources of information. Provenance, which means the origin of something, is particularly important in journalism and mass communications. Only by looking at the primary source of information can we be sure that a secondary source is accurate. As a future communication professional, you need to develop an instinct to find and cite primary sources.

Rabbit-holing Bill Self’s words

Let’s consider an example of how primary and secondary sources work in journalistic research.

Read this Kansas City Star article about KU men’s basketball facing off against Syracuse and consider this question: Is this a primary or a secondary source?

Answer: It is a secondary source because it quotes the head coach, Bill Self, extensively about the upcoming game and the status of his players.

Another question: If you also were writing an article or report about this game, or about the team at this point in the season, would it be acceptable for you to use this article as the source of Bill Self’s quotes?

Answer: No, because that would make you a tertiary (that means, third) source of Self’s words. As a journalist, you want your work to be the secondary source, not further down the source chain.

So how can we trace Self’s quotes to the primary source? The process of tracing back a primary source is sometimes called rabbit-holing. Like a rabbit, we need to enter a tunnel of sources that, ultimately, will lead to the primary source.

The caption of the primary image of the article states that Self spoke to reporters on Nov. 30, 2017. In other words, Self held a press conference on this date.

We know that there were plenty of other reporters at this event, so one of the local news stations might have recorded the press conference. But this would still be a secondary source, and we want to find the primary source.

We know that KU Athletics has its own public affairs team that handles media relations and communications for the team. So if we search for “KU Athletics press conference Syracuse,” for instance, we might be able to track down first-hand evidence of this press conference.

Lucky us, on the KU Athletics website, there is a transcript and video of this press conference. This is the primary source, the end of this rabbit hole. If you want to quote Bill Self in your article or report, this is the source you will read and listen to, and then cite.

Who cares the most, and can pay for, being primary?

When tracking down primary sources, you should try to think of who has a vested interest in preserving the information, and who might be able to afford to compile the information.

Think about it. KU Athletics would probably have the most interest in putting a recording of the press conference out there. Doing so would ensure that the press would be able to report on the upcoming game, even if they weren’t able to send a reporter to the press conference. It also would ensure that reporters accurately relayed what Self said. By putting the press conference online, KU Athletics minimizes the chance of a news outlet misunderstanding and misquoting its star coach.

But all of this publicity and accuracy comes at a cost. To record and transcribe a press conference requires a lot of expensive video and audio equipment. You would likely also have to pay several people to set up the press conference, communicate its happening to the press, and record the whole thing. Then you need money for a person or program to transcribe what was said, before you fork over some money for people to double-check the transcription, write a press release, create the web posting, and send a notice out to folks that it was all available. Did we mention that you would have to do this as quickly as possible so that the information was still timely and accessible to the press to publish on their deadlines?

Being a primary source, or being the custodian of a primary source, requires a vested interest and resources. Considering these two elements can help you efficiently identify and track down primary sources.

Clickbait and Content Farms

Two other initial questions that anyone who comes across new information on the Internet should ask are: Is this clickbait? and Am I on a content farm?

Clickbait is the intentional act of over-promising or otherwise misrepresenting — in a headline, on social media, in an image, or in some combination of those — what a reader is going to find in an online story, according to Techcrunch. The Washington Post also described clickbait as a teaser headline without context to what a news story is really about.

Why is clickbait not credible? It’s because the quality of a clickbait article is secondary to the article’s ability to attract readers.

Clickbait is not about quality information, but about advertising rates and making money from advertisements. Online publishers charge advertisers based on how many people view (that is, click on) these publishers’ content. The more traffic that a web page generates, the more money a publisher can charge for the advertising space on this web page. Clickbait drives up traffic, which drives up advertising rates, which drives up publisher revenue.

Online publishers that specialize in clickbait are sometimes referred to as operating content farms or content mills. A content farm is a website that exists solely for the purpose of generating advertising revenue from content that produces a lot of interest and web traffic. The more that a content farm can masquerade as a legitimate information source, the more likely it is to dupe readers into clicking on its content.

Examples of content farms abound. Columbia Journalism Review reported that Slant magazine incentivized their writers to generate clickbait. The magazine offered writers $5 for every 500 clicks their articles generated. RealNewsRightNow is a website that parodies  media outlets by posting information that reads like legitimate news headlines. The website was created solely to generate advertising revenue. And in the months leading up to the 2016 presidential election, two men made thousands of dollars by posting fake news stories about both the Democratic and Republican presidential nominees on their clickbait website LibertyWriters.

The insidious problem with clickbait is that because this content is engineered to match what people search for, and because it generates a lot of web traffic, clickbait content tends to rank high in Google search results. Since we tend not to look very deep in search results, we often end up reading clickbait.

Activity: Get Primaried

Below are two secondary sources. Select one of these sources and locate the primary source from which the secondary source was created. You must provide the hyperlink to the primary source and explain how you found and evaluated its credibility.

• This is a Lawrence Journal-World article about Lawrence commissioners debating regulations on where and when food trucks can operate in the city. This is a secondary source.
• This is a KCUR radio story and its transcript about a study on the potential cost-savings of using Uber in place of ambulances. This is a secondary source.

Activity 2: Take the click bait

Evaluate the credibility of Slant magazine , which was identified as a click bait farm in this chapter. You may consider examining one or two particular articles in addition to reading through the website’s staff listings and terms of service. Try to find as many cues as possible that would indicate Slant is a click bait farm. What are these cues and how do these cues affect your opinion of Slant’s credibility?