Research

Online searching: Flawed queries can lead to poor decision-making

Eye-opening research conducted by Assistant Professors Carola Grebitus and Rod Roscoe shows that we may not be as adept at online searching as we’d like to think.

2017-06-06

We move fast, overlook bias in websites, and seem satisfied with little diversity in our sources.

Does this sound familiar?

You need information, grab the nearest device, and plug in a few search terms.

Minutes — maybe seconds — later, you find what you’re looking for and move on. Asked and answered, right?

Not so fast, says Assistant Professor of Agribusiness Carola Grebitus and her colleague Assistant Professor of Human Systems Engineering Rod Roscoe

Enlightening research conducted by the pair shows that we may need more guidance in our online searching capabilities. We rush, miss prejudice in websites, and seem convinced with scarce variety in our references, according to their paper, “Online information search and decision-making: effects of web search stance,” published in the journal Computers in Human Behavior.

We are less inclined to take the time to question and be skeptical, Roscoe says. “We love that easy answer. We are also guilty of narrowly-focused searches."

“You want to buy something, and you go to one website, you click through to something else, and then click through to something else,” Grebitus says. “You go in one direction most of the time.”

The bottom line, according to the research:

“The combination of flawed digital literacy skills and mixed messages creates a treacherous environment for decision-making. When information seekers make decisions based on lower-quality searches and biased information, the chances of dissatisfaction or harm are likely increased.”

The research brought together Roscoe, an assistant professor in the Human Systems Engineering Program of The Polytechnic School, and Grebitus, an assistant professor of food industry management at the W. P. Carey Morrison School of Agribusiness, for a genuinely interdisciplinary collaboration. The pair conducted their research with the help of undergraduate, graduate, and doctoral students Adam C. Johnson, Joseph O'Brian, and Irfan Kula.

“That’s one of the great things at ASU. We include students in this kind of research, and they have this learning experience,” Grebitus says.

Online obstacle course

Roscoe and Grebitus started with the premise that online information searches expose us to a broad range of websites and conflicting ideas and perspectives. In the study, they evaluated how the holistic stance of a web search — positive or negative — influences our purchasing decisions when researching a particular product.

They conducted this study against a backdrop of research showing that many obstacles stand between us and a robust and reliable online search.

• Many of us lack strong digital literacy skills, which can result in online searches that yield inaccurate data or overlook critical information.
• Both overt and subtle efforts influence our web searches. Everything from blatant advertising and celebrity endorsements to site design and interactive experiences can impact decision-making. These digital promotions are not limited to just commercial sites; news and informational sites can sway us depending on how they present data and narratives.
• When we search online, we land on multiple sites that give conflicting information and arguments. Does bottled water “harm the environment” or is it “safer than tap water?” It depends on the search and site.

For their study, Roscoe and Grebitus wanted to focus on a product that was both accessible and subject to debate, Roscoe says.

The ubiquitous plastic water bottle, particularly relevant in sunny Arizona, fit the bill. The topic was of particular interest to Grebitus, a senior sustainability scholar at the Julie Ann Wrigley Global Institute of Sustainability.

Though not as controversial as some subjects, such as vaccines, water bottles generate plenty of discussion over their environmental, health, and economic impacts. As the study noted, even a quick online search for bottled water turns up numerous websites that present both positive and negative stances.

Searching, sites, and stance

For the study, the team recruited 109 mostly college-age participants. They were asked to estimate their knowledge about bottled water, the frequency of consuming different types of water, and drinking water preferences. They were also asked about their online searching skills and beliefs in the accuracy of online information.

Participants were asked to consider three alternatives concerning bottled water. One alternative was simply not to buy any bottled water. The other options included buying bottled water and considering different attributes such as price, type of water (purified, natural artesian, mountain spring), and type of plastic (no label, plant-based plastic, post-consumer waste).

With just 10 to 12 minutes to search online, participants were told to find information related to bottled water that would guide their choices about bottled water purchases.

Some participants assigned to an “environment condition” were given these instructions: “Imagine that you are shopping for a six-pack of 16.9-ounce bottled water and trying to make the most environmentally friendly purchase.” In their searches, they were told to “gather information that helps to guide your choices about environmentally friendly bottled water purchases.”

Participants without the “environment condition” received very similar instructions, except references to the environment and environmentally friendly, were removed.

Researchers analyzed the internet searches, looking at query terms, site visits, and stance. They also placed the websites into five categories: aggregator, commercial, forum, informational, and news.

On average, participants conducted six queries and used eight different terms. Participants tasked with the environmental goal recorded more questions and search terms than participants without that goal. Search terms ran the gamut, from artesian well and mountain spring to pollution and plant-based, to types of plastic and comparison.

The study showed that participants opted for more superficial searching, possibly a result of the time constraints.

“Participants used very few terms that tapped deeper background knowledge or analysis,” according to the study. “For example, when evaluating bottled water products, it may be useful to learn about industry regulations or safety reports. However, participants incorporated less than one of such terms in their searches, on average. Although they may have encountered this information on visited sites, participants did not seek such knowledge directly.”

The team made additional findings:

• Query terms influenced the types of sites obtained in a search, which in turn shaped the overall search stance. Both aggregator and commercial sites were commonly encountered, and participants’ search strategies had a “meaningful impact” on the kind of information — and possibly the quality of information — they obtained.
• More than one-third of participants' visits were to commercial sites, and about one-fourth were to aggregator sites that offer a curated collection of links and articles that mix opinion, journalism, and advertising. So participants spent most of their search, nearly 58 percent, viewing the most potentially biased sites.
• When participants plugged in environmental and analytical search terms, they were more likely to visit sites such as nytimes.com (The New York Times) or foodandwaterwatch.org that bills itself as an advocate for healthy food and clean water. Participants were less likely to visit sites such as bottledwater.org (operated by the International Bottled Water Association) or dasani.com (a commercial site) that advocated for bottled water purchases.
• Aggregator sites were more likely to present bottled water as harmful to the environment, economically problematic, and damaging to one's health.
• Participants were more likely to buy bottled water when they visited websites that emphasized environmental, economic, or health benefits for bottled water. These sites took a positive stance.
• Participants who encountered a more negative stance were less likely to purchase bottled water.
• When asked to think specifically about environmental impacts, participants were less inclined to buy bottled water. If they did choose to buy, they tended to choose bottles made from recycled plastic and were less likely to buy water from mountain spring sources.

According to the study, the most critical finding related to the influence of a search stance — positive or negative — is on our decision-making. Even a roughly 10-minute search could nudge a participant in one direction or another, Roscoe says.

Though researchers focused on water bottles, the lessons learned in the study apply to any decision-making using online searches. Whether we are searching for a recipe or information about vaccines, we should be concerned with the quality of the answers we find, Roscoe says.

And that begins and ends with how we go about getting those answers. “It’s still the same search process,” he says.

In the know

To boost the quality of online searching, Roscoe and Grebitus offer several recommendations:

Know sources.Have any idea where your information comes from — retailers, government, public institutions, advocacy groups, and more. “Be a bit more aware,” Grebitus says. “Are the first two hits in your search from advertisers?”

Be mindful. Seek out different perspectives and look at the evidence. “It comes down to who has the evidence-based website, not the slickest website,” Roscoe says.

Check yourself. Be aware of your bias, which can cause you to avoid dissenting voices and information. “We like to confirm what we already believe. We don’t want to challenge our beliefs,” Roscoe says.

Education matters.Try to inform yourself about a topic. With the water bottle example, for instance, learn basics about recycling such as the differences between type 4 and type 2 plastic. “The more knowledgeable we are, the more we can evaluate the evidence,” Roscoe says. “If you don’t know anything about recycling, you can’t make an informed choice.”

Think first. Don’t take information at face value; critical thinking is necessary. “Don’t just read information, but read it with a purpose,” Grebitus says.

Agribusiness