Information systems professor puts product recommendations through the paces

Associate Professor of Information Systems Zhan (Michael) Shi has addressed product recommendations from a new angle: How should platforms select various products for recommendation?

As potential product buyers scroll through seemingly endless options, they encounter price-based, sales-based, or human-curated recommendations. Many researchers have evaluated product preference predictions for accuracy.

Associate Professor of Information Systems Zhan (Michael) Shi has addressed product recommendations from another angle: How should platforms select various products for recommendation? The findings were published in the June 2020 Information Systems Research in a paper titled: “An Economic Analysis of Product Recommendation in the Presence of Quality and Taste-Match Heterogeneity.”

The pursuit was propelled by a previous project. “My co-author, Raghu Santanam, and I worked on a project with our PhD student Chen Liang (now an assistant professor at the University of Connecticut) where we used an Apple AppStore dataset to study the impact of editorial recommendations,” Shi says. “Our focus was to empirically examine the spillover effects of platform recommendations (on products that are not recommended), taking the recommended products as given.”

The group also pondered different questions. How did the AppStore editors pick apps for recommendation? What is their goal? What are the criteria? Since a tiny amount of information had been disclosed by Apple publicly, Shi and Santanam examined theoretically how a market-operating platform, such as the AppStore, should select products for recommendation. “We realized there might be an opportunity to write a modeling paper on this topic,” Shi observes. “We thought it was important given the significant influence of the platform over the market outcome.”

The product recommendation gap

The research team designed the paper to provide a systematic analysis of the implications of platform recommendations. The model yields predictions on the types of products that should be picked for optimizing specific market outcomes. The analysis allowed them to compare the various implications of product type and popularity-based recommendation strategies. By considering product heterogeneity in both vertical and horizontal dimensions, the researchers illustrate there are very different impacts on platform recommendation strategy. Optimality cannot be guaranteed by using the observed price and sales signals.

Since the product space on the majority of market platforms is quite large, Shi and Santanam assumed there is an infinite number of products and each consumer is looking to purchase — at most — one product. That search process includes a search cost to consumers to evaluate consumption utility and price for any product. To reduce the burden on consumers, market platforms utilize product recommendations often under listings such as “Best Sellers” or “Interesting Finds.” One specific example: Apple’s iOS App Store. The platform publishes rankings of the most-downloaded apps and highlights editor-curated lists.

Everything else being equal, products that are not recommended would be less visible to consumers. Considering there is a gap between gains for recommended products and potential losses for non-recommended items, Shi and Santanam zeroed in on evaluating the equilibrium market outcomes, and those included trade-offs that come from a platform recommending specific products.

That boiled down to answering three questions. How do platform recommendations impact equilibrium consumer surplus and producer profits? What type of products should the platform pick for recommendation under different objectives? Are frequently used recommendation strategies based on price and sales optimal in terms of platform-level outcomes?

That required the research team to examine consumers’ responses when they have incomplete product information. As consumers look at product utility and price while sampling products, there is going to be a stopping point.

The problems with bestseller lists

Shi and Santanam examined situations where products vary in quality and/or taste match. By starting with a scenario where consumers sample products in a random order, the researchers were then able to change the consumer search by putting the recommended products at the top of the search list. That comparison illustrated a theoretically optimal recommendation strategy for a platform. “Due to the nature of the model, the conclusions are technical,” Shi notes. In general, we found the platform’s strategy depends on how its interest is aligned with consumer surplus and product profits.”

A handful of variations often combine to affect product recommendations. “We found the quality and taste-dispersion dimensions can interact to affect the overall effectiveness of product recommendation strategies,” Shi says. “Conditioning on taste dispersion, recommending high-quality products increases both producer profits and consumer surplus. Conditioning on quality — recommending high-taste dispersion products — may increase or decrease producer profits depending on the joint effect of profit margin and purchase probability.”

Shi and Santanam also address several practical implications beyond the immediate scope of product selection for recommendation. Their result regarding sales-based recommendations complements existing knowledge about market effects of platform — published sales rankings, such as bestseller lists. Existing research and literature have illustrated that bestseller lists not only reflect consumers’ buying histories, they also often directly influence consumer behavior. A bestseller list in a crowded field can lead to the bestseller list being used as a product-discovery method.

The resulting environment favors already-successful products, which tends to stifle product variety. It leads to a ‘rich get richer’ field for sellers. “To the extent bestseller lists are used by consumers as a product discovery channel, we conceptualize their role is essentially to promote the best-selling products in the consumer search sequence,” Shi says. “In this sense, bestseller lists can be seen as sales-based recommendations within our framework. Our finding shows that, in addition to having drawbacks for product variety and equality, the effects of bestseller lists can be suboptimal, even in terms of pure market efficiency.”

What’s next for product recommendation research?

Given these conclusions, Shi and Santanam highlighted a handful of ways they can build upon this current research. Platform recommendations could be incorporated as part of the market outcome rather than as an intervention. Considering that Shi and Santanam have evaluated only creators who are producing and selling their products, a longer-term effort could include examining how platform recommendations will influence creators’ decisions on variables such as entry, investment, and innovation.

“The project prompts me to start thinking about the broader question of how the platform should provide information to market participants to optimize market outcome,” Shi concludes. “Highlighting recommended products is one specific form of providing information. I hope to generalize the modeling framework to encompass more practices and for us to be able to analyze the impact and provide some unified understanding.”