Students

Podcast: Deciding how to decide

How does your organization make decisions? Thomas Davenport, a professor of Information Technology and Management at Babson College and author of "Competing on Analytics: The Science of Winning," has taken a systematic look at the way decisions are made. He suggests that companies would benefit by examining the kinds of decisions they make, and the alternatives for approaching them. In this two-part conversation with knowIT, Davenport discusses analytics, the changing role of the executive and the future of decision making.

2010-03-31

How does your organization make decisions? Thomas Davenport, a professor of Information Technology and Management at Babson College and author of "Competing on Analytics: The Science of Winning," has taken a systematic look at the way decisions are made. He suggests that companies would benefit by examining the kinds of decisions they make, and the alternatives for approaching them. In this two-part conversation with knowIT, Davenport discusses analytics, the changing role of the executive and the future of decision making. (Transcripts follow) Part One: [podcast id="1"] Part Two: [podcast id="2"] Transcript: Part One knowIT: In your recent Harvard Business Review piece, "Make Better Decisions," you talk about changing not only how decisions are made, but also how they are executed. And that sounds like a pretty fundamental reform of how organizations operate. Is this reengineering the corporation 3.0? Thomas Davenport: Well, I would like it to catch on certainly, in the way that reengineering did, and I think in the same way that reengineering involved a systematic look at business processes, I would certainly like the world to engage in a systematic look at their decision making. It is hard to know what will catch on and what won't. I think reengineering was partly a result of the Japanese. They seemed a lot more competitive than we were, and they thought that might be part of the secret sauce. I would certainly argue that we need it as much as we needed reengineering of our processes in the past. knowIT: Yet, you write that business people, while they are aware of the kind of decision making alternatives that you talk about, they are not adopting them. And you talk about the representative companies that are adopting these positive decision-making processes, but not all companies are. Davenport: Yeah, yeah. I think some of the things I talked about, they are not particularly aware of yet. I mean, the neurobiology stuff, and the wisdom of crowds stuff, and the behavioral economics stuff -- but I think that is just really getting starting in managers' consciousnesses. I think I said in the article that the groupthink idea was identified by Irving Janis over 50 years ago, and so not much excuse for not being aware of that. The whole idea that you would have a role established to propose decision alternatives, I think was established by the Catholic Church in the 15th century. So, we ought to be aware of that one. I do not know what it is that keeps these innovations, if you want to call them that, from being adopted by decision makers, but we have got a long way to go, I think. Diffusion of innovation seems to be quite ineffective in this particular case. knowIT: It's interesting to me, because the results that you talk about, the results of your study, the economic and financial results that these companies see from changing the way they make decisions, are dramatic. So, if the results are dramatic, you would think, I would think, that executives would be jumping onboard. Davenport: Well, they are. Interestingly enough, I would say the analytics work that I have done was, to me, surprisingly popular and even independent of anything I did. The world seems to be moving in a more analytical direction. Decision making though -- which I view as kind of an extension, analytics are one approach to making an effective decision. Decision making, I think, is all tied up in organizational politics and people's feelings about the prerogatives of being a Senior Executive. So, I do not know ? maybe it is just that executives do not want somebody mucking around in their decision processes. I do not know why we continue to make the bad decisions that we do, but it is clearly, it almost brought down our economy over the last few years. It is clearly something organizations need to address. knowIT: Along those lines, when I read your work I think about whether executives are jealously guarding the way that they make decisions currently, you call it their Black Box of decision making, and if we're institutionalizing a process for making decisions, does that suggest that these highly paid executives are less valuable? Davenport: That might be one conclusion, which would be, I think, a reason why executives might resist this. Obviously, there are a lot of pressures on executive pay already, and maybe this will just add to the chorus. I think that orchestrating an effective decision process, for me, would be as a shareholder of a company, or something like that, would be something I would be willing to pay a lot for. One of my friends who, I was talking to her, she is a Professor at the University of Texas, about how this idea might get propagated or might not. She is more legally focused than I am, and she said, maybe shareholder lawsuits will be the key: that if you could demonstrate that a company used a poor decision process, that might be an argument for helping to win a shareholder lawsuit, in a particular decision domain. So, I do not know exactly what it will take, but again, I think we really need it. knowIT: If we look into the future, if we make a logical extension, do we see human beings making decisions less frequently, or do we see the decisions that human beings make becoming less important, because there are these institutionalized processes for making decisions? Davenport: Well, that is already happening to an amazing degree, and one of the things that I think has been interesting over the past decade or so, is there has not been talk about automated decision processes, for example. There was a lot of talk about them in, I want to say the 1980s. I think that artificial intelligence was on the cover of Time Magazine in the 1980s, but then everybody was talking about it, nobody was doing it. Now, nobody is talking about it, but everybody is doing it. So, there are an increasing number of areas -- financial services (it is all over the place) pricing in various industries, health care is increasingly becoming semi-automated, anyway -- where part of the decision is being made through a highly structured process that is supported, and in some cases totally automated, by computers. So, I think, increasingly the issue will be how do we balance the human contribution to a decision, and the automated contribution. It is interesting, I just finished a research project, looking at 57 attempts to improve decision processes. That Harvard Business Review article was kind of the early returns from that project. A surprising number said that they were looking at this issue, I think it is in the 50 percent range, said that they were looking at this issue of human overrides to at least a semi-automated decision process, which suggests that a fair number of organizations are engaging with that issue already. knowIT: That's fascinating to me that, as you say, so many companies are doing just that. Davenport: Yes, I wrote about it a few years ago with Jeanie Harrison in a Sloan Management Review article, and for whatever reason, it wasn't highly popular. So, it is really appearing all over the place. We were talking about it yesterday here in Phoenix, for a gathering of people interested in business intelligence, this idea that increasingly a lot of these decisions get automated, and is it a continuum from humans being supported by information, to the decision being entirely computerized. I don't think it is a smooth continuum, but clearly all of those points along that continuum are happening today. knowIT: Now, you mentioned financial services and I know that in mortgage underwriting there is an IT decision process there. Davenport: Yes, hard to find a human being in this country who will actually look at your application. You have businesses, like LendingTree.com, that will give you four loan offers in a minute or less, obviously that has to be computerized, and it is. They have two computer-based decision models, one to figure out whether you deserve a loan or not, and the next to figure out which are the banks that are most likely to actually send it to offer you the loan. So, more and more that is happening. Less so in other countries, but where you have a well established credit score system, and a lot of online information, it is, basically, a snap to decide whether you deserve a mortgage or not. knowIT: So, what happens when the underlying assumptions or whatever goes into creating that automated system, when those assumptions or inputs are wrong? Davenport: No, that could never happen in this country. [laughter] Davenport: No. Yes, we obviously experienced a bit of that lately and that is the tricky thing about these automated systems, there is no better way to lose money than to put in an automated decision system in a key part of your business, and then have it no longer fit the world. So, I think the key challenge for managers going forward is knowing what are the underlying assumptions in that model. If your model for issuing a mortgage depends on the idea that housing prices are always going up, and you start to realize that is not a tenable assumption anymore, you better change that model very quickly. I have talked to banks that realized that too late and are effectively out of business or acquired now, in part because of that. I think there is a responsibility on the part of the analyst, the quantitative modeler, to point out that there are underlying assumptions here, and to identify clearly what those are, to make them very explicit. There is a responsibility on the part of the executive to look at them very seriously and say, is that a true representation of the world. A famous statistician, George Box, once said, all models are wrong but some are useful, and so what we need to decide is, is it still useful, or is it more wrong than useful. knowIT: So you talk about the importance of analysts in terms of creating these automated models for decision making and also in terms of gathering the evidence and the data that drive decisions. But it's my impression, at least, that a lot of these analysts are not necessarily entry level, but lower level employees. Certainly they're not the executives; they are not the folks making millions of dollars a year. Yet it seems their role is really so crucial, it's really the foundation of this new way of decision making. Davenport: It is for any company that believes in analytical and facts-based decision making, it is a very critical role. This is another one, I think, of the interesting things in the study that I have just done, a number of the companies are saying that these analysts are no longer just the kind of back room, quantitative people who kind of gather the data, run the models, and so on. It is much more important that they effectively communicate these assumptions and iterate back and forth with the decision maker. They need to build the trust of the decision maker. They need to work with the decision maker on communications with stake holders, about the decision. They need to kind of help frame the decision in the first place. So, it is a much broader role for the analysts than we previously envisioned, I think. One of the companies I talked to said, "We need people who can tell a story with data," which is a rare skill these days. I think more business schools need to be churning out people that can do that effectively. Part Two knowIT: Now you also talked, in your research, about deciding how to decide. You give the example of Air Products and Chemicals, a company that decides whether decisions will be made by a sole decision maker or after consultation with a group or by a majority vote. You also give the example of the Educational Testing Service, which has created this centralized deliberative body for decision making. But to me, those sound ripe for log jams for getting ideas bogged down in committee. I think whenever you ask a bunch of people in a room to talk about something and give their input, that decisions are rarely made. So, my question is how do companies deal with that problem? How do they overcome log jams? Especially if decisions need to be made quickly. Thomas Davenport: Sure. Sure. Well, I think there are several pieces to it. One is, you don't have big deliberative bodies for decisions that aren't really critical to the success of your organization. Another is, you put a clock on your decision, [laughs] so you know how quickly you need to make them. And I think that's one of the problems in many organizations is nobody keeps track of "How long does it actually take to make the decision?" And the other is -- one of the things Air Products has done, and I think is one of the simpler, it can be politically difficult, but, not conceptually difficult -- is they assigned fairly clear decision roles. They use this racy model. Bane Consultants are advocates of one called "Rapid" which was in a Harvard Business Review article in 2006 called "Who Has the D?" Which is what the D in Rapid...the D in Rapid is decide. The person who actually decides who you have review and advise and all these other different roles. So, I think if you establish decision roles clearly. If you have clearly established, in advance, that this is a decision worthy of a lot of consideration and debate and so on. And, if you know how long it takes you to make it. All of those things would mitigate in the direction of preventing excess bureaucracy. But, I do think that one of the things I'm not sure about is, this whole idea of "deciding how to decide" could really get out of hand. So, I think you have to be careful with it, that it doesn't become too bureaucratized. I think there was actually already a Dilbert cartoon about [laughs] deciding how to decide. Not taken from my article, but it does lend itself to bureaucratization, I think. knowIT: So it seems like when we talk about these human-intensive processes of decision making that that stands in stark contrast to automated decision making processes. Are there companies or industries for which either is a better fit or is it a matter of situations that call for human intensive or automated decision making? Davenport: Yeah. I think it's very situational and that's one of the reasons you have to decide on how to decide. One question is, how many times are we going to be making this decision? How quickly does it need to happen? If you're going to be making a decision, if you need it, virtually in real time, and you're going to be making it over and over again, like issuing mortgages or whatever, it's nuts not to try to automate it now. Given the technological capabilities that are available. If you're not going to be making it very often, if there's not a whole lot of time pressure, then it probably doesn't make sense to automate it. But, I think it's all very situational which is one of the reasons why we have to look at our key decisions and say, "What are the key attributes of them?" And that would give you some rationale for whether to automate it or not, how participative it should be, what kind of human capabilities we need to draw on. Is it your rational brain, your emotional brain? Just so many options these days. Vic Vroom at Yale has done some work, just on the participation dimension alone, the one that Air Products was wrestling with, and he basically has a little expect system that helps you decide how participative a particular decision should be. But, I think we need to expand the criterion and add more variables to the expert system so that we can take a lot more factors into account. knowIT: You've talked about the emotional brain and I'm intrigued by that idea. When is emotional decision making called for? Davenport: Well, it's interesting, there appears to be a set of neurobiological findings suggesting that people without emotional brains can't make decisions at all. You know, through lobotomies or accidental injury to the head, we wipe out the parts of the brain that are responsible for emotions -- those people can't decide. So, you could say, I guess, for all human decisions, you need the emotional brain. In that sense, I think there are certain contexts in which the emotional brain works really well or any part of the human brain. If there's a lot of experience, the brain seems to be quite good at flooding neurons with dopamine when things go well from a decision. So, that kind of is a training function that lets a brain learn from experience. Some people ... There's a good, popular, science book called, "How We Decide" by Jonah Lehrer that talks about some of the circumstances. One of the reasons I like it, it starts with our quarterback in the New England Patriots, Tom Brady, who's not actually doing all that well this year. But, normally, he's really good at picking out receivers and it talks about how his brain might allow that to happen. It's really all ... The rational brain doesn't have time to work in that case. It's all emotional brain stuff. So, really quick decision making ... William James once said that if you see a bear and you run, you know, your emotion brain decides to run. It's your rational brain that decides a little bit later, you're running because you're afraid and that was a bear. [laughs] So, if you have a real need for speed, and you have some experience, and your emotional brain has learned in the past from similar situations -- those tend to be the circumstances under which it's really quite adept. knowIT: So, clearly IT will play a role in terms of automated decision-making processes. How do you see that role evolving? Davenport: Yeah. Well, all the analytical decision making is obviously very informed by data and technology. The automated stuff usually comes from either taking an algorithm and putting it into, kind of, a scoring environment where you automatically generate a score and make a decision. "Is the score high enough to proceed? Yeah? Great. Go ahead, give 'em a mortgage," or whatever. If not, too bad. Or rule engines which are what people now call the type of artificial intelligence approaches that has a series of "if ... then" statements. Kind of Boolean algebra stuff. And then, the other area where technology plays an increasing role is in the whole "Wisdom of Crowds" domain. We now realize that there are certain areas where getting multiple large groups involved in a decision can yield a pretty good outcome. The IOL electronics markets is probably the most familiar example of an effective prediction market that does a much better job at deciding who's going to be president than many other forms of prediction. And technology plays a pretty important role there as well. So, I'd say those are probably the three areas. Depending on how you define Information Technology, I think before very long, we'll probably have devices that will tell us how effectively our brains are working. Is that Information Technology? I'm not sure. That would be a broad version of the term. knowIT: So as we think about the role of IT and decision making and how that role might evolve, where does outsourcing come in? Obviously there was this huge wave of IT outsourcing, then business process outsourcing followed. Do you see a sort of decision making outsourcing in the future? Davenport: Certainly I think there may be aspects of it that can get outsourced. A lot of companies have problems digesting all the information that they have at their disposal coming out of their operations. And so they could, well, and some already are starting to hire people to analyze it for them and provide them with insights that might guide decisions as opposed to just, kind of, doing the back office stuff. Some Indian firms that I've worked with are now in the business of doing analytical consulting and analytical process outsourcing, if you will. Which is not quite decision making, but it's certainly closer to decision making than the typical outsourcing stuff was. So, I think there is a real need. Companies are just drowning in all the data that they have and they want insights, not data. So, I think there will be more and more of that. And I think data providers are increasingly being asked, not just to provide data, but to, you know, "Give me insights." I've talked to Nielsen, IRI, Bloomberg, various people are all saying, "We've got to get into the insights business and not just in the data provision business." knowIT: Practically, how does that work? How can you generate insights if you don't really know what the question is? Or do these work closely with a business to find out what that question is? Davenport: In some cases it's a matter of, you know, it's a consulting engagement. Where you work closely with a business to find out what their needs are. In some cases it's kind of ... I have a friend who runs a little company that generates through rule engines and algorithms, that generates computed English text from financial data that will tell you what's happening with the financials of a particular company's stock. The idea is ... There are a lot of companies where there's no equity analyst for them at all. We have a lot fewer equity analysts than we had in the past and a lot of stocks are totally uncovered by analysts. And so this might provide insights to investors that are based on financial data that they couldn't interpret themselves, but could be interpreted in, printed out in English prose. The versions I've seen have been pretty good English prose, actually, that explain what that company is likely to do, whether they're likely to pay a dividend, whether they're treating the business as a cash cow and not reinvesting, et cetera. knowIT: That's fascinating to me, but we still go back to the idea of assumptions, that in order to interpret the financial data into text, there are certain assumptions that are made. Davenport: There are assumptions and ... There are always assumptions and, in many cases, we don't know what they are. So, that particular company, I think is not going to identify its assumptions. But, I think for managers who use a lot of these computerized systems, you can't just advocate responsibility anymore. You've got to say, "Are those assumptions reasonable?" and "What are the assumptions?" It's got to be a lot more pushing back than maybe we've done in the past. I think that's what we learned ... one of the things we learned from the financial crisis. knowIT: That's very interesting. It sounds like there is some good potential for data driven decision making to help us avoid those kinds of problems in the future. Davenport: Yeah. I mean, you could argue that those things were data driven decision making already, they just weren't very good data driven decision making. [laughs] So, Robert Shiller, the Yale economist, who I think was fairly unique in predicting both of the last bubble-burstings that we've had has said, "You know, it's got to be every manager's responsibility to know how quantitative models work and know when not to pay any attention to them." And I think it means a lot of managers will need to be retrained if we're going to do that very effectively. knowIT: That makes sense. This research that you've been talking about is fascinating. Can you give us a sneak peek into the newest book? Davenport: Sure. It's kind of a bridge book between competing on analytics, which is all about how companies competed on their analytical capabilities, and this work on decision making. It's how companies can make more analytical decisions. How they can establish greater analytical capabilities, not necessarily to compete on them. But, we think that everybody can benefit from being more analytical and facts-based in most cases, anyway. And having intuition be the last resort when you can't get data or when the decision has to be just instantaneous or something, rather than the first resort. So, it's about decision making, but it's also about ... It's particularly about analytics. And it does mention the fact that you need some human faculties in there, as well, every now and then. [laughs] knowIT: I look forward to it. Davenport: Thanks.

KnowIT