Our work is only possible through the generosity of private philanthropy. Find out how you can support our mission and can contribute to our work.
Join the HJS mailing list and keep up to date.
Speaker: Richard Nisbett, Author of Mindware
Chair: Alan Mendoza, Director, Henry Jackson Society
Host: Viscount Ridley
Date: 12th April 2016
Committee Room 1, House of Lords
Well, good morning, good day, good afternoon everybody. I’m Matt Ridley. Thank you very much for coming. It’s a tremendous honour, certainly for me to have Richard Nesbitt here, I’ve kept an eye on his work for many years, and I’m looking forward to hearing him talk. The good news is you can look at him rather than his pictures, because the pictures are not going to work. But that will no doubt improve the presentation I’m sure, because he is an excellent speaker. He is professor of social psychology and co-director of the culture and cognition program at the University of Michigan, and his research interests are in social cognition, culture, social class, and aging. He’s recently released a book “Mindware: Tools for Smart Thinking”. Richard, welcome to the House of Lords
Professor Richard Nisbett
Thank you, thanks very much. I was delighted to be asked as speaker, I was delighted there was such a group as the Henry Jackson Society. I’m old enough to remember when Scoop Jackson was a Senator and those were the days when it was necessary to say that American foreign policy was bipartisan. No one necessary feels it necessary to say that today so far as I know. Jackson was a Senator at the time of another great centrist Senator, named Dan Patrick Moynihan, and he said something that’s relevant to what I want to talk about today, which is the central conservative truth is that it is culture not politics that determines the success of a society. The central truth is that politics can change a culture, and save it from itself. And I think there are some things we can do which would change our culture, that would make it better, and that’s what I’m going to talk about today.
The industrial revolution changed everything. By that I mean everything. Wealth in Europe basically was unchanged for 2000 years before the industrial revolution. Since then it’s taken off astronomically. Our lifespans have increased by more than double since the beginning of the industrial revolution, and the industrial revolution produced a revolution in the way we think. This is not widely understood. If you were to ask someone in 1760, the average worker on the street, some questions that you would assume that anyone could answer you would not get a very satisfying answer because he probably couldn’t read, probably had no ability to deal with abstractions, no understanding really of how to categorise, or how to make inferences about some object or animal on the basis of its category membership, very little ability to reason logically outside the very narrow confines of his daily life, and no ability, very little ability to do hypothetical reasoning. So if you asked someone, told someone “I have a cousin who lives in North America, and in North America all the bears are brown, I got a letter yesterday from my cousin in North America, that said he had seen a bear, what color do you suppose it was?”. The person would say “How should I know? Ask your cousin who saw the bear”, just if P then Q was not part of the logical repertoire. If you were to ask “how is a fish like a bird?” the person would say “A fish is not like a bird, a fish can swim but it can’t swim, a bird can fly but it can’t swim, a bird can eat a fish but a fish can’t eat a bird”. If you would ask someone “How do you think you would think about that if you were French?”, “That’s ridiculous, I’m not French, I’m English, I can’t be French”, and since that time, with the three Rs, and the training in logic and abstraction and categorisation and so on, there have been huge changes in the way people think in everyday life, not just on the job but in everyday life. And we can trace those changes through IQ tests. IQ between the beginning of the 20th century when IQ tests began to be given and the end of World War II increased by eight or ten points, which is a lot. After the end of World War II, IQ in Britain and the US has increased by more than 15 points. That’s a standard deviation, for those of you who speak standard deviation. That’s enormous. Someone who scores 115 right after WWII would be considered natural material for a university. That same person working on that same test today would get a score of 100, and would not likely be a candidate for university, maybe some little bit of post-secondary education, not much more. That’s IQ test scores, how about actual intelligence? Not quite the same thing.
IQ tests include subtests that we clearly are going to say are tapping intelligence. If you ask a child “Why is it that doctors go back and get more education?” a smarter child can tell you why, and a less smart child can’t. If you were to ask a child “How are revenge and forgiveness alike?” a smarter child can answer that question and a less smart can’t [sic]. And the percentage of children that can answer questions like that has increased enormously in 70 years. So how did this happen? School mostly, better school and more of it, and at the turn of the 20th century the average person in Britain and America had had a very few years of grammar school. Today 40% of people in Britain and the US have a tertiary degree. So an absolutely enormous change which is for one thing given a change in vocabulary of a standard deviation, equivalent to an IQ increase of 15 points, and of course the larger the vocabulary the more concepts you have, and the more concepts you have the better you can think. School has also improved, I’ve looked at the curriculum recently for preschool, and they’re doing things that are really equipping children with concepts and abstractions, with logic and inductive rules. It’s hard to believe, but at the turn of the 20th century it was understood by university professors that you couldn’t teach students calculus until their senior year, and now it’s routinely taught in high school.
The culture has also changed. There was a golden age of tv in the US, it was the 50’s, it was a golden age, and I love lucy was great, it still holds up, but you weren’t going to get smarter from watching it. Now we have series that are like a Dickens novel, where you have to retain information about plot and character, you have to make inferences in order to stay abreast of what’s going on. Computer games have contributed something to our intelligence. And that’s all been terrific, the culture rose to meet the challenge of the industrial revolution, but we’re in the midst of another revolution now, namely the information revolution. And now what’s necessary is thinking tools to deal with data rather than just things. So we need to be able to collect data, to code data – that is decide what something means and then, if at all possible, assign a number to it. We have to be able to analyse data, we have to be able to manipulate the environment so as to generate reliable information, and we have to able to use data appropriately in order to make effective decisions. The tools for doing this are relatively new, but they’re available right now largely just for the fields for which they were invented. This is true for statistics in particular. My joke is that statistics are taught in university so as to prevent, if at all possible, its escape into everyday life. What are some of these tools?
They are statistics and probability theory. To be an effective member of our society in this age I want to maintain that you really need to understand the concept of a sample of a population, a sample gathered by a concept of randomness, a law of large numbers, normal distribution, standard deviation, statistical significance, regression to the mean, base rate, and correlation. That’s at a kind of minimum. In terms of scientific methodology what’s needed is a concept of a randomized controlled experiment. The concept of a confounding variable, of self-selection, of independence of observations vs dependence of observations, a concept of a natural experiment, and some idea of how to find a natural experiments in the world, and the concept of an artefact. And from decision theory what’s needed is some rough idea about cost-benefit analysis, concepts of opportunity costs, sunk costs, and loss aversion.
Now most of you have heard of all of these concepts, I’m sure the great majority of you have heard of most of these concepts, and some of you undoubtedly use these concepts in your work. But not so much in your daily life, and not nearly as much in your work as you might. I want to talk about six examples of what could be done with these tools. Two from statistics, two from scientific methodology, and two from decision theory.
So suppose I told you I have a friend who’s an executive, and recently he interviewed someone for a manager position, who’d come to him with extremely high recommendations from his previous employers, he had a great record at his previous company, but he interviewed the guy, and he didn’t have any interesting commentary to make about my friend’s business. It just didn’t seem terribly sharp. So my friend told his colleagues “I don’t think we should pursue this guy, I just don’t think he’s material for us”. It’s something that happens all the time, every day, perfectly reasonable. But suppose I told you about a football coach friend of mine, who went to observe a practice for a particular player, a forward who had a great point scoring record, had terrific reviews from his coaches. And he watched him during a practice and the guy missed several points he should have gotten, he just didn’t seem in control of the ball. So he went back and told his friends “I don’t think we should pursue this guy, I don’t think he’s material for us”. Now I suspect that produces something of an alarm bell ring, not quite the same as the first situation, you wonder if that’s really a terrific judgement. And why do we see a difference between those two situations? I would I argue it’s because we understand that there is error, substantial error, associated with any brief observation of someone’s athletic performance. And because of that we realise that we have to have a much larger sample in order to make an accurate judgement about the population of that person’s ability. The coaches judgement is actually a better judgement as it is than my executive friend’s judgement. Because the 30 minute unstructured interview correlates no better than 0.1 on a scale from 0 to 1 with anything that has ever been looked at: performance in college, performance on the job, performance as a physician, performance in the military. And to give you an idea of how weak an indication that is of someone’s talent. It would increase the likelihood of picking the better of two candidates from 50-50, a coin flip, to 53%. Almost no gain at all. Incidentally, I’ll point out to defend this, the implication is that you should not interview. The reason that you should not interview is that you’re not going to be capable, we are not going to be capable (I don’t want to excuse myself from this at all), we’re not capable of confining that information to its proper range of a tie-breaker at most. There’s typically, for the kinds of things I’ve been talking about, the kinds of performances, the folder contains enough information to allow you to pick with 65-75% accuracy, which between two candidates is actually better. So that’s an enormous gain. You’re going to throw it away if you interview. So what is it that’s necessary to understand the executive’s decision when we already understand the coach’s decision, and recognise it to be not great? We have to understand that sample values resemble population values as a direct function of their size, and an inverse function of the expected error of each observation. We need to be able to think in the information age of every observation that we make about anything, someone’s height, distance to Pluto, how friendly Jane is, how honest Bob is, we need to be able to think of observations as truth scores, that is the real value of the distance to Pluto corrected by the error associated with observations of that kind. Which is presumably zero for your height, or for the distance to Pluto, but as you’ll see in a minute it’s not remotely zero for most of the things that matter to us in everyday life.
So to give you an idea, we did a study with University of Michigan freshmen where we had previously determined how well you can predict a kid’s score on a spelling test from knowing another score on another single test. And we found out how much our reliability people believed they had from these kinds of observations. If Bob got a better grade on the first test of the year, what do you think the likelihood is that he got a better grade on the second test of the year? Or if Jane got a better score in the first basketball game of the season, what’s the likelihood that she got a better score on the first game of the season what’s the likelihood that she got a better score on the second game of the season. Or Billy got a better score on the first twenty spelling tests, how likely is it that he would have got a better score on the second set of 20 tests, or basketball. And, as it turns out, we know what those values are, we’ve collected the data, we know what the values actually are. It turns out that people were remarkably accurate about the degree of predictability, they estimated, and we were able to go from their probability estimate to a correlation estimate. They estimated a correlation of 0.5 for both basketball and spelling, and that’s correct. It’s certainly a lot better than not knowing anything but it’s not perfect. The predictability however as you from one to one, to twenty to twenty, that probability increases way beyond 95%. If you know the odd test scores, predicting the even test scores, the odds become absolutely enormous, reflecting the fact that there is really a very high correlation if you have enough data. Ok, so how does this compare, and people do understand that more data is better than less, it’s just that they don’t understand the degree to which it is true. They estimate correlations of about 0.7 when they should be going past 0.95. But a very high degree of attributes can be thought of as personality traits. So we found out how well you could predict honest from behavior in one situation to another situation, or behavior in twenty situations to another twenty situations, and how well you can predict friendliness. People believe that the predictability of friendliness or honesty from one occasion to another amounts to a correlation of 0.8, which happens to equal a probability judgement of 0.8. In other words: “Jane was more friendly at the party I saw her at last night, how likely is it that she’ll be more friendly than Joan at the committee meeting tomorrow morning? I estimate that, we estimate that, to be 0.8”. Which is colossally wrong. The predictability of any behavior that’s characterised as a measurement of a trait from one observation to another observation almost never exceeds a correlation of 0.2, that’ to say probability about 0.56, and is typically even less than that, 0.1 is the typical correlation. So people are wildly mis-calibrated with respect to how well people’s behavior can be predicted and many other studies show this kind of failure. The problem is framing any kind of observation as truth core plus error, and making an estimate of the error, and adding in this qualification in there that when it comes to human behavior there is always gonna be error, probabilities somewhere, for personality traits huge amount of error.
So, let me ask you, how many people “speak correlations”, so, not a lot. Everybody speaks probabilities though, so I can ask in terms of probabilities. What do you suppose the degree of agreement between any two reviewers of a paper written in Psychology, and submitted to the journal? Do you think that if Reviewer A rates Paper 1 as better than Paper 2, then what’s the probability that Reviewer B will do that?
[Some discussion with members of the audience suggesting 60% up to 80%]
What do you think that the correlation would be for solid state physics proposals for the National Science Foundation? How many people think only a 60% chance that if Physicist A says Proposal 1 is better than Proposal 2 that Physicist B will agree with that?
[Some discussion with members going from 70% up to 90%]
Actually in both cases for Psychology and for Physics the probability is 0.6, if you want to really know which is the better proposal. For example if you’re an analyst looking at business models, you’d better ask more than one person, you’d better ask more than two people, in order to get up to a level of 70 or 80% that you might want, you have to have very many reviewers. Ok, so that’s the law of large numbers.
In the US there’s an expression describing the fact that the best freshman baseball player, in the national baseball leagues, is typically not the best player the second year. And there is a name for this phenomenon, it’s called the sophomore slump. And that same concept is applied to second novels. Great first novel, second one not so good. It applies to music albums. And if you ask people “why do you suppose that is that the best player in the first year isn’t the best in the second year?”. I can assure you that if you ask a college freshman who hasn’t had any statistics that person will always give you only deterministic answers. They’ll say something like “Well, the pitchers make the necessary adjustments, or maybe the guy gets too cocky and slacks off the next year”. But let’s think about how somebody gets to be the best player the first year. Now I have to ask you to imagine the picture, I don’t have the normal distribution curve, but I’m sure you know it. How do you get to be all the way out there? Well, by having a lot more talent than the average person, that’s for sure, but there’s error associated with an observation. Everything else had to be right, the guy got particularly good coaching, ideal for him, just happens that his first three or four games went extremely well, but his confidence, he got engaged to the girl of his dreams. The next year the great dice roll gave the guy an injured elbow, took him out for a few games, and I’m sorry to say his fiancée jilted him. Somebody else got lucky on the great dice roll and became the best player this year. In other words there is nothing to explain about the sophomore slump. It’s an inevitable expectation given that an observation like the performance in one year to performance in another year has error associated with it. A substantial amount.
Suppose that I were to tell you I have a friend who’s a manufacturer’s representative. Her name is Katherine. She loves her job, she likes to travel, she’s something of a gourmet. But she finds that she likes to go to restaurants when it’s been highly recommended to her, but she finds typically that when she goes back to a restaurant she has had an excellent meal one time the subsequent meal is typically not as good as the first. Now if you ask a freshman why that is you’ll never get anything but a deterministic answer “well, maybe the chef’s change a lot, or maybe her expectations got too high, and they could only be disappointed”. If the student has had only one statistics course, they’re likely to say something like “Well maybe it was just by chance that she got such a good meal that first time”. Which is correct as far as it goes. But if someone has had several statistics courses they’re likely to say something like this: “Do I think there are more restaurants in the world where you could get an excellent meal every time or more restaurants in the world where you would get an excellent meal only some of the time, if I think that there are more of the second kind, and the person got an excellent meal the first time around, I have to assume it’s the second type of restaurant, and so she’s going to be disappointed”. There is nothing to be explained causally. My favorite example of the “regression to the mean problem” and that’s what this is. Any extreme score, on re-observation, can be expected to be less extreme, to the extent that there is error affecting that observation. My favorite example comes from Danny Kahnemann who wrote this wonderful book called Thinking Fast and Slow who was talking to a bunch of Israeli flight instructors, who told them what Psychologists had learned about instruction, he told them for example that it was much more effective to tell them what was good about their good performances, and why it’s the right thing to do, than it is to critique what was bad about their bad performances. Pandemonium breaks out. The instructors say:”I sure know what it’s like in your business Professor, but I can assure you it’s different here, it’s the complete opposite. If a pilot makes a good maneuver and we tell him it was good it’s not as good the second time around. On the other hand if the pilot makes a terrible maneuver and we scream at the guy, odds are it’s going to be better the second time around.” In other words we find that flying an airplane has error associated with it, and to the extent that that’s true any extreme performances come back to the middle.
So, let’s switch to scientific method considerations. Now here I like to show a picture of a Ford pickup and a Volvo station wagon, and tell you that the Insurance Institute of America has looked at the number of traffic fatalities per model year for each of the very large number of vehicles, and found that the fatality rate is vastly higher for the Ford pickup than the Volvo station wagon. Now, if I were to just tell you that you might say “well I’ve heard about Volvos”, and then this is the best part of my whole talk, which you’re going to have to miss [as the projector isn’t working]. I put up a slide which is a quiz for you, I ask you to identify which automobile each person would be driving. One is a suburban matron, and the other is a twenty-five year old cowboy. We don’t assign automobiles to people at random. We don’t say “Billy you’re going to be driving a lovely powder-blue Volvo station wagon”. Billy chooses his own vehicle. And the problem here is the self-selection problem. The person chooses the car, but the person has many other things associated with them and other attributes which are actually entering the picture, it makes more sense to think about this as a little odd but to think of the self-selection point as a point of the autos because auto is what we are trying to predict about, it is as if the autos were selecting the type of driver that they get which comes under the kind of circumstances that the auto was driven, it comes under the number of miles the auto is driven. This self-selection principal is that the subject chooses not only the subject’s level at any given variable but the level of a host of other variables that correlate with that variable and it could be any other of them variables that is actually producing a relationship between the target variable and something else that you are interested in.
Men who take vitamin E are less likely to get prostate cancer. Many people of a certain age, many men of a certain age might go out and buy some vitamin E, but before you do that let me point out that the people who take vitamin E are self-selecting, nobody assigned them, they have selected that variable for themselves, what else do we assume goes with that? Well people who take in vitamins probably have more education, they think it is a good idea to take vitamin E, they probably have more money, have gone to a doctor who has told them to take vitamins, they probably have a better diet because they care more about their health and have more money to do something about it, better cholesterol levels, exercise levels better and blood pressure levels and everything else. So the only way we are going to find out whether vitamin E is a good idea is to do an experiment. Randomly assigning some people to take vitamin E and for some people not to take vitamin E, and we will wait a while and measure who will get prostate cancer and who doesn’t. We have cancelled out all of the differences, we are going to have on average the same education, the same amount of money, the same amount of diet everything is cancelled out by this randomized control procedure, and when you do that you find that you are more likely to get prostate cancer if you take vitamin E than if you don’t.
I will assure you that most of what you read in the Times about medical and health matters is wrong. At the very least they don’t have the evidence to match what they are claiming and quite often it is literally wrong because if you do the experiments you know what is the case. I have discovered that there is a name for this in public health which is the healthy user bias – people who do one thing that is perceived to be healthy but do all kinds of other things. So if you read in the Times that a little bit of cyanide every day is better for your heart condition then 10/15 years from now you find people taking a little bit of cyanide and they were actually doing better because of all of the other things presumably associated with healthy practices. So the notion here is multiple regression analysis, the person who does these studies, non-experimental studies is looking at the correlation between various health practices and health outcomes, is likely to tell you something like we control age, gender and social class and so on. The only control for those things is the indefinitely large number of other things which are associated with it. How do you measure how much a person cares about health? I don’t know of a way to measure that in a terribly valid way. How about social class, they say they control for social class but what do they do, do they look at the amount of money that the person makes, do they look at the prestige of the occupation, did they look at the degree of education, did they combine those three things in one way? It turns out that how you measure social class can completely flip the findings here.
A finding that is comparable to the prostate cancer one is hormone replacement therapy for older women which for many years was presumed to be good for cardiovascular health so someone said lets do the experiment and when you do the experiment it turns out that hormone replacement therapy is bad for cardiovascular health. It turns out that the way you chose to control for social class throws the findings from a very slight positive effect or negative effect to a hugely beneficial effect which is mistaken.
Second, methodology principal when Barack Obama declared the Presidency in 2007, he was invited to go and have a big meeting at Google and their employees and the first question which Eric Schmidt asked (as a joke) was ‘What is the best way to sort 30 million interties’ and Obama replied ‘well I think bubble sort would be the wrong way to go.’ Eric Schmidt reared his forehead and the audience broke out in applause because that in fact was correct. Then Obama went on to say that he believed in data, he believed in evidence and he would align the government accordingly. In Google they have a derisory term for the way that businesses make most decisions, which is you get the hippo – that is the highest paid person’s opinion and that is the way to make it. At Google they do something else called AB testing and it is what is better blue border or the red border, what gets most clicks? A man called Dan Seroca put this into operation for the Obama campaign. I did have another lovely picture which I can’t show you which had 3 pictures that this man generated for Obama – a turquoise portrait of Obama, a black and white picture of Obama and family or a 5 second video of Obama giving a speech and then there were three legends beneath it like learn more and two or three other things. Which do you think would be most effective here, I find I don’t have intuitions about what that legend should say or what that picture should be and as they say, which they also say around Google, assumptions tend to be wrong. Google discovered something, the first people to discover were social psychologists and pity the poor social psychologists, because we know that predictions of human behavior in normal situations aren’t worth beans. Assumptions tend to be wrong, constantly finding that our beliefs about the human entity are wrong. Now the best combination of picture and legend was 40% more effective than the worst and that amounts to a lot of money and a lot of votes.
The AB test still applies in all walks of life really. Imagine that you are a grocery chain operator and to arrange groceries they have always been arranged in the US and I assume it is the same way here, that is sodas in aisle 6 and bread in aisle 8 or the way they do in Japan which is by type of food so all the ingredients for miso soup, or the ingredients for Italian food and so on. You might think o the Japanese way is probably better for the Japanese and the American way is probably better for the Americans but actually the Japanese way is better for the Americans, better for the retailer because people buy more food, they remember o I need Romano cheese which they would of otherwise forgotten and better for the customer because when they get home and say O I forgot the Romano cheese and then drags something out of the freezer instead of eating an wholesome meal.
A lovely set of experiments was done recently in this supermarket in Texas, which happens to be my home town, with groceries trying to find out how can they increase fruit and vegetable sales. Now that is an important thing to do if you are a grocer because the profit margin on fruit and vegetables is higher than on anything else and it is also good for the health of the customer because fruits and vegetables are better than most other things that they might buy in supermarkets. It turns out if you simply say the average person spends $12 or whatever it is on fruits and vegetables then that substantially increases the amount of fruit and vegetables bought, or even better you could put a sign in a cart saying ‘place fruit and vegetables in front of cart’ and that doubles the sales of fruits and vegetables.
In personal life you are only going to find out the answer to certain things that are actually fairly important by doing experiments. You are not really going to know if coffee is on balance a good thing for you to do because it increases your alertness or a bad thing to do because it makes you jittery or obnoxious to other people if you are doing it haphazardly, because you had coffee yesterday because your husband made it for you, you don’t have it today because you are in a hurry, you have it tomorrow because you are very very tired etc. Unless you flip a coin when you come into the kitchen in the morning to find out the answer to that question.
Now here is where politics comes in. Things that we do that are not a good idea or may or may not be a good idea which cost a great deal of money and have big impacts on people’s lives. The US has spent $200 billion on Headstart which is a pre-kindergarten program for poor women and kids. We don’t know whether that money did any good or not. No-one ever did a random assignment of kids to conviction, the money was shoveled out of the door and people put into these classes. We know that you can make a big difference with pre-kindergarten programs or the particular good ones, people have done the studies, we don’t know how long that maps onto what Headstart does. If it had been 50 years ago that the program had started, a few million dollars you could have found out what is really effective. So if societies where in the habit of experimenting they would save a lot of money and a lot of grief.
Speaking of grief, on 9/11, 9000 grief counsellors descended on New York. Now grief counselling is a great idea. The person meets the leader with a number of people in a small group, they report what their experiences are, what their feelings are, the leader explains why their feelings are perfectly natural and ordinary and in the not too distant future, they are going to find life a lot less stressful. What actually strikes as a very good idea – it isn’t it prolongs the stress to put people through those things, here is something that social psychologists have invented a month after the disaster you had a person behind a brief essay that would be subsequent distress. It doesn’t seem to me like it should have that affect but it does, assumptions tend to be wrong.
So my third example comes from a program started by prisoners in New Jersey who thought it might be a good idea to try and show kids who are at risk from delinquency how awful jail is. So they brought some junior high kids in, they say it is terrible here it is unbelievably boring, the food is horrible you are at constant risk of physical abuse, including sexual abuse. The effect of this program has now been examined, and overtime the effect of this program is to increase the likelihood that these at risk kids will end up committing a crime and be incarcerated. And the cost benefit analysis is that for every dollar spent on what is called ‘scared straight’ programs there is a 140 dollar of cost incurred in the form of incarceration costs and crime costs. So as a society we are not in the habit of doing anything as like as many experiments as we should do and it doesn’t matter that it is obvious to you that scared straight programs don’t work, the social psychologists say sorry, obvious is not going to do the trick.
Speaking of cost-benefit analysis, switch the pitch to decision theory, many years ago my wife and I bought a summer cottage. We were at the absolute edge of our finances, we didn’t have the money to buy furniture so I thought well I will build my own furniture. Now a person with carpentry skills that would be a good idea – that’s not me. So I thought I will have to go to a class so I went to a class on carpentry and after 15 hours I had a box. I thought this is going nowhere I have to do something else so I did the second best alternative recognizing that I was paying a huge opportunity cost which is benefit to the next most valuable act you could be taking. The next most valuable action was actually vastly more valuable than the action I was taking. The best action to take was to eat peanut butter and pizza for a few months, buy cheap furniture and as we got more money, buy more furniture. I was not paying for me to do something like taking a furniture building class paying a huge opportunity cost.
In case you don’t know, economists are different species from the rest of us. Economists are constantly calculating opportunity costs – should I be doing this now, should I be listening to a lecture or is there something else which I could be doing now, I always think that economists must not be terribly happy as they are always saying I could always be reading a novel, I could be doing this. Economists don’t do anything that they could hire a small boy to do. They don’t mow their lawns, they don’t weed their gardens, they don’t fill out their own income tax returns. Now my just saying this to you is probably not going to change your behavior but let me just encourage you to think more in terms of opportunity costs for things. Imagine you run an office building, you need an office yourself so you decide that you will use an office in your own building, feels like it is free and an accountant might tell you it is free but an economist would say you are possibly paying an opportunity cost for it. If you could get an equally good office for less money than the rent you would pay for an office in your building, then your next best alternative is better than the one you have chosen.
Final example, my first year of graduate school I did a project on personality and something that repressed what it was because I spent a huge amount of time on it, analyzing the data my stomach was sinking lower and lower and it looks like there is nothing there. I went to my advisor hoping that he could do something about this and he looks at the data for an hour and says ‘I am sorry kid, sometimes you win and sometimes you lose’ it is not in here. That’s unacceptable so I did what I did my first year of graduate school so I dive back into the data and did more analysis, did a couple of subsidiary studies – still nothing. I was trying to rescue what our economists call sunk cost. I was trying to make that current in my head spin be worthwhile. And the economists would say, here is a thought experiment, suppose someone gave you a crappy day and asked you to analyse it, would you be doing it, no you have got to be kidding – if that is the answer then you shouldn’t either.
One final example that categorizes the sunk cost problem. Suppose you bought a ticket a month ago to a football game and tonight is the night but the star is not playing, nothing hangs on the outcome. If you ask most people whether they would go to the game or not, most people would say ‘I don’t know but I would hate to waste that £100’ to which the economist says, honk wrong, you can’t waste it you don’t have it. What you can do is pay twice, once for the ticket and once for the team. Now an economist telling you that, that is probably not going to be enough, probably need a psychologist with you as well to say ‘because you are going to be motivated.’ Even if you are not an economist you should try to understand that you are not going to rescue costs, you are going to find it very unpleasant, the £100 for you sitting there vibrating in your mind and you want to make that somehow not be painful, so you say to yourself lots of times I have gone to games I wasn’t expecting much but they have turned out to be quite interesting, I have nothing on TV tonight, and the psychologist would say no here is the thought experiment that you have to do. Suppose you didn’t have tickets for the game and a friend called you and said I have tickets for the game tonight, I can’t go would you like them, if your answer would be great I will be right over then by all means go to the game. But if your answer would be, you have got to be kidding, the stars not playing, nothing hangs on the outcome then there is your answer, don’t go.
Well governments are in the habit in trying to rescue some costs, I gave you the example of a talk like this to people in the CIA recently and they came up with a wonderful example of governments trying to rescue costs that are sunk and cannot be rescued. So when they have a project that looks like it is going nowhere they replace the manager. No matter how good they think the manager is because the new manager is not going to have the sunk costs that the old manager did on the program like the old manager did.
Well I wish I had more time to give examples of this but maybe in discussion afterwards, I know we have a limited time, maybe you can tell me examples may be put into use by blood or bodies. Thank you.
Thank you Richard that was absolutely fascinating talk, but I am a little bit worried about time and I can’t remember how long we have got the room for (until 14:00) so we may well get kicked out but should we rapidly take a couple of quick questions.
It is about knowledge and the quality of decisions made. When I go to the referendum voting booth in a couple of months’ time, do you think that I should face a test of 10 items which I can tick yes, no or whatever should my vote be either discounted or at least weighted in calculating the eventual outcome.
Well if you are an American these days you are very nervous about the people who are making the political decisions in this country, mainly the ordinary voter but I don’t think you should have a test, no, education is the best we can do, save a society from itself by having more education. For example some idea of some of the political consequences in what happened in Europe in the 1930s might have made some difference to us in the US.
If the opinion polls are looking at the issue of human behavior in this unique situations, what’s your view of opinion polling?
We have just had a general election where the opinion polls were way off – by about 6 points. What they did is discard an accurate poll because they thought it was an out liar and that was the type of thing what was going on.
Well we have a guy in the US who is Mr Statistics and in the last two presidential elections he has called every single state correctly.
In your experience how widespread is the kind of betting tools and mechanisms that you are talking about in international business? Google for example use a lot of these but are they the exception?
I am very glad you asked the question because the bottom line from what I am saying is it is a new age, totally different requirements from what we had in the industrial age. The tools are there, they are being used only to a tiny fraction of what they could be being used. I can generate problems like I just gave you right now and say gee I would like to have solved that problem using that rule we just don’t do it. The industrial revolution in 1760, there were very few literate people in Britain by 1840 there was 70% literacy in Britain, that is enourmous and all of the things that go with that, the logical capacities and so on, we haven’t come close to that so far with the tools that are necessary with the information revolution.
My question is around education improving thinking. The is more information out there now, more data, more people able to access that data, so presumably what we need is better skills in order to go and analyse that data. Do you see evidence of the educational systems are evolving in a way that will give young people these tools and if not what should we do?
Great question, If I would have had another 2 minutes I would of addressed that. I do think it is something that is relevant. I generate tests for people in these everyday life kind of things and I give them to students and I watch what happens for the first 2 years of education for students at the University of Michigan. I look at what happens to them as a function of their majors, those who have had a psychology major increase by 70% and it is huge. People who have had social science, other social science increase by about 40% which is huge. Physical science students learn nothing. Humanities students learn nothing about these kind of things, those fields actually learn more about logical thinking, they do better than the behavioral science students. I looked at what was the effect of 2 years of graduate school, psychologists get an extra 70% on top of what they had, physicians to my surprise, get about a 30% gain and I didn’t expect that. So I went to medical school for a few days and soon the mystery disappeared, the professor might say something like ‘patient has symptom A and symptom B now do a differential diagnosis between disease 1 and disease 2’ and the student says ‘sounds like disease 1 to me’ and the professor says wrong. It is wrong because diagnosis of disease 1 is very expensive and highly unreliable and treatments for it are not very effective and rare. One guy gave the best slogan I have ever heard which is when you hear hoof feet think horses not zebras.
Just a quick related question on the information revolution because I think it is so inaudible. The thesis of authors inaudible… may be adversely affecting the process in a sense that we may be prioritizing a kind of superficial multi-capacity over where we measure judgement. Do you think that this point that there could be any truth to that?
Well like everybody I think about that a lot but I don’t know my work habits have changed I mean I am one of the worst offenders of email, there is by the way you may be interested to know, a program you can put on your machine which makes it impossible for you to check your email more frequently than every hour or two. I haven’t done it yet, I know I should. It has changed, I have just written 3 books in the past few years, one of them being Mindware which is all about what I am talking about today in an attempt to be able to get people to use these kind of tools in everyday life. I couldn’t have written the book I wrote 25/30 years ago because I can look things up, it is unbelievable I mean to do a 20 hour library search may take 2 seconds. Scholarship you can do vastly more of it and vastly better which may be substituting to some degree these defects in thinking that may very well be there. I certainly think I skip more rapidly from one thing to another than is a good idea but I don’t know.
Ok so since most people aren’t going back to school and they are making decisions and we know that every observation because they are associated with it, what do you think is the best place of intervention? Do we control for expectations, do we add information to any approach, can we reduce a probable error? How do we intervene because we are not going to correct the thinking that most of the people making them decisions have.
Well intervention early is going to make a difference, one thing I didn’t talk about and you have given me the opportunity to is when I first started doing this work, we used these intuitive tools which were very powerful for many of the things we do, they just lead us down a garden path for some things. I am going to see if I can teach people in such a way that believing it wouldn’t work, that you can’t do it. The instructional stuff which I was just referring to college majors and so on, showed that that was wrong, I mean you really can change it but then it became clear that you really can change it, much quicker than that. We are able to teach the law of large numbers purely in an abstract way, defining sample, defining population, defining error, never giving concrete examples at all and people can immediately apply it to the kinds of problems I was talking about here. Or you can give them concrete everyday problems without doing any abstractions at all just saying here is how I think about this problem, what do you think? That is also quite effective, I mean the two together are as you would expect better than one by itself. And these effects last in a totally different context because weeks later we call people up in the guise of an opinion survey and ask them about some public policy or some kind of decision they have to make and we know what statistical rule they ought to be applying to get the right answer to these questions and there is substantial savings of gain. Now why don’t people do this in their classes, I mean why don’t statisticians give examples like I am giving or economists? Economists, I hate to say it but at the University of Michigan, senior’s ability to apply cost-benefit analysis to the problems we give them is not effected by the number of courses. When you ask professors, you could give examples, you could teach this stuff, they say well it is hard enough anyways, it is just what you have to do and I can’t spend time with these kinds of examples. First of all it is much less time than they might think, you already get huge gains and secondly I think it is that they are at an educational aid, I think they would literally learn better if they substituted some everyday real life examples in place of some other formal methods. You would get people understanding the formal material better but I don’t have proof of that.
Given we can now go to the internet nowadays for a lot of substantiate information which we couldn’t before, that should free up space in the education system to teach the principles which you are talking about. I wonder if we should draw a line under it at that point, we have had some very good questions as well as a wonderful presentation, which was absolutely fascinating. The name of the book is ‘Mindware’ and thank you very much to the Henry Jackson Society for organizing this, to Alan, thank you very much to Richard for a fascinating talk and please continue the conversation.