https://youtubetranscript.com/?v=5p5YEvi8CHQ

So, the first thing I want to show you is this thing called a Galton board. Now, this was worked out by Francis Galton, who’s actually quite an interesting person. If you hear about Francis Galton, you’ll often hear that he was the first person who tried to measure human intelligence, but that’s actually not true. What Galton tried to do was measure human eminence, and he thought of eminence as, I suppose, something like high position in the cultural dominance hierarchy. And this was back in Victorian England in the late 1800s, and so eminence would have been intellectual achievement, financial achievement, cultural achievement, achievement broadly speaking, but really considered within the confines of a dominance hierarchy. So at that point in time, the English regarded themselves as the prime nation and ethnicity on the planet, and then the aristocratic Englishmen regarded themselves as the prime human beings in that prime country and ethnicity. And so it was really an idea that would be more associated with dominance hierarchy, uppermost dominance hierarchy position than anything we would conceptualize as intelligence today. And Galton was a polymath. He was a genius in many, many fields, and he got very interested in the idea of measuring human differences. And he tried to determine whether the differences that he could measure, and some of those things were like reaction time and height and physical fitness and I can’t remember all the other things that he attempted to measure, but he wanted to see if any of those things could be used to predict eminence. they couldn’t, partly because eminence is not the kind of category that you can make into a scientific category. And partly because his measurement instruments in many ways weren’t sensitive enough, he didn’t make enough measurements and the statistics weren’t sufficiently sophisticated. But the idea that you could measure elementary attributes and that you could use them to predict something important was still an important idea. And it’s one of those examples in science, sort of like phrenology. I mean, everyone makes fun of phrenology now. The idea that you could read someone’s character by mapping out the protrusions and dips and so forth on their skull, it sounds ridiculous to modern ears. But there was again, there was some idea behind it, and that was the idea that cortical functions could be localized and that cognitive functions could be differentiated into separate functions. We still do that. We think of emotions and motivations and personality traits and intelligences, even though intelligences is not a very good idea. The idea that you can differentiate human psychological function and that that would be related in some way to the underlying neurological and physical architecture, that’s not a stupid idea. An idea can be very intelligent at one level of analysis and not so intelligent at the other. So I would say the problem with the phrenologist was actually a problem of operationalization. They had the theory right in some sense, but they didn’t have the measures right. Now one of the things Galton came up with was the normal, he came up with this thing called a Galton board. And a Galton board demonstrates how a normal distribution is produced. Now basically what you, the normal distribution is an axiomatic, it’s an axiom of modern statistics. And every system has to have its axioms. And the axiom of normal distribution is basically the idea that around any measurement there’s going to be a set, there’s going to be variations in measurement, and those measurements are basically going to be random. And then another axiom is that extreme outlying measurements are going to be rare and small outlying measurements are going to be more common. So you can imagine if I measured any one of you a hundred times with a, let’s say with a tape measure, I’m going to get a set of variables that are not always exactly the same. And so there’ll be an average and then there’ll be some deviation around that average. And if I looked at a hundred measurements that I took of each of one of you, then I would get a normal distribution around that distribution. And that’s sort of the fuzziness of the mean. Now that might be partly because my measurement instrument isn’t very accurate. It might be partly because at some point when you’re being measured for the 50th time you’re slouching a little bit or maybe other times you’re standing up a bit straighter or maybe I measure you in the morning sometimes and in the evening at other times and you’re actually taller in the morning than you are in the evening because your spine has a chance to decompress at night so maybe that adds half an inch to your height in the morning. And so there’s going to be shifting in movement around the central tendency, around the average. Now if I, turns out if I measure all of you, the same thing’s going to happen. What we’re going to get is an average height, which would be the average height of the people in this room, and then there’s going to be a distribution, basically a normal distribution. Those of you who are much taller than the average are going to be much fewer than those of you that are close to the average. And those of you that are much shorter than the average are going to be, there’s going to be far fewer of you than there are going to be those of you who are just somewhat smaller than the average. Now generally the idea of a normal distribution is predicated on the idea that the variation around the average is actually random. And that’s an important thing because the variation around an average is not always random. Now psychologists will tell you, and so will most social scientists who use classical statistics, that the normal distribution is the norm, or maybe they’ll say more than that, is that it’s the standard case that whatever set of variables you measure is going to come out in a normal distribution. And that means that you can apply all of the stats that you’re going to learn to decompose the world and reconstruct it to your data sets because the data sets will fit the assumptions of the statistics. The problem with that is that it’s often wrong. Now I asked you guys to buy the Black Swan, and we haven’t talked about the Black Swan much yet and I’m not going to test you on it, but I would recommend that you read it, especially if you’re interested in psychology, because you’ve got to watch your axioms. And the idea that the normal distribution might not always be correct and certainly might not be a correct description of your data is a fundamentally important idea. Because if it’s wrong, if your data isn’t normally distributed, then the phenomena that you’re looking at isn’t what you think it is and the statistics that you’re going to use aren’t going to work. Now this came as quite a puzzle to me at one point because it turned out that when we produced the creative achievement questionnaire, and you’d think I would have known better by this time because it wasn’t that long ago, it was like 15 years ago. The data never came back normally distributed. It came back distributed in what they call a Pareto distribution, which is everyone stacks up on the left-hand side and the curve drifts off to the right and that’s one of those curves where almost everyone has zero or one and a small minority of people have very high scores. Now when you get a Pareto distribution, there’s something going on that’s not random. And that doesn’t mean you can ignore it. Now one of the things that psychologists will do sometimes if they get a distribution that’s not random is they’ll do a mathematical transformation like a logarithmic transformation to pull in the outliers and to try to make the data fit the normal distribution again, assuming that there’s some kind of measurement error or that the scale’s got a logarithmic function. It isn’t always obvious that that’s a useful and appropriate thing to do because it’s the case in many situations where the extremeness of the distribution is actually an accurate representation of the way that that phenomena behaves in nature. Now I want to show you how a normal distribution works. Have any of you ever seen a demonstration of why a normal distribution is normal and why it takes the shape it takes? Has anybody ever seen that? No, it’s weird, eh, because you’d think that given its unbelievable importance, especially in psychology, where everything we do is measured and has an average and a standard deviation, that there’d be some investigation into why we make that presupposition. There’s a great book, one book I would recommend for those of you who might be interested in psychology as a career, there’s a book called A History of Statistical Thinking. Now you’d think, Jesus, if there’s not anything more boring than statistics, it’s got to be the history of statistics. But it turns out that that’s really wrong. I mean, the history of statistics is actually the history of the social sciences, and not only of the social sciences because statistics became not a… statistics was actually initially invented by cosmologists who were measuring planetary position. And then the statistical processes that have been used to underpin modern science have been tossed back and forth, weirdly enough, between social scientists and physicists over about the last 200 years because the physicists ended up having to describe the world from a statistical perspective, right? Because if you go down into the realm of the atomic and subatomic particles, what you find is that things behave statistically down there. They don’t behave deterministically. And it’s the same at the level of the human scale. You know, we behave statistically, not deterministically. So you know, sometimes you hear this old idea that psychology has physics envy or something like that, but it turned out that for much of the history of the development of statistical ideas, physics had psychology envy, which I think is quite funny. So anyways, if you look at this book, Historic History of Statistical Thinking, you can see how the idea that populations could have behaviour and that populations could be measured and that the idea that states, like political states, could be measured and that human behaviour could be measured. And I didn’t realise until I read this book that how revolutionary the idea of statistics actually was because, of course, it’s part of the idea of measurement. So I had this very interesting client at one point. He was an old guy and he’d been a psychologist and a financier and a statistician and he was in love with mathematics. He was one of these guys for whom mathematical equations had this immediate glimmer of beauty and he made this very interesting little gold sculpture that I have up at my office that’s a representation of what people claim to be the most perfect mathematical equation ever constructed. So he made this little gadget. It’s like a religious icon almost and that’s really what he thought of it. And I’m afraid, foolishly enough, that I can’t tell you the name of the equation. Some of you might know it. It relates I, E and pi. Is there anybody in here who… Yes, what is that? Yes, and what is that equation? Okay do you know why it’s such a remarkable equation? In one equation. So he thought that in some sense this equation summed up the magnificence of the mathematical universe and so it was so funny because he also made little pins that people could wear of this equation that were also made out of gold. There was another person in our department at that point and I was wearing this pin around and she said, oh well, she told me what the equation was. She pointed at that and she got all excited because it was this perfect equation that related all these fundamental constants to one and zero. So anyways, he taught me a lot of this while he was a client of mine. He was very much obsessed with mathematical ideas. He couldn’t get them out of his head. It was a true obsession and a useful one. But he taught me a lot of things about statistics that I just never knew at all. One of them was the Pareto distribution which just staggered me that I didn’t know it. I only learned this stuff about five or six years ago and it just made me feel like a complete moron because I’d gone through immense amount of psychological training and I was measuring things like prefrontal ability and intelligence and personality and creativity and then in the creativity measurement I stumbled into these Pareto distributions. I thought they were a bloody mistake. I didn’t know what the hell they were. I’ve also found economists who didn’t know about the Pareto distribution which is a pretty bloody bizarre thing. Like it really is a strange thing. So anyways, one of the things he showed me is how the normal distribution comes about and I had a student who did a PhD thesis on Galton. So I knew something about Galton at this point. He was doing part of his thesis on the history of the measurement of intelligence and Galton was a key figure in the establishment of that sort of measurement. But this is one of the things Galton invented so let’s take a look at it here. Okay, so that’s your basic rock and roll Galton board apparently. So let me show you a picture of a Galton board so you get a better sense of exactly what it’s doing. So there’s a good one right there. So basically all that happens with a Galton board is that you have a bunch of pegs on a board, the board is horizontal, and you drop balls down it, marbles or whatever it is, and the marbles come in one place and then they bounce down the pins and they distribute themselves in a normal distribution. Now the reason they do that is because there are far more ways of getting down the middle than there are of getting to either side. Now you see because to get to the only to the right side say the ball has to go right, right, right, right, right, right, right, right, and then fall in the little cup and then on the left it has to do exactly the same thing. There’s only one way it can do that, right, but to get into the middle it can go left, right, left, right, left, right, right, left, etc. There’s a far greater number of pathways for the ball to go from the middle down to the middle than there is for the ball to go to the sides. And so all that means fundamentally is that the probability that a given ball is going to land in the middle or near the middle is much higher than the probability that a ball is going to land in the extremes. And so it’s just a description of how random processes lay themselves out around a mean. Okay, so that’s a normal distribution. Now if you have normal distribution one and you have normal distribution two, you can overlap them, right, and if they overlap perfectly, the mean and the standard deviation are the same, then there’s nothing different about them. You know, because one thing you want to ask is well, how do you know if two means are different? And the answer is you don’t know unless you know what the means are measuring and you don’t know what the means are measuring unless you know the variation of the measurement. And so you have to know the mean and the standard deviation. So the standard deviation is like the width of the mean in some sense. And all statistics do, generally speaking, is take one normal distribution and another and slap them on top of each other and then measure how far apart they are, how much they overlap, and correct that for the sample size. And then you can tell if the two distributions are different, significantly different. So what you would do is you’d run a whole bunch of normal distribution processes by chance and you’re going to get a distribution of distributions in some sense, and then you can tell what the probability is that the normal distribution that you drew for group A is the same as the normal distribution that you drew for group B. And that’s all there is basically to standard group comparison statistics. And that’s how we decide when we have a significant effect, if there’s a significant difference between two groups, or better still, if you randomly assign people to two groups and then you do an experimental manipulation and then you measure the outcome, the means, then you can test to see whether your experimental manipulation produced an effect that would be greater than chance. And then you’d infer that, well, there’s a low probability, one in 20, that you would produce that effect if you were just running random simulations and therefore you can say with some certainty that there’s an actual causal effect. That’s with an experimental model. Now, you know, why do you think, all right, let’s say, let me give you a conundrum. So I used to study people who were sons of male alcoholics. Okay, now there was a reason for that. We didn’t have women. Why might that be? We didn’t use women. Why? Why would they be confounding apart from the fact that they’re women? That’s supposed to be a joke, you know. Jesus. It’s a better joke than that. Okay. Well, it could be that there’s some sort of emotional coping mechanism that differs between the way that men deal with it. Because a father-daughter relationship is different than a father-daughter relationship. Okay, well, so that’s a possibility. That’s not a bad answer. It’s not the answer that was appropriate for our research, and I’ll tell you why, because you couldn’t infer it from the question. We were interested in what factors made alcoholism hereditary. Because it does seem to have a strong hereditary component. What we basically concluded after doing a tremendous amount of research was that people who were prone to alcoholism, at least one type of person who’s prone to one type of alcoholism, got a very, very powerful stimulant effect from the alcohol during the time that their blood alcohol level was ascending in the 10 or 15 minutes after they took a drink. Especially if they took a large drink fast, or multiple large drinks fast. You can probably tell, by the way, if you’re one of these people, if you want to go do this in the bar the next time you go, go on an empty stomach, take your pulse, write it down, drink three or four shots fast, wait 10 minutes, take your pulse again. If it’s gone up 10 or 15 beats a minute, look out. Because that means alcohol is working as a psychomotor stimulant for you. And we found that for many of these people that was an opiate effect. What seemed to happen was that when they drank alcohol fast, they produced probably beta endorphin, although we were never sure. It can be blocked with naltrexone, which is an opiate antagonist. The other characteristic of that pattern of alcohol consumption is that the real kick only occurs when you’re on the ascending limb of the blood alcohol curve. So first of all, your blood alcohol goes up, and then it goes down. And generally when it goes down, it’s not pleasant. That’s when you start to feel hungover. And hangover is actually alcohol withdrawal, by the way. So it’s like heroin withdrawal, except it’s alcohol withdrawal. But it’s generally not pleasant for people, so they usually sleep through it. Or it puts them to sleep. But if you’re one of these people who get a real kick on the ascending limb of the blood alcohol curve, then you can just keep pounding back the alcohol, and it’ll keep hitting you and keep you in that position where you’re on the ascending part of the blood alcohol curve. And you can probably tell if you’re one of these people if you can’t stop once you get started. So if it’s like you have four drinks quick and it’s like, man, you’re gone until the alcohol runs out or until it’s four in the morning or until you’ve spent all your money or you’ve been at the last bar in town or that you’re sitting on your friend’s bed after everybody’s gone home from the party and you’re still drinking, you might suspect that you’re one of those people. And if you are one of those people, well then you should watch the hell out because alcohol is a vicious drug and it gets people in its grasp hard and it’s hard for them to escape once they do. People also drink to quell anxiety. So now the problem, because we were looking at the genetics of alcoholism, it wasn’t easy to study offspring of alcoholic mothers. And the reason for that is they might have consumed alcohol during pregnancy. In which case, and that’s a bad idea especially, there’s certain key times in pregnancy where even a few drinks are not good. And that turns out, if I remember correctly, that turns out to be the times when the fetus produces the bulk of its hippocampal tissue. And so anyway, so if it turned out that daughters or sons of female alcoholics were markedly different from the general population in some manner, we wouldn’t be able to tell if that was a consequence of alcohol consumption during pregnancy or if it was a genetic reflection. So what we wanted to do was study sons of male alcoholics. And so their mothers actually couldn’t be alcoholic. And we wanted their fathers, the best subjects had an alcoholic father and an alcoholic grandfather and at least one or more alcoholic first or second degree male relatives. And they couldn’t be alcoholic and they had to be young. So because obviously if they were 40 and they still weren’t alcoholic, then they probably weren’t going to be alcoholic. So we wanted to cast them between say, well it had to be 18, which was in Quebec. That was even a little late probably, but that’s the best we could do from an ethical perspective. So we used to bring these guys into the lab and get them quite drunk. The National Institute of Alcoholism and Alcohol Abuse pretty much put a stop to that research because we used to bring them in and we’d get their blood alcohol level up to .12 or .10. It had to be pretty high. It actually looked like the real physiological effects seemed to kick in when people hit legal intoxication. So you don’t really get the opioid effect until you pop yourself up about .08, which was the legal limit for driving at that point. So we used to get, some of these guys were pretty big. They’d come in, they were maybe 230 pound guys. To get them up to .1 or .2, you had to give them quite a whack of alcohol. And then what we usually do is we let them sober up till .06, about that, and then we’d send them home in a cab. got all ethical on the whole situation. They wouldn’t let us send them home until they hit .02. It was like, well if you’re 240 pounds and we’ve just nailed you with enough alcohol so that your blood alcohol level hit .12, you’re going to be sitting in our bloody lab, bored to death, feeling horrible for like six hours or seven hours. And you’d be pretty damn irritated about it. It wasn’t obvious how we were supposed to keep the people there. It’s like, well, can I leave? No. I’m not paying you if you leave. It’s like, that’s going to produce real positive outcomes with like drunk people in the lab. That’s going to work really well. Of course, then they’d never come back either because it was such a bloody awful experience. So I stopped doing that research partly because it became impossible. Anyways, we did find out a lot. We found out that there was this one particular pattern of alcohol abuse that seemed to be hereditary. So we couldn’t study it women. Anyway, so now there was a problem with this line of research. And the problem was, well, alcoholism comes along with other problems. So this was correlational research, right? We’d take, we’d pick a group of people and we’d match them, the sons of male alcoholics, we’d match them with people who weren’t sons of male alcoholics. So they were still sons, they were still the same age, but their fathers weren’t alcoholic. Now here’s the problem. What should we match them on? Age, gender. Well, you can’t match them on number of drinks, obviously, because you want the people who are alcoholic in one group not to be, you don’t want alcoholics in the second group. So do you match them on number of drinks? Well if you don’t, then you don’t know if the effect that you’re measuring is a consequence of the genetic difference or on the number of drinks per week or drinks per occasion that the people had, right? That would be a confounding variable. Do you match them for antisocial personality? Or antisocial personality symptoms? Because lots of people who are alcoholic, they tilt towards the antisocial side of the spectrum. So do you control for that? Well you don’t know because you don’t know if antisocial personality is part of alcoholism, like it’s part of the same underlying genetic problem, or if it’s a secondary manifestation, or if it’s a consequence of drinking. You don’t know any of that. Do you match them on depressive symptoms? Do you match them on schooling? Do you match them on education? Do you match them on personality? Do you match them on other forms of psychopathology? Do you match them on what they drink? Do you match them on how many drinks per occasion they drink when they drink, etc.? Well the answer to that is, you don’t know. And so what that means is there’s actually an infinite number of potential covariance. Because you don’t know what differences there are between the two groups, or the differences that are relevant to the question at hand. Now that’s actually one of the big problems with psychiatric research. In fact, it might be a problem with psychiatric research that’s so serious that it cannot be solved. You know, so if you take kids who are attention deficit disordered, say, horrible diagnostic criteria, and you match them with kids who aren’t, and then you look to see what makes the ADHD kids different, what do you match them on? Well you don’t know. And that is, so usually what happens is the people who do psychiatric research finesse this a bit. They match them on the important variables. Age, physical health maybe, education. But the problem is, you actually have no idea what the important variables are, and there are an unlimited number of them. And so that’s actually why random assignation to groups is so important. Now if I take all of you, and I say, well, let’s look at the effects of alcohol. So what I would do is I’d say, you’d come towards me and I’d do a golden board sorting. You go to the left, you go to the right, you go to the left, you go to the right. This is non-biased separation of the two populations. And then maybe I’d give one group two ounces of alcohol and water, or in coke say, and in the other group I’d put a few drops of alcohol on the top of a glass of coke so it smelled like alcohol and tasted like alcohol when you had your first drink, and then I’d put you through a whole battery of neuropsychological and personality tests. Which I did, by the way, to a bunch of people when I think it was the first publication I had back in about 85 or something like that. Random assignation gets rid of the necessity for the infinite number of covariance, right? Because you’re all different in a whole bunch of important ways, but we could assume, as long as I threw half of you in this group randomly and half of you in that group randomly, that all your various idiosyncrasies, no matter how many there are, would cancel out. And that’s the massive advantage to random assignation. Now one of the things you’re going to notice when you go through psychology, especially you do psychopathological work, is that the studies are almost always correlational. And they’ll control for relevant covariance, that’s what it’ll say in the paper. But the problem is, you don’t bloody well know what the relevant covariance are. If you did, you’d already understand the condition, you wouldn’t have to do the damn research. So you can make a pretty strong case that all psychiatric research that is studying psychopathological groups compared to at control, it’s all not interpretable. And it turns out in that kind of research that who the control should be is the killer. Because you don’t know. Picking the psychopathological population is easy, you just pick them according to whatever diagnostic checklist you happen to use. But when you figure out who to control them against, it’s like, siblings? I mean, what do you do? How do you match a population? The answer is you can’t. So anyways, the reason that we didn’t include women was because of the uncontrollable potential confounding effects. And the only way that you can do that is to assign randomly. And that’s an important thing to remember. It’s why experimental designs are way more powerful than correlational designs. The problem is, frequently in psychology, what you see are correlational designs. Now I think if you’re careful and you dig around and you’re obsessively careful, you can extract information out of correlational studies. But it’s no simple thing to do. You see this problem come up too when you hear about studies on diet. It’s like, well you should eat a low-fat diet or you should eat a high-fat diet. Usually what happens is they track people across time who hypothetically have been having one diet or the other. But the problem is you don’t know what the hell else makes those people different. And the problem with that is there is an infinite number of potential things that make them different. And that’s a big problem. The probability that you’ve picked the one thing that makes people who have a high-fat diet different from other people and that it happens to be that they have a high-fat diet, and that’s the only difference. It’s like, yeah, no, not at all. Definitely not. So okay, so anyways, random assignation gets rid of the problem of the infinite number of covariates, and that’s worth knowing. And then the other thing that we’ve just figured out is that if you make a measurement, you’re going to get a distribution around the measurement. And if you do that in two groups, you can check out the overlap between the two groups and you can determine what the probability is that that overlap is there as a consequence of chance. Now you might say, why not set your probability level to like 1 in 10,000? So you could be absolutely damn sure that the two groups don’t overlap. So why wouldn’t you do that? Why wouldn’t you set P equals .0001 instead of P equals .05? So that would be 1 in 1000 instead of 1 in 20. Yes? Because you might include that there’s no effect when they’re… That’s exactly right. You’re basically damned if you do and damned if you don’t, which is a very important thing to remember about life. Because generally when you make a choice of any sort, there’s error and risk on both sides of the choice, right? It’s really, really useful to remember that because people always act like their current situation is risk free, which is never, ever the case. Yeah, so people were trying to figure out, well, how do we balance the risk of finding something that doesn’t exist against the risk of not finding something that exists? The reason it’s 1 in 20. And why is that? It’s because someone made that guess 40 years ago or 50 years ago and it’s just stuck. There’s no reason. And so the other thing to notice is that you want to be careful about the P equals .05 phenomenon. Because one of the things you’ll see is that people treat any experimental result they get like it’s significant if the probability is .05 or less, and it’s not if it’s .06 or greater. And that’s not smart. Because the cutoff is arbitrary. You need to know three things to interpret an experimental result. It’s like you can’t calculate the area of a triangle without knowing three things. I think it’s three things anyways. You need to know the number of people in the study. You need to know the size of the effect, so that would be the relationship between two or more variables that you’re interested in looking at. And then you need to know the probability that you would find that effect size among a population of that size by chance. And you cannot interpret one of those numbers without the other two. It’s not possible. And now, psychologists, statisticians among psychologists who have a clue, have been jumping up and down for 50 years trying to get psychologists to report all three of those every time they report anything. Which seems logical, right? The effect size is the difference between the two means divided by the standard deviation of the pool group. Well, you need both of those because it’s the standard deviation that tells you how damn big the numbers are. Because what does 70 mean? Well, it doesn’t mean anything. 70. It’s like if I just came out here and said that, I could say 70 and 40. Are those different? Well, what the hell does that mean? It doesn’t mean anything. You need to know what the units are. And the distribution gives you the units. And so once you know what the units are, you can say, well, here’s how big the difference is between these two groups and here’s the probability that that would be acquired by chance. And so that’s how confident you could be that it’s an actual difference. And so you want to always report effect sizes. And most of you are going to be taught to look at the damn probability. And that’s stupid. It’s like when you want to know how tall, how much difference there is in height between two people, you want to know how much difference there is in height. You also might want to know to what degree that’s there because of chance. But it’s the effect size that’s the critical variable. Now you can’t understand the effect size without understanding how many subjects were in the experiment and also the probability that you would acquire that by chance. So you want to keep that in mind. And when you’re reading scientific papers, you want to be looking at the effect size. How large an effect size is this? Now all the effect sizes are, you can transform one into another. So you have correlation coefficients which go up to one. You have the square of the correlation coefficient, which is the amount of variance that’s accounted for, and that also goes up to one. And you have your basic correlation coefficients, or sorry, your standard deviation effect sizes, which is mean one minus mean two over the standard deviation of the entire group. So large effect sizes in psychology, I’ve talked to you about this before, are correlation coefficients of about 0.5 or above, or r squareds of 25% of the variance, which is 0.5 squared, or mean one minus mean two of, say, half a standard deviation or greater. And you can derive the conclusion that those are relatively large effect sizes by looking at the distribution of the effect sizes across the published literature. And most of you will be told estimates for effect sizes that are actually way too large, because those were guesses too. How big’s a large effect size? Well some statistician guessed 50 years ago. So it’s just like the 0.05 rule. It’s arbitrary. That doesn’t mean it’s stupid, but it’s arbitrary, and you don’t want to get stuck on it like it’s some sort of fact. Okay, so now let’s look at Pareto distributions. That’ll do. Okay the first thing you’re going to see about a Pareto distribution is it’s definitely not normal. Now one of the things that I mentioned to you before, but I want to hammer this home because it’s unbelievably important for determining how to understand the way the world lays itself out and how to interpret the way that people distribute themselves in terms of their success across time. Now this is a fundamental law, and I’ll show you how it works. So the law is most of you get nothing and a few of you get everything. Okay so that’s the law. Now you might say, well that’s because of, you might attribute that to various things. So for example, if you’re a left winger you attribute it to the inequities of the social structure, and if you’re a right winger you basically attribute to the fact that, well most people are not that good at anything so it’s no wonder they end up with nothing. So they’re not very smart and they’re not very hard working so they don’t get very much of whatever it is that they’re after. Okay so we’ll take that apart a little bit but before we do that I want to show you an animation. Now you’ve got to watch this one carefully. Oh yes that’s… Okay yeah this thing moves very very quickly. So we’ll go back to the beginning and… Alright now. So here’s the deal. Each of you gets ten dollars. Okay so that’s a non-random starting place right? You’re all starting in the same place, that’s definitely not random. Okay now here’s how you play this game. You flip a coin, you turn to your partner and you flip a coin and if it both comes up heads then you win and they lose. And if it comes up tails and heads then you lose and they win. And if the winner gets a dollar from the loser. Okay and so you can imagine that this is a simulation. So basically it’s like your apes trading bananas. If you give away a banana then you’re done with the banana. So one of you will walk away with eleven and one of you will walk away with nine. And then you turn to someone else and just wander around the room and randomly trade. Okay so what happens? Well the first thing that happens is this. Okay so we started the graph with everyone at ten. Now you’ve done a bunch of trades. We don’t know how many trades you’ve done but a fair number. And so what happens, what you see happening is some winners are starting to emerge. Right on the right hand side. Those are the people who’ve won every single trade. Maybe they’ve traded ten times so now they have twenty. And then there’s the people on the other side who’ve lost nine out of the last trades. Now what happens when you lose nine? What’s the big problem that you have? What happens? You only have one dollar left so what happens if you lose another trade? That’s right you hit zero. And zero is not a number like any other number. Zero is like the black hole of numbers. You fall into the zero hole and that’s it. You’re not in the game anymore. And poverty is like that. It’s like that. It’s kind of a trap. And it’s very, very difficult to get out of. So and it seems to be in part because once you fall off the charts enough a bunch of things start conspiring against you. So for example, you don’t have enough money to buy a large amount of decent food cheaply. So you have to buy expensive junk food in the short term. So that might be one possibility. Or let’s say you end up on the street. Well you can’t even get a job then because you don’t have an address. And you can’t get social security because you don’t have an address. Like things start to conspire against you very badly. Or maybe you’re unemployed and you’ve been unemployed for a year and a half because it’s been a prolonged downturn in the economy. Well if you’re 17 who the hell cares? But if you’re 50 that might be that for you, right? You’ve hit zero. You’ve been out of the market for 18 months. Nobody’s going to hire you. And so you’ve hit zero. And the problem is we don’t know what the hell to do when people hit zero. It’s difficult to pry them out of zero and throw them back in the game. Now you could say well what if you just re-armed them with money. And that would actually work if the game was truly random. But it isn’t obvious that the game is truly random. And that’s where things get weird. So anyways by this point in the game there’s some winners piling up and there’s some losers piling up. What’s the difference between the people who have 19 and the people who have one? Well we know the one people are going to be wiped out. They’ve got a 50% chance of hitting zero. What about the 19 people? They’ve got a much better chance of not hitting zero because they have to pay a lot more. Right exactly. So they’re in this place where they’re basically sitting pretty. They can fail nine times and that just puts them back to where they were to begin with. So okay so we keep playing. Well so you see what’s happening is that with repeated random play the normal distribution turns into a Pareto distribution and most of the population stacks up at zero. Okay this old guy I told you about he had a theory of social structure that was predicated on this. There’s a bit of a Marxist twist to it. He thought that what happened is that so if you ran a game long enough most people will stack up at zero and a few people have everything. Okay so let’s say now you’re down at zero. What’s your best strategy? Well you might say well we should reset the game. Because if you’re at zero and it’s basically a random game, if you wipe out that game and you put another one in place there’s some probability that when the next game starts playing you’re not going to end up at zero. And that was his theory of revolutions fundamentally. Once the game had played itself out until resources were maximally distributed it didn’t cost the people at the bottom anything to be revolutionaries because they had already hit zero. And so one of his hypotheses was that one of the things that political and economic systems have to do, they have to figure out how to do, is to make sure that the people who end up on the zero side of the distribution don’t have nothing to lose. Because if you have nothing to lose, God only knows what you’re going to do next. Now it’s a big problem because merely shoveling money down there is not likely to change the outcome very much. Yes? Yes? I was about to say that’s a little bit like the gambling policy though. Like if you, because once you reset the game your probability of ending up at zero is exactly the same as the first time. Yeah but the probability that any given person will end up at zero is the same. But the probability that you’ll end up the same isn’t. Because at the beginning of the game you have just as much probability of moving up to the top as you do of moving down to the bottom. Once you’re at zero though you don’t get to play anymore. So at zero your probability of moving up is zero. Whereas at the beginning your probability of moving up is 50%. So when the gambler’s fallacy is merely that if you keep dumping good money after bad you’ll win it back. Because let’s say you’ve lost ten times in a row you think, well I’ve lost ten times in a row it’s 50-50. I must have a virtually 100% chance of winning the next time. It’s like well no you don’t because probability doesn’t have any memory. That’s the gambler’s fallacy. So alright. Now one of the things we might ask ourselves is, this is something that I discussed with him in late because he really thought of this as a random process. And you know because he thought, it’s very very interesting and very complicated. We know for example that the IQ distribution in this room is approximately normal. And so we can assume that it’s a consequence of random factors. So what do we mean random factors? Well there have been random genetic things going on in your lineage since life began. And here you are, you’re smarter than him perhaps. Why? Well we’re going to eliminate the environmental effects for now. Just forget about them. We’re going to assume that everybody’s been raised in an environment where they had enough to eat and where they have enough resources, informational resources so that their intellect can capitalize. So none of you people have been starved for information or food. So I would say in many ways the important variation in the environment has already been ironed out for most of you. Not completely. Okay so it’s random occurrences in the past in the evolutionary history that’s put you wherever the hell you are in the IQ distribution. So you’re the beneficiary of random forces. So he thought of the problem as being random all the way down to the bottom. Now the problem I have with that is, wait a second, we have evidence that some things predict where you’re going to end up in the distribution. And so what are those things? So IQ, conscientiousness, emotional stability, openness if it’s a creative product, and then some other smattering of personality features depending on the particular domain. So for example if you want to be a salesperson some extraversion is useful. If you want to be a manager it tends to be better if you’re disagreeable to some degree rather than if you’re agreeable. If you want to be a nurse or a caretaker then agreeableness is useful. But the big performance predictors across time are intelligence and conscientiousness. Now if the damn game is random, which the statistics seem to indicate or at least that you can model it using processes that model random processes, why in the world do IQ and conscientiousness, we’ll just stick with those two for now, why in the world do they predict success? So does anybody have any ideas about that? Let’s start with IQ. I mean for example if you apply your IQ to the stock market, if I distribute to you all a bunch of money and I measured your IQ and I said okay put together a portfolio, you get to pick 30 stocks, you can’t sell them for a year, you get to sell them at the end of the year, what’s the probability that the high IQ people would pick a better set of stocks than the low IQ people? The answer to that as far as I can tell is zero. It’s not going to, because you can’t pick stocks as it turns out. All the information is already eaten up. And the stock market is an interesting analogy of the environment, right? You know what I mean? Because the stock market is basically an index of the continually transforming human economic and social environment. It’s basically random. You can’t predict the damn thing even if you’re smart. So why is it that smart people win across time? And then again, why is it that hardworking people win across time? No but that’s only a restatement of what I just said. It’s not a causal account. You’re basically saying that the high IQ people make better choices. Yeah, but that’s, if it’s a random environment, how can they? Money, we can view it as opportunities and genetics which make somebody adaptable in multiple environments. And each coin flip as each subsequent generation of that lineage. So that there is an increase, like the more good adaptable genetics, emotional stability and situational wealth you have, the more likely your next generation has, you know, like they have to lose a lot more to get to zero. Okay, so what about a given individual though? Forget about it playing across time. And that’s also a weird thing because some of the single-celled organisms that were your ancestors three billion years ago are still single-celled organisms. Whereas here you are, you know, same environment roughly speaking. So there’s this tremendous branching of possibility across time. And you might say, well IQ is adaptive, which is a terrible word, adaptive. It’s like, yeah, okay, so how do you count for all the single-celled organisms? They’re just not that bright, but there they are. There’s more of them by weight than there are of us people by weight. So I puzzled this out a bunch of different ways and you can think about this and see what you think. I mean, the first thing that seems to me to be the case is that you don’t have to play one game. You know, like imagine that you’re sitting around with your friends and there are 50 board games going on and you can keep a hand in each of them. Well then, at some point in one of the board games you’re going to start to amass some success, right, just by chance. If you’re playing 50 board games and 25 of them after playing for some time you’re going to be at the top of it, you know, you’re going to be moving towards the top 50% rather than the bottom. We might say, okay, play 50 games, play 20 turns, throw away the ones that you’re doing the worst at, the 5% that you’re doing the worst at, and then just keep doing that until you end up with the one game that you win. And maybe you can do that better if you have high IQ because you’re faster. So maybe all that happens is that if you’re smart is that you can play many games faster and as a consequence of that you can choose the ones that you seem to be winning and stick with those. And then maybe what happens with conscientiousness is that you actually stick to them. So that means that you actually don’t have to predict the future in order to master it. It means that you have to simulate multiple futures, keep your options open, and sort of play dynamically as the environment unfolds. So now it might be more complicated than that because it’s also possible that in some environments, at least for local periods of time, the game is actually not random. You know, so it’s funny. Do you think, is monopoly random? What do you think? You’ve all, everybody here has played monopoly I presume, right? Is there anybody who hasn’t? Okay, so what do you think? Do the smart people win monopoly more often? I think it’s the people that kind of have. You do, you think, so you would say it’s the conscientious people that win more often. They have to decide what each move will lead to. Yeah, well it’s pretty clear that you can lose stupidly in monopoly, right? So I’m not sure if you can win intelligently. You know, you can avoid obviously bad choices. Like one choice would be don’t buy any property and just hold on to your money. That seems to be a losing strategy, right? So that’ll wipe you out. So maybe another thing that smart people can do is that they don’t do things that are self-evidently, that make it certain that you’ll lose. It might be something like that. So, alright, so one of the things we might ask ourselves too is, given that IQ and conscientiousness do predict success, how much success do they predict across time? And the answer to that is they predict a substantial amount. If you combine IQ and personality, I’ll show you the equations. These were formulated in the 1990s. So we’re going to look at this here. So let me walk you through it a bit. So this is a spreadsheet I put together quite a long time ago. Get this a little bigger so you can see it. So here’s the elements of the equation. So you need to know, first element in the equation. How many people, so we’re going to say, well how powerfully can you predict the performance outcome of people across time? And the value we’re going to use is dollar value. And dollars are basically, they’re sort of like a, obviously they’re the universal currency because you can trade money in for virtually anything. So we use dollars as our standard of value. The first question might be, well how many people do you want to predict for? So let’s say we’re going to take ten. So we put that in this part of the equation here. Ten people. Then you might say, well over what period of time do you want to calculate their success? And one decent answer is five years because people seem to stay in the same job position or career position for about five years, although that’s shortening. So that’s another thing that you guys should keep in mind. You’re probably not going to be in the same position in your life for more than about five years at a time. And so you have to plan and plot for dynamic transformations in your career. And you have to do that in a way that works to your advantage, which partly means you have to keep options open and you have to be able to say no. So okay, the next thing you might ask yourself is, well how high a predictor, what’s the R for your predictor, what’s the correlation coefficient? And we put together a neuropsychological battery. I published the results of this with someone named Daniel Higgins quite a long time ago. We used neuropsychological battery looking at dorsolateral prefrontal ability, which we thought of as something potentially separate from fluid intelligence, which probably just turned out to be quite a good measure of fluid intelligence. And we also used conscientiousness and we found that when we validated that in a… we validated it in a factory and we looked at the performance of the administrators and managers in the factory because they had relatively complex jobs and we had access to seven years of their performance records, which someone else had gathered independently of us. And what we found was that for those people who had performance records that were more than four years old, we could predict their performance at about .59, about, about .6. Tremendously powerful prediction. Now one of the things we found was that the degree to which our predictions were accurate increased with the increase in the number of years of performance data we had. So for example, if I want to predict how you do next year and I get a performance… you enter a new job and I get a performance index after your first year. I’m not going to be able to predict that very well from your intelligence and your conscientiousness because the measure of your performance actually won’t be very good. Because it turns out that you can’t even figure out how well someone is performing in a job until about three years. So it turns out that if you take a complex job on, you get better and better at it over three years if you’re going to get better and better at it. And so we can’t tell how well you’re doing until about that period of time. So that’s another thing that’s useful to know, by the way, when you guys go off to find your next complex role in life. You can’t really expect to be good at what you’re going to do until about three years. After about four years, additional experience doesn’t seem to matter. So that’s also worth knowing. So you’re going to feel like a bloody moron for the first bit of your new job and that’s because you are. If you keep at it, you’ll accrue experience and expertise quite rapidly over a three year period and that’s probably the right amount of time over which to evaluate your performance. Because you need to know that, right? Should you be freaking out if after six months you’re not doing a good job? Well, you’ve got to see how you’re doing compared to your peers. Hopefully you’re not doing a worse job than them, but if it’s a complex job, it’s going to take you a long time to master it. So anyways, our predictor was 0.59, which we were pretty damn thrilled about. And we’re comparing that, we’re going to compare that to a comparison predictor of zero for now. Zero is random. So if I was going to predict your trajectory through life, let’s say I want to predict your industrial productivity. I’ll take ten of you. I’m going to predict your industrial productivity over the next five years. That’ll be the goal. Or we can do this, we’ll do this a slightly different way. Let’s see, so that’s the selection ratio. The last one is the selection ratio. So let’s say I wanted to select you guys for a position at a, let’s say you’re going to be managers in a relatively complex corporate organization. There’s about 110 of you or something like that in here now, maybe let’s say 100. Say I’m going to pick 10 of you. That means my selection ratio is right there. It’s one in ten. You can transform that into a standard score of 1.76. Now it’s important to know how many people you get to choose from because if you’re going to hire one person and you only have one applicant, there’s not a lot of sense doing any selection because you only have one applicant. So I don’t care how much you know about that person, it’s not going to be helpful if you have to hire them. Now in this situation we’re going to assume that I can screen all 100 of you and only recommend 10. So we’re going to assume you have to put in a variable for job complexity because the relationship between intelligence and conscientiousness increases, the relationship between intelligence, conscientiousness and predictive power increases as the job becomes more complicated, which is exactly what you’d expect, right? So if you can do a job by rote, all that IQ and conscientiousness will predict is how fast you can learn it. But if it’s a dynamic job where you have to make decisions on a non-stop basis, then IQ and conscientiousness are going to predict your performance over the long run. So we’re going to assume that you’re in a complex job. We’re going to assume that you have an annual salary of 4 million more than they would have if they would have just picked randomly, and that would be a productivity increase of 104%. So that’s graphed here. So with random selection, 50% of the people you hire are going to be above average and 50% are going to be below, and using a good psychometric battery you can get that to about 80-20. And the consequence of that is 400,000 more by hiring one better person. Right, it’s 100,000 a year. It’s actually more than you’re paying them in salary. And now the reason for that, there’s a bunch of reasons. The first reason is that there are tremendous amounts, there’s tremendous differences in individual productivity. Now one of the things that’s weird about these formulas is this formula is predicated on the idea that productivity is actually normally distributed, but actually productivity is pre-dodistributed. So what that means is that the basic consequence of using this sort of prediction is probably higher than this formula indicates. Now there’s a variety of reasons to know this. One is obviously the practical idea. The practical idea is of what use is it knowing something about the people that you’re going to hire or work with? And the answer to that is it depends on what you know. But if the things that you know are valid and they include personality and intelligence, it’s not only valuable to know it, it’s virtually vital because the success of your enterprise will depend on the people that you hire and associate with. The second thing is, I think, that knowing this should change the way that you look at the world. If you understand that the outcomes in life are distributed on a pre-dodistribution and that there are inbuilt temperamental factors that play an important role in determining that, one of the problems that you have before you as modern people is what the hell do you do about that from a conceptual, social, political, and economic perspective? Because nobody who’s currently considering how societies are structured pays any attention to this sort of thing. They don’t assume that there are differences between people with regards to their life outcome chances, not of this sort of magnitude. And so none of the social policies that we have in place reflect any of this. And so I would say, like our psychometrics, our 21st century and our political and economic theories are basically 17th century. And it’s not a good thing. And you guys are going to suffer for this or benefit from it because as society becomes more technological and as it transforms more and more rapidly, the degree to which the pre-dodistribution is going to kick in is going to increase. Now you already see that. That’s why the separation between rich and poor in industrialized countries is becoming increasingly severe. Now that’s modified by some degree by the fact that the middle class worldwide is growing like mad. And so that’s a really good thing. But still the long-term play is the pre-dodistribution. And it’s partly because an intelligent person is one thing, but an intelligent person with infinite computational power, that’s a completely different thing. And that’s where we’re headed. So I don’t know exactly what to make of all this sort of thing politically, but I know that to the degree that we ignore it, we’re going to have very unhappy and unstable societies. So we’ll see you Thursday. On that happy note, we’ll see you Thursday.