Zeroes and Ones： Into the Depths of Computation ｜ Jim Keller ｜ EP 272 (1TmuJSbms9c)

https://youtubetranscript.com/?v=1TmuJSbms9c

I knew this wasn’t going to work. So you’re in a space. The direction I’m going is not going to work. So you know how mosquitoes work? Mosquitoes are fun. So they detect two things. They detect water vapor and carbon dioxide. So mosquitoes will fly along in a direction as long as the water vapor and carbon dioxide are staying the same or going up. But as soon as it starts to go down, they change direction in a random direction. Right. And within a couple of turns, they’re aiming right for a mammal. They can bite. It’s kind of colorful. So if you know you’re going in the wrong direction, a change in direction can maybe is just as bad, but there’s some chance it’s good, especially if you’re somewhat smart and you have some experience. Right. So there was a whole bunch of- So even a random move is better than no move if the outcome is certain failure. And so that is some justification for taking a risk. Hello, everyone. I’m pleased to welcome Jim Keller to my YouTube channel and podcast today. Jim is a microprocessor engineer known for his work at digital equipment, AMD, Apple, Tesla, and Intel. He was a professor at the University of Toronto and he was a professor at the University of Toronto and he was a professor at the University of Toronto and he was a professor at the University of Toronto and Intel. He was co-architect for what were among the earliest of 64-bit microprocessors, the EV5 and EV6 digital alpha processors designed in the 90s. In the later 90s, he served as lead architect for the AMD K8 microarchitecture, including the original Athlon 64 and was involved in designing the Athlon K7 and Apple A4 through seven processors. He was also the co-author of the specifications for the x86-64 instruction set and hyper transport interconnect. From 2012 to 2015, he returned to AMD to work on the AMD K12 and Zen microarchitectures. At Tesla, he worked on automotive autopilot hardware and software designing the hardware three autopilot chip. He then served as senior VP of Silicon Engineering, heading a team of 10,000 people at Intel. He is presently president and CTO at Tenz Torrent, building AI computers. He’s also my brother-in-law and we’ve talked a lot over the last 20 years. He was a friend of mine before he married my sister and we’ve known each other for a very long time since we both lived in Boston. So I’m very happy to have you, to talk to you today, Jim. I’m really looking forward to it. So thanks for agreeing to do this. Sure thing. So let’s walk through your career first. It takes some unpacking, your resume takes some unpacking to be comprehensible, I would say. So let’s start with digital equipment. You were working on very early stage sophisticated microprocessors. So tell me about that. Well, the long story is, I graduated college as an electrical engineer with a bachelor’s and I took a job in Florida because I wanted to live on the beach. And it turned out to be a really interesting job was at Harris and I spent like two years working in the labs, picking up electrical equipment and doing some networking stuff and some visual design. And at some point, a friend told me I should work at digital. So I read about digital and I literally read the computer architecture manual for the VAC 780 on the plane on the way to the interview. And then I interviewed them with a whole bunch of questions because I just read this architecture spec, which I didn’t know that much about to be honest, but I was kind of a wise ass as a kid. And they hired me because they thought it was funny. And so I got my architecture education working on the VAC 8800, working for a guy named Bob Stewart, along with some other really great architects. I spent about seven years in that group, learning to be a processor architect. And then I spent a little time at digital research in California for six months, and then went back and joined the Hudson team where they were building alpha processors and became co-architect with Pete Bannon on EV5 and then with Dirk Meyer on EV6. Now those chips- I was about 15 years and we worked on, I would say three very successful shipping products and then a couple other products that didn’t get to market. So those chips from what I remember were remarkably ahead of their time, but that didn’t seem to save Digital Equipment Corporation. Yeah. Is that fair to say? That’s definitely fair to say. But digital literally made the world’s fastest computer in the years they were going out of business. And there was a complicated market dynamic. Digital was very successful building mini computers in the mini computer age, which replaced the mainframes to a large extent, but they missed the boat on the PC revolution and would say workstations. And they had a big expensive computer mindset right when computer prices were falling much. So we were building really fast processors for server and high-end workstations, and the market had kind of moved on and let’s say a lot of crazy things were going on inside the company at the time. Yeah. Well, we’re going to return to the topic of crazy things going on inside computers, but it’s interesting to note right there. So that’s a situation where a company has a great product, but doesn’t know how to launch it into the marketplace or is blinded by its own preconceptions. It can’t even say necessarily what it has. I think it’s more than that. Gordon Bell was CTO and he was a really brilliant computer architect, but he also had really good, let’s say, observational skills. And like midway through the backstay of the hundreds, he decided the technology we were using was too low, too late and redirected the program and it became a very successful product because he knew what was going on and he made decisions like that. When he left, I think digital became an argument between business unit managers, not about technology. And alpha was a great technology, but it went into business units that were aiming at high prices and high margins, not market penetration and not basically keeping up with the software revolution that was happening at the time. So it was, Ken Olson was a great manager, but he wasn’t a technical leader. And without Gordon Bell, the company kind of lost its way. When companies lose their way, they fail. And they failed fast, too. They went from record profits to losing a billion a year and they kind of rectified that for a little while and then it shall drop off again and over. Yeah, well, one of the things I’ve been struck by watching your career and talking to you over the years is exactly that, the rate at which a company that appears dominant can disintegrate and disappear is really something quite stunning. I think Fortune 500 companies tend to last no more than 30 years. That’s approximately the span. And that’s not a tremendously long period of time. So there’s always dominant companies and there isn’t always a handful of dominant companies, but the company that’s dominant tends to shift quite quickly. So we’ll return to this a couple of times as the classic S curve in economics. You start out low, you solve a problem, you ramp up the plateau and then you fail. And this dominates business, it dominates humanity at some level and it plays out over and over. Now it sounds like you didn’t have, so how is it that you managed to do this job? You intimated when we were talking that you weren’t really trained for it. And so you were trained as an engineer. You had an, is it a bachelor’s degree in engineering? And how prepared were you as a consequence of your degree for any of the jobs that you undertook? Training is highly overrated. So a good engineering degree is math, physics, basic understanding of science and some smattering of communication skills. You can probably do a great engineering degree in two or three years if you’re dedicated to it. You know, the things that stretch my brains the most when I went to college is with math and mechanical engineering. And Penn State, I went to Penn State and they used mechanical engineering and math as two of the weed out courses to find out if you had the chops or the gumption to get through engineering. So there was a fairly high failure rate there. But mechanical engineering is a really interesting discipline because you have to think, think about solving math problems spatially. Like, you know, do things like, how do you calculate the force on a rotating accelerating object? Like it’s somewhat complicated. And it makes you really think. So you have to learn to think. And in engineering school, you never answer the multiple choice question. You learn stuff and film is and stuff, but then you calculate and to understand what the result is. And the problem sets are like little design exercises. And that’s quite good at math. How much of it do you think is pure screening, let’s say, what wouldn’t be pure screening, but fundamental screening for conscientiousness and IQ and how much of it is learning to think, how much of the education process is that? Like if you’re going to hire an engineer, are you hiring fundamentally on the basis of IQ and you get smarter people from the top schools? Or do you think that the engineering training actually does prepare people for a technical career? Well, it depends on the engineer and it depends on the school and depends on their approach. So my IQ isn’t super high compared to really smart people. I mean, it’s high enough. When I went to college, it took me about a year and a half to learn how to think properly. And I found for me personally, I had to do the work on a regular basis early. I didn’t study for finals. I wasn’t the kind of person that could pick up a book, understand and get an A the next day and then forget about it. I’m not, I can’t do that. So I had to learn how to do the work, go through the mechanisms, automatize some of the basics so I didn’t have to think about them so hard, but literally let my brain work on this stuff so then I could use them to go problem solve. And especially in engineering, there’s lots of different kinds of engineering. There’s like highly technical stuff where you turn the crank like a skilled lawyer might. But there’s other stuff where you have to be really creative. If you have an unsolved problem, nobody solved it first. And as an engineer, you have a skillset, right? But you have to apply it creatively. And there’s lots of high IQ people who aren’t creative and there’s low IQ people that are creative. And you find in a big engineering team, there’s a real diversity of personality types. There’s open-minded people, conscientious people, gregarious people, and it takes many different kinds of people working together to do something sophisticated. I’d say, like some of my senior classes in engineering was just going a little deeper on stuff I already knew. Like I could have left after three years and been just fine. But I do think the work I did did help me be an engineer, but then the problems I saw, I worked on after I graduated college. Like in school, most of the problems you’re given, there’s a known answer, because they’re in a book, and you’re in a room with 20 people and they’re doing the same stuff. When you’re an engineer working in the company, they never have two people the same thing to do because that’s a waste of money, right? And when you start engineering, you’re given relatively small tasks by your manager or supervisor. But as you go along at some point, depends again on who you are, you’re working on stuff that you don’t even know how to deal with. There’s no answer in the book. So it’s, but it’s not like physics, right? Like physicists are a funny bunch. I realized this the other day that physicists, they’re supposed to work on stuff that’s unsolved. Whereas engineers, there’s a big repertoire of engineering and then it’s reduction of practice. And then the world’s complicated. So you go build a new bridge that’s never been built before. It’s not like bridges are unsolved problem. This particular bridge hasn’t been solved before. There may be unique challenges to it, but it’s not like physics where you’re looking for an unknown particle or, you know, there’s a pretty big dividing line between engineering and pure science. Engineers typically work in domains where there’s many, many knowns and unknowns are problems of the combination of reality, complexity. Whereas physics in principle, they’re working on stuff that’s fundamentally unknown. As soon as it’s known, they have to move on because then it’s engineering. Like, you know, translate the unknown into engineering and engineering applies known concepts to unknown problems. Okay, so you, okay, so now you went from digital to AMD and you learned how to design microprocessors. So at AMD, you worked on the K8. Yeah. At that point, AMD was losing ground to Intel. Yes. And so how did you fix that? So basically Dirk Meyer and I were co-architects of the 86, the third alpha chip. He left about, he left a digital to AMD about a year before I did. He started the K7 project. When I joined, I started the K8 project and then helped and then worked with him significantly on K7 as well. And how did we do it? Well. Yeah, so those were 64-bit chips that you guys designed to compete with the Intel chips that have dominated the home computer market at that point. Well, so there’s a funny thing, which is, like at some level building fast computers isn’t that part. Right? So you have to have a goal. Like, so a lot of designers, so they have a design and then the easiest thing for the next one is to go look at that design and make it like 10% better, 20% better. Right? But every one of those designs has limitations built into it. Like it’s sort of like, if you buy a two bedroom house, you can add one bedroom. You can’t add eight bedrooms. Right? If you want an eight bedroom house, you have to build a different kind of house. Right? So every design you build has kind of a, you know, a range that it can play. And, you know, you build the first one and you know you can make some improvements, but at some point the improvements don’t really help that much. Right? And so AMD, they had a design called K5, which for complicated reasons didn’t work out that well. And they lost ground to Intel. Before that, they had literally the 386 and 46 AMD copied Intel’s designs. They were clone manufacturer. The K5 was their first design and didn’t work out that good. And then they bought a company called NextGen, which had K6, which is an okay design, but it wasn’t competitive against Intel. And then K7, Dirk was the chief architect of, and he designed a computer that was competitive and the head of Intel. And some of that came from our work at Digital on UB5 and UB6. Dirk worked on UB4 as well. And some of it was just saying, in this day, you know, we have this many transistors, but you get more transistors every generation. So you can basically imagine you’re building a house. Suddenly you have way more bricks and way bigger steel beams. So your idea about what to build has to scale with that. And then K7 was a 32 bit chip and then K8 was a 64 bit chip, you know, somewhat related to that as it turned out, but also it was built to be thicker. And what I did is I wrote the performance model. I came up with the basic architecture and I started to organize a team around building it. And while we were doing that, we also wrote the thing called hyper transport, which became the basis of essentially all modern server computers or what’s called two socket servers. We wrote that in 98 and 2002, or 2022 they’re still building them that way. When you say you wrote it, what does that mean? What does the process of writing that entail? What is it that you’re writing and how do you do that? I’m dyslexic. So I wrote a complete protocol spec about how two computer chips talk to each other in 18 pages, right? Which is relatively terse. And there’s a couple of pictures and computer protocols are pretty straightforward. There’s a command, there’s the address you’re talking to, there’s the data you’re moving, there’s some protocol bits that tell you how to exchange commands, right? And then Dirk took the spec and said, you mind if I slush this out a little bit? And three days later, he sent me a 50 page version of it, which clarified all the little bullets. And then that specification, we literally used to build the interface between K8 chips. Right, so there’s a couple levels of design. What sort of impact did that have? What sort of impact did that have on the broader world? What’s the significant? It’s very difficult for non-engineers to understand any of this. AMD market share. It’s so underground. AMD market share and server went from 0% to 35%, which was a huge impact to the business. And it became essentially the standard because apparently Intel had a version of that, but it didn’t go to market. But after Opteron came out to market, Intel built a similar version, similar protocol about how to connect a small number of processors together with that kind of interconnect. And then that, let’s say design framework became standard in the industry. So if you go into like a Google data center and you pull it out, there’ll be two sockets with an interconnect between the two of them and each socket will have memory attached to it. And they call it the 2P server or two processor server. And it had a really big impact. We didn’t do it because we thought it was gonna have a big impact. We did it because we thought it was a better way to build computers. And at AMD, we were somewhat resource constrained. So we couldn’t build a thing that looked like a big IBM server. So we built was basically a small server with a minimal amount of interconnect between it. So it was a little bit of creativity by constraint, Steve Jobs line. And what function does those servers have, again, in the broader world? What are they doing now for people? Well, it’s basically the entire cloud. It’s all Google, all Amazon, all Facebook, all Microsoft is there. But here’s the interesting thing. When we built them, the big server guy, servers used to be backplans like this big with multiple CPU slots, multiple memory cards, multiple IO slots. And the server manufacturer thought the server was oriented around the back. So IBM, HP, Dell, they all turned this down. But all the little starters at the time like Google, were using PCs as low cost servers. And we made this, basically you could take a PC board instead of putting one computer on it, you could put two, which radically saved the money. So when AMD made those kinds of servers, it was a way lower entry point for server class technology. And the little startups at the time used it. And then over 15 years disrupted all the big server manufacturers. So it’s one of those, I couldn’t say we planned it, like the constraints and we had a target market, we didn’t know that it was gonna become essentially how servers should go for 20 odd years. But it happens. After AMD went to Apple, you worked on the A4 through seven processors. Well, I worked at two startups that did processors for networking, Sidebyte and TSME. And that was probably about five or six years. And then I joined Apple in 2008. So I guess I was, no, it must have been eight years. I was AMD 98, 99, 2000, and then I worked at startups for about eight years. And then I went to Apple. Yeah, and worked on mobile processors. Tell me about those chips and what you did at Apple. What that did. I had some friends that were working at Apple and they wouldn’t tell me what I was gonna work on. So when I interviewed there, they said, oh, you should just come here, it’ll be fun. And I didn’t actually know what I was gonna work on. They had a group called Platform Architecture run by a guy named Mike Colbert, who was like the unofficial CTO of Apple. He worked for Steve Jobs. And he had a group of architects that looked at what Apple was doing and figured out what they should do next. And I worked on a MacBook Air definition. Like I wrote the power management spec and did some other architectural work, which ultimately was an end-to-end chip called MCP89. And then I was one of the chief architects of four generations of SoCs, what’s called A4, A5, A6, A7. And we did a lot of stuff there, but the division was… You know, mobile phones. And SoCs are what? Yeah, system on a chip. Oh, yes. So pack a computer into a phone, you have a little piece of silicon about that big, and all the components, the CPU, the GPU, the IO, are all on the same chip. And when they first started building phone chips, they were considered to be very slow, low cost, you know, very integrated chips. And we thought if you looked ahead, because technology shrinks about every two years, and about six or eight years, we’d have enough transistors on a phone chip, that it would be more powerful than a PC at the time. So we started architecting computers, interconnects and other functions, so that when we have enough transistors, we could literally have, you know, a high-end desktop in a phone. And Apple’s DNA is, you create the product that kills your turn product. You create the product that? That killed, so every company has a great product, and they worry about competitors coming in and kill it. And Apple wanted to be the first to kill their own products. So Steve Jobs saw phones and tablets would replace PCs. And he wanted to be the first to do it. He didn’t want somebody else to do it to him. And did you know Jobs? No. I’ve seen him a couple of times, I said hi to him twice. I felt like I knew him pretty well. Everybody at Apple did. Like when Steve wanted something done, everybody knew the next day. My boss, Mike, talked to him every single day, multiple times sometimes. He used to joke, like we’d walk into Mike’s office and he’d be holding the phone out like this. He goes, Steve, he’s a pest. We’re like, yeah, Steve, so what? But Mike could translate what Steve wanted into engineering stuff, and Steve trusted Mike a lot. And like he could translate the vision into engineering. And Steve’s judgment on stuff like this is spectacular. So, and did you have any sense, do you have any sense of why that is? I mean, Jobs was famously, obviously originated Apple, and then was famously brought back in to save them when they were in danger of extinction, and then in fact did seem to save them. And you never know when you hear about these things from the outside, how much of that is sort of the mythologization of a person, how much of it is, you know, this person was really singular and unique. Yeah, definitely singular and unique. I mean, your psychological parlance, as we talked about, he would be considered high in openness and disagreeableness, right? And I think negative emotionality, like he was a very difficult person. But the solution to, you know, things could go really bad and being disagreeable was, I’m gonna make it as great as possible. And he was willing to take the risk for that. You know, his public persona was very well practiced. Mike used to say, the worse the practice for the Apple keynotes, the better they would go off. Like he was throwing iPhones, you know, at one of the iPhone pre-launch practices, because nothing was right. But then when he showed up and, you know, his persona of, you know, technical explainer, let’s say, you know, that was very real. That’s what he wanted to portray. He believed every single bit of it. You could tell. Though, you know, any engineers that, when I joined Apple, I watched some of his early keynotes when he came back to Apple and changed the Macs. You know, it’s inspiring. But, you know, it’s also super tough, right? Because he went into a company that was very dysfunctional, had a whole bunch of, you know, engineering groups doing basically random stuff. Let’s say senior managers who felt like they owned their product lines and knew what they were doing. And, you know, Steve wanted them to do what he wanted them to do and they didn’t want to do it. And I’m pretty sure he cleaned house pretty thoroughly. And he famously reduced the product lines. And, you know, who knows how many products, like four, you know, there was consumer and professional. So do you think it was that disagreeableness? I mean, we hear all the time now in the modern world about the necessity for empathy and so forth. And that’s the agreeableness dimension. And you’re making the claim that Jobs was low in agreeableness and that he was able to kill off malfunctioning projects. And that’s not exactly a nice thing to do. And disagreeable- So imagine you go to a room full of people who dedicated their life in the last five years of their work, five years of their careers, building products that you can’t sell. And you say, we have to do something completely different. And everybody’s, you know, every day as an engineer, you work on something, you embrace it, you love it, you care about it. Like engineers are very emotional people somewhere in their pointy little souls, right? So, but if it’s not working out for whatever reason, you have to do something different. And if you listen to everybody, you’ll never change anything, right? It’s difficult to get people reoriented. Now, another line you gave me, I don’t know where it came from, is you run fastest when you’re running towards something and away from something. Yeah, that was from animal experimental literature. If you threaten a rat and offer to reward simultaneously, it will run faster towards the reward than if you just reward it. Because you get all your motivational systems on board that way. Yeah, yeah. So Steve was very good at the vision. We are going to build this beautiful computer, right? And you better goddamn build it now or you’re going to die. So that was his, okay, so the openness, the openness, that’s the creativity dimension. That gives him the vision. He’s extroverted, can he communicate enthusiastically? He can certainly put on the act. I have no idea if he’s extroverted or not. I never saw him be extroverted in any natural setting. You know, I’ve seen him walk around. Like I said, I’ve seen him in the cafeteria. He was visible on the Apple campus even until his last days. He didn’t like to be bothered. Like he didn’t go up to Steve and say, hey Steve, how’s it going? Right, well that would be reflective of basic disagreeableness too, right? You know, it’s hard with, it’s hard. Sometimes people can communicate very effectively, communicate a vision because they are high in openness. Extroverted people are enthusiastic and assertive. So they tend to be verbally dominant and can inspire people because they generate a lot of positive emotion, but that can be mimicked by openness. So- Yeah, so you have to remember, like Steve was part of Pixar and very much part of Hollywood and you know, creating movies and creating personas and characters and archetypes. So he was super well-rounded in how that stuff works and what works and doesn’t work about it. Right, and he had an unerring eye for beauty and elegance. Remarkable. And he cared about it. And he would fight for that. And that’s hard. It’s very hard to fight for beauty and elegance. And I suspect it’s particularly hard. Perhaps, maybe I’m wrong about this, but I would think that would be a hard sell to at least a subset of engineers. Yeah, so another, this is explained to my boss at Tesla was, I worked both for Elon and for a guy named Doug Field. And Doug said, there’s this, no, there’s this productivity graph versus order. So at the origin is zero productivity and chaos, right? And then as you add order to your design methods, your productivity will go up. And what happens with engineers is they understand as they get better processes, they get better trained, they get better at working together. Every single thing that makes the whole organization more orderly improves productivity. Unfortunately, that peaks at some point and then too much order, productivity goes down. And so then, you know, as I say, any idiot can see, you should be at the peak, you know, enough order to really be effective, but not so much order to grind to a halt. But why can’t you stay there? And the reason is, is once order takes over the organization, it’s unstoppable, right? It feels good. You get even better at doing what you’re doing. You get even more organized. You micromanage your time even better. You close out all the creativity, you’re not open to change. A whole bunch of bad things happen. You shut out the disorderly people who actually know how to make a change and do something creative. And your organization dies. So successful- Do you think that Jobs was conscientious as well? You know, like was he in early? Was he working 18 hour days? I know he was up in the middle of the night because he called Michael on that stuff. Well, the point is both Steve and Elon were counter forces, but order, right? You have to be really strong to avoid your organization getting captured by order. Well, order also has its remarkable air of moral virtue, right? Because it’s pure and it’s efficient. Everything about it feels good. Hmm. You know? But it’s like alcohol, the first drink feels great. The second feels okay. The third one not so good. But you know, you keep remembering what the first one did, so you drank it. Right? There’s lots and lots of processes, you know, where some is good, too much is bad. But the counter force, the more is weak. And that’s the thing that, you know, so Steve was interesting because he was simultaneously super creative and had visions, which could inspire people, but he also prevented the company from being over organized and preventing him from doing what he was doing. And that’s hard because people, like I said, they get committed to what they’re doing. Yeah, well, it’s an open question. I can imagine that the creative process has a productive component and then a culling component. And the productive component looks like it’s associated with openness, but what the culling component is, is open to question. And it does seem to me that at least upon occasion, it’s low agreeableness. It’s the ability to say, no, we’re going to dispense with that and to not let anything stand in the face of that decision, which would also include often human compassion. Yeah, and people have different approaches to it. Like jobs would call things so they weren’t beautiful or they weren’t great. You know, Elon Musk was famous for getting the first principles and really understanding that fundamentally and culling from like a standpoint of knowledge. Yeah, and you’ve asked me like, what makes an engineer great? Like, so you have to have the will to creativity. Like, now there’s lots of engineering jobs that aren’t creative. Like you need a skillset, you can exercise the skillset. But if you’re going to build new things, you need to be creative. But you also have to have a filter good enough to figure out what’s actually good and bad. Like I know a lot of really creative engineers and they find the new thing excited and they go down the rabbit hole and they, you know, they can work on it for six months and nothing to show for it. So you have to have that conscientiousness. I don’t know if it’s conscientiousness, it’s agreeable, you know, that taste on how. Well, the conscientiousness would keep you working in the direction that you’ve chosen and doing that diligently and orderly. The low agreeableness, well, that’s the open question because agreeableness is such a complicated dimension. There’s obvious disadvantages and advantages at every point on the distribution. I mean, disagreeable people are often harder to work with because they don’t care much about your feelings. But one thing I’ve noted about working with disagreeable people is you always know what they’re thinking. And if you want someone to tell you what’s stupid and wrong, they’re perfectly willing to do that. Yeah, I used to wonder, so Dirk Meyer was a disagreeable manager, but he could tell you what was wrong with what you were doing in the way you would go, okay. Like he was very unemotional about it. Like he’d go, Jim, I really like this and this, but this isn’t working for shit. Like, what are we gonna do about it? And you’d just be all shocked as a matter of fact. I would say when I was younger, I was a lot less disagreeable. I’m fairly open-minded and I like to create new stuff, kind of stuff, things. But then I saw enough things fail over the years because we didn’t make the, let’s say the hard choices about something. And then you hate to work on something for two years and not have to go away because at some point you realize you’re doing a couple of wrong things and you didn’t do something about it when you could. And so as a manager and a senior leader, I’m somewhat famously disagreeable. Part of it’s an act to get people to move and part of it’s my belief that I can’t have people dedicate themselves to doing bad things for very long because it’ll bite us. Yeah, well, we’ve talked a little bit about this too, about the moral dilemma between agreeableness and conscientiousness. They’re both virtues. Agreeableness seems to me to govern short-term intimate relationships like that between a mother and a child. And it involves very careful attention to the emotional reactions of another person and the optimization of those in the short-term. But conscientiousness looks like a longer-term virtue and they come into conflict at some point because sometimes- Yeah, they come into conflict in the midterm. Yes, yeah. It could just be how our brains see the future, but it’s like if you’re managing a group and you have to fire somebody, it’s hard, right? But do you wanna fire five people now or everybody later? Like once you’ve internalized that and taken responsibility for that decision, then making management, leadership, position choices is always hard, but it’s so much better to make them and then succeed than it is to fail because you couldn’t make the hard calls. Yeah, well, it isn’t obvious at all who’s got the upper hand, someone who fires early out of necessity but is accurate and looking carefully or someone who is willing to let people drag on. I’ll give you two counter examples of that. So Jack Walsh in his book, Straight From the Gut, a weird thing, he said, once you have a doubt on somebody, you never act fast enough, which took me years to really believe that. And then the other weird one is people say, hey, I have this organization of a hundred people and there’s five, five people that aren’t working out, but I’m not sure who they are. So I’m gonna be really careful because I don’t wanna accidentally fire a good person. Right, that makes sense, right? You got five bad people, maybe you figure out who two or three of them are, but there’s this other group of five or 10, you’re not sure which ones are the wrong ones. Here’s the sad truth, there’s a lot of people in the world. You’re better firing too many than too few. And how did you come to terms with that emotionally? I mean, look, we have a mutual friend who fires people with quite great regularity and I’ve talked to him and he scores very high in disagreeableness. And I talked to him about firing, which he’s done a lot of, and he was actually quite positive about it. He said, I don’t fire anyone who I don’t think is causing more trouble than preventing. And so by firing the person that I’m firing, I’m actually doing a very large number of people, including potentially that person of favor. It didn’t bother him, but he was temperamentally wired that way, I would say. I would say, you know, digital equipment went bankrupt because they had bad people who didn’t fire. I’ve seen many groups fail because they couldn’t clean house, right? And the impact on the greater good equation is super easy. You wanna save 90 people or lose 100. So that’s true. The thing that took me a while to realize that the world needs shaking up all over the place and individuals do, right? A lot of people who are not doing too good, they need a wake-up call. You give them a bad review and they kind of shrug it. They’re like, what are you gonna do about it? You know, it’s like a spoiled kid, nothing, right? But when they actually get fired, they really have to do some soul searching. And then the fact that if you’re doing something good, there’s always a queue outside the door of more people. Now, here’s another way to think about it. Take a group of 100 people and rank them from top to bottom. Human beings, by the way, are really good at this. You have four managers in the group, except for the managers and individual friends. They’ll tend to rank the 100 people the same way. I’ve done this experiment many times. But we’re really good at ranking. And there’s a little bit of what are you ranking for? You’re ranking for creativity, productivity, conscientiousness. But if you set the criteria right, people rank pretty well. If you have 100 people in your group and there’s 50 people outside, the distribution of those 50 people is around the average of the team, right? So there’s this idea that you fire the bottom 10% of a team because the random people you hired will be better on average than the bottom 10% of your team, right? It’s just math. Unfriendly statistics. Right. Yes, I get the argument, right? Problem is that every company that does that, first it gets gamed because managers hire bottom 10 percenters. So when they get to fire the 10%, they don’t have to fire their friends, right? And it also really is hard on morale. Like people bond. And there may be people in the bottom 10% of your group that are the social glue of the organization. They may be inadvertently taking out the stuff that makes the team work. Right, well, that’s a measurement error too, right? It means that your criteria for competence aren’t broad enough. Yeah. That’s tough. Good point. Maybe you’re ranking a little wrong, but impact on morale is high. Teams, generally speaking, Rory Rigg was CEO of AMD when I joined. And we had a big layoff, which we had to do because we were running out of money. We were broke. And when we all just settled, we landed on just the right amount of people for the money we had. And he basically read us to riot, and he said, he said, guys, teams have to grow. When you cut, you always cut further, and then you grow. People aren’t happy unless they’re growing, right? It’s like when you prune bushes and stuff, you don’t prune the bush to where you want the bush. You prune the bush past that point so it grows out and it looks nice, right? Things have to grow. It’s really an amazing dynamic. Well, yeah, well, and it’s never clear how much death there is involved in growth and the pruning analogy is exactly that. And this is harsh stuff, obviously, but you’re looking at one collapse or another. It’s right, that’s the thing. It’s not harsh. Because it’s beautiful when you prune your bush and it grows back beautifully. It’s great when you rebuild an organization that’s really strong and powerful because you made the right calls, right? Like this isn’t just negative stuff. It’s hard stuff to do that creates something really great. Like when I joined AMD in, what was it, 2013 or something, like they had two product lines, Bulldozer and Jaguar, and they were both failing. And I canceled both products. Like, yeah. Okay, and so what was the human cost of that, I would say both to you and also to the people that were involved? Well, I did the math on it. It’s like, we needed to be building eight bedroom houses and we’re trying to add six bedrooms to a two bedroom house in one case. It was never gonna work. Yeah, so you saw that as doomed to failure. And the other one was structurally screwed up. It seemed to be the right ballpark for the performance we should get, but the way it was engineered and built was sort of like, you let the plumber do the architecting and the house looked like shit. And it was difficult for complicated reasons, technical reasons, there was no path out of where they were. And when I realized I had to cancel them, yeah, it was sleepless nights here. We had revenue on that. We had people committed to it. People really liked it. When I canceled it, especially on the Jaguar team, a significant number of people quit because they were angry about it. There was some pretty big organizations. There was some management we had to let go. The best architect at the time, well, one of the best architects at AMD was, he really was my way or the highway. And he was, you know, he could not communicate what he was doing, so I let him go, which is a strange thing to do. But he’d ranked the organization, you’re ranking near the top, and I let one of those guys go because he was ineffective working with the team. And how did you justify that to yourself? And how did you check yourself against stupidity and ignorance and, you know, self-interest? And how did you know that what you were doing was right? Well, I’m a little lower in conscientious than I should be for a senior leader, first of all. So I knew this wasn’t gonna work. So you’re in a space. The direction I’m going is not gonna work. So you know how mosquitoes work? Mosquitoes are fun. So they detect two things. They detect water vapor and carbon dioxide. So mosquitoes will fly along in a direction as long as the water vapor and carbon dioxide are staying the same or going up. But as soon as it starts to go down, they change direction in a random direction, right? And within a couple of turns, they’re aiming right for a mammal. They can bite. It’s gotten colorful. So if you know you’re going in the wrong direction, a change in direction can maybe is just as bad, but there’s some chance it’s good, especially if you’re somewhat smart and you have some experience, right? So there was a whole bunch of- So even a random move is better than no move if the outcome is a certain failure. And so that is some justification for taking a risk. Now, there’s an incident number of fail directions, but you’re somewhat informed, right? And then the other problem is like, when you build a house, it has a foundation. Once the foundation is built, it’s very difficult to change the top of the house a lot. Like if you have a foundation for a two-story house, it’s hard to make it into an eight-story house. So when we cancel those projects, we consciously reset some design methodologies, some team organizations, some leadership, some let’s say, we’ve said we’re going to have the best in class leadership, design methodology, and some of the architectural tools. We’re just gonna take those as givens. Now that the land has been cleared, we had the opportunity to go back and do that. And it was interesting in the design teams, it turns out there was some very good pieces in the two processors they had, but they weren’t working together organically like they should. And let’s say the framework of the design wasn’t big enough. And then the tools over the years have evolved into lots of little local improvements, but it wasn’t really the right tool set. Now AMD leapt forward when you did this, and they were the only competitor to Intel in a realistic sense. And so these actions on your part were part of what made that company thrive and kept competition within the microprocessor world. These decisions didn’t have trivial outcomes. Oh no, it had really great outcomes. And the really cool thing was when we did that, we didn’t really bring in outsiders. That design was entirely based on people who worked at AMD at the time. So what we needed was to clear the plate a little bit, to reestablish some first principles about how we were doing things to have a better goal. There was a little bit of head knocking on getting like the methodology straightened out. You know, I was, let’s say fairly disagreeable about how we were gonna get through that because people kept saying, oh, it’s too hard to do this. Well, is it any good? No. Well, if it’s no good, it doesn’t matter how hard it is. You have to do it, right? If you’re gonna drown, you don’t go, well, a mile is too far to swim. So I’m just not gonna swim. You’re gonna drown, right? If you’re a mile offshore, you’re drowning. Swimming a mile is the requirement, right? And I’ve explained that like a million different ways. When something is pretty good, you know, the world’s divided into three things. Things are good, so you’re happy. When things are bad, you fix it. And then there’s the middle ground, it’s not great, but it’s not killing. Those are the ones where human beings have a really hard time improving, right? So by canceling the projects and declaring everything bad, everything could be improved. All right, and people would bug me about it. It was like, Jim, this wasn’t that bad. Well, was it great? Was it gonna win? No, all right, it’s bad. I defined everything that’s not great, bad. If I moved a little on the continuum, everything 5% or more away from great was bad, period. And how did you, like, how did you come to decide that was a good criteria? What’s that? Why did you decide that was a good criteria? All the time we were competing with Intel and on a whole bunch of metrics, like literally, their CPU had twice the frequency, twice the performance per clock, a whole bunch of metrics were so good. So we plotted them all. Like, you know, I had all bunch of data on this stuff, but also as a mindset, like if it wasn’t best in class, like a computer has a whole bunch of things in it, there’s something called a branch predictor, which predicts which way branches are going. And ours was way better than ours, we can measure it. So we did. There’s a thing called a memory system. Their memory system was way better, it was twice as fast. So it was like, well, we need to be within 10% of them on everything or we’re gonna get our assets. And it would be really nice if we were better on some things. So we measured all their stuff. Like if you’re gonna compete, you know, like in basketball, you don’t just play yourself, you know, you try to see what the other teams are doing, what are they good at, what are they bad at? Every coach is great at doing analysis of all the competition. And then, you know, you win two A’s, you meet the competition with what they’re good at, and then you have some secret plays that you’re better at than surprising. Given recent SCOTUS wins, it feels like the pendulum may be swinging back to a time when the nuclear family was situated at the center of American life, where real conversation, learning and growth began at home. President Ronald Reagan said in his farewell address that all great change in America begins around the dinner table. Well, all great meals in America begin with Good Ranchers. Good Ranchers cares deeply about providing families with steakhouse quality beef, chicken and seafood meat at a reasonable price. Their mission is to bring people to the table, making those shared moments with your loved ones easy, accessible and delicious. Good Ranchers ships 100% American meat, born, raised and harvested in the US, right to your door. And if you subscribe, your price is locked in for the life of your subscription. Great food creates great conversation, and great conversation makes great change. So start bringing people back to the table with high quality American meat. Go to goodranchers.com slash Peterson to get $30 off your first order plus free shipping. That’s goodranchers.com slash Peterson. So you went from AMD to Tesla. So, and we talked a little bit about Elon Musk. And we talked a little bit about you, about him and your experiences at Tesla, but also and what you did there. Sure. So let’s talk about Elon Musk to begin with. Sure. Well, he’s a pretty public guy. So I don’t know that I can add that much. Well, kind of what you said about Steve Jobs, was it true? Yeah, it’s mostly true. Like Elon’s the real deal. He’s a really good engineer. He has this belief that he can learn deeply and he’s practiced it and does it. I’ve seen him do it. Like he gets into lots of details. Not all. He has done five impossible things. Yeah, it’s pretty respectable. And he likes the details and he has a good eye for it. Usually before Tesla, when you do a technical presentation, usually you have some kind of methodology for presenting the problem to an executive. What’s the problem? What’s the background of that? Elon is solution person. If I don’t like your solution, I don’t care about the stupid problem in your data. Just forget it. No. What’s the solution? There’s a solution, great. Great. Then everything else is backup. What’s the data behind that thing? Oh, we’ve got the data. What was the problem? Like, how did you figure this out? Oh, here’s the original problem we found. Like he likes, he has a reverse order for most people. Most people tell you the story. It’s a problem. Here’s how we figured it out. Here’s the background data. Here’s how we built the solution. Here’s the solution. Here’s the next step. Like that’s a typical technical presentation. It’s not a bad idea. Elon hated this. Like stop with the bullshit. What’s the solution? And can you give me an example of that? We had a problem with low resolution camera images, right? And we were trying to improve like how the computer perceived roads and like low light conditions. So we started with, well, here’s the resolution of the camera. Here’s the light sensitivity. Here’s the images we’re getting. And he was like, what the? Do you have a solution or not? Well, yeah, we do. We have the software that does this. Well, page 12. Why is it on page 12? Page one of it and we’re in a really good place. Here’s the old image. Here’s the new image. Here’s how we did it. Right? That’s what he wanted. Now partly is a high bandwidth. So what I’m wondering is what was effective about that? Was it that there was an ethos then that the most important thing that you had to bring forward wasn’t a problem, but it was a solution. And so that people were striving constantly to generate and communicate solutions, which seems like a good strategy. Well, no, I think he’s worried. So the engineer sees a problem and they started investigating it. And then as you’re investigating, you get to develop understanding and your understanding of the problem is good. Like you would think that, right? No, you think that. Lots of people think that. Elon doesn’t think that. Elon wants a solution, right? And if you’re falling in love with your understanding and you’re falling in love with your little details. Well, that also feels like work. You know, you mentioned earlier that just because it’s hard doesn’t mean it’s useful. Right. And focusing on the solution. You are focused like crazy on the solution. The best way to do that. So then you have two states, right? If there’s a problem, there’s two states. You have a solution and you don’t have a solution. If you have a solution that’s on page one, we’re all happy. Yeah, well, I’ve seen in the- If you don’t have a solution, why the hell are you talking to me? Why aren’t you finding a solution? Yeah, yeah. Well, I’ve seen in the software projects, other projects that I’ve been involved in too, that focusing on a solution, I think this is along the same lines as what you’re discussing is, well, then you get to product a lot faster. It’s like this thing has to exist and work. Maybe it won’t solve. And the problem with the problem is that you can indefinitely investigate the problem and expand it. And also that that feels like work, but it’s not saleable. Yeah. Yeah, it puts in the forefront of your mind, that constant, you need to be creative pursuing the problem, but also make sure you’re really on track to the solution and you’re just not falling in love with the problems. People call it admiring the problems. Some engineers are great at admiring problems. Like I’ve worked with lots of people to come in with a 12 page presentation and then when I’m done, I’m like, did you guys just give me a whole bunch more data on the problem? He’s like, yeah, we’re really getting to the bottom of it. It’s like, no, you’re not, you’re not getting anywhere. Well, the bottom of a problem is a solution because why would you just investigate the problem, right? I mean, your destination- No, no, they just keep getting deeper and deeper in the problem admiration and nothing happens. Okay. I’d say three quarters of engineers would be perfectly happy to do that their whole life. So you explained this to me, complex mastery behavior. So humans are very, we like to learn, right? We don’t like to do dumb repetitive things, right? But we like to do things that are complicated that we’ve mastered, that take skill and insight, but you can have complex mastery behavior just analyzing problems. Right, definitely. Right, and there’s careers for that. Some people find they’re really good at it. They’re analysts, they generate lots of data, but if you’re in charge of solving problems, you know, that period needs to be focused, short, concise, and you need to move on to solutions, right? I think that’s probably why I’m not so temperamentally fond of activists. Are they problem admirers? Well, that’s what it looks like to me. It’s like, well, that’s a, and this is one of the reasons I admire Elon Musk. It’s like, well, you’re concerned about the environment? Well, why don’t you build an electric car then? Right, well, so a large number of people, and this is my favorite thing I learned, and it was working with mechanical engineers at Tesla, was they think the world’s made out of silly punty. Right, they used to design, when we were building Model 3, they designed the part and they knew the joke about how they’re gonna make it. Are they gonna, you know, CNC it, like mill it, or are they gonna injection mold it, 3D print it, stamp it, make it with a hammer, you know, cut it out with scissors, you know, carve it out of a block. They had this cool machine that could carve 3D models out of clay. Like, it was funny. Like, so they could design things in their heads and on computers, and then go build any damn thing they want. Like if you were looking at complicated mechanical assembly, there’ll be some screwed aluminum thing, it would be milled somewhere and then drilled, and there’s screws going through it, and there’ll be some little tabs sticking off of it that holds another thing. Like they can make stuff, right? They think they can make anything, right? And there’s a whole bunch of people in the world that don’t think they can make anything. They think the world is what it is. I had a friend, he had a rattle in his dashboard, and he didn’t know what to do about it. And I was asking him where the rattle was, and you know, and I was thinking, and I was talking to him about, like how the dashboard’s made, and he goes, Oh, I get it. You think the dashboard’s made of a whole bunch of parts that are put together some way. I thought the dashboard is just the dashboard. Like he didn’t conceptualize it as there’s this outer piece and there’s inner brackets and there’s radio, and there’s these things. Because mentally I can’t help but see the whole thing in 3D, and then I’m wondering which piece is loose and where it is. Right, and then how to fix it. Like I’m not mechanical engineering creative, but I’m visual. And for people who get stuck on activism as problem description, they don’t think the world can change. Which at some level makes sense. For human evolution, like it was pretty much the same for a million years. Like it’s weird how good we are at change. And my best theory on it is from zero to 20. Like your brain is going through radical change because you’re going from silly putty and not knowing much to being pretty smart. So you have to change and adapt really fast. And then humans are adapted to deal with each other and humans are fairly crafty. You have to deal with that. But the lifestyle of most people from, you know, series 30 deaths, it’s fairly static. And so we have this funny capacity for learning rapidly exponentially and then dealing with slowly changing environments, but we’re not naturally adapted to rapid change. In the modern world is, well, especially in engineering is rapid change. So, okay, so- There’s a couple of guys who was just so funny. You never knew what they were up to. Like one day, like they were working on the interior for the car and they made this crazy looking model, which kind of looked like a car, but it turned out it was a thing you could move around and had the attachment points for all the interior parts. So you could basically, it looked like a weird skeleton, but it had the attachment points that you could adjust and you could build all the interior parts and put a Tesla interior together right in the middle where the engineering desks were. It was really cool. And let them go build it and think about it. And then it was in the CAD model in the computer, you could see it, but it didn’t always work out in real life. Like we have a scale problem. When you look at something that’s small, even if you scale it up perfectly when it’s big, sometimes that’s just what you thought and sometimes it doesn’t work. And so you wanna do, computers like to change things fast, but like real scale models that you sit in and live in and get a human experience for it. And it was really fun for that to just show up and be like, holy cats. So I did a similar thing. We took all the electrical subsystems and motors and laid them out on two tables, two big tables covered with all the electrical parts of a Model 3. We stared at it and once you see them all together, it’s crystal clear that could be a lot better. Cause you know, there’s your three motors that look almost the same, why isn’t that one motor? There’s these two parts that are completely separate separate assemblies, but if you build it together, you could have one thing, do both things really much more naturally, a lighter weight. Right, and by laying that all out in front of you, you didn’t have to do the mental work of representing that. You could do the mental work of seeing how all the parts interconnected and what might be. Yeah, yeah, Doug Clark is architect I worked with when I was a kid at Digital, called it the interocular traumatic test. When you look at it, does it bug you? And a lot of things when you really lay them out like that, you go, oh, we’re not doing this right. And Elon likes that kind of stuff. Like, you know, you’re almost afraid to show them like when you laid it all out, you looked at and go, this is crazy. So it was like, do we show Elon or not? Cause he’ll look at it and think this is crazy. Like who built this car, who did this? So I wanted to talk to you too, I wanna talk to you like I’m someone very stupid and in this particular regard I am, I really don’t understand how computation works and you’re a microprocessor architect and you build computers. And so I’m, and I listened to a discussion you had earlier this month and there was a lot of it I couldn’t follow. I thought it might be helpful and interesting for you just to walk through for me and for my audience, how a computer actually works, what it does and how you build it and then what it would be like to design and to architect a microprocessor. Yeah, well, it’s somewhat hard to describe but there’s a couple of simple things. So let’s start with the easiest thing. The computers have three components really, memory, programs and input and output. Right, those are the three basic things we always build. Right, so memory is like the DRAM or the disk drive, place where you store data and it’s just stored and it can have different representations. We currently use ones and zeros. Like, so you can take any bit of information and describe it as a sequence of ones and zeros and it’s stored in silicon and either static and dynamic memories are on disk drives, which, you know, there’s a couple of technologies for that. So does memory make sense to you? A place to store information. Well, it does, although I have some difficulty in understanding exactly how the transformation is undertaken to represent things in zeros and ones. I mean, I- I’ll give you a simple example. So if you shine a light on a photo detector, right? So the light comes in and it’s a stream of photons, right? And the photo detector counts the photons, literally. So every photon that hits it or a couple of photons hit it, they cause some electric charge to move and that causes the circuit to wake up and say, I saw some photons. So say you’re trying to evaluate how strong that light is. So it can be anywhere from nothing to super intense, right? And then you might say, well, let’s put that in a range of numbers from zero to a thousand, right? So, and then you’re trying to lay on it and so you count the photons for say, you know, microseconds and then you translate how many photons counted into the number, right? So, and so just imagine as the light varies up and down, the count, the number coming out of your photo detector is varying between zero and a thousand, right? And we use base 10, but you can translate that to binary, which is base two and now you have ones and zeros. So you’ve basically now translated a light ray optical information to a count. And so virtually everything can be represented by a count apparently. So just think of your computer, it has a camera, which is essentially doing just that. When you get different colors, but first you go through color filter. So you have a green filter and a red filter and a blue filter, that’s enough to represent the color spectrum. And then you have a little photon countering underneath that and it counts for a little while. Then you ship out the number and you reset it and count again. Those of light varies, you’re getting that. There’s a pretty big grid, like a modern cameras, 12 million photo detectors in it. Actually 36 million because it’s got different colors, but well, it depends on how they build it. Like they might have different pixels be different colors and then interpret the colors. Like a keyboard is really simple. So the fundamental issue to begin with is that you reduce everything to a count and then you represent that count in base two. It’s base two, right? Zero and one. And you can do the same thing with sound. You can have a sound detector that basically counts intensity of sound waves and the keyboard, well, you have a little grid of keys and when you push a key, it sends which key was that? That was key number 27. So now you encode- And it’s either on or off. On or off. So D might be count number 26 and F is 27 and G is 28. So everything gets encoded into a number. And then computer is like binary numbers for technical reasons, but that’s not a big deal. So memory, so input and output is the first thing. So the computer is built around the memory. So the input and output systems, so the inputs, all your photo detectors, sound detectors, keyboard detectors, and there’s amazing numbers of sensors these days. You can detect gravitational waves. Maybe you can detect photons. You can detect electrical waves, sound waves. You can detect temperature. It’s very, very, you turn all that stuff into a number and then your input device writes that into memory. And memory stores information. Memory used to be small and expensive and now it’s big and cheap, but it’s still just memory. And nothing happens to it in the memory. It just gets stored. When we look inside the computer, it’s a big memory and it’s full of numbers. Now, the person who’s writing the program is telling the input and output. Here comes the input video stream. Put the video stream address, one million. So all the memory is addressed. And the address typically starts at zero and modern computer goes up to billions. Okay, so walk through that again, the addressing. So what’s exactly the function of that? Well, you wanna know where the memory is. Okay, right, you need to, okay, fine, yeah. So basically your phone probably, I don’t know, has eight or 16 gigabytes of memory in it. Maybe, yeah, four or eight, I don’t know. So billion, eight billion bytes of information in there. And when you’re designing your programs, you kind of lay out, well, here’s the operating system that’s going, here’s what the input and output buffers are, here’s memory we’re gonna use to run some program. So, and all that’s addressed. So you can think of the addresses, it’s just like a post office, right? So every house has a postal address, it’s a street address and then your house address. And so you can find every person. And the address corresponds to the physical location in some sense to the physical location of the, and how are the zero and ones represented in the memory? It’s literally a voltage that’s either high or low. And zero is using ground, which is zero volts. And modern, DRAM cells probably stored at 1.1 volts. And in a DRAM cell, it’s a capacitor that’s holding electrons. So basically when you store the cell and either you drain all the electrons out, so it’s zero volts, or you put a bunch of electrons in so that it holds a one volt. So it’s literally a number of electrons in there. There’s a couple of ways to make memory cells. There’s another way, which is called a bistable element where you have what’s called cross-couple inverters, but that’s too complicated to explain. And then the memories are usually built in rays. So there’s an X, Y. They take the number and you say, I’ll take the bottom half of the number and figure out which row it’s in and the top half of the number of which column it’s in. And where the column and the row overlap, then I’ll write my new data of a one and zero in that spot. It’s literally that simple. So if you look at a memory chip, you’ll see this array of bits with little blocks on two edges. Usually, one side’s the row, one side’s the column, and then the bottom is what they call the sense sense when you read it back out again. So a memory process is you activate the row and column to a spot, which gives you the address of that bit. And then you drive the bit in and charge up or discharge that cell. And then it holds them. And it’s super simple. You can build a memory with a pegboard. You could build a memory, I mean, literally did. Way back when there was something called clur memory where they had essentially the XY grid. And at each little place, there was a little magnetic bead, which when you put the current through, you put the current in the same direction, you could make it be north to south and the opposite direction, south to north. So you basically remagnetize the little beads. So there’s lots of ways to make memory, but currently the really dense memories are called dynamic memories, where you literally put charge in there. And then there’s fun stuff that happens like flash cells, the cells got so small that the electrons from the quantum effects tunnel out occasionally. So you put 25. There’s some doubt about where the electron actually is. Yes, and sometimes it could literally jump out of the cell once it jumps out and doesn’t come back. So they got down to like 25 electrons in the cell and they would wander off over a couple of hours and you have to refresh them. So periodically you go back and you read the data before too much of it escaped and you write it back in. So it’s called refreshing the memory. But the DRAMs hold more charge than that. And then the flash guys figure out how to stack the cells. So modern flash chips, there’s an XY grid, but there’s also a Z dimension. They’re like 256 layers thick now. So it’s like a three-dimensional memory. But the simple thing still is it’s a linear range of addresses where you put some data. Okay, so that’s memory. So the next component. Is programs. This is the compute part. So a simple program is A equal B plus C, right? So the data at address A, so when you write the program, you tend to use what they call variable names A, B and C. But there’s a tool called compiler, which will sign A as the address 100 and B the address 101 and C the address 102, right? And then the computer when it’s running says, do what I told you to do. So you see this program A equal B plus C. So you get B, you get C, you add them together, you put it in A. And typically what happens is you have what’s called a local memory or a registered file. So you get the data from memory into the registered file. You do whatever operation you’re told to do like add. And then you put C back in memory. And what are the range of operations? Or is that too broad a question? What are the fundamental operations apparently arithmetic? I’ve done this. Like the number of operations that a computer does, like the construction steps can have a hundred or 500 or a thousand different instructions. But the most common ones are load data from memory to the processor or the program ones, store memory back. Those are your first two instructions. And then add, subtract, multiply, divide, clear, very simple, it’s written. Then there’s what’s called logical operators and or not. It’s stunning to me conceptually, thinking through this, that computers which can produce whole worlds in some sense can do that as a consequence of zeros and ones and arithmetic operators. Sure. Well, your brain is doing something interesting like that. There’s no magic to it. So the key to programs is abstraction layers, right? So at some low level, like I understand computers from atoms up to operations, which is fairly broad range, but there’s lots of people who can do that. Yet I understand them like the surface of the keyboard. Yes. Yes, monkey with military helicopter, basically. Pete Bannon had an interview question. So, computer scientists would say, tell me what happens when I type a key, right? Cause you can talk all day, I could talk all day about it. Cause the key is at the position which encoded the number, which got sent into the memory. There’s an interrupt delivered to the processor to say, there’s new data in memory, go take a look at it. But you can describe that at many, many levels, right? So it’s a good place to start, it’s not bad. All right. There’s an interview question. There’s an interview question. Yeah, right. Some people, by the way, are stumped. They go to college and they can’t tell you what happens. The key is clicked, which is weird. So, back to the computer. So, what’s the basic operations? You have subtract, multiply, divide, you know, clear, set the one, and, or, not, XOR. You can take a number and you can shift it around, you can mask it. And so, different architecture. How are those operators discovered, Jim? I mean, I know there’s arithmetic operators and then is that just the question of how was their arithmetic discovered? But there’s a logic. Way after math. So computers, you know, at some level, they’re doing arithmetic. Like, it’s not very sophisticated. And I’ll get to a little more complicated version of this. And by the time we invented computers, people had pretty good idea of number theory. They’d figured out that base 10 was just one of the bases. You could have two, three, four, five, six, seven, eight. People had, the philosophers had worked out what logic is. You know, if this is true and this is true, then this is true, this is true, or this is true. Like, the logical operators are real. Like, there was a whole bunch of them. Yeah, it’s the realism of them that’s stunning to me. Yeah. So, there’s the basic operator set and then there’s something called control flow. So computers, typically, they put a program like add, you know, A equals B plus C, you know, D equals E plus F, F equals E plus A, you know. And you typically put that in what’s called program memory, but it’s just part of the memory of the computer. And you have a program calendar, which, you know, I don’t know what’s called calendar, but the thing that points at the next instruction to execute, and it’s the fault thing is do this instruction and then do the one right after it. That became like the way computers are built. That’s an arbitrary choice, by the way. You can have every instruction tell you where to get the next instruction. Here’s a bunch of things you can do. But for simplicity, people said, this piece of memory has programs in it, start at the first instruction, and then do the next one and the next one, next one. The program counters, but that’s not good enough because then you would just start at the first one and then you go to the end of memory and be done. So there’s something called control flow. So a program- Called, sorry, called control? Control flow. So imagine you wanted to add up a list of 10 numbers. So your first instruction says, I’m on the first instruction. Then you say that the sum equals the current sum plus the next number. Increment the counter of how many instructions I had, increment, count by one, and then test. Is the counter equal to 10? If yes, keep going straight. If no, go back to get the next number. So you created a little loop. Right? So, and it turns out that computer scientists invented a whole bunch of kind of loop, what they call control flow constraints. Do this while X is true. Do this until the counter gets to a number. Right? So you can create little, basically sub programs in the program. Right? And then there’s a couple you can test. Like, hey, I need to decide if this is a dog or a cat. So if it’s a one, go look at here. If it’s zero, go look at that. Right? So that’s conditional branch, loop branches. And then somebody famously invented sub routines. You notice how as he was writing the program, he’d write this little routine, but it would be used a bunch of different times. So rather than putting the code in multiple times, it was like define a word, and then whenever I need to use that word, I don’t have to put the whole definition for the word. I just put the word. So subroutine is like a local definition of something or a local computation that’s used multiple times. So your top level program might be, go to the subroutine that counts up numbers, comes up numbers and come back. Now go to the subroutine that checks whether it’s your bank balance or not, come back. So the program, this becomes sequential operation, control flow, like doing loops. Let’s say do this until something’s done, and then conditional branches, it says, depending on value, do this or that, and then subroutines to do something atomic. And that’s essentially all the program. Operations, loops, conditional branches, and subroutines. That’s it. Now, why can computers construct worlds? So I still remember when, so if you look at your screen, your computer in front of you probably has two or four million pixels on it. So that seems like a lot, right? And when they first started in televisions, when they lit up screens, they were scanning the little electron beam across a phosphorescent surface and lighting and modulating the intensity of the electron gun to make the little phosphorescent brighter. Right, and it was writing, it writes one line at a time at an incredibly fast rate by human standards, and we saw that as continual images and then continual motion. The phosphorescent was designed to decay at the rate, so by the time you came back to it, it had just gotten a little dimmer, and then you wrote it with the next value, so it didn’t flicker. So your eye has some persistence. So the electron hits it, it makes the phosphor light up with some photons of the right color, and then it slowly decays, and you scan down, and it gets back and writes it again before it’s too dim. And so the screen on a phosphor-based television, is it analogous? It’s analogous in some sense to the binary representation? The dot-and-one-raw? That’s entirely analog. So it’s digitized in the sense, it’s discrete, I’d say, in the sense that each little pixel, you can see the little phosphorescent screen. Right. Especially notice that when they went to color TVs, because they have a red, green, and blue thing in there. Right, right. And they would hit them with the red. And they’re essentially either on or off? Well, they have a range, so that beam is a variable intensity. Right, so now modern computers work differently. So the screen in front of you has a little, it literally has an XY grid, and it can address each one of those things. Right, so you don’t shoot a beam at it anymore. You have an XY, you decode. It’s almost like the screen looks like a big, flat memory, but instead of storing ones and zeros, it’s storing color. But they have the same kind of decay property, and you write some new color, there’s a bunch of stuff. Now here’s the wild thing. Computers are now so fast, you can run a 10,000 line program for every single pixel on that screen. Right. So what, and what does that imply? Well, it turns out for a whole bunch of reasons. Like if you want to make something look really good on the screen, so the world’s relatively continuous, right? So if you look at it, there’s all this light reflecting around, there’s all these things going on. There’s no little pixels in the surface of your table. To make a discrete grid look that way, you have to combine the colors of, you have to do a whole bunch of stuff. You have to pretend you’re shining lights on it. You have, there’s a reflection from one surface to the next one. And it turns out when you have thousands of instructions per pixel, you can start to make those pixels look realistic. Right. The operations, if you go look in the pixel program, like it looks so beautiful, you think that’s incredible, but if you look in the pixel program, it’s load the data into the register, add it to a number, test it against the number, subtract something, or something called clipping, like make sure that the pixel doesn’t get brighter than this and dimmer than that. It’s all simple operations. Like there’s nothing in the computer that’s like, do a pixel operation, right? Well, there might be a sub-routine named that, but underneath it, it’s just the same old stuff. Computers always do load, store, add, subtract, multiply, divide, branch. And so- Okay, so how far have we got? I’m listening to so many things. I’m having a hard time keeping track of the order. You mentioned earlier that computers consist of four elements. I believe that’s what you said. Memory. Yep. Input and output. Yep. And compute. Okay, three. So I was counting input and output separately, but okay. And have we gone through all three of them? Okay. So memory is just a place to store bits. Yep. Input and output is typically the way to, it depends on what you’re doing. You might just send bits from one place to another, but it might also be, you could say input, input and output in the computer and sensors are slightly different things. Like sensors turn analog real world signals into bits. Into digital. Right. And then programs basically transform the data in some way. And programs is basically operations like add, subtract, divide, and then branches that we either let you build loops or make decisions. And then the hardware that lets you do subroutines to break the program into pieces. And that’s pretty much it. So to some degree, you take the world, you transform it into on-off or yes-no, billions of those. And then you manipulate the yeses and nos or the zeros and ones, and that can produce almost any sort of phenomenon that you can imagine. Yeah. Yes and no is not a very good, ones and zeros is better because then it’s a mathematical representation, a digital representation of an analog reality. Something like that. And is the analog reality analog all the way down or is it digital at the bottom? It’s quantum at the bottom. So there’s something called the fine constant, which makes the universe look discreet, but it’s a very, very small number. Right. So, and there’s a fun fact, which is- Is that the Planck length? Is that associated with the Planck length? Yeah. And that’s the smallest possible length, I believe. Like the mass of the universe is 10 to the 40th and the Planck length is 10 to the minus 40th. And there’s a physics thread about the mystery of why those things are 10 to the 40th and 10 to the 40th. All right. So let’s move from that to, I’m going to ask you, these are the questions- I just want to say, so- Yep. Yep. The thing that makes computers do what they do is abstraction layers. So at the bottom, there’s atoms. So there’s engineers who know how to put atoms together in a way that makes switches, which we call transistors. Right. And those guys are experts at that stuff. Right. And they just, they can operate at that level. Then there’s another thing where you take multiple transistors together and you basically make what’s called logic gates, which literally do the ans and ors and inversions. Right. And then that’s an abstraction layer. We call it, you know, the physical design library or something like that. And then people take those and they may come up into adders and subtractors and multipliers. This is a well understood Boolean mass. How do you add two binary numbers? So you make those. Then there’s another abstraction layer that says, are they going to take multiple operation units and put them together to make part of the computer? Right. And then you make, there’s a bunch of those blocks. And then that thing runs a program, very simply. And there’s a small number of people who write programs at the low level, but then there’s people who use what’s called libraries where they, you know, they’re doing some higher level program. And so they’re going to do a matrix multiplying and they’re going to do this and that, but they don’t actually write that low level code. So there’s, you know, there’s a stack of abstractions. And when something gets too complicated, you split the abstraction layer into two things. There used to be when people wrote a program, there’s a program called a compiler that translated your C program or four term program into the low level instructions. But it turns out there’s too many languages up here and there’s too many instructions here. So now they translate it from the high level language into an intermediate representation, which is sort of, let’s say, a generic program. And then there’s another thing that translates to intermediate representation and the specific computer you have. But that just keeps going higher and higher. Like a lot of programmers, they use, you know, frameworks that can do amazing things. Like you could literally layer a program that says, search the internet for a picture of a cat, sort by color, output to my printer. Like there’s a language or that’s a program. Search the internet, holy cow, that runs a trillion lines of code on a hundred thousand computers, find a cat. That’s a really expensive, that’s a really complicated program. So how much of the radical increase in computation power is a consequence of hardware transformation? And how much of it is a consequence of the increasing density, let’s say of these abstraction layers? Well, so this is where, you know, there’s a really creative tension or dynamic interplay. So when computers first started, they were so slow, you ran really simple programs, A equal B plus C times D, right? And we’ve been going up the math hierarchy. So then you could run a program that did what’s called, you know, matrix math, like, or linear algebra, systems of big equations, and then matrices, and then more complicated ones. So as the computational power went up, you could dedicate more and more stuff to, you know, that kind of computation. And then similar thing happened on abstraction layers. Like it used to be, if you bought a million dollar computer, you hand wrote every line of code because you didn’t want to waste time on the computer, well, it’s like overhead. But today, you know, that million dollar computer costs 10 cents. You don’t really care how many cycles you use, you know, parsing a cat video or something. And so the computation capacity, let the abstractions at the programming level increase a lot. So somebody had a graph about how many bytes does this take to store the letter A? Like it used to be one, and then word for windows, it’s like 10 kilobytes per letter. Because the letter has a font, it has a color, it has a shadow, you know, there’s a whole bunch of, and that’s fine. Like the computer with a million dollars for, you know, a thousand bytes of memory, you wouldn’t store a letter A like that. You’d put it in one byte, but now you have gigabytes and terabytes of storage, who cares? You probably already know that there are data brokers out there selling your internet data off to companies who want to serve you a targeted ad, but you might be surprised to learn that they’re also selling your information to the Department of Homeland Security and the IRS. Mask your digital footprint and protect yourself with ExpressVPN. One of the easiest ways for brokers to aggregate data and tie it back to you is through your device’s unique IP address. But when you’re connected to ExpressVPN, your IP address is hidden, making it much more difficult for data brokers to identify you. ExpressVPN also encrypts 100% of network traffic to keep your data safe from hackers on public wifi. You can download ExpressVPN on all your devices, your phone, your computer, even your home wifi router. Just tap one button and you’re protected. Make sure your online activity and data is protected with the best VPN money can buy. Visit expressvpn.com slash Jordan right now and get three extra months free. That’s expressvpn.com slash Jordan. Okay, so you walk us through the basics of computation. Now, can you shed some light on, like I don’t understand what you do as a computer architect. Like when you go to work, when you’re working on a project, what is it that you’re actually involved in doing? Make ads go faster. So I’m a fairly low level engineer, you know, low level in terms of the abstraction layers. Like I understand the higher ones, but you know, I talked to the people who make transistors and NAND gates and OR gates. And they talk to the people who know the atoms. Right, so, and I hardly ever talk to the atom people, but I know something about atoms. So I build, when I’m architecting stuff, the functional units and then how they operate together at the low level that runs programs. But I don’t write programs. I build, I’m an architect of the computer that runs programs. Right, and then it used to be, you could look at a computer and you know how a program works. You run the first line, the second line, if there’s a branch, the branch unit, then you branch it out. Right, and the computer would literally have that in it. Such an instruction, load the data, do the operation, if there’s a branch, execute the branch, if necessary, change the program counter. So, you know, people, you know, there was a period of time where computers had like five stages and then each one of them could say, that’s the branch, that’s the fetch unit, that’s the load unit, that’s the add unit, that’s the branch unit. Right, but modern computers are more complicated than this. Right, because computers like that would do one instruction every five cycles. And modern computers, the fastest one I know about, is doing 10 instructions, 10 instructions a cycle in parallel. Right, and this is difficult. So the best way- Unpack that, unpack that. So if you write a program, since you write, right, when you write, you write linear narratives. Right, you write a sentence that makes sense, followed by another sentence. Right, and so as you’re writing along, sometimes the one sentence defines the meaning of the next sentence, right? And then group it in paragraphs, you might call those subroutines, right? And sometimes the paragraphs have to be ordered, and sometimes the paragraphs, the order doesn’t matter much, right? So programs are written by human beings and they’re written in the same linear narrative. So if you want to go faster than parsing the instructions one at a time in order, you have to do some analysis and say, all right, I got two sentences, are they dependent or not? If they’re dependent, I do them in order. If they’re not dependent, I can do them in parallel or any order, right? And you start, so the modern computers, when they’re reading the programs out, they’re analyzing the dependencies and deciding what can happen in order, what has to happen in order for correctness, correct understanding, and what can be reordered. And then it turns out, there’s many places where you say, if there’s an error, go here, but there’s hardly ever an error and you can predict that really well. So you say, I’m going, you’re reading along and you say, here’s a point where I’m not sure which, should I read the next sentence or should I jump to the next paragraph, right? So a modern computer predicts that. It doesn’t wait for you to fully understand all the sentences up to that point so you know exactly where to read them to. So imagine, so now you’re reading this book and you’re reading sentences in dependency order, which means you haven’t, so you get to a branch and you haven’t read all the sentences before that and understood them. So you don’t know where to read the next paragraph or the next chapter, but we predict what’s gonna happen and we just keep on going. And how does that tie into the process of designing the? So the goal of modern computers is to go fast. Well, let me say there’s three kinds of computers. There’s computers that run very simple programs in order, right? They just do exactly what you told them to do and they tend to be small and simple, but they’re so small and simple, you can make a chip with a thousand of those computers on. So when you build a GPU that does a little program for every pixel on your screen, each one of those pixels gets its own program. It’s very simple, but you sort of say the first thousand pixels you run on these thousand computers. So like a modern GPU has currently like six or 8,000 processors in it. And they literally, you do the first 6,000 pixels and then the next 6,000 pixels and the next 6,000 pixels and they do that fast enough that you can run fairly big program on every pixel on the screen for every screen refresh time. So you have simple computers that do stuff in order, right? And then you have, let’s say computers that are, they’re designed to run complicated long programs as fast as possible, right? And that’s where you parse the instructions carefully and you figure out what order you can do them in and when possibly reorder it, right? And the reason to reorder it is because if this doesn’t depend on this, I can do it in parallel. Now I can do two things at a time. The next thing I can predict that I can do it in parallel, I can do three things. And like I said, the computer in your desktop is probably doing three to five things at a time and the best I know of is 10, right? And that’s because, and there’s other sophisticated predictors in there. So to do that, you have to fetch large groups of instructions at a time, you have to figure out where the sentence boundaries are, figure out if they’re dependent or not, figure out if you can predict where the next instructions are coming from when you hit branches. And it turns out that’s fairly complicated. The difference between a little computer that does, let’s say one instruction at a time and a complicated one that does 10 instructions at a time. It’s a hundred times more complicated, right? And from a, what’s the best way to do lots of instructions, complicated computers are not efficient, but there’s so many applications where people care how fast it is. So when you’re like clicking on your webpage, you want that to come up as fast as possible. So the part of it that’s, let’s say, what’s it called? You know, the logic of the webpage, it’s probably a serial narrative written by human being. So you have to, you run that on a complicated computer that does it out of order and predicts what to do as fast as possible. But when you render the screen itself, that runs on large numbers of simple computers and they call the pixel. Right, and then there’s a third kind of computer which we’re starting to invent, which is AI computers. And that’s what you’re working on now. Yes. For 10’s torrent. Yeah, and there’s a really good talk by Andre Carpathi called Software 2.0. So all the first two kinds of computers, simple computers and complex out of order computers, they’re running programs written by humans. Right, and you look at the code, it’s literally a declarative statements about operations and where to go. And it’s serial, it’s a linear narrative. Yeah, the different thing about AI computers is you use data to train the weights in neural networks. To get you the desired result. So instead of the programs are no longer written by humans. Now it turns out there’s components of the AI stack that are written by humans, but at a high level you use data to train them. So you have a big neural network and you want to detect cats. So you put a cat picture into the network when you start training and the output is gibberish. And you compare gibberish to what a cat does and you calculate the difference in what the network said versus the desired result, which is the word cat. And then they do something called back propagation, which is mathematically sophisticated, but essentially takes the error and partition it across the layers of the network, such that you’ve sort of bumped each neuron a little closer to saying cat next time. By taking the bigger at the end, distributing it across, it’s called back propagation. And then you put another cat in. And if you have the right size network and the right training methods, after you show the network a million cats, when you put a cat in, it reliably says cat. And when you put a picture, it’s not a cat, it reliably says not a cat. Right? And you never wrote any code that said anything to do with cats. And can you understand what it is that the computer’s doing now that it’s recognizing cats? A little bit. So people for years worked on visual computing, and they were trying to detect things like cats. Right? And cats have a whole bunch of artifacts. They have round eyes, they have pointy ears, they have fluffy hair. So you could detect, it was called feature detection. You would say, this will be a cat if I see the following colors, the following amount of fluffiness, the following number, you know, two pointy ears, not three, one or two round eyes, depending on the view. Right? So you could write code, and the problem with that is, well, now the cat has an arbitrary orientation. So you have to, you do your feature detect on the picture, and the features have to search the whole image, and you have to rotate around, you know, and it’s sort of, and every single thing you want to detect, you have to write a unique program for it. You’re done with cats, now you go to dogs. And then what about the dog that has certain pointy ears? These dogs have round ears and cats have pointy ears. You know? So it was sort of endless thing. Right, right. Same thing with speech. Endless detail, by detail construction. Yeah. I had a friend that worked on speech recognition years ago. So you break speech into, you know, the phonemes, so you can see those, and then they have frequency characteristics, and you can differentiate vowels from consonants. So those people working on speech were doing a whole bunch of analysis of analog wave forms of sound. And they were making some progress, but it never really worked. And then they train neural networks by, you put the word in, and you have what’s called supervised learning. So you play a language where you know what all the words are, and you keep telling the network how to correct. And with like a billion samples on a big enough neural network, it can recognize speech defined. And if you train it with a broad variety of accents, it can work across accents. And then it turns out the bigger they made these networks, the more information they can put in. And then on the cat one specifically, they found, so when they first had a neural network crack the cat problem, I forget, it was like 50 layers deep. And if you look in the layers, you could see that it was detecting point ears and eyes, but it was also detecting a lot of other things. And some things we don’t know. Yeah, well, if we see the back end of a cat walking away, we still know it’s a cat, and it pretty much lacks eyes and pointy ears from that perspective. And the funny thing, if you take an object, like in light, right, it’s like a phone, you can project the phone onto a flat surface. Say that’s a projection, right? And as you move it around, you get different. It’s a shadow. Shadow, but think of it as a projection, right? So that’s a projection of a light source on a flat plane. It’s a fairly simple projection. But what if you had a light shaped like a cat and you shine that on the phone, what would the projection look like? And it turns out mathematically, there’s an arbitrary number of projections. You can, like we think of projections in three dimensions because we’re three dimensional creatures, right? But there can be lots of projections. And then you can have the projection project on another plane. So that the neural networks are doing is they’re teasing out all the details of what that is. And some of the projection planes give you what’s called, sizing variants or rotation variants. Like you could recognize a cat, no matter which way it’s pointing. Like your brain is a little specialized, like the faces. Right, it likes them to be vertical. It’s better with right side up. But with a little bit of work, you can recognize an upside down face, pretty well, unless you have a problem. Okay, so we could do two things here. We could either talk about your, no, let’s go into your, you were an engineer and then you were a manager. And you’ve worked in lots of companies, some of which were incredibly creative, some of which were thriving to an incredible degree and some of which were collapsing and irreparable. So what have you learned about what makes companies work? And more importantly, what have you learned about what makes them not work? And maybe what do you do then? Sure, well, that’s a fine question. Well, first of all, there’s like, I’ve noticed that many people have noticed, this is not just me, that people like in engineering fields, people kind of bucket towards, technical people and management people. And it’s not that there aren’t good technical managers or there’s not good managers or technical people who can manage, right? Right, but that’s an intersection of two skills, say. Yeah, but generally speaking, most people are one or the other. And it’s like when you wake up in the morning, or you wanna solve a problem, or do you wanna organize the problem? Like, are you worried about your schedule and your head counts and how things are getting done? And did you hit the milestones? Or are you working on technical problems? And people, and in engineering fields, it’s often there’s the fellow track with the technical leadership position or the director of EP track, the management leadership. Right, so I’m a technical person. But you took on management roles repeatedly. Well, I did because I found out that if you’re, generally speaking, the top of the organization is the manager, the VP. And as a technical person, no matter how high you go, you’re an advisor for that person. And I decided consciously after I worked at Apple, that I was gonna be a VP and have everybody work for me, because then I can do it. Right, so then my skill set is somewhat unusual, and I’m not the only one, obviously, but I decided to get on the management track so I could build the computers I wanted, because sometimes when I wasn’t the leader of the group, some managers at some point would decide they own the next decision and they would make some random decision. I’d be grumpy about it, and there’s nothing I can do about it, because people work for them, not for me. So that’s, it was a conscious thing, and I hired a consultant, Ben Katrao, to help me reframe how I approach this. Now I’m still a technical person, but I found that, turns out there’s a whole bunch of really good technical managers that I like to work with that like to organize stuff. And I would say, I maintain my openness and low conscientiousness and disagreeable behavior, and I have people work for me, work on my team, or work with people that manage better. So even though I’ve been a manager, you know, EMD was 2,400 people total at the end, and Intel was 10,000. My staff was, you know, 15 or 20 people. And usually half of them are real managers and half of them are technical leaders. That’s how I solve it. And there are lots of companies running away. A lot of times founders tend to be technical people, but people working for them are not technical. Or they’re stronger on the management side than the technical side. But for everybody, you need to decide who you are. Like I had a great technical manager at EMD, and one day he was a little amassed because I was looking to the, you know, a couple of the really technical heavyweights that’s all we’re probably. He said, you know, I’m pretty technical. I said, yeah, I know. And he said, are you technical compared to Jim and the Barb? And he goes, I guess not really. He said, I know, I really like, you know, what I want you to do is you’re running this project. You have 150 people working for you. You make all the technical decisions you can, but when it’s out of your wheelhouse, we got serious experts and you have two choices. You can call them or I can call them. And he later told me, he said, I found that it was a lot better when I called them than when you called them. And successful thing. And he was technical. He was really good at making good decisions, but he wasn’t the strongest technical person in the group. So that’s the first thing is, you know, figure out who you are. I’ve seen a lot of people fail in engineering because at some point they think I’m technical, but I want to get on the management track, but they’re bored by management and they don’t have a plan to deal with it. And so they start- Yeah, well, you weren’t bored by management. And so- I joke that I decided to see the organization that’s computer architecture problem and treat- Well, that’s exactly what I was going to ask. What transformation did you have to undertake to- So one of them was, what do I have to do to be effective? Right, so that’s, you know, I hate to work on failed projects, right? And then the next was the organizational problem itself is an architectural problem. And then I kept, you know, for myself, well, I’m a funny kind of, if something has a solution and it’s being confidently driven, I’m not that interested in it. I like problems. And so in a big organization, there’s a million problems and I start sorting them by priority and then solving some of them or handing them out to the right people. So there’s a whole bunch of technical work to do on that. And then I’m fairly good at skill assessing people who are technical, either for management or technical positions. And then, you know, giving them work. I like autonomy in management. So if somebody’s competent and they can do it and they understand it. Then gave me a bunch of books to read. And one of the frameworks is goals, organization, contract and teamwork or capabilities, I guess, we usually solve for that. So is the goal super clear? Do we have the capability to solve the problem? Is there a contract between me and the groups doing it? So they know what to do and what their goals, you know, box are in, right? And do they have to, you know, is the organization that, like a lot of times, you know, there’s a joke that start up, start with a problem and build organizations to support it. But on the second, third system, the organization defines the problem rather than the problem defining the organization and then it breaks up. Yeah, then the organization becomes the problem. Yes. Yeah, they constrain the problem and then become the problem. Well, we had a number of discussions while you were doing this about ethics. And I mean, you said that you go, you look at the problems. Well, that’s hard, right? Cause you have to know enough to know what the problems are. Then you have to be willing to look at the problems. Well, then you prioritize them. Like you skipped over that very quickly, but all of that’s extraordinarily difficult, I would say both cognitively and emotionally. Sometimes it is and sometimes it isn’t. Like when I joined A&P, the CPUs were less than half as fast as the competition and they had no plan to catch up. So that wasn’t that hard. No, but what would be hard there, I would presume is figuring out how it could be that such an obvious problem had gone undetected and unsolved. And then… No, actually one of their architects when I was working at Apple told me that they believed that CPU performance had plateaued, it wasn’t going to get any faster. And they were going to work on adding features to the rest of the chip. And then Intel came in and said, we think computers are going to get five or 10% faster every year. And they did it. One had one goal, which is, things slow down, the other has a different goal. Five or 10% isn’t a lot, but you do that 10 years in a row. And the other guys weren’t. So that wasn’t that complicated. Like Elon famously said, he tells everybody secret plans and nobody believes him and then does them and they still don’t believe him. And then they’re like, oh, shock. So Intel publicly said they were going five or 10% faster every year and then he said, no, they’re not. You know, and the results were at some point that the gap got bigger and bigger. And you know, the people in AMD were committed to their plan. I don’t know why, it’s interesting how these things get internalized. And then you start, even when they, you know, at some point, you know how it is, it’s cognitive dissonance. You say you’re going to do something different, but you’ve learned how to do this other thing really well. You keep doing it. Right. And then- Well, you build a whole machinery around it. Yes, exactly. You know, they had a big machine that did all kinds of stuff that was perfectly useful. Right. And good people doing it. Like I said, we didn’t hire any people to build then, but we did refactor, you know, you know, reset the goals, refactor a whole bunch of engineering. Okay. So while at AMD, you were successful twice. And so, and the success was both building a chip that was competitive. So you had to put together the teams to build the chip, but also to transform the internal structure of the company so that that became possible. And then also to communicate that to your customers. And so what’s the problem set there? I didn’t communicate with the customers. I, you know, because, you know, computer were all performance cells. You know, it’s the- Okay, so that’s the first thing you brought to the table, performance cells. And here, we’re going to break that down. Here’s the measurements. Here’s the measurements. And then there’s lots of public benchmarks. Like everybody tries to game, but generally speaking, there’s a really big community of computer users and they know what they want and they know it’s fast. Right. And you know exactly how a computer works. So you can actually say, once you decide that what faster is better, does that work on all of the elements of design? Well, it’s a little complicated, too, right? There are certain things like you can make it faster. Nobody would care. Like, yeah, there’s some judgment calls in there, but it’s not that complicated. Today on phones is a thing called geekbench. And you get a number at the end. You know, is your geekbench score 100 or 50? 100 is better. Right, right, right. The people who made the benchmark tried to pick the components of your phone experience such that the geekbench number represented whether the phone was faster or not. And then whether you care or not is another question. Like for the current applications, if they got twice as fast, you might not notice, but as the computer gets faster, so new applications are possible. And on the phone where it’s possible, it’s great, where it’s not possible, it feels slow and laggy, right? So performance wins and, you know, different form factors like a notebook or a desktop or a phone have different amounts of power. They can live within the budget. Okay, so you had a goal. You had the measurements in place. You decomposed that into tasks. You assigned competent people. What psychological factors got in the way? Like, how did you see companies? All of them. Yeah, fair enough. All of them. But what did you see specifically, once you have a good plan in place, that doesn’t necessarily mean it’s going to be implemented. And so what are the mistakes that people make that you saw in large companies that doom the companies or that stop them from transforming internally? So there’s a couple of very separate problems. When somebody with a good set of ideas says, I need to transform this place, like, are the goals proper, right? And then you want to say, do I have the capability in the team to do it? Like I worried when I went to AMD, I wouldn’t have enough experts in certain things to do it. I’d have to go hire 50 people to fix it. But it turns out there was, I did plenty of, you know, there was plenty of good people, actually some really great people. So I was like, you know, pretty quickly checked off the capability box. And then you start wondering, well, why the hell aren’t we doing the right thing? Well, the problem was the goals were wrong and then the organization was wrong. Right, and then generally speaking, if those aren’t right, so to begin, so maybe to begin with, the goals weren’t unreasonable and no one knew, but then across time, the fact that one set of goals was better than the other. The belief that computers weren’t going to get much faster was a bad goal in a world where the competitor believed they were going to get a lot faster. Yes, and could do it. And that became incrementally worse across time to the point where it became cataclysmic. Yeah. So you got to get the goals right and you got to establish where you have capabilities. You know, those are the kind of fundamentals. But then the organization building is hard because somebody will tell you, so-and-so is a great manager. Well, is he? Or, you know, like a lot of times there’s somebody that looks like a good manager, but you just have three people working for us. It’s really good. The problem with that is when things are going well, the empty suit manager with his good people supporting him, they look like they’re making lots of progress, but when they run into hard problems and the technical guys don’t want to do it, they go to him and he makes a random decision or does something dumb or doesn’t believe them. Like that happens a lot. The technical guy goes to the empty suit manager and says, you know, I think this isn’t working, we need to change. And he says, now we’re fine. We’ll just put our weight through it, right? So you get these weaknesses in your organization because you don’t have skill level. Like I said, I’ve worked with a lot of really good technical managers who know when they can make the decision and they know when they have to come to somebody who’s more of an expert. That’s great. And it turns out some people are so good at that. They can operate way higher than you think because they’re not technically strong. They’re super good at translating and making judgment calls like that. So you got to start building your organization and then there’s stuff about how you build teams. Like some groups are what you call functional, but all the people who do software in one group and all the people who do hardware in another group, all the people who do atoms in another group, right? And then the managers, but if the thing you’re building needs a little of all three of those things, you know, it’s called a functional organization versus product organization. You might want a team with a couple of programmers, a couple of hardware people, a couple of atom people in the same team. So they all have one goal, as opposed to the functional group says I’m making the best software in the world. Was it the right thing for this product? They go, I don’t know. It don’t work on the product, I work on software. So I’m generally speaking, you know, product focused. So if you only have like five of some discipline, you tend to make a little functional team like that. And there’s a couple of things in computer design which are functional. But generally speaking, I like product focused organization. So everybody’s like, they’re all working together on the same thing. They may have different disciplines. So AMD- Now you’ve encountered all sorts of frustrations. Sorry, you’ve encountered all sorts of frustrating situations when you’ve gone into companies that, where you’re trying to put together a good product. And so what do you see as particularly counterproductive? And what have you learned like how to conduct yourself so that you can be successful? Weak leadership, people who can’t make the technical decisions they have to. That’s a big problem. Functional organizations where people are optimizing for the function, not the product. Bad goals is one of the worst things. Some organizations have real capability gaps. Like, you know, they think they have the right people, but they don’t. You know, some managers play favorites. They think so-and-so is really good and they’re not. Yeah, so that’s a real functional analysis. The company just can’t do what it needs to do. Yeah, so, and we’re still analyzing. Here’s a group. They’re actually from someplace. There’s a belief that we’re gonna build this product. It has to be a great product. And how do you do the, you know, basic blocking, how to make that successful? Right, that’s different than the Malays that overtakes big successful companies, which you can generically call bureaucratic capture. Right, that’s a different problem. Like a company that’s bureaucratically captured will manifest all kinds of bad behavior in your organization and product development. And then, you know, some big companies where the, you know, the bureaucracy is taken over, there might still be groups that are really doing a great job making great products. You know, so there’s, you know, I think there are separate spaces and I understand both of them pretty well. And again, you know, the way it’s all big complicated problems, you have some abstractions about what you’re dealing with. So, you know, a framework like Goal’s Organization Capability and Contract is a super clear message for evaluating what the hell’s going on and then making changes. You know, very specific changes to it. Goals are clear, you know, if you’re not clear, nothing else matters. Get the goals clear, right? Capabilities, are they good? You don’t have the right capabilities, nothing will save you. You have to have the ability to do the job you’re doing. You know, does your organization serve the goals? That’s a big problem. That’s a painful one. That’s because that’s when you start changing who works for who and what the boundaries are. But you have to do it. Okay, so let’s tackle it this way then. So you’re going to pick someone who has optimal attributes to what would create and operate within a highly functional organization. What are you looking for in that person? What’s crucial? Well, people are fairly diverse. You know, that’s the funny thing. So engineers need to have this will to create if they’re technical leaders, let’s say. And then they have to have the discernment to make decisions about whether they’re actually making progress towards the goals or just wasting their time on something cute, right? That’s the thing. Technical managers, they need to know how to run a program. They need to know how to hire and fire. They need to know how to structure work. They need to know how to evaluate how long it’s going to take, how to evaluate whether people are making progress. There’s a whole bunch of things, but then people have very different styles. Some people are very extroverted. I worked with this woman, she was great. She would just have these team meetings and she would really get out there and energize the team. And another guy in the same building was very low key and he would wander around and talk to people and have a really good sense of the team, like an introvert versus extrovert style. But they both worked. They were both very competent. And they were both, to me, you know, really good technical competency. They weren’t my technical leads, but they were technically competent enough to make the decisions and know when they had to punt the decision up. So who do you not want, who do you not, okay. So, I mean, that kind of goes along with the management literature. You see that you want people who are intelligent, especially for complex jobs so they can learn. You want people who are conscientious because they work hard and they have integrity. Then with the other dimensions, it looks like there’s a fair bit of variability, although too much negative emotionality can be a problem. I think that’s because it’s associated with depression and too much anxiety and so on. But there’s diversity in the other personality dimensions and that might be task specific. But what sort of person do you not want to work with? Fakes. There’s lots of fakers out there. You know, they have sales attributes. They’re extroverted, agreeable. You know, they want to say everything’s good all the time. They’re not sufficiently concerned about disaster and digging in the stuff. They may have some kind of narcissistic personality problem. But there’s- So they’re imposters. They’re mimicking competence. Mimicking competence, that’s a problem. There are people who literally- They take credit from other people. Yeah, I kind of put that in a separate boat, but there’s people who take credit for the team. I realized early on there’s two kinds of managers. Some people put it up and some people put it down. Like as a manager, I often tangle with the people I work for, but I always took care of the people who work for me. But some other managers, you know, I had this one guy who works for me. I thought he was great. And then I walked by, I’m neaning what he was having. He was abusing his team and they hated him. I fired him because he always said the nice things to me. And, you know, he did it on those people. So it’s, yeah, there’s a bunch of weird stuff that happens with management like that. Like you have to be excited. Like if you’re a senior manager in a high tech thing, there’s many people in the group that are smarter than you. And you have to promote them and put that forward. You can’t be uncomfortable because somebody’s smarter than you. When I was in A&D, I had six senior fellows. I think they were all smart. They weren’t as generalists or something, you know, and they didn’t have my interest in architecture of organization. But man, it was smart. People were good. I could talk to them all, you know, I could keep up with them over sometimes, but, you know, I was more than happy to promote them as the smart guys. Like in any way. Why were you confident enough, do you think, to allow you to be surrounded by people that you- I grew up in a beautiful climate. Like I’m above average smart, but I met people who were so smart. Like I knew Butler Lamson, who’s, you know, famous unrated IQ and his wife was smarter, was a joke that he spoke at half Lamson because his wife was the smart one who spoke really fast. But I, at a fairly young age, I was competent in getting things done and work with people that were smarter than me, but they liked my, you know, I’m an engineer and I build stuff, you know, those rocket scientists, they think it up and then they hope somebody would build it for them because they’re off on the next thing. So that’s, you know, a belief I have. You know, it wasn’t always easy. I still remember working on EV5 and I went to the digital research lab and there’s half of those and these super smart people. And I started describing what I was doing and I would describe something for about two minutes and then they would spend five minutes taking it apart and analyzing how it could be like way better. And then they’d ask me the next question. After an hour of that, I felt like, oh my God, was this beating the desk. And they were like, this is great, Jim. I was like, you thought that was great? They’re like, yeah, we’re glad you’re doing it. So, I’ve always had that attitude since, you know? But yeah, it’s hard on some people when they realize how smart some people are. So what- I make up for it because I’m open-minded and I worked my ass off for many years and then I’ve dived in lots of things. And then, you know, I’m not afraid to ask dumb questions. I, you know, like a lot of people protect, they’re trying to reject who they are so they don’t ask the right questions, they don’t learn it. And I’m like, I know you don’t understand what the hell’s going on. I’ve done that in the room of 50 people. And they’re like, well, we thought you should know. It’s like, well, I don’t, but I’m not gonna believe until I do. And then they give all the information and then I’m smarter than I used to be, right? And, you know, so that takes a certain, you know, mental resilience and sometimes it’s very hard on them. But, you know, again, it’s sort of like, you know, you fire the people you have to fire to save the group and save the product, maybe save the company. Yeah, because the net good is really high and it’s the right thing to do. So exposing yourself is the right thing to do, ironically. Now it’s hard and you know. Well, if you admit you’re stupid, then sometimes you don’t have to stay that way. Yeah, it’s hard in some sick organization. Sometimes it’s not safe. And I feel for people who are in places where they would really like to be more open and can’t because, you know, organizations, they have political and bureaucratic or hard on people that are actually trying to do the right thing. And Lauren, I totally understand that. It takes a while, but, you know, the psychological safety thing is it gets overused and gets a bad rap, but having an organization where it’s actually safe to open your mouth and talk and ask questions and occasionally look stupid, you know, and fumble a little bit and have your peers like support you with that and be happy for you when you learn stuff. That’s really important. It’s hard to do, you know, and so, and there’s great attention because, you know, as a leader, you have to be just real enough to do the hard things. While we’re still creating an environment where people can open up and do that. I would say I’m mixed. I have mixed reviews on that topic because once I feel fine things that are wrong and people are doing the wrong thing, you know, I have to get to the bottom of it. And some people, it’s never happened to them before. Like the people haven’t really taken what apart. You know, they got A’s in college and they got good reviews and they rose to their Peter principle and competence point and all of a sudden they’re doing something they’re over their head and they know what to do about it and have a lot of practice. So, yeah, it’s a funny thing. So I’m gonna close with a question about your current venture. You’re now working with a company that does AI computing and what do you hope to do that you can talk about? Well, so I was an investor in this company when it first started, Abhija Ajay, who’s the founder, worked with me at AMD and I always thought he was an especially smart guy and I liked his approach to building AI computation. I’m really intrigued about computers programmed by data. Right, I think it’s more like how our brains work. Our brains are really weird, right? Because we think in this linear narrative we have this little voice in our head but we know we have 10 billion neurons and they’re collecting away, exchanging small amounts of brain transmitters and electrical pulses. It’s bloody hilarious the gap between what a neuron looks like and what a thought looks like. And so there’s a really interesting opportunity to make big AI computers that are actually really programmable. So one of the things we’re doing is we’re building the software stack that lets you build a neural network that you want and then program and get the results you expect reasonably well, as opposed to having a very large army of people tweaking it. And so there’s a bunch of architectural interesting things to do. And then as a startup, which we have chips at work, we started production, we’re gonna start selling them. There’s a whole bunch of work to do on how to engage with customers and a lot of customers we’re talking to are super smart. There’s all these AI software startups with really smart people that have some problems that basically have to computer for a million times faster than you usually solve. So there’s like a huge capacity gap on what they wanna do. So participating that is fun. I like that kind of thinking. And your goal, so you go into an organization, you have a goal for the chips. What’s your goal for this organization? Oh, we’re gonna be successful selling AI computers to a large number of people. You know, significantly better performance, better programmability and lower cost. And there’s a bunch of innovation work to do around that to make that really possible and doable. Like the AI field is relatively new. The computers that run AI today are relatively clunky. And to me, you know, need a lot of work and refinement so that, you know, from the idea that you wanna express in the program you’re writing to the result you want, better and cleaner. Okay, so one final question for anyone who’s listening who would like to pursue engineering as a career or let’s say who wants to be successful within the confines of a big company. What advice do you have for people? What have you learned that you can sum up? Yeah, look straight in and ask me that. First you have to know yourself a bunch. Like, what are you good at? Like, you can’t get really good at something you’re not into and you’re not good at. So you have to have some natural talent for it. And then you have to really spend some time figuring out what you like. Like I read this thing, it was interesting. Like people think of college as expanding their possibilities and the university itself has so many options you think that would expand your possibilities. But once you pick one of them and you study it for four, eight, 10 years, you’ve narrowed your possibilities, right? You’re kind of stuck with your discipline and you pick that 20, which I think is crazy by the way. Like I think if you want to be an engineer, a good general engineering degree, like mechanical engineering or electrical engineering will give you thinking skillsets. I’m not a huge fan of people getting PhDs unless they really, really know they love it, right? And then take some jobs where, you know, there’s an opportunity to do something for a year or two and then do something else. Like my first job out of school was a random job. I worked on like five different projects in two years while I was there, you know, fixing hardware, building something, debugging something. I learned a lot in the digital. I had many different roles even though I sat at company for 15 years. I wrote programs, I did logic design, I did testing, I did lab work, you know? And so I got to see a lot of different things and get a feel for what I really liked. And I worked with smart people that, you know, I had a lot to learn from. I’m working hard when you’re young is really useful. You know, some people are like, well, you know, it’s like the 10,000 hour problem. And if you want to be an expert, you need to do that a couple of different times on different things. And you can’t do it unless you really love it. A friend of mine’s wife said, what do they put in the water? All she does is talk about work. Yeah, so you figure out what you’re competent at because you need that. Figure out what you’re interested in. I mean, men and women seem to pick different occupations, not based on their competence, but on their interest. And so interest is a very powerful motivating factor. Oh, that was it. I’ve been in a lot of places where the best engineers were women. So, you know, we know that numbers are less, but there’s plenty of really great women. Yeah, it certainly doesn’t make it impossible. It’s just an indication of the, what would you call it? Of the impact of interest as a phenomenon. It’s important as well as competence. Yeah, maybe. And so a diverse range of experiences. Yeah, don’t over index on something before you’re really sure that that’s something that you’re really gonna like or be great at. Don’t be afraid to ask stupid questions if you don’t know what you’re doing. Yeah. Try to work with good people. Work in organizations where, if everybody hates the company you’re working in, move somewhere else. You wanna work someplace where the energy is good, people are excited about what you’re doing and why. Like sometimes you might be in a company that has something going wrong, but your group is gonna change it. That can be really fun, but you need some camaraderie, some hope, right? When the goal’s clear. Right, so that’s an adventure. You have a destination and the camaraderie along the way. Yeah, and there’s so many places where you’re doing so many wild things, being stuck in a company you don’t like that’s been going nowhere for 10 years, man. You don’t have that many 10 years lasting your life. Make sure you’re actually getting it, especially if you’re getting a different experience. Somebody said, you have 10 years experience or one year of experience 10 times. Right. Now, sometimes you work on the same thing and you refine it and you become the expert, but then you should feel like you’re making progress and expertise. Right. But if you’re just kind of going through the motions over and over and doing the same thing. Then it’s time to fire yourself under those conditions. If you’re bored, you’re not moving, right? Like engineering’s not boring, right? It’s relatively exciting. Yeah, I think that’s actually a pretty good rule of thumb. If you’re bored, you’re doing it wrong. Yeah, something’s wrong. Yeah, something’s wrong. Funny, like in AMD, we had this group that did tests and it was kind of dysfunctional and there was a couple of managers and nobody liked it. And at some level, the test engineering wasn’t the hardest thing. So, but I decided, that’s stupid. Why isn’t our test group the best in the world? But we were organized around it. We had a really great leader. We had a good team. I told them I wanted it to be really great. And I told the engineers to stop complaining about it. They had a problem come to me and we’ll fix it. Within two years, people were coming to me and it’s like, man, the test guys are killing. Yeah. Yeah, they went above and beyond. They made it something of value. You know, it was great. It was super fun. That’s a really good place to end. Cool. Thanks, Jim. Hey, good seeing you, man. Much appreciated. Thank you for taking the time. All right. We’ll talk soon. Yeah, cheers. Bye.