Transcript of Episode 33 – Melanie Mitchell on the Elements of AI

The following is a rough transcript which has not been revised by The Jim Rutt Show or by Melanie Mitchell. Please check with us before using any quotations from this transcript. Thank you.

Jim: Howdy, this is Jim Rutt, and this is The Jim Rutt Show. Listeners have asked us to provide pointers to some of the resources we talk about on the show. We now have links to books and articles referenced in recent podcasts that are available on our website. We also offer full transcripts. Go to, that’s Today’s guest is Melanie Mitchell, professor of computer science at Portland State University and external faculty at the Santa Fe Institute.

Melanie: Hey, Jim, it’s great to be here.

Jim: Yeah, it’s great to have you here. We go back a ways.

Melanie: Absolutely, many years.

Jim: Indeed, indeed. Melanie is the author of several books. I often recommend her book, Complexity: A Guided Tour, as a really good introduction to complexity science. And also, I have some really nice history with Melanie’s books. One of them was quite historically important to me. When I first started stumbling into … not even knowing that it really existed, complexity science, the very first book I read was John Holland’s book on genetic algorithms. The second was Stu Kauffman’s Origins of Order. But the third was Melanie’s book on genetic algorithms. Well, I don’t even remember the name of it to tell you the … oh, yeah. An Introduction to Genetic Algorithms.

Melanie: Yeah, exactly.

Jim: Yeah, and I really like that. In fact, it was the one that made this whole genetic algorithms thing clear to me. So that when I retired from business in 2001, I literally went down the hall and started writing hybrid genetic algorithm neural nets. And frankly, a lot of it’s stolen. At least the genetic algorithm part’s stolen right out of your book, so a very important part actually of my intellectual trajectory. So, thank you for that.

Melanie: And that’s great to hear.

Jim: Yeah.

Melanie: You’re welcome, great to hear.

Jim: Yeah, Melanie’s also created and taught introductory level online courses on complexity science. A little Googling showed me that none of them seem to be active at the moment. Did I miss something? Do you have any of your courses up?

Melanie: Yeah, they’re on And they’re always available, but they’re in session … one of them is about to end its session.

Jim: Okay, yeah. Well they all said something like, “Not in session,” or, “Not available,” or this or that. But we’ll get the link from Melanie and put it on the episode page. So as usual, a few people will want to follow up on any link or reference. On our episode page, we’ll have a link to one or more of the courses that’s available. And those things have gotten just amazingly good feedback. Anyway, today we’re going to talk mostly … and I’m sure we’ll talk about other things along the way. Her new book, Artificial Intelligence: A Guide for Thinking Humans, a reader should find it very accessible. I don’t believe there was a single mathematical formula in the whole book, thus Melanie obviously learned Hawking’s law. Every equation halves the sale of the book, and that’s multiplicative.

Jim: While on the other hand, that does not mean the book is lightweight. She addresses many of the most important issues in AI today, and she gives some very nice examples explaining some of basics of neural nets. And even some of the more advanced topics like convolutional neural nets and recurrent nets in a way that I think people will find deeper than you usually find in such intro books. But yet not in a way that’s kind of technologically off-putting, so a good recommendation for people looking for an introduction to where the world is with respect to AI today.

Melanie: Thanks for that great introduction.

Jim: You can send me some bitcoin at the following address, right?

Melanie: Yeah, right.

Jim: No, really it was a really good book. Okay, let’s jump in. One of the points you make early on in the book is that much of the media … and truthfully I’ve even found this with some very good academics outside of computer science departments. Think that deep learning is artificial intelligence, and that is all there is to AI. And you start out by telling us how that isn’t true, and maybe a little bit of the history of symbolic and subsymbolic AI.

Melanie: Sure. So at the very beginning of AI, the requirements field maybe back in the ’40s and 1950s, people did not use learning as a method to create AI. At least not for the most part. They used symbolic AI, and which people manually programmed in rules for intelligent behavior. That turned out to not work very well, because intelligent behavior is way to complex to be captured in a simple set of rules that … cooked up by a computer programmer. So in the ’90s, starting in the ’80s and ’90s, people started using more statistical approaches. Where the idea was to learn from data, computer programs learned from getting lots and lots of examples. And there were many different ways to do that.

Melanie: But only in the last decade or so has the method of deep neural networks taken over, and it’s been so successful that it’s kind of taken over the entire field. And everybody is using it in some form or another. And a lot of the media, and as you say, people outside of the field, equate deep neural networks with AI. Equate machine learning with AI, equate deep neural networks with machine learning. Yet all of those things, that are kind of subfields of a much broader set of techniques which you might call artificial intelligence.

Jim: Yup, and you talk about hybrid systems. And in reality, in a lot of the real interesting things going on out there in the world, are hybrids. There are partially, maybe substantially deep learning. But they have other attributes as well, for instance self-driving cars, and do you have other things that you could point to that are examples of hybrid systems?

Melanie: Yeah, I would say that most of the systems that are used commercially are hybrid. One well known example is the program by Google DeepMind, that learned to play Go and chess. So, AlphaGo is the famous Go playing program. AlphaZero is a program that learns to play chess and other games. And those involve both deep neural networks and more conventional technique called Tree Search, which is a more symbolic kind of technique. So, those two systems combined together really are what give those systems the power.

Jim: Okay, good. Another thing you address, and for those people who would listen to the show, we recently had Gary Marcus on the show. And he talked about the same thing, is the hype problem in AI. That people overpromise and underdeliver. And it seems to be going on today, but it wouldn’t be the first time. There’s been a history of AI springs and AI winners. Could you maybe give us a little bit of the history of the AI hype cycle?

Melanie: Sure, AI hype has been around for as long has AI has been around. And it come not only from the media, but from AI researchers themselves. Who either are hyping because they’re trying to get more funding, or because they actually believe that we’re very close to big breakthroughs in mimicking human intelligence, or possibly both. But there’s been this kind of recurrent cycle where there’s been a lot of optimism and people making big predictions about what’s coming very soon. Say self-driving cars, which maybe we’ll talk about a little bit later. But then there’s a bit disappointment wherein the predictions don’t come true, where people overestimated where the field was. And so, you go from these so called AI springs, in which there’s huge optimism, lots of funding thrown at researchers and lots of startup companies, venture capital. To AI winter, where a lot of the funding dries up. People become more pessimistic, realized that it was harder than we thought.

Melanie: And that cycle then keeps going. It’s kind of a boom and bust cycle in AI. And when I got my PhD in 1990, we were in the middle of an AI winter. And I was even advised not to use the term artificial intelligence on my job applications. Which now is kind of the opposite, everybody’s using the word artificial intelligence, even if they’re not really doing it.

Jim: Yeah, it’d cost you $100,000 a year to leave it off your resume, right?

Melanie: Exactly.

Jim: Yeah, I remember those cycles. In fact when I was a young entrepreneur in the early ’80s, a lot of my college friends from MIT had gotten involved the Lisp machine era of AI. Which was hugely funded, companies going public, et cetera. And a bunch of the tried to recruit me to come into those companies. And I went and talked to them, and listened think them. Being at that point kind of a relatively experienced corporate IT guy, working on some advanced problems, but nonetheless working with actual corporate IT people. I made the assessment, “Any industry based around the Lisp computer language is just too cognitively loaded to be realistic for corporate America.” I concluded at that time, and irrevocably, that that whole Lisp machine Symbolics corp and all that stuff was just not going to work. I was right, and it kind of spiraled down into that AI winter that you were talking about there in the early ’90s.

Melanie: Yeah, well I have to say that as a pregraduate student, I learned to program in Lisp on a Symbolics Lisp machine. And that’s by far the best machine, the best user interface. And the best programming language I’ve ever used. But I understand that it wasn’t going to scale up in corporate America. But for research purposes, it was great.

Jim: Yeah, probably one of my friends was involved in the things that you like. But as I said to him, “You need an IQ of 130 basically to be good at programming Lisp.” And I can tell you that IQ 130s are fairly thin on the ground in corporate America, unfortunately.

Melanie: Yeah.

Jim: But on the other hand, IQ of 130’s kind of your baseline PhD science and engineering kind of person. So, it’s a good fit in that world.

Melanie: Ha! Well, we can talk about IQs. The IQ is a measure of intelligence, when we talk about what is intelligence later on.

Jim: Yeah, it’s I know, a controversial topic. And obviously that’s way too reductionist of a way totally about it. Nonetheless, I do push back a little bit about people saying, “There’s nothing useful in IQ.” It’s not everything, but it’s also not nothing. But we can talk about that later, if you like.

Melanie: Okay.

Jim: And certainly, it seems like today we are well into the hype cycle. I’ve never seen a hype cycle quite like this one in AI. And of course, maybe it’s different this time. What do you think?

Melanie: Well, that’s a good question. As you say, we … it seems to be the peak of a new hype cycle. Where deep learning has been extraordinarily successful, and there’s a huge amount of money being put into startup companies using deep learning. And even big companies like Google, Facebook, Apple, IBM, et cetera putting a huge amount of money into AI, and deep learning in particular. But I think people are just beginning to really see some of the limitations. Self-driving cars is a very good example, people assumed that we would have self-driving cars widely deployed by now. And even 2020 which is just a couple weeks away, has been sort of the year that people have predicted that we’re going to see millions of self-driving cars on the road. That human drivers are going to be a think of the past and so on.

Melanie: But it’s seeming more and more like that’s way overestimating how soon we’re going to see that kind of technology. So, there’s been a lot of predictions of a new AI winter. And I wouldn’t be at all surprised if that happens.

Jim: Yeah, I could say in my other role as occasional private equity investor in early stage companies, I’m starting to see the ridiculous. Two years ago, it was blockchain and such, that was tacked onto every business plan whether it needed it or not. Now you can see a beer brewery, “Oh, and we’re going to use AI!” Okay, well yeah, right. We’ve been making beer successfully for a thousand years without it, and I doubt you needed it large … for small scale microbreweries. So, that’s generally a sign that we’re getting late in the hype cycle. Where people are just relentlessly stapling AI to almost any other kind of business plan.

Melanie: Yeah, AI, blockchain, and quantum computing, those are kind of the three buzzwords.

Jim: We got the mother of all buzzwords, right?

Melanie: Yeah.

Jim: If we want to extract money out of naive investors, let’s string those together into some semi-plausible sounding story.

Melanie: Right.

Jim: Oh, my goodness, interesting. Another area of a lot of confusion in the general public around AI is around strong … or AGI versus narrow AI. And a lot of this kind of big talk about AI, going even as far back as the 1956 Dartmouth conference, were sort of talking about strong AI. Or artificial general intelligence or human level AI, what can you say about that versus the AI we actually have today?

Melanie: All … throughout the history of AI, every AI program that’s ever been created has been narrow. In the sense that it’s able to do essentially, one thing. Like with have machines that can play chess at a Grandmaster level, but they can’t do anything else. They can’t play checkers. We have machines that can do speech recognition. Now actually amazingly well, but they actually don’t understand any of the speech that they’re recognizing. And they can’t say, analyze that speech for its sentiment whether it’s positive or negative. You have other programs that can do … analyze speech for sentiment, but they can’t do anything else. So, we’re in a world of narrow AI that’s gotten better and better at narrow tasks.

Melanie: But really at the beginning, the goal was general AI, AI that can do the sort of range of general tasks that humans can do. Of course it’s debatable how general humans are. But that’s kind of the ultimate goal. And people talk about AGI, artificial general intelligence or human level intelligence. Or as you say, strong AI. All these terms refer to a goal rather than to anything close to what we have today.

Jim: Yeah, that it certainly would seem to be correct to me. And you and I both know an awful lot of people, including me, who got involved with AI at one time or other. It was all about the hope for this human level AI someday.

Melanie: I think most people who got into the field, at least up until recently, that was their goal. They got in because they were really excited about recreating human-like intelligence. Because they were interested in intelligence, in the phenomena of intelligence and understanding it. But it seems that our own intelligence is actually invisible to us. We do so many things so easily, like this conversation we’re carrying out. You and I are just chatting and talking, and we have no idea sort of how our brains are doing that. And it turns out that it’s much harder than we thought, than people have ever thought to recreate that in machines. So, that’s been a recurring them in AI. That people are figuring out more and more how complex intelligence really is, in a way that’s almost invisible to our conscious processing.

Jim: Yeah. And one of your chapter titles, I believe was Easy Things Are Hard, right?

Melanie: Right.

Jim: That’s something that’s been pointed out many times. I think you may have also mentioned this one specifically, neither of us could multiply two 50-digit numbers very quickly. And yet a $1.50 calculator does it without any problem at all. And yet the biggest computers in the world, our smartest AIs can’t pass the Steve Wozniak test. Which is to drop a robot into a random American kitchen and have it figure out how to make coffee.

Melanie: Exactly. Exactly, that’s a well known paradox in AI.

Jim: Indeed, and yet there are some people who think that we’re knocking at the door. You quoted Shane Legg, a very smart guy from DeepMind, who recently said he thought we’d be at the human level of AI in the mid-2020s.

Melanie: Yeah, he’s since walked that back a bit as you can imagine. But he’s not the only one. Mark Zuckerberg, the founder of Facebook, believed that within the next five years we’d have something like human AI. And he was going to have his company invent that. I think founders of Google also were very optimistic. And they hired Ray Kurzweil, maybe the most famous AI optimist, to make general AI happen at Google. So there’s a pretty … it’s big different of opinion, even in the technology community about how close we are to having something like that.

Jim: Yeah, and maybe it’s like fusion-based electricity. It’s 20 years in the future and always will be.

Melanie: That’s what people say. And I think the reason for that is that we don’t understand how complicated our own intelligence is, because we’re so unconscious of most of it. Most of the intelligence that they … the sort of things that allow us to be generally intelligent happen below the level of our consciousness.

Jim: Absolutely, in fact my day job, to the degree I have one, is the scientific study of consciousness. And the more I learn about it, the more I realized how amazingly minor the actual conscious frame is itself. I used to think it was doing 1% or 2% of the work, but it’s a lot less than that actually.

Melanie: Ha! That’s interesting.

Jim: Yup, yeah, the actually bit rate being processed in consciousness is on the order of 60 bits a second. That’s 60 bits a second. And yet we know that until … probably still to this day, it’s true. That the actual processing at the bit level in the human brain is larger than any computer on earth. We’ll pass that number soon. But think about that, 60 bits per second versus the most powerful computer on earth, the thing that lives between your ears. So that shows you how much is happening down in the lower parts of the brain, where we really still don’t have that much insight.

Melanie: Yeah, that’s why I thought it was amusing when Andrew Ng … who’s a former Stanford and very big name in AI. He came out with this prediction that anything that people can do in a second or less, machines will be able to do as well within the next few years. And I was thinking, “Well, he really has no clue what people are doing in that second or less. They’re doing a lot.”

Jim: Yeah. The one you alluded think earlier, which is still a great mystery, is how is it that humans produce speech, right? Making your tongue wag is fairly well understood, but how is it that we choose the words we do? I mean, it all happens completely unconsciously. We have no insight into our sentence creations mechanisms, it’s black box machinery from the conscious mind. And there’s one … that’s something we do in one second all the time.

Melanie: Yeah, and that machine can’t do very well at all.

Jim: Yup. Let’s jump ahead a little bit, because you mentioned it, and that’s self-driving cars. And you point out that one of the things that bedevils many of these kinds of high-stakes, high-value problems is the long tail problem. Why don’t you tell us about that, and then more generally what you think is going on with self-driving cars? Because you were right, even big companies like Ford was saying 2020 was the year you could go down to the Ford dealer and buy a level 5 self-driving car. We … obviously, isn’t going to happen.

Melanie: Yeah. So the way that these cars are created, the sort of self-driving cars, is at least in their vision systems, they use deep learning. Which involved learning from millions of labeled examples, like some human has labeled a video that a car might see when it’s something like that when it’s driving. They’ve labeled every object in it. And the car learns like, “This is a trash can. This is a stop light. This is a stop sign. This is a pedestrian. This is a dog.” And it learns all those objects, but the long tail problem says that there’s lots of things in the world that are very unusual. But there’s so many things that are unusual, that something unusual is likely to come up.

Melanie: So, there’s this idea of … think of a statistical distribution where there’s lots of normal usual things that you would see. But many, many very unusual things that are in this tail of that distribution. So, I gave some examples like you’re sitting at a stop light for five minutes. It’s a red light and it doesn’t change. Okay, that hardly ever happens. But it happens occasionally. Or there’s a snowman in the middle of the road where you’re driving, should you run into it? There’s all this crazy stuff in the world that can happen, and machines don’t learn about it. So the question is, what do they do when they’re in a situation that they haven’t been trained for?

Melanie: Well, we humans have this thing that we call common sense. We have it, you can argue the degree to which different people have common sense. But we all have this ability to take something we’ve learned and apply it appropriately for the most part in new situations. And self-driving cars really lack that common sense. So, they can do really well in most situations. But there will be situations that occur for every self-driving car that it can’t deal with. So, we don’t have level 5 autonomy. Which is where we can just sit in the back and drink wine, and read the newspaper while the car drives us, because something weird might happen that it can’t deal with.

Jim: Yeah, I got to say that I find anything less than level 5 uninteresting, and probably annoying and dangerous.

Melanie: Yeah, that’s a problem. I mean that this is the big trade-off in self-driving cars, is that level 5 means the car does everything in all circumstances. Level 3 or 4, whatever we have now with things like Tesla or the Waymo and all these different self-driving sort of prototypes, it’s more that the car can usually manage. But occasionally, the human needs to step in. Well humans are very bad at paying attention, as we know. And people don’t pay attention. And they don’t step in when they’re needed. Because stuff happens very fast. So we’ve seen a lot of accidents that involve that very thing, where a self-driving car runs into a situation it can’t deal with and the human isn’t paying attention. So, it is very dangerous.

Jim: Yeah, and interestingly Google with their Waymo, even before it was Waymo, they came to that view. And at the time it seemed radical, they said, “We’re going to build a car with no steering wheel. Because we … our analysis, our cognitive scientists have told us that the handoff between computer and human is never going to work.” And to the degree they stick to their guns on that, they may now be caught on, “Okay, they were right about that.” But then there’s also the long tail problem on the other side.

Melanie: Yeah, I think the way that it’s going to be dealt with is by redefining what it means to be self-driving. So that we’ll have self-driving cars, but only in very restricted circumstances. In parts of the city where everything is completely mapped out, and there’s maybe restrictions on what … where pedestrians can go. And it won’t be available in certain kinds of weather. It’s going to be restricted so that the infrastructure that we human build will kind of meet self-driving cars halfway, so they won’t have to deal with the long tail problem. Because we’ll try and prevent any weird things from happening. But they won’t be able to run in all circumstances.

Jim: Yeah, the weather’s one that’s … I’ve always wondered about. Having lived most of my life in places with real four seasons of weather and having dealt with the fact that you can barely see out the window because of ice and fog, and hardcore sleet coming down and all this sort of stuff. You’d have to have a hell of a lot of extra processing, redundancy, and sensors to be able to safely drive a car in kind of adverse real winter weather. The kind you might find in the mountains back east.

Melanie: Yeah, absolutely. Humans aren’t that good at it either, as we know. And humans get into lots of accidents. And some of the self-driving car people have argued, “Well, self-driving cars aren’t perfect. They’re going to get into accidents. But maybe they’ll reduce the overall number of accidents.”

Jim: Yeah, that could well be the case. I make that point. Which is if the goal was that self-driving cars had to be perfect, they’d never get there. But all they have to do would be better than humans.

Melanie: Well, I think that’s maybe not true. That humans will not accept self-driving cars that are going to be fallible. That are maybe just a bit better than humans, that kill people. Because they get it … they do the wrong thing. And I don’t know whether that’s fair or not, but I think people sort of hold a higher standard to the technologies than they do to other humans.

Jim: That’s probably true. And even just at the human scale, the humans aren’t that bad. Gary Marcus and I actually looked this up while we were online on our call a month or two ago. And it turns out that the current fatality rate in the United States is about one fatality per 100 million … I think the actual number is like 80 million miles driven. And 80 million miles driven is only about what all the self-driving car tests in the world so far have added up to, so there’s no real statistical signal at all. In fact, if there is a statistical signal at all. It’s that it’s the opposite, is that they’re still a lot more dangerous than humans. Humans aren’t that bad actually. They’re not that good, but they’re not that bad.

Melanie: Okay, but I’ll push back a little bit, because looking at fatalities is one thing. We have a lot of stuff, technology in cars to prevent fatalities like airbags and seat belts, and all of that. But there’s a lot of accidents that happen with humans, that aren’t fatal. And so, humans do get into accidents much more often than that. And maybe the number of accidents would decrease with self-driving cars.

Jim: That’s a good point. So we’ll … clearly somebody, I’m sure that people like Waymo and Uber, et cetera have done this. That a figure of merit, what is the economic utility, what level of wonderfulness in a self-driving car versus a human? But then we still have to get to the question that you also raised, which is maybe we demand a higher standard from automation before we’re willing to try it?

Melanie: Yeah, and one of the reasons is that we understand … I think it’s a matter of understanding. And I understand why people get into accidents, they’re not paying attention. You kind of see situations where somebody acted unpredictably or something like that. Because I’m a human, and I can kind of model other humans. But it’s harder to understand some of the accidents that self-driving cars get into. For instance when the car, the Uber car, it didn’t stop for that pedestrian that it killed in Arizona. Or the Teslas in autopilot that have been running into stopped fire engines and police cars, we’ve seen a bunch of those examples. And it’s very … the kind of mistakes they make are pretty different than the kind of mistakes human make. That might be a reason that people will trust them less. Just because, I would say they’re less predictable than humans.

Jim: Or at least less explainable, and I think this is really important for our audience. I think one of the things that makes particular the neural style, like the subsymbolic deep learning style, potentially problematic in that way is that they are at least so far pretty opaque black boxes. And it’s really difficult to get any explanation of why they did what they did. Unlike symbolic systems, could you talk about that distinction a little bit?

Melanie: Sure. Right, so deep neural networks are these very complicated programs. They’re software for the most part, that involve millions of simulated neurons with even more millions of simulated connections between those neurons. And it’s just a bunch of equations and numbers. Very hard for a human, even the person who programmed that network, to figure out what the network actually learned and why it’s making its decisions. And looking at a long, long list of numbers is not an explanation of why this car say, ran into this fire truck. So this idea that neural networks in particular, or machine learning more generally, is hard to explain decisions that it made. It has become a big issue in the field. There’s kind of a subindustry of people trying to make them more explainable, more transparent. How can you trust these systems when you can’t explain why they’re working? It’s a little bit hard to say that we’re going to trust them in all situations.

Jim: Yeah, that’s where the older symbolic systems had an advantage. They may have been brittle in certain situations, but we’ll come back to the brittleness of neural nets too. But because they were written out in essentially logic statements, one could more or less figure out why they did what they did.

Melanie: Yeah, that’s exactly right, and you can think of humans as kind of a combination. Where we have obviously, our neural networks in some sense. We have hundreds of millions of neurons and trillions of connections between the neurons. And that’s what’s in our brain, but we also have language on top of that. Which is a symbolic system, which allows us to explain our decisions. We’re not always explaining them sort of truthfully or with veracity, but it’s an approximation. And it’ll … it makes us kind of trust other humans. Because we sort of kind of know how they think. But these machines just are black boxes, and it’s really hard to get inside their heads if you will.

Jim: I don’t know if our … was it your book? Or whether it was something else I read somewhere else, but apparently other … not the US. Because we’re usually laggards in such things, there’s some countries that are considering legislation to require plain English explanations of results from AI systems that have human implications. Say for instance, being turned down for a loan or approved for mortgage, or something like that.

Melanie: Yeah, that was in my book. I talked about the European Union’s GDPR legislation.

Jim: Ah, okay.

Melanie: Which part of it has the right to an explanation. If your life has been impacted by an algorithm in some way, by an algorithm making the decision, you have the right to an explanation. But that kind of gets into a murky philosophical territory of, what counts as an explanation? I could give you my … the source code to this network and all of the data and say, “Well, that’s the explanation.” Well, obviously that doesn’t tell me much. Maybe all the information is somehow in there, but it doesn’t tell me much. So, how do you decide what counts as a reasonable explanation?

Jim: Yup.

Melanie: And it’s not clear that we’re going to be able to build explainable systems that have the same kind of success, in terms of accuracy, as these deep neural networks.

Jim: I can give you a real world example. I sometimes do a little bit of Facebook advertising for my podcast episodes. Frankly, more to learn demographic information who’s interested in it than to drive a huge amount of traffic. But it’s interesting that I’ve had episodes, completely innocent science-only episodes turned by the Facebook advertising machine. By what … after I raised unholy hell, they finally admitted was a completely arbitrary black box signal which they couldn’t explain. And they had to reverse by human intervention.

Melanie: That’s strange, yeah. I know. Well, shows how we’d still need humans in the loop. And it’s frightening how much we’re kind of in many cases, giving autonomy to these algorithms to make decisions. About humans, like are you allowed to get on this plane? “No,” well why? Well, because the algorithm says, “No.” I mean …

Jim: Yeah, not only is it … I have to say that I’m just utterly pissed off that no one could explain. Because particularly, the one particular episode that kept getting returned, turned down again and again was the most pure science, nonpolitical, noncontroversial episode of them all. And so I said, “What the hell’s going on here!” And nobody could explain it. But the other thing that makes this even more potentially problematic … and you talked about this a lot. You give some beautiful examples. Is while old expert systems had their own kind of brittleness, deep neural nets are surprisingly brittle and can be easily fooled as well. You show some great examples where using adversarial software that intentionally generates trick images. Neural nets that were trained to pick out and classify images, a panda versus a houseplant, versus a steam kettle. You could give it pictures that kind of looked like static. They all said, “Ostrich.” I thought that was really interesting.

Melanie: Yeah.

Jim: And it might we’re talking a little bit about how at least at the present point in time, neural nets can be adversely defeated in perverse kind of ways.

Melanie: Right, so people have shown many ways in which you can fool neural networks, even the best deep neural networks that people are using today commercially. That if you’re sort of clever enough and know something about the way these networks work, you can create images. Either images that look exactly like the original image … just to humans, nothing’s changed. But you’ve secretly changed very slightly a few of the pixels, and now the network thinks as you said, it’s an ostrich rather than a school bus. Or else you can generate these very noisy looking images that don’t look like anything to a human, but the network thinks it’s object. It’s very, very confident that it’s some object. So, those are called adversarial examples.

Melanie: And they’ve been shown not only in vision, but also in … for speech recognition. You can fool your Alexa or your Siri system to think that some white noise is actually a command to it. And you can fool language processing programs to misread sentences, to mistranslate them in a way that you have targeted. It’s there’s all kinds of ways in which you can manipulate these networks. It really shows that they are not perceiving their inputs in the same way that we humans perceive them. That they haven’t learned the concepts that we meant to teach them. That they’ve learned something else, that allows them to statistically correlate inputs with outputs and do very well at it. Except they can be fooled in these very nonhuman like ways.

Melanie: And that’s caused a lot of concern in the community. There’s a big area of sort of adversarial learning now that people study and try and build defenses against these attacks. But it turns out to be quite hard.

Jim: Yeah, and you allude to it in passing. But I come to think quite strongly that the reason that we see this brittleness in these neural nets is because they’re almost exclusively feed forward. Meaning that signal … start at one end and pass through to the other end with no or very minimal backchannel. Yet we know in things like the human visual perception system, there’s a massive amount of feedback. And I think you quoted, and other people have said similar things, maybe 10X as much feedback as feedforward. And further in a lot of AI research where they introduce a little bit of recurrency, it tend to be very short-range. A storage loop, one loop. While in the human brain, some of the feedback is from very high levels to very low levels. It strike me that until we start including much more real long-range recurrency into neural networks, it’s going to be essentially impossible to solve that problem.

Melanie: I think that’s probably true. The problem is that neuroscientists don’t really understand the role of all those feedback connections in the brain. And if you try and include that kind of thing in neural networks, they become much harder to train. It’s one of those things that’s going to require some advances in neuroscience. To really get a better handle on what all those feedback connections are doing and how we learn their weights, and to try and apply some of that to artificial neural nets.

Jim: Yeah. And you mentioned, we think at least, that we don’t have algorithms to solve those weights. When I would argue that we’re kind of like the person that lost their card, keys. But they’re looking under the street lamp, because that’s where the light is. Even though that’s not where they lost their keys. And that the equivalent of the street lamp is the obsession with the gradient descent type algorithms for tuning weights. They work really good when they work, but there’s a lot of things like highly long-range recurrent architectures where they don’t work at all. Yet we do have other methods, and you alluded to them again just in passing.

Jim: One of my favorites, probably because it’s what I personally work with the most, is evolutionary solutions to neural weights. That will work on arbitrary architectures, it doesn’t really matter. And it can actually be used to discover not just weights, but architectures too. And there’s been a growing, still tiny, but growing amount of work that shows for in many problem domains … or in some problem domains. I should many yet. Evolution works as good or maybe better than gradient descent, and yet it clearly works in domains where gradient descent can’t get any traction at all. Do you have any thoughts on whether evolutionary approaches might be able to open up these kind of macroarchitectural questions?

Melanie: Well as someone who wrote a book about genetic algorithms, I’m really excited to see some renewed interest in this field. People have been looking at genetic algorithms in some form or another since almost the beginning of the AI age. But they never really worked that well on a wide set of problems. And people kind of decided that, “Oh, evolutionary methods don’t work.” So, there was not that much support for them, not that much interest. But I’ve seen in the last like few years some renewed interest in that area. And some renewed successes that kind of come like the success of deep learning, because of the amount of computing that’s available to us now. So it may be that evolutionary methods were a good idea all along, but just never had the amount of computing that was needed.

Melanie: I was recently at the Neural Information Processing Systems Conference, which is the biggest international machine learning conference. And there were several keynote lectures that mentioned that ideas from evolution and ideas from artificial life should be looked at again, in the context of AI and machine learning. So, I think there’s … it’s going to come back. And I think that that’s maybe one of the next frontiers.

Jim: Yup, I’m with you. It seems to me that it will allow us to explore architectures, that the gradient descent, feedforward people just can’t get to. And that will be a good thing. And I really think you’re right, that a lot of it’s going to be about the fact that people are going to wake up one day and go, “Holy moly, highly parallel stuff has gotten really, really cheap. Clock speeds on chips aren’t much faster than they were 10 years ago, but the number of cores per dollar continues to grow at Moore’s law. If there’s anything in computer science that’s embarrassingly parallel, it’s evolutionary algorithms, right?

Melanie: Yeah, exactly. And companies like Google and Uber are taking this very seriously, and kind of looking at evolutionary methods now. So, that’s kind of exciting.

Jim: Yeah, Uber’s had Jeff Clune out at their place for quite a while. And it’s funny, he says, “Oh, I don’t use the word evolution.” But you know that he still understands that’s an important part of his toolkit.

Melanie: Oh, yeah, and another guy, Ken Stanley is out there. And who does use the word evolution, and has been working on evolutionary algorithms applied to neural nets for the last 30 years. So, I think that there’s some interesting stuff in that area.

Jim: Good, I’m going to go and see … look at the NIPS proceedings this year and pick out some of those evolutionary papers. And see what’s happening, because it’s something I’ve been waiting for for a long time. And again, in my own little way, I fool around with it at the margin. It’s surprising how often you can actually solve a problem. I used to … back in 2001, 2002 when I first started talking about them. I would say that evolutionary neural nets are the weakest possible method, but also the most general. They can solve most anything, though not necessarily very well or very rapidly, with better techniques like the Stanley stuff. From originally UT, I guess he’s down in Florida now, at least his academic home. The availability of massive, truly massive amounts of parallel computation. Its time may come.

Jim: Kind of in the same way that the deep learning, there really wasn’t that much magic in the algorithms. What really made the difference was people realized you could implement gradient descent on very cheap parallel architecture, in this case GPUs. And also somebody tried what would seem like … or a [inaudible 00:43:18] ridiculous transfer function, ReLU, and it turned out to work great! And it turned out to be particularly efficient on GPUs. So, sometimes those kinds of fortuitous fairly simplify technological underpinnings can really change the fortunes of a whole technical approach.

Melanie: Yeah, I don’t if you follow Rod Brooks on Twitter. But he’s a super well known AI robotics researcher. And just like a week ago, he had this long Twitter thread where he talked about how he’s betting on evolutionary computation as the next big thing in AI. So, that was pretty interesting.

Jim: That is interesting. Actually I don’t know if I follow him or not. I don’t think so. Because I don’t see … yeah, robot. Yup, I’m going to follow him.

Melanie: You’ll see his thread where he says if you want to revolutionize AI and make a lot of money, try evolutionary computation.

Jim: Cool, maybe it might be … and get me enough to go back to work!

Melanie: Yeah, there you go!

Jim: Because of all things I love, evolutionary AI is it. But I’m too old and too rich, and too lazy. So hell with it, leave it to the kids today.

Melanie: Yeah.

Jim: Oh, yeah, very interesting passage in your book, where you played deliverer of other people’s arguments about why AGI might never happen. That somehow thinking was something that only humans could do. You want to bring forth some of those arguments and then your views about them?

Melanie: So, what … I quoted some of Alan Turing’s arguments in his original paper on the Turing test. Where he is basically … lists a bunch of arguments against machines thinking. But then I talked about sort of more recent discussions about it. Where the idea that, could a machine actually be thinking? Versus is it just simulating thought? Whether that’s a actual great question or really an error in categories. So, I asked my mother. I kind of quoted a conversation with my mom in the book, where I talk to her. And she’s really adamant that machines can’t think, even in principle. Because brains are for thinking and machines are for doing mechanical things. Ergo, machines cannot think. Really it’s kind of a deep seeded philosophical almost religious view about thinking.

Melanie: And I think that in a way, that comes from the notion that there’s something more to thinking than just something mechanical, like our neurons getting input and firing. That there’s something more to it in the brain. And it’s really a complex systems issue, where to my mind it’s an issue of emergence. That we have all these very relatively simple, nonthinking components, namely neurons and connections, and so on. Out of that emerges this complex phenomena that we call thinking, that is not easily reducible to the sum of its parts. So, maybe … I don’t know if that’s exactly what you want me to get at. But that was a lot of what I was trying to get across in that chapter.

Jim: Yup, I think that’s really good. And the other thing, I think you again alluded to it in passing, but something I’ve kept my on pretty heavily. Is that I suspect at one level, some of the critique might have been right. In that all the focus just on what’s happening in the head may have left out some of the important parts of cognition. And the idea of embodied cognition and socially embedded cognition may actually be important for us to understand what human type cognition is really all about.

Melanie: Yeah, that’s a really important discussion. So most of AI throughout its entire history is … kind of seen intelligence as metaphorically, the brain in the vat. That you have this thing, this machine, a computer or whatever it is. That doesn’t have a body, is not interacting in a social group or in a culture, or anything like that. But it’s thinking and it’s rationale, and it’s perfect and it’s infallible, and all of that stuff. And that’s kind of the idea of superintelligence. That you’re going to have some machine that’s going to be … it’s never going to need to sleep. It’s never going to stop paying attention, because it’s bored and all of that stuff.

Melanie: But another view is that intelligence, at least in humans and all the animals that we know of, it’s embodied. That we … it relies very much on the body that you have, and the way that body interacts with the environment. And all these things like needing to sleep, getting bored, having emotions are actually a big part of intelligence. And this whole social aspect is very big too. So I think that view, it’s been around also, for forever. But it’s getting some new traction, as we see really the kind of limitations of the brain in the vat view of AI.

Jim: Yeah, one of the guys I really like following, for a long time, is Antonio Damasio. Kind of his core book is The Feeling of What Happens. And he’s a both neuroscience, but also where the body and the mind meet kind of scientist. And he makes a lot of really strong arguments, that it’s particularly around consciousness. Without the body, there really is no consciousness, without the sense of the body. I think we’re finding perhaps that that’s more and more true the deeper we dig, certainly encourage people to look in those directions.

Melanie: Yeah, but I think some people in AI would argue. And it’s with you and say, “Well, we don’t need consciousness. A machine doesn’t need consciousness to be intelligent.”

Jim: I would agree with that by the way.

Melanie: Okay.

Jim: Yeah, yup, I agree with that. And here’s my view on that, what … and now because this happens to be exactly the work I do. Is that almost certainly consciousness is not necessary for intelligence or even super intelligence. Which is probably a good thing, right? Because many of the dire scenarios about the computers taking over seem to have accidentally brought in the concept of machine consciousness, in addition to machine intelligence. However I suspect that mom nature discovered a very clever hack. In that consciousness happens to solve one of the deepest problems that AI researchers run into again and again, and again, which is the combinatoric explosion.

Jim: My model says that consciousness is essentially a fairly simple machine that forces decisions, more or less arbitrarily. But they’re not arbitrary, and this machinery of consciousness has been tuned over 200 million years at least, to be able to produce quite decent results in the real world. But it does it by basically brutally chopping with an ax all the possibilities that our computer programs get themselves tied up into shoelaces with. And that’s really what consciousness is for.

Melanie: Interesting. I don’t know if I agree with you that you could have intelligence without consciousness. But I think consciousness also is one of those words that people mean different things by.

Jim: Yeah, that’s one of the worst parts of that, of the consciousness science. It’s you have to spend two days figuring out what each person you’re talking to actually means by consciousness. I mean, you find some people that think only humans have consciousness. And if they’ve ever had a dog, they obviously don’t mean the same thing I do. Which where I say not only does a dog have consciousness, but so do all mammals. All reptiles, all dinosaurs, all birds, et cetera, and maybe even amphibians. But of course, that’s a big jump from humans only to, “Oh, maybe amphibians.” But we’re clearly talking about something different. So whenever you have a conversation about consciousness, you have to be very precise. We don’t have time to do that today, so we’ll have to moving on.

Melanie: Yeah.

Jim: Let’s go onto our next topic, which again you did a very nice job of laying it out for the layman. The idea of reinforcement or Q-learning, why don’t you tell people briefly what that is?

Melanie: So before I talked about supervised learning, this idea that neural network or other machine learning system gets labeled examples. Where I … a human says, “This is a dog, this is a cat. This is a lion, this is a tiger,” and so on. Well reinforcement learning is a way to learn without labeled examples, by allowing a system to take actions in some environment and then either get rewards or punishments occasionally. Not for every action, but occasionally. So when you’re playing chess, you might learn to play chess by making moves at random. And getting either a reward when you when win the game, if you ever win it, or a punishment when you lose. And then deciding how to adjust your probabilities for making these different moves according to that reward or punishment.

Melanie: So, there’s been a number of algorithms that people have developed for this so called reinforcement learning. Which is really a form of the kind of animal training people used to do to get rats to learn to follow mazes or do other kinds of tasks, by getting rewards like food at the end. Now if you translated that into computer programs, you can make these sort of numerical quote on quote, rewards that the system gets. And reinforcement learning, again it’s very old idea, but only recently has it seen a lot of big successes. The biggest one being the DeepMind’s machine AlphaZero that learned how to play Go and beat the world’s best Go players.

Jim: Yup. I think, and the big thing from my mind, is that the rewards could be some considerable distance from the action. Unlike supervised learning, where either you got the picture right or you got the picture wrong, right? In reinforcement learning, it might be like the game move, Go where there could be 300 moves before you found out if you won or lost. And we get back to the old allocation of credit problem, how do we in some principled way decide which moves contributed to the win or to the loss?

Melanie: Right, and that allocation of credit problem is what these reinforcement learning algorithms like Q-learning solve. And they solve that having the system not learn to predict whether it’s going to win or lose at each move, but to try and predict from one move to the next to get its predictions to align. So, it would take a little bit more technical explanation to convey that idea. But it’s a very simple idea. And as long as you have enough computer power so that the machine can say, pay games with itself while it’s learning for millions and millions of games, it actually is extremely successful in the game world. It’s not totally clear how well it’s going to translate to more real world problems, where you don’t have the very clearcut rules of the game and the very discrete idea of moves and winning or losing.

Jim: Yeah, and I think that point, that’s a very, very important point. In fact that work I’ve written, mentioned that I did back in 2001, was essentially self-play with genetic neural nets that learning how to play Othello. And they learned pretty well. So, it’s certainly doable. But the point about why it was able to work was there was only a limited number of choices at each turn. And typically, those numbers are surprisingly small. In checkers, it’s something like five or six. Chess, it might be 20. I think the estimate in Go, it might be 250. But the real world, it’s much, much higher than that. Even some games are much higher than that.

Jim: I also do some work on a very advanced military type war game called Advanced Tactics: Gold. And my rough calculation is that each turn, there’s 10 to the 60th. That’s 10 to the 60th power possible moves, and there’s no way anything like the kinds of Q-learning and reinforcement learning we have today could even start to get any traction at all on problems at that scale. And yet humans clearly can.

Melanie: Right, humans are very good at taking a situation that they’re in and sort of dividing it up into discrete concepts. We see a whole system of pixels in our vision, and we’re able to see objects, right? We say, “This is this object, this is that object,” and we see relationships between objects. And we kind of describe the situation in a very finite way. So if we could get computers to do that in a kind of more human like way, maybe they’d be able to learn using reinforcement learning in the way that we do. But it’s really a combination between perception and learning, and being able to chunk the world or to core screen the world in the right way.

Jim: And there’s something else too, that may be even beyond that. And what keeps coming back when I think about these problems and these … and AlphaZero’s a fine example. But the huge number of data sets that they have to have to get experience, somehow humans are able to generalize what they know from one domain to another, often called transfer learning. The greatest chess master of all time may have played 50,000 games of chess. AlphaZero would … hardly even proved at all after … I know they … maybe not quite true. But that certainly would have gotten nowhere near its level of expertise in anything less than many millions of games. So, there’s something going on in humans that’s maybe qualitatively different. That we don’t even yet know the answer to how to get to, beyond just core screening and objects, and things of that sort.

Melanie: Yeah, and I think that’s right. People are able to abstract very well. There’s a movie about … it’s called Searching for Bobby Fischer. Did you ever see that movie?

Jim: I know of the movie, I don’t believe I ever saw it.

Melanie: Yeah, it’s about a kid, a child chess prodigy who’s learning to play chess. And it’s kind of about his life. But there’s one scene in the movie where he’s being taught by this chess master, is his teacher. And they’re in the middle of the game, and then the teacher sweeps all the pieces on the board onto the floor and says, “Okay, put them back the way they were.” The kid can do it. This is evidently something that chess experts can do. That because they see the board not as just a collection of however many pieces, but they see it in terms of high-level concepts. So, the can reconstruct it because they didn’t have to memorize every single piece’s position. They meant … just knew the concepts. They could recreate it in terms of the concepts, so where the concepts are like small configurations of pieces.

Melanie: So somehow being able to parse the world that way into this set of high-level concepts is one of the things that we can do, that we haven’t figured out how to get machines to do. And that’s something that I’m particularly interested in, because I think that’s the heart of the ability to do abstraction and analogy.

Jim: Yeah, that’s a perfect transition to my next topic, which I just have the word understanding written down.

Melanie: Right.

Jim: You use that word a lot of times. And people castigate me sometimes when I ask, “Well, how close are you to language understanding.” And they’ll, “Well, understanding, what does that even mean,” right?

Melanie: Right.

Jim: But I think you and I both agree that it does mean something, even if we can’t define it as crisply as we’d like. I want you to take a whack at, what does it mean to understand? You started to get to it a little bit there earlier, but maybe take it a little further?

Melanie: Yeah, it is a difficult word. But I think it’s a really important one when we’re thinking about intelligence. So you can think of say, reading a story, a little paragraph or something about some event that happened. So, machines can do that. And you can even ask some questions about the story, and sometimes they can answer then. But the question is, did they understand the story in the way that we do? And often what psychologists talk about is our ability to construct internal mental models that simulate the situations that we encounter, that we read about. Or that we imagine and that we can use to predict the future. That it sort of brings in knowledge about the situation that we’ve had, stored knowledge about the world. So probably, I should give an example here.

Melanie: So, say that you read a story about a young kid running a footrace barefoot and winning. That’s a sketch of a story. Well, what do you understand about that story? Well, you know a lot about what a race is. And you know, did the kid want to win the race? Yes, of course the kid wanted to win the race. Because people compete in races, because they want to win. Did the kid come in first? Yes, that’s what winning means. Did the kid’s feet hurt after the race? Well, probably if he was barefoot. They hurt. Because when your foot encounters rough surface, that can hurt it. Was the kid wearing socks? No, of course that being barefoot means you’re not wearing socks.

Melanie: So when you just have all this infinite knowledge about the way the world works, that allows us to sort of run these mental models that can allow us to answer questions about the things that we’re faced with. So, understanding is a lot about modeling and being able to use those models to sort of access knowledge and predict the future. That’s kind of one attempt at trying to get some handle on that term.

Jim: Yeah. And one area where this really comes out is in the area of language processing, which of course is a huge area of investment right now, and scientific and academic research. And yet, you gave a bunch of great examples about the tools that we have today. They don’t really have anything like this kind of understanding. Maybe could you take people through some examples of the very impressive things that we have today in language processing, and how they don’t really show understanding?

Melanie: One of the more impressive things we have are translation programs. Like Google Translate that can take in text and translate it into any number of languages, hundreds of languages. But the program when you actually give it a passage, it’s not understanding the passage in the way that we humans understand it. And we can see that in some of the errors that it makes. So, it makes errors like if I … one of the examples I had in the book was a little story about someone in a restaurant. And the man stormed out of the restaurant. The waitress said, “Hey, what about the bill?” In one the cases, Google Translate translated bill into French as legislative document, right? Even though the context was a restaurant. Because it didn’t understand the story, it didn’t understand. It didn’t have a mental model of what a restaurant was and what a bill is in the context of a restaurant.

Melanie: Well, sometimes these programs can take into account some context. But they’re very far from the kind of understanding that we humans have, that lets us answer questions or … about what’s likely to happen when you’re in a restaurant and somebody mentions the word bill. And we also know things about where if the person storms out of a restaurant, they probably didn’t like their meal and things like that, sort of basic facts about the world and about other people. So, Google Translate’s really good at sort of the statistical mapping from one language to another. But it makes mistakes that no human would ever make, because of its lack of understanding.

Jim: This need for deep understanding and common sense is … it depends how … many people have been talking about it for years. And you mention it in your book. And it’s something I’ve kept out of the corner of my eye at least, so I don’t know anything … as much about it as I’d like to. Is the Psych Project down in Austin, could you tell people a little bit about what that is? What it may have achieved and what it may not have achieved so far?

Melanie: So, Doug Lenat is the founder of the Cyc project. He’s an AI researcher from way back. And he’s one of the first people that I know of in the field who really took this idea of common sense seriously. Who said, “We’re not going to get anywhere in AI if we don’t have machines with common sense.” That is the invisible knowledge that we all use in the world to understand language and other things. But his idea was that we would have this sort of encyclopedia of common sense. And that’s why he called his project Cyc, short for encyclopedia. The common sense would be logic-based language that humans, his graduate students or later his employees, would basically type in all of the knowledge that a machine would need.

Melanie: Such as, example, a person cannot be in two places at one time, okay? Or every person has a mother, or just statements like that, all of knowledge. So instead of being an encyclopedia like the Encyclopaedia Britannica or Wikipedia or something, this would be an encyclopedia or knowledge that is not written down anywhere. So he’s been working on this for, I don’t know, 40 years? Something like that, and it’s really hard to say how far they’ve gotten. Because I don’t think that … I think a lot of it’s proprietary. But it certainly doesn’t seem like the project of trying to manually input all of common sense, or having the machine deduce new statements from the statements that it’s already been given, is going to solve the problem.

Melanie: Because the system again, it doesn’t have the kind of mental models that are needed to kind of get the right knowledge out and predict the future usefully. And I think the other problem is that a lot of the knowledge that we have about the world is … we don’t even know we have it. And so, we can’t type it into the system. That a person can be … can’t be in more than one place at a time is sort of like that. But how many statements are there like that that we know? That would be really hard to list all of them. So, I don’t think it’s a very sustainable approach. But I do think that Lenat what very prescient in his notion that common sense is really going to be the core of intelligence, and I agree with him.

Jim: Yeah, I’ve heard some people say that maybe Lenat is just … took on a huge job that is doable. But it’s just bigger than he thought it was. So, maybe it’s 10 times or a hundred times bigger than the effort that’s gone into it. And that maybe it would be worth spending five billion dollars to build Cyc out, I’m not sure.

Melanie: I’m not sure either, and I think there’s other research groups now that are getting to the common sense business. The Allen Institute for AI which was founded by Paul Allen, a cofounder of Microsoft is now focusing on common sense. And I think they have not exactly an idea of Cyc, but they have this kind of crowdsourcing platform to try and get common sense assertions from the online world. They’re trying to mine common sense assertions from the web. Again, I don’t really think this is the approach that’s going to lead to common side effect. But it’s worth a try, I guess. Nobody really knows the right approach.

Jim: Another one in that genre, it was ConceptNet, where they tried to opensource this kind of stuff.

Melanie: Yeah, MIT. And people do use ConceptNet, and people do use Cyc for various projects. And they’re useful, right? They’re kind of know bases. And Google and Apple, and other companies have what they call their knowledge graphs. They’re not exactly common sense knowledge, but they’re sort of general knowledge about the world they use in search. All these things are useful, but none of them are kind of getting to the heart of what we mean when we talk about common sense. So DARPA, which is one of the big funders of research, has its own version of the common sense in AI project.

Melanie: They have this program called Foundations of Common Sense, that the grand challenge is to develop a program with a common sense of an 18-month-old baby. So they’ve enlisted cognitives in developmental psychologists, to work with AI people on getting machines to kind of learn the basic intuitive physics of the world. In the way a baby would learn it, but by watching videos and being in virtual reality, and all that kind of stuff. So, that’s another approach that it was pretty interesting. I’m not sure how it’s going to work, but we’ll see.

Jim: Yeah, my friend Josh Tenenbaum at MIT is working on approaches that sound a lot like that, aiming at the 18-month-old or two-year-old, incorporating intuitive physics. Folk physics if you want to call it that, to the degree they can assess what a 18-month-old actually does know. And again, it’s interesting. Well I don’t know if it’ll get there or not, but it is good that people are working on these things.

Melanie: Yeah, I think it’s great.

Jim: For our last topic, I know this is one that’s near and dear to your heart. And you’ve been working on it since you were a graduate student. And that is the other way to ground AI might be through what are called analogs, analogies, and metaphors. Talk a little bit about your thoughts in that space, maybe tell the story about your Copycat program.

Melanie: Right, so I got into AI. I was originally physics, math, astronomy, that was what I was working on as an undergrad. But after I graduate, I read Gödel, Escher, Bach, Douglas Hofstadter’s book about AI. And about consciousness and so on that inspired so many people to go into AI, and it also inspired me. And I actually sought out Hofstadter and convinced him to take me on as a grad student. The project that he gave me was to build a program that could make analogies. He had developed this idealized domain of analogy making that involved analogies between strings of letters. So, here’s an example. If the string A-B-C changes to the string, A-B-D, what does the string I-J-K change to? Most people would say I-J-L, the rightmost letter in the string changed to its successor in the alphabet. So, change I-J-K to I-J-L.

Melanie: And say, “Okay, same original change, A-B-C to A-B-D. What does I-I-J-J-K-K change to?” And there, people would say, “Oh, well I-I-J-J-L-L.” That is, they group the letters. So they don’t follow the original rule literally, but they make a little change. Because now we don’t have a string, a sequence of letters, we have a sequence of groups of letters. Then I can go on and on, and give you hundreds and hundreds of these kinds of analogies that require different changes. What we call conceptual slippages, like from letter to group. Or you can do other ones where you go from successor to predecessor, or any number of conceptual slippages you can make.

Melanie: So the idea with this project wasn’t necessarily to focus on strings of letters, but to use the strings of letters as kind of an idealized domain think explore much bigger issues. About how you take the knowledge you have and apply it to new situations, and how you see abstract similarity and so on. So, I built this program called Copycat. Because when you do the same thing to a new situation, you’re being a copycat. And Copycat was able to do lots and lots of these letter string analogy problems. But very much, it was kind of a mixture of symbolic and subsymbolic AI. And it only got so far, and then I started working on other things.

Melanie: But I still feel like a lot of the ideas in that original project … which was done in the 1980s and 1990s. I think like genetic algorithms and like neural nets, there’s a role for those ideas that kind of coming up again 30 years later. So, I’m excited about thinking about those idea again and trying to apply them to more general domains.

Jim: Do you have some examples of things you’re currently working on?

Melanie: Well, one of the things I’m working on is visual analogies. So if you want to say, recognize a visual situation. And one of the examples we use often is a person walking a dog, okay? Well, you could try and recognize that there’s a person. They’re holding a leash. The leash is attached to the dog, and they’re both walking. You could maybe recognize all those things in say, an image or a video. Okay, and so now you’ve learned let’s say, what walking a dog is. But then I show you a person walking 10 dogs or a person running with a dog, or a person riding a bike with a dog on a leash, or any other kind of variation on this theme. And humans are really good at taking something they’ve learned and applying it analogously in new situations.

Melanie: But this is something that machines, as you said earlier, are not very good at. We have this problem of transfer learning, where systems learn in one domain and they can’t transfer what they learned to a new domain. But transfer learning is another word for analogy. And I think that this big problem of being able to apply knowledge to new situations by making analogies is what’s going to get us out of the problem of the long tail problem, that we talked about earlier. And the problem of transfer learning, that so many people are struggling with now.

Jim: Interesting, now the other very closely related field is the area of metaphors. Particularly a lot of us know the work of George Lakoff. I remember when I first started reading Lakoff, I said, “Oh, he’s got to be exaggerating. It can’t all be metaphors.” Meaning language, but he’d take down some very interesting pathways. And you realize that it’s hard to say a sentence, or and certainly to say two sentences, without using at least one fairly significant metaphor. Can you tell us a little bit about what that’s about, and what might be going on in that space?

Melanie: Lakoff and Johnson wrote this book called Metaphors We Live By. And their idea was that our language, just our everyday languages, is filled with metaphors. So as one example of if I say, “She gave me a warm welcome.” That, we’re using the word warm metaphorically. Because it wasn’t literally hot in the room or anything, but it’s more of a social metaphor. The theory is now we’re actually interpreting it by using our model of physical temperature, our mental model of physical temperature. So they go into all these different examples of how we talk about abstract concepts like time and money, and love, and we understand them in terms of physical metaphors.

Melanie: So, that’s a really fascinating study. It’s kind of linguistic metaphors. And I’ve seen a lot of followups that I found really convincing, that show that we really do structure our concepts in terms of very physical metaphors. We talk about if, you might say, that speech was very uplifting. And we say, “Okay, up is good, down is bad.” Or if you say something like, “I fell in love,” we think about falling. Being out of control and all … we apply these physical ideas to much more abstract concepts. So, I found that really fascinating. I think it’s something that if we have a machine that can understand language, it’s going to have to be able to understand this kind of metaphor by drawing on mental models of these more physical concepts.

Jim: Yeah, I know one of my favorites, kind of a double metaphor is, “Let’s move forward with this project.” Forward, I would argue, and I think Lakoff does, comes initially from moving forward and backward in our body. Which then was projected on to time, forward and backward in time, which is a metaphor from our body walking forward and backward. And a project moving forward, as in making progress … is that another projection probably from the projection of time. And so, we have the stack of metaphors that actually get us to the language as we actually use it.

Melanie: Yeah, that’s a good point. And it’s invisible to us that we’re making these metaphors. We don’t even notice it, right? When we talk. But there’s been some work in psychophysics and neural imaging, and other psychology kind of experiments that show that people really are the … activating the neural areas that encode the physical concepts. That we’re not aware that we’re even referring to. But unconsciously, they’re actually quite active and influence the way we understand things. So, it’s really interesting.

Jim: Now, can you see a commonality or maybe something that brings together the idea of analogies and metaphors?

Melanie: I think they’re related. Certainly analogies are about seeing similarities, sort of abstract similarities between different situations. And metaphors are about, I don’t know, bringing in often physical concepts into abstract situations. And perceiving them as being similar in some ways. The origin of many metaphors is a little obscure, I don’t know how some of them came about. But that kind of gives some clue to how thinking in general and origin of language, and even maybe the origin of abstract thinking. So, I think it’s a fascinating field. But really very interdisciplinary, linguistics, psychology, neuroscience and AI I think are all needed to try and make sense of it.

Jim: And even throw in biophysics, right? To the degree that the groundings of the metaphors are in biophysics, it’s be really nice to know how those work.

Melanie: Yeah.

Jim: Besides your work, who else should we be looking at in the world that’s … let’s just limit the work to analogies. Since that’s really closer to what you’re working on today, who else should we be following?

Melanie: Well, there’s a lot of different approaches to analogy. There’s a group at Northwestern, Ken Forbus and Dedre Gentner, and they’re colleagues who have looked at a more symbolic AI approach to analogy making. And that work is very extensive. Very different from the work that I’m doing, but definitely very prominent in the field. There’s actually some of the people at DeepMind, the group that did AlphaGo and AlphaZero are looking at how to do analogies with using neural networks. And they did some interesting work on visual analogies. I’m trying to think, there’s quite a few people who are doing some interesting work in this area. If I can think of any more, I’ll definitely send you some links to post.

Jim: Yeah, that’d be great. Because that’s what … one of the things people say they find most useful, is use the podcast as a way to find other things to learn about.

Melanie: Yeah.

Jim: So, that’d business wonderful if you could send those to us.

Melanie: Sure.

Jim: Well Melanie, I think that’s about … we’re up on our time limit now. And I got to say, this has been a wonderful conversation. And I can strongly recommend that people go out and get your book, and been a great conversation!

Melanie: Well I’ve really enjoyed it, thanks so much for having me on.

Jim: Yeah, I didn’t necessarily go everywhere I thought it might, but that’s part of the fun of these things.

Melanie: Yeah, well I think we covered a lot.

Jim: We certainly did. Production services and audio editing by Jared Janes Consulting. Music by Tom Mahler at