The following is a rough transcript which has not been revised by The Jim Rutt Show or Seth Lloyd. Please check with us before using any quotations from this transcript. Thank you.
Jim: Today’s guest is Seth Lloyd. Seth is Professor of Mechanical Engineering at MIT. He was also chosen for the very first class of Miller fellows at the Santa Fe Institute. The Miller fellowships are our most prestigious visiting position at the Santa Fe Institute. Seth is the author or co-author of over 300 science publications, that’s holy shit. That’s a big ass number. And is the author of a popular book, Programming the Universe.
And in fact, Seth is a returning guest. We had him on in EP 79 titled, On Our Quantum Universe, where we talked about his book, Programming the Universe. So welcome to the Jim Rutt Show, Seth Lloyd.
Seth: Thanks for having me, Jim.
Jim: Yeah. This should be very interesting. This is a topic that I have been interested in for 25 years and have never had a solid answer in my head about it. And that is how do we measure complexity? As it turns out, there are lots of ways to measure complexity. Why is it so hard to say this is the measure of complexity?
Seth: That’s a good question. When I started working on measuring complexity back in 1986 or ’87, my supervisor at Rockefeller University, Heinz Pagels came in and said, “Hey, Seth, let’s come up with a mathematical measure of complexity. So you can be like Spock with a little funny camcorder and coming and go, “Whooo-ooo.” “Oh, this is very complex.” “Oh, no. It’s not complex at all, right?” “No. There’s a 73 complexity, 74.”
And I answered, I said, “But isn’t complexity supposed to be just things that are hard to characterize and measure, so maybe it’s not possible?”
Jim: Yeah. I have to admit, as a baby complexity dude, when I went out to the Santa Fe Institute in 2002, I assumed I would just sit down with someone and they would tell me the secret to how to measure complexity. But of course, I sat down with 20 people, I got 20 different ideas.
Seth: Yeah. Well, actually, yeah, I was a professor at Santa Fe Institute starting in 1988, I started writing about measuring complexity. And I gave a talk there called 31 Measures of Complexity in honor of the Baskin-Robbins ice cream company with its 31 flavors of ice cream. And I think there were considerably more than 31 measures of complexity. But the world is complex and there are very few things out there that are truly simple, and even things that are truly simple, like maybe an electron is truly simple, but it still requires a lot of complex theory to understand it.
Jim: And three electrons becomes complex, right?
Seth: Oh, yeah. Absolutely. Yes. There’s the famous three-body problem of gravity, just even in ordinary Newtonian gravity, if you have the earth and the moon, that’s fine, and that you can figure out what happens. But if you have the earth, moon, and then a satellite going around and they’re all interacting with each other, that becomes the three-body problem, and then all bets are off.
Jim: And that’s about the simplest thing that we could describe, but it gets way worse from there.
Seth: Yeah.
Jim: Think about the complexity of the metabolism of bacterium, for instance.
Seth: Yeah. Exactly. And so the metabolism of bacterium is a great example because that is clearly, really complex. There are thousands of chemical reactions taking place, there’s all these feedback loops, they’re all interacting with each other, and they’re trying to give the bacteria the energy it needs to do it’s bacterial things, like having sex, and eating, and reproducing, and stuff like that.
And by any measure, it should be very complex. But how do you actually measure that? Do you measure the number of different interlocking chemical interactions? Well, that would certainly call it to be complex. It’s probably at least that complex. Do you have to measure the level of every single chemical in the bacterium, including in different places within the bacterium?
Now you’re starting to talk about very large amounts of information required to specify what’s going on. But coming up with one number to measure the complexity of the metabolism of bacterium is a tough thing, not even clear it’s the right thing to do, not even clear it’s morally correct.
Jim: So fundamentally, the problem of measuring complexity is that complexity is more a very broad class of things rather than something you can put your finger on and say, “This is an electron. It can be applied to many different kinds of systems, which are qualitatively different.” Is that fair enough?
Seth: Exactly. And what I discovered back in writing about these 31 Measures of Complexity, it was actually pre-internet, and I just went to the library, the basement of the library at MIT for a month, and I just looked up everywhere I could find about measures of complexity.
And what you find is that pretty much every field out there has its own set of measures of complexity that work well for it. I mean, for example, in computer science, the main measures of complexity are, how many logical operations does a computer have to perform in order to execute a particular program? Those kinds of questions are… You can give a quantitative answer to them, but then it becomes hard to take that, and number of logical operations, amount of memory space, how do you translate that over to the bacterium and its metabolism, right?
Jim: Yeah. In fact, again, back in the days when I was a very baby complexitory and one of the earliest books I read was Origins of Order by Stuart Kauffman. And of course, he lays out there his famous thesis that at least the kind of complexity he was interested lived at the border between disorder and order, and-
Seth: The edge of chaos, as they call it.
Jim: Yeah. Though I stopped using that because chaos has a more specific mathematical meaning, which clashes with his thinking, I think, where he really meant disorder and order but got shorthanded to the edge of chaos, right?
Seth: Right. I mean, that’s a very nice notion and it captures one of the features about complexity, why it’s hard to make a measure of complexity. So you could say that something that is very ordered, very regular, and simple, like if you take physically a salt crystal with sodium chloride, sodium chloride, sodium chloride, sodium chloride, sodium chloride, it’s a very simple, there are a lot of atoms, Avogadro’s number in roughly a gram, but they’re ordered very simply.
And so actual description is simple and it’s highly ordered, you say, “Okay. It’s probably not very complex so it might support complex vibrations and things like that.” And if you have the same thing for a number, there could be a number that’s like one, one, one, one, one, one, one a billion times, that’s highly ordered. And you’d like to be able to say, “Okay, if you coming up with a measure of complexity, you should say, ‘Well, those very ordered things are not complex.'”
And indeed, things that are very ordered like that can be described very simply in the sense, for instance, in one, one, one a billion times, there’s a very short computer program that just says, “Print one a billion times.” And it will produce a billion ones.
So it’s got kind of simple what’s called algorithmic complexity. It’s algorithmically simple. And the salt crystals the same way, can be described very exactly, precisely, and simply. But it’s not the description length and how hard it’s described can’t be the only thing because if you take something that’s completely disorder, like the state of the molecules in this room at a microscopic level, they’ve got a vast amount of entropy, which is actually information required to describe where this molecule is, where it’s moving, where this other molecule is, where it’s moving, where this other molecule is, where it’s moving.
You’ve got a gajillion molecules in this room and it requires a gajillion bits of information to describe it, but in some sense, it’s very simple. I mean, it’s like highly disordered but so what?
Jim: Yeah. The example I like to use on that, because we’re actually now talking but I’m going to go through a list of about 15 different measures of complexity.
Seth: Oh, great.
Jim: I took your 31, I did some other research and came up with a group of 15 or 16, and each one has a different set of pros and cons. One we’re talking about now are algorithmic complexity, also known as Kolmogorov complexity, if I pronounce that correct.
Seth: Exactly.
Jim: And as we were saying, it seems to accurately catch the sense that one, one, one a billion times is pretty simple because you can write a three-line program to generate that.
Seth: Right.
Jim: On the other hand, you would use the example of molecules in the rooms. The other one I use because people have seen it a lot, is static on their TV screen. Right? Essentially random pixels. The program to describe any such program is essentially just an elucidation of all the pixels.
So it’s like if you have a thousand by a thousand raster, it’s a million essentially statements about on or off. And there’s no way to compress that if it’s truly random. And so it has the highest Kolmogorov complexity, but it doesn’t at all touch on the kind of complexity Stuart was talking about.
Seth: Exactly. Right. We want things that are complex should require a lot of information to describe, but they shouldn’t be random. If you just flip a coin say heads is zero and tails is one, and you just come up with a bit string, 0110010101 and keep on going, that has very high algorithmic complexity because it has no pattern in it.
So you flip it a billion times, and the shortest program to reproduce that is just something says print 01101011 et cetera, with a list of all the bits in it. So it’s got huge algorithmic complexity. It’s just some random number. Nobody would think that it’s particularly complex and some more intuitive way of complexity.
Jim: Now until we get to another measure which was popular at one point, think probably less so today, and that’s Shannon entropy.
Seth: Yeah. So Shannon entropy is really, I think that one should probably just call it information. This has a beautiful history in the sense that back in the 19th century when James Clerk Maxwell and Ludwig Boltzmann and Josiah Willard Gibbs were trying to figure out what entropy was.
Entropy at that point was like, ooh, some weird feature of systems where hot, systems have more entropy and cold systems have less entropy. And so if you’re trying to build a steam engine, you got to… And entropy doesn’t decrease and it tends to increase.
And it’s some measure of randomness at microscopic randomness. And they came up with a great formula for this, Maxwell and Boltzmann did and Gibbs [inaudible 00:10:20] to describe what entropy was. And then about 50 years later in the 1930s, Claude Shannon, he was an engineer at Bell Labs at the time, he came up with a formula to try to measure information for communications.
For communicating by telephone lines or radio, and he came up with this mathematical formula, and it was exactly the same formula as the one that Maxwell, Boltzmann and Gibbs had come up with half a century or more earlier. And so it realized that actually entropy is the information required to describe the positions of atoms and molecules. And conversely, Shannon entropy for a bit string, is the amount of information required to describe that bit string if you take into account all the regularities in the bit string.
Jim: Yeah. So it’s kind of a way to compress the amount of regularities, but also the irregularities in describing the system.
Seth: Yeah. Exactly. Right. Because actually Shannon was very interested in the question, if you have a message that you want to use your bandwidth as efficiently as possible. And so when you’re encoding information, you want to compress it in a way that can be decoded at the end.
And so for example, this bit string, which is a billion ones, is easy to compress because actually I can just say, “Hey, consider the bit string, which has a billion. I’m now going to send you a billion ones.” But instead of sending them all one by one, just like say, “I’m going to send you, here’s a billion ones.” Right?
Jim: Yeah. Exactly. All right. Let’s go on to another one, which is Charles Bennett’s logical depth.
Seth: Well, that’s a beautiful measure of complexity. So this is actually, and this was the logical depth when I was trying to look for a physical measure of complexity, which is what Heinz Pagels had challenged me to do when I was a PhD student. And I was reading about these different measures, Kolmogorov complexity, Shannon information, entropy, and then I read Charlie Bennett’s beautiful paper about logical depth.
So Charlie said, “Oh, look, things that have low Kolmogorov complexity but are easy to produce like a billion ones, those aren’t complex, but things that are like just flipping a coin a billion times, creating a random bit string has very high Kolmogorov complexity, so that shouldn’t be complex either.” But he said, “But what about something like say the first billion digits of pi, 3.1415926, et cetera?” Now all I can remember, my way from remembering the digits of pi is you count the number of letters in the following phrase.
It goes how I want to drink alcoholic, of course, after the heavy labor is involving quantum mechanics.
Jim: Ah. That’ll get you about a dozen digits in. Around MIT, you just ask in your class, “Hey, did anybody know the first a hundred digits of pi?” And people will start to spouting on [inaudible 00:13:04]
Seth: Oh, yeah. Of course. Of course. I know more. I know a thousand.
Jim: Get a life, dude.
Seth: Yeah. Really.
Jim: Get a girlfriend.
Seth: So pi is a very specific number. The digits look kind of random, but there are lots of patterns in it because a very specific thing. So what Charlie said is, let’s look how hard it was to produce pi, assuming that we’re starting from a short program for producing it.
I mean, a famous program for producing pi, which was known by the ancient Greeks. Right? You inscribe a regular polygon inside a circle, and then you inscribe a regular polygon with the same number of sides outside a circle, and then they knew how to figure out the size of those triangles. So you have an upper bound to the pi, and you have a lower bound to pi. And as the number of little triangles gets bigger and bigger, it gets closer and closer. So they actually figured out ways to get increasingly good rational approximations to pi that way.
Jim: It doesn’t scale very well. The old sieve of Eratosthenes, right?
Seth: It does not. Right. It does not scale very well, but it is still pretty efficient. And so what Charlie said is let’s look at the shortest program the simplest way or one of the simplest way of describing something. In this case, the first billion digits of pi.
And look at how many steps a computer has to take in order to produce the output. And if it takes a large amount number of steps, then we’ll say that it has high logical depth. And this is nice because if you look at one, one, one a billion times it’s easy to produce because you just had your computer printing and prints out a billion ones. It takes a billion steps, but it’s the same length as the program.
And similarly, if you take just a random bit string, 01101101, et cetera, my students tell me, “I always come up with the same random bit strings, so it’s not a very random bit string.” and then the program is print 0110110111, and a list of all those digits. That also runs very fast because it’s just whips out the digits. But to print out the first billion digits of pi by a short program, like the Sieve of Eratosthenes will give you a very short program, takes quite a long time. As you mentioned, it’s not very efficient to produce pi that way. So then these first billion digits of pi would be, or first billion bits of pi would be considered to be something that’s logically deep.
Jim: And that applies presumably to other things that are similar nature. Things that people who read in popular science may have run across are the famous patterns of cellular automata, right?
Seth: Oh, yeah. Absolutely.
Jim: That our Melanie Mitchell did a lot of work on, and Stephen Wolfram stole some of her work, probably, and then continued on with his own work and many other people. And what was it? 121, one of the famous patterns that very, very looks extremely complex, and indeed, there’s no way to produce it except to do all the steps.
Seth: Yeah. That’s right. I forget which cellular automaton it is. I guess 126 or something like that. We could look it up, which is computationally universal in the sense that if you put in the right initial program for the cellular automaton. And cellular automaton, of course, it’s just a computational structure that is homogeneous.
So you just have a bunch of cells, each cell gets updated as a function of its own state and the state of its neighbors. And so it’s like a computational version of a gas or something like that with molecules bouncing off of each other. And depending on the rule that you have, they can either be very simple or they can be very chaotic and very random, or they can do things which looks really complicated or complex.
And this one rule that you’re alluding to, I believe is the rule which was discovered many years later to actually be arbitrarily complex in the sense you could program any computer program into it. So for instance, you could program the cellular automaton to calculate the first billion digits of pi.
Jim: I just looked it up. It’s got old rule 110.
Seth: 110. 110. Thank you. So that actually has the Stuart Kaufman’s Origin of Order book. There you have simple rules, but they can either create very regular behavior like one, one, one, one, one, one or they can create very random looking so-called chaotic behavior, but some of them have this, which sit in this region between order and chaos as this edge of chaos concept. And these can produce arbitrarily complicated kinds of patterns.
So these patterns that they’re producing are also effectively logically deep, because if we agree that the first billion digits of pi are logically deep, and you can program the cellular automaton to produce the first billion digits of pi after a very, very long time evolution, then it can do arbitrarily logically deep things.
Jim: Now the next one on my list, two characters named Gell-Mann and Lloyd, effective complexity.
Seth: In my PhD thesis, when I was trying to make a physical measure of complexity, logical depth as we just describe it is a measure of complexity that refers to bit strings and computers. It’s like if you think of how long does a computation that starts one of the shortest programs that produce a bit string, how long does it take to do that?
So Heinz Pagels and I, for my PhD thesis, we defined something which we called thermodynamic depth, which was a physical analog of logical depth. And basically we said the logical depth is how much computational resources does it take to describe how something was produced from a simple description, and thermodynamic depth is simply how many physical resources like free energy that you need in order to make things happen? How much free energy did you have to consume and burn up in order to put this system together, starting from the way it was actually put together?
This actually is nice because you can define it mathematically precisely and physically precisely. And then it’s hard to say what is the logical depth of the bacterial metabolism? That’s not clear. But we know that the thermodynamic depth of a bacterial metabolism is humongous because it actually took billions of years and a whole lot of evolution and gajillions of bacteria had to sacrifice their lives in order to get to the point where natural selection would produce this amazingly complicated bacterial metabolism.
So we know that these bacteria, they’re very thermodynamically deep. Those fit in nicely, and you can make them, you can really join them up the physical definition and the computational one, then you can join them together via the physics of computation, the physics of how computers work to show that they’re essentially the same thing when they’re overlapped.
So effective complexity, this is Murray Gell-Mann and I came up with this idea. It was another one of these occasions like Murray, who I know that you knew Murray well, and I worked with him closely for many a year. He could be very frustrating as brilliant genus, noble, laureate, discoverers of quark go. He could still be very frustrating to talk with.
It would take a long time. I’d be like, “Okay. Murray, we’ve been working on this for a year and a half, I think we’ve got good results. Let’s publish it.” “Oh, no, no, no. It’s not good enough. We have to wait for another six years and then fail to publish it entirely, right?”
So anyway, yeah. So effective complexity, remember I was just describing about the difference between logical depth and thermodynamic depth. So logical depth is a computational measure of complexity, and thermodynamic depth is a physical measure of complexity, but one which is inspired by the computational measure.
So effective complexity is a measure that combines these two ideas that we were talking about, physical notions and computational notions. And we’ve actually talked about both of these notions because the physical notion that it uses is entropy, random motions of atoms and molecules. And the computational notion that it uses is this idea of Kolmogorov complexity or algorithmic information.
So if I say the effective complexity of the molecules in this room as I wander around is we divide up the description of these molecules into two parts. One is the entropy, which describes all their individual motions, their random motions and stuff like that. But the effective complexity is the algorithmic part to describe them.
It’s like, “Okay. Here’s the percentage of nitrogen, here’s the percentage of oxygen, here’s the percentage of carbon dioxide, here’s the temperature in the room, here’s the pressure, here’s how much air is flowing in, here’s how much air is flowing out.” And then we describe, and we can describe in a more macroscopic sense what’s happening in this room.
That amount of information is much smaller than the amount of information at the random motions of the molecules themselves, which it’s a vast number, like 10 to 30 bits or something like that. But the effective complexity of the gas in this room is going to be something more on the order of a few 10 or a hundred thousand bits where you describe what the molecules are, how the molecules interact with each other, what kind of chemical reactions they can go through, what their temperature and pressure are at a more macroscopic level.
So effective complexity means, “Okay. We don’t care about all that random stuff, that’s just random, but we do care about the non-random stuff. We need to describe it.” So basically, Murray and I came up with a procedure for saying, “Okay. What’s the non-random stuff that we need to describe and what’s the random stuff we need to describe?” And the effective complexity is the non-random behavior, which of course can get arbitrarily complex.
Jim: Now if you were thinking about bacterium, how would you make that division?
Seth: Yeah. So that’s a very good question. So I mean, you need to say, “Okay. What do I mean by a bacterium? Do I mean a specific type of bacterium?” Yeah. Let’s say it’s E. coli of some sort. And let’s say it’s E. coli, let’s say just like to sacrifice myself, E. coli taken from my own gut, right?
And then, okay, it’s E. coli taken from my gut, and here the diet, here’s what it’s been eating today because here’s what I had for breakfast with my toast and eggs and jam and butter and coffee and stuff like that. And so here’s what the E. coli took in. And then the effective complexity for the bacterium is the description you need to make of the organization of the metabolism of the bacterium that allows it to do what it has to do.
Take in food, the type of food it’s taking in, how it metabolizes it, what these chemical reactions are, how much energy it gets from it, what it uses that energy to do, and which in the end for bacteria is reproducing. And then you would need to describe that process, which is a complicated process, right?
Many thousands of chemical interlocking, chemical interactions, thousands of species of chemicals, you probably have to describe the metabolism. You’d have to describe much if not all of the DNA of the bacterium in order to do that. So it could take the chemical reactions, the types of chemicals could take hundreds of thousands or millions of bits.
The DNA takes a few billion bits, but then you’re not interested in how the little atoms and molecules are moving around in the bacterium. You want to say, “Okay. Here’s the cell wall, the lipid membrane surrounding the bacterium. Here are the structures inside of it that are opening up channels to take in information and food and to put out other kinds of information and waste.”
So you want to describe what those channels are, but you don’t care the exact molecular configuration of them or how the molecules in them are wiggling around. So the effective complexity of a bacterium would still be a very large number, many billions of bits of information, assuming we have to describe pretty much the whole DNA of the bacterium, describe how it’s working, and all the mechanisms that it entails.
So it’s a large number, but it’s much, much smaller than the Avogadro’s number amount of information that would be a billion, billion, billion, billion bits that would take to describe all the motions of the molecules and atoms moving around inside the bacteria.
Jim: Now when you do that separation between the randomness and the structural components, is there a principled way to do it or is it inevitably going to be subjective?
Seth: Yeah. So that’s why I would say in order to define the difference between the important information, the effective information, and the unimportant one, you have to define what’s important. You have to say, so we could be subjective for that. For the bacterium, we say this E. coli lives in Seth’s gut surrounded by other bacteria and other crap.
Jim: Lots and lots of crap. High-grade crap, mostly, right?
Seth: Lots of crap, but it’s got to reproduce. Right? It has this whole function. And in order to reproduce it has this whole necessary metabolic function. Actually, if it gets screwed up, it doesn’t reproduce. So once you say, “Okay. We want to define it. It is a bacterium. It lives in my gut. Here’s what its surroundings are like. Here’s what it needs to do in order to take in food and to reproduce, make new bacteria.”
The effective complexity has to encompass at least that. Then you get a measure, which it is subjective in the sense you have to say, how much information do you need to describe to say, “Well, the bacteria is living this. It’s like taking in glucose and these fats and these other things, and then it’s reproducing.” It is effective for a purpose. So once you describe the setting and the purpose, then you can define effective complexity precisely. You make the distinction between information that’s important for that bacterium and information that’s not important for that bacterium.
Jim: Yeah. I think the big takeaway from me from this discussion is that not only do we have many kinds of complexity measures, which are more appropriate for some applications than others, but even in the case of one complexity measure, there’s still a fair bit of what is the purpose question before you can actually run the calculation.
Seth: Yeah. Right. I mean, actually, you, Jim and I are both long-term associates with the Santa Fe Institute and people there blather on about complexity all the time. That’s why the place was founded in the first place. And so you don’t necessarily have to talk about complexity just because it’s there. Is it useful to talk about complexity?
And so actually with the bacteria, you could say, “Well, the bacteria has to have some kind of minimal metabolic complexity in order to do what it needs to do.” And that’s the kind of point that you want to make, that you want to tease out. It’s like the bacteria is trying to perform all these different tasks. It needs to perform all these different tasks. So what’s the minimum effective complexity to describe the mechanism whereby it’s doing that?
Jim: Let me throw out something, see what you think. Let’s suppose we want to use the lens of ecology. How does a bacterium or a class of bacterium fit into an ecosystem of other micro and macro organisms? Then that perhaps informs where you want to draw the line for the regularities that you’re concerned about because you’re not really concerned about the inner jiggerings of the chemistry, but you are interested in the membranes and the inputs and the outputs and who eats who and that kind of stuff.
And that would give you a different, perhaps informing where you want to draw the line for regularities as opposed to if you want to define how does metabolism work in the mitochondria, for instance.
Seth: Right. Exactly. Right. And in fact, if you go and read Murray’s and my papers about effective complexity, we use frequently an idea that was developed in the 19th century by people like Gibbs and Maxwell, which is the idea of coarse graining. Coarse graining actually refers to photographs, prior to digital photographs where you have grains of silver nitride in the photographic film and the finer the grains, the better the quality of the photograph, and the coarse of the grains the less good the quality of the photograph.
So a coarse grain description of a system like a bacterium then say, “Well, here’s the information we’re going to leave out. We’re only interested in these features of it.” So it’s important into doing effective complexity to define the level of coarse graining.
Jim: You can confirm this for me. I’ve noticed in my many talks with Murray that coarse graining is one of his universal lenses. He uses it a lot for all kinds of things.
Seth: Absolutely. I’ve used it for many things. And it’s true that I missed Murray very much since he passed on about five years ago, a little more. I haven’t had as many talks about coarse graining since then, so I’m glad we can talk about coarse graining.
Jim: Let’s go on to another quite different set of domains where it’s useful, fractal dimensions.
Seth: Yeah. Right. In looking at complex systems, a very important part and intellectually historically important as well, is the relationship between the field of nonlinear dynamical systems, including chaos. This includes from the mathematical perspective, this includes a study of fractals, which are patterns like the patterns and things like snowflakes. If you look at them very closely they’re self-similar.
So you look at one scale and they look very similar at a smaller scale and at a larger scale. One of the most famous such fractals is my former colleague, Ed Lorenz here at MIT. We actually when I first moved here in 1994, we were next door neighbors in an MIT-owned apartment building.
And Ed, he was in the Earth and Planetary Sciences program, and he noticed that the equations of motion for the weather were chaotic in the sense of a tiny little difference now would make a big difference level later. So that they were intrinsically unpredictable, but they were not entirely unpredictable because this nonlinear dynamics would drive the dynamics of the weather to some what’s called a strange attractor, which is a fractal structure that has this funny self-similar structure.
So the dynamics was always confined to this funny, strange attractor. And the fact that it was confined to that, even though it was unpredictable could tell you a lot about how the weather was going to behave. And the weather, particularly here in New England is famously complex. They say here if you don’t like the weather now just wait five minutes and you probably like it even less then.
Jim: Yeah. I actually took a meteorology course and they had a prediction contest that ran every day, and the winner, the year I took the course was the guy took the weather from the previous day from Albany, and that was the winner.
Seth: There you go.
Jim: Rather than doing any modeling or analysis or anything else.
Seth: That was a wise idea. Exactly. There you go. That’s actually probably a best prediction of what the weather’s going to be like in 10 minutes is probably pretty much like what it is now.
Jim: Exactly.
Seth: We’ve been talking about these measures of complexity for a much broader set of measures of complexity, like the kind of things studied at the Santa Fe Institute. But the theory of nonlinear dynamical systems, which also has notions of complexity with these fractal structures and strange attractors and various ways in which a system will start out behaving in an orderly fashion and then get more chaotic.
There’s a lot of overlap with them. Oh, gosh. It’s a long time ago now. I think it’s probably 20 years ago, I was on a panel about complexity at Davos, the World Economic Forum. And I was having a debate with Robert May who was a famous professor about complex systems and dynamical systems at Oxford. And Robert May started with saying what he was doing. He said, “Well, lots of people talk about complexity and there’s a very nicely defined theory of low dimensional, nonlinear dynamical systems including fractals and chaos, and this is a very precise science. And then there’s a whole bunch of other stuff, which is bullshit.”
And so he talked about this for a while, and then when it came to be my term to talk, I said, “Oh, gee. Thank you, Lord May.” Actually, when I knew I was debating, Lord May. Lord May? And then, “Who’s Lord May?” And when he showed up, it was Bob May from Australia. He wasn’t a lord when I knew him. So when I got on, I said, “I’m glad that Bob described these two branches of complexity because what I’m going to tell you is going to be entirely about what he would call bullshit.”
Jim: Interesting. So what domains would fractal dimensions be useful in?
Seth: Yeah. So fractal dimensions are, I mean, you can see actually from the way that we’ve been describing complex systems, if a non-linear dynamical system like the weather, let’s just keep on with the weather because that’s a good one. If the thing is completely unpredictable and chaotic, it’s going to look just completely random looking, right?
It’s going to take a lot of information to describe. It’s going to have a very high algorithmic complexity, but at some level it’s not going to be that interesting. Whereas the actual weather requires quite a lot of algorithmic information to describe, but not all that much because you’re just describing the Navier-Stokes equations give the nonlinear dynamics of these driven fluids in the atmosphere.
And then solving that is hard. It’s logically very deep because it takes a long, long time to on supercomputers to start with your description of the dynamics. Today, it predict what’s going to happen even 10 days in the future, actually, after 10 days, it gets to be very hard indeed because of the chaotic nature. But it still makes it much more predictable. So the fact that it’s complex, but it sits on a much lower dimensional structure, these fractal structures means even though it’s unpredictable in some ways it’s still very predictable in other ways.
Jim: Yeah. In other words, you could not calculate the weather by calculating the motions of atoms, but fortunately, you have these fractal structures at various scales that you can use in part of your simulation.
Seth: Right. Exactly. Things like the weather are hard to predict, but they’re not intrinsically unpredictable. Something like the very random motions of molecules and atoms, basically anybody who’s ever played pool, in pool, it’s tough enough to hit a ball with the cue so it goes off in that direction, and then it’s even tougher to hit a ball like this so it goes off in that direction, and hits this other ball and drives it in this other direction because a combination is harder.
And the reason is that collisions of pool balls is chaotic, and an error that you make in the angle on the first one gets doubled in the motion of the second one. It gets doubled again in the motion of the third one. So all it takes is a couple of collisions for things to get very uncertain. This by the way was pointed out by James Clerk Maxwell back in maybe the 1860s or something like that. There’s still things that you can predict.
Jim: Let’s do this one as briefly as we can because it’s one that comes up in every one of these lists that I went and looked at, but it’s funky in its own ways, and that’s the Lempel-Ziv complexity.
Seth: This is a very nice idea. Remember this problem that Claude Shannon was originally trying to solve at Bell Labs, which is if you take a message that has regularities in it, in Shannon’s place, it was statistical regularities. How do you compress it? People are very familiar with this. And of course, at a rough-and-ready way long before Shannon, you look at the letters in English and they come with different frequencies. So E is by far the most frequent one, then I think it’s T, then A, then I, O, N, then S, H, R, L.
Jim: Morse code took advantage of that actually.
Seth: And Morse code took advantage of that. Right. Exactly. And E is dot.
Jim: T is dash.
Seth: And T is dash. And actually the way that Morse figured this out, he and I went to the same high school except a couple hundred years apart. He didn’t go and count the frequencies, but he just went to typesetters. And the typesetters have all this type with the lead types of the different letters, and they have them organized in a particular way, and already they’re organized in a way that the ones like E and T are going to be closer for them to do so they don’t have to reach as far.
But then they also have different numbers of Es and Ts. They have a lot more Es than they have Ts, right? Because they want to have the right amount of type. And so he just counted the type that the typesetters had. And he figured out very accurately what the order of letters and frequency was. And then he came up with codes such that more frequent letters had shorter codes.
Jim: And by the way, we could have calculated the thermodynamic depth of that information. Right?
Seth: Absolutely. Yeah.
Jim: All those typesetters, all those years of rough heuristics, and they end up with a pretty close number.
Seth: Yeah. And you could say what Shannon did as he made this mathematically precise by figuring out, you can compress a message that has these kinds of statistical regularities by a certain amount, but no more. And he gave a very simple mathematical formula for that.
So Lempel-Ziv-Welch complexity is a method for coding where ideally with coding, you’d say, “Oh, let’s go through the message. Let’s count all the letters and then the words and look at their different frequencies and combinations of words. And then we’ll have a complicated procedure. We’ll assign shorter codes to letters that occur more frequently, and then we’ll have more complicated codes that’s assigned shorter codes to words, combinations of words that appear more frequently.” But LZW, Lempel-Ziv-Welch, does this automatically.
And it actually, you just go through the message and you start with a dictionary, which has certain codes for certain letters, but it kind of crude and it doesn’t assign short codes for frequently occurring letters or combinations of words. But what happens is as the coding takes place, it’s adaptive and it learns which combinations of letters are occurring with greater frequency.
And when it finds some combination of letters that’s occurring more with greater frequency, it assigns it a shorter code in a very sweet and automatic fashion. And in fact, the person who’s receiving this coded information, LZW is set up in such a way so that even though they don’t know what the information is, and they don’t even know to begin with how the codes are being assigned, they actually can deduce what the codes are.
And so the encoded LZW message then does the same job that Shannon would like you to do, which is construct something that’s highly efficient. In fact, mathematically the mathematical proof for LZW is that asymptotically as the length of the message gets really long, then it attains this Shannon bound for how efficiently you could encode. And you can encode on the fly and decode on the fly and attain this optimal Shannon bound for the efficiency of your communication channel. And I say that’s kicking ass.
Jim: That’s pretty good. And that’s LZV kinds of algorithms are obviously used in file compression algorithms on our computers and such. Right?
Seth: Right. Exactly. So LZW is a basis for anybody who uses zip or GIF, like zipping files and using GIF is like, that’s using LZW. It’s a straightforward procedure and it does a good job.
Jim: Cool.
Seth: You think, “Why don’t you just keep on doing it until it just gets compressed and compressed and compressed?”
Jim: Yeah. I remember having that thought when I was like 13. I go, “Wait a minute.” Right? And then you realize, you can’t.
Seth: Yeah. So what happens then is that, actually, this is a cool thing about coding and randomness and complexity, in general. So if something has got a lot of statistical regularities, then it’s highly compressible. But the encoded form of this, the compressed form. By definition-
Jim: Can’t be.
Seth: … [inaudible 00:40:20] more efficient, so it’s going to look much more random. So actually if you do LZW twice, then-
Jim: It gets bigger.
Seth: … you don’t get anything better.
Jim: Or it gets bigger.
Seth: [inaudible 00:40:27] it gets bigger. Right.
Jim: Let’s move on to our next one, and this is from another one of our colleagues, Jim Crutchfield and Young. That’s statistical complexity and the work that particularly Crutchfield did with his e-machines.
Seth: Yeah. Epsilon machines. Right.
Jim: I mean, epsilon machines. Yeah. Yeah.
Seth: Yeah. Yeah. This is a very lovely measure of complexity. It’s closely related to effective complexity. What Murray and I were working on, not surprisingly, because we learned about Jim’s measure from Jim. Basically it says suppose we have some very long message or texts, and it has a bunch of statistical regularities in that, and we want to construct the simplest computational machine that will reproduce this message with the same statistics.
So we’re not just this message, but messages that have the same statistics for words. Under the assumption that the message we’re given is representative of a whole sample or ensemble of messages. We’re just given one out of many that could happen.
And so what they do is they have a beautiful procedure that the simplest conceptual computational device is called an automaton. And we were talking before about cellular automaton. Cellular automata or automatons that are spread out homogeneously in space, but an automaton is just a device that has different possible states like three states, zero, one, and two.
And what it does is it moves from state to state, and when it moves from one state to another, it gives a particular output. Like let’s say there are three states, zero, one, and two, when it moves from zero to one, it outputs an A. When it moves from one to two, it outfits a B. And if it moves from two to zero, it outputs an A again.
And so you can see that this automaton is just going to go around this little circle 012012012, and it’s going to be producing a string that goes A, B, A, A, B, A, A, B, et cetera, or it can be probabilistic and it can branch, in which case it’ll produce things with probabilistic strings.
And Crutchfield and Young, their epsilon machine, the epsilon machine is the size of the simplest automaton that will reproduce a message with the same statistics as your message that you’ve trained it on up to accuracy epsilon.
Jim: Very interesting. Yeah. I did talk to them quite a bit when I was there.
Seth: Yeah. Yeah. In fact, it is very relevant for the current time because if you think of these large language models, which are causing all the stir amongst people and simply be causing a lot of fear amongst people for reasons that I actually, I understand why they might feel fear, but it’s not fear that I feel myself.
Jim: Yeah. My reaction to that is I’m not going to fear anything that’s a feed-forward static model.
Seth: Yeah. They’re basically these linked attention mechanisms where these general purpose transformers that they’re based on. I mean, that’s a fancy words for a particular automaton where you deviate up how it’s going to look at this message in terms of this so-called attention mechanism. There’s an attention mechanism that pays attention to the fact that it’s a period at the end of sentences, and says, “Aha, look. Hey, dudes, there’s an end of our sentence there. Let’s pay attention to them.”
They’re like, “Oh, yeah. Okay. Yeah. Right. Okay. It’s like the end of a message.” Right? These large language models are basically a whole bunch of attention messages vying for attention with each other, but there are basically simple kinds of automata. And they’re big. Right?
I mean, it turns out that they’re trained on the one that’s trained on the whole corpus of Wikipedia or something like that. I mean, these are very large models. They take up a vast amount of computer space. They take up a huge amount of energy to train. In fact, if you think of the carbon footprint of an information processing organism, these large language models have the largest carbon footprint of any information processing organism we have around, and they’re still really dumb.
Jim: Indeed.
Seth: They better get better or basically we’re just going to warm the world by 40 degrees Celsius and we’ll just have to be run by large language models who are still really stupid.
Jim: Yeah, and compared to human brain which is about 20 watts, right?
Seth: Yeah. Exactly. I mean, I’m not saying that humans are smart. Right? I mean, please, but I mean really…
Jim: Yeah. Just read the news if you want to disabuse yourself with the notion that humans, at least in the collective are smart. Do you want to say anything else about thermodynamic depth?
Seth: Thermodynamic depth, it’s the most physical of these notions. It was inspired by logical depth. It’s related closely to ideas of effective complexity. So it ties in very well with these notions of, for instance, this statistical complexity, Crutchfield and young and epsilon machines and effective complexity.
So I think it’s certainly not the only possible measure, but it is a good measure of the physical resources that were required to produce what you see. And that’s a reasonable type of measure of complexity. So in my 31 Measures of Complexity article, I’m sure you saw, I divvy things up, but there’s measures of how hard it is to describe. Right?
So entropy, Maxwell, Boltzmann, Gibbs entropy, Shannon entropy, Shannon information, algorithmic information, Kolmogorov complexity, those are all measures of how hard something is to describe, and they’re very closely related to each other. And then there’s measures like how hard is it to do something?
Like in the theory of computational complexity, how many elementary logical operations does your little old computer have to do in order to do a particular computation? Like calculate the first billion digits of pi. And then spatial computational complexity, how much memory space does it need to do this computation? Those are very practical measures of how hard is it to do, or in the case of thermodynamic depth, how much free energy, how much energy do you have to burn up and dissipate to apply to actually make something happen, right? That’s a very closely related physical notion to these ideas of computational complexity.
And then you have things that combine description length, and this effort it is to do something. Thermodynamic depth combines, oh, we look at the shortest description of a thing, and then the shortest program to produce it. We say, “How hard is it to produce this thing from its shortest program?” Thermodynamic depth is the same.
We look at effective complexity. How hard is it to describe the functional parts of a system? We’re talking about bacteria and biological systems for effective complexity. But a really nice application, which is actually why I’m sitting here as a professor of mechanical engineering at MIT, is application to engineered systems.
You say the effective complexity of something like a car is the length of the blueprint and the descriptions required to make that car, including how do you manufacture the alloy needed to make this particular part of the car? You don’t care about every individual atom is, but by god, you care that it doesn’t rust out. You care that the engine runs in a particular way.
So in a case like that, the effective complexity is very nicely defined because you say, “I want a car. Here’s what it’s supposed to do. Here’s its functional requirements. What is the effective complexity for attaining those functional requirements?”
Jim: Let’s move on to our next one, which takes us in a quite different direction, into a different class of thinking about complexity. And that’s the fairly broad class of mutual information.
Seth: Yeah. So mutual information is information, it refers to systems that are complex in the sense they’re made up of multiple subsystems. And mutual information is the information of that the different parts of the system possess in common.
So if I have just a bit, a zero or a one. That’s one bit of information. If I have another bit, zero or one, that’s another bit of information. But if I say, if I specify that the two bits are the same. So they’re zero, zero or one, one, then each bit on its own has a bit of information because each could be zero or one, but together they only have one bit of information, not two.
And so the mutual information is one bit. It’s the sum of the information of the individual pieces minus the total amount of information. So it’s a measure of how much information is shared between these two parts of the system. And so people have often and the mutual information is a kind of symptom of complexity. That is to say, it’s very hard to imagine a complex system that does not possess a whole bunch of mutual information, but it doesn’t have to be.
You can have a system that have a lot of mutual information is not complex. For example, suppose I have a billion bits and they’re all either zero, zero, zero, zero, zero, or one, one, one, one, one. Now they’ve got a lot of mutual information because they’ve got a billion bits of mutual information because they’re all the same bit, right? But you wouldn’t call them these billion bits of particularly complex system.
On the other hand, if you look at a bacteria and it’s metabolism and all the complex parts that are acting there, there’s a vast amount of mutual information between these different parts of the metabolism of bacteria because there’s all this communication going on and all these chemicals being exchanged and free energy being pumped around.
So there’s a very large amount of mutual information, but because the bacteria is performing this very complicated process in order to do what it’s got to do, which is to say, get some food and reproduce, then it’s very complicated. But the mutual information, the large amounts of mutual information is now a symptom of the fact that it’s complex. And so it’s probably a necessary condition to have a lot of mutual information, but insufficient to be complex.
Jim: Gotcha. And that gets me to my next one, which is a neighbor, and that doesn’t really normally show up on lists of complexity measures, but I thought I’d throw it in anyway see what you’d think. You may not know about it and that’s to Tononi’s this integrated information.
Seth: Oh, yes. I do know. In fact, I was visiting University of Wisconsin a few years ago and Tononi sat me down and wouldn’t let me leave until he showed me 200 transparencies, 200 slides about integrated information. Yes. I do know about integrated information. What would you like me to say about it?
Jim: Yeah. Well, how do you think that relates to things like mutual information and the degree to which it is necessary or sufficient for different kinds of complexity?
Seth: So integrated information, I would call it a more intricate form of mutual information. So it measures not only how much information is shared between the different parts of the system, like when I do this, all the fireworks go off. It’s a form of mutual information, and it’s a form of mutual information that complex systems like brains or bacteria have a lot of, but something like a billion bits that are either all zero or all one do not have a lot of.
A nice way of thinking about it, it’s in a system that has many different parts. It’s a degree to which you can infer the operation of the different parts from each other, and also just in a dynamic way. But like mutual information, integrated information is anything we’d like to be likely to call complex like a bacterium or the operations of a brain or something like that have a lot of integrated information, but there are plenty of things that have a lot of integrated information that we probably wouldn’t call complex.
One of the feature, I’ll use an example from my friend and colleague, Scott Aaronson. If I have an error correcting code, an error correcting code has a feature that you can, even if you mess up a lot of the bits in a system, you can still reconstruct the message because it’s encoded in a way that’s redundant.
And so every part of the system has in it this information about what the message is. And by that very nature, this error correcting code has a lot of integrated information. But error correcting codes can be quite simple. The way that they integrate and have this redundancy can be quite simple. And in Tononi’s case, he actually claims… I mean, he says, “Okay. Consciousness…” I don’t know what he means when he says consciousness.
He explained it many times and I didn’t get it. That’s all right. That happens to me a lot when people try to explain what they mean by consciousness. I think it’s a problem with consciousness. Hard to explain. And at least hard to explain to me, let’s put it that way. So he states that, okay, things that are conscious clearly have a lot of integrated information. Absolutely. For sure. Our brain has a lot of integrated information.
I’m not sure that the bacterial metabolism is conscious, but it certainly does complex information processing and it’s got a lot of integrated information. So Tononi just states that anything that has a lot of integrated information is conscious. And I simply do not believe that error-correcting codes are conscious. That’s in fact one of the least conscious things I can think of.
Jim: Yeah. And of course, he even says that a photoelectric cell attached to a light switch is conscious at one bit. Right?
Seth: Yeah. Yeah. There’s a name for this, Panpsychism.
Jim: Yeah.
Seth: That everything is consciousness and everything is conscious, and we’re all part of the gigantic universe of consciousness, et cetera. And I actually believe everything carries information around with it, and everything’s processing information. That’s why we can build quantum computers just like, “Hey, electrons, atoms, photons, they all have got information and we just massage them in the right way.” They’ll process that information for us. But that doesn’t mean it’s conscious. So I think the problem here is this actually that-
Jim: It’s a definition. Yeah.
Seth: Yeah.
Jim: It’s an area I studied a lot actually, is the science of consciousness and the first thing you always have to do is, “What the hell are you talking about, dude?” Right? And I actually had Christof Koch Koch on one time and we-
Seth: Oh, yeah. Christof. He and Tononi collaborate on this stuff. He’s like-
Jim: Yeah. We basically argued about IIT. He couldn’t convince me that anything they had a high pi was by definition conscious [inaudible 00:54:41]
Seth: No. But on the same time, if you talk with folks like David Chalmers, and by the way I’m not… Well, to say that the fact that I disagree with some of these folks doesn’t mean… I have tremendous respect for all of them and what they’re saying is very interesting. It just happens to be wrong. So yeah. I think the problem is more that maybe there isn’t something, when we say consciousness, maybe it’s a word that doesn’t actually have a referent.
Jim: Yes. John Searle, the philosopher does a great job on this.
Seth: Oh, yeah. Yeah. Yeah.
Jim: He basically says, “You guys are all confused. You can’t put your finger on something and say this is consciousness. Consciousness is a process like digestion, it involves the tongue and the esophagus and the liver and the stomach and various intestines, and this whole process is a biological system. In the same way, consciousness is a biological system that uses the perception areas, the object ontologies, the episodic memories, and a bunch of other things.”
And so think of it as very much as a process that serves a biological purpose, that pays for its energy expenditure and the information content and DNA. And don’t think you can just say, “That is consciousness.”
Seth: Yeah. Absolutely. That’s a beautiful description. And actually there’s a rather more practical version of this, which I got from my anesthesiologist. Which is, okay, I’m going to get my hip replaced. And he says, “Okay. Do you want general or local?” I said, “Oh, I didn’t know you could have your hip replaced with local anesthesia.”
He said, “Oh, yeah. Sure. So first we give you a spinal so you don’t feel anything as if you’re giving birth. And then we give you this thing to knock out your perception of being awake. And then we give you this other thing to knock out at a memory you have, and we have this other thing that knocks out the perception of pain.” And they’re like five different things. And in order to make someone sufficiently unconscious to replace their hip, you have to knock out all these different things that interact with each other.
As you say, it’s just like the digestion except digestion for the brain. So it’s not like there’s just one thing that is consciousness there.
Jim: Yeah. I still got a few more on my list. Let’s do two more and then I will let you young man depart.
Seth: Thank you.
Jim: Network complexity.
Seth: Oh, network complexity. Yeah. It’s actually quite broad. A very important class of complex systems are networks, right? The communications network, the network of neural connections in our brain. The power grid is a famous complex network which actually is all screwed up right now because of its complexity. And it turns out that we’re having a tough time integrating renewable energy in the power grid because it’s very complicated.
So network complexity refers to, I would say, a set of ideas about how you deal with complex networks. First of all, the structure of the network is like… Well, let’s just take the power grid for example. Right? There are power plants. They come in all kinds of different sizes. They come in all kinds of different kinds. Some of them are coal power plants, some of them are nuclear power plants, some of them are wind farms, some of them are solar generators.
They’re all connected to the grid in different ways. And then you have the electricity in the grid and that energy is spread over very long distances in a complicated way that it’s spread through all these different transmission lines of all different kinds until it eventually gets to your home.
This engenders a lot of dynamical and sometimes quite unforeseen behaviors. Because one thing about these networks is they tend to have regimes in which they can be kind of chaotic, which is bad. You actually don’t want the electrical grid to go chaotic, so you try to tune them to regimes in which they’re not chaotic. But then actually it turns out pretty much with any complicated electromechanical system like a network or a car, when you’re trying to drive it at its limits and keep everything under control, that’s the point of which a little change, just a little bit of stuff can make it, “Rahhh.” Go crazy.
Jim: Yeah.
Seth: So I mean, this is why when you trying to drive non-chaotic systems to their limits, you can drive them to a point where they’re at this edge of chaos. And so you can get this complex emergent and often unpleasant behavior emerging.
Jim: Yeah. I remember when I was at the Santa Fe Institute, we had a woman from Battelle Institute come out who had studied the grid as a complex system. And she said, “The bad news folks is if you overlay the topologies and behaviors of the network with the fluctuations of demand, I can show you that any arbitrary level of failure will occur at some point including a complete collapse of the whole grid.”
Seth: I like it. I like it. Yeah. Yeah. That’s the kind of nice theorems that mathematicians like to prove about the grid, but I’m glad that what we knew to be the case is actually empirically is actually mathematical theorem. That’s the Santa Fe institute for you.
Jim: Absolutely. All right. Let’s go on to one last thing and that’s multiscale entropy.
Seth: Yeah. Multiscale entropy is a very interesting question. This actually has to do with Murray’s favorite topic of coarse graining. So a coarse graining says, remember coarse graining is you look at a system at a particular scale, which means that you just toss out all the information below of certain scale. Like the molecules of gas in this room. It’s like, “I’m just going to look at… All I really care about is the temperature, the pressure that there’s enough oxygen and stuff here. And I don’t care where all the molecules are moving.”
So you look at things at different scales. So if you look at something at a particular scale, like say a very coarse grain scale, then there’s not a lot of information to describe it. And then you describe it at a finer scale and now there’s a lot more information to describe it. And you have to look at how that smaller scale information also goes up to the larger scale.
And then you look at a series of scales. And so a complex system by almost all of the definitions that we’ve been talking about are typically systems that have a lot of information at each scale. So you look at it at any scale and a living system is a great example of this.
Like look at you and me, we’re made up of billions of cells. And then if you look at us, we have all kinds of macroscopic behavior, conversations about complexity and stuff like that. And you look at a smaller scale, then the scale of the gut or the cells themselves, and it’s still super complicated. Like an individual human cell is very, very complicated, even if you look at some really tiny mechanism within a human cell, let’s say the mitochondria where energy from food gets turned into energy to run around with and win the mountain bike race in the Olympic Games.
I mean, I would say mountain bikers are very worried about mitochondrial energy. That’s extremely complicated just within a cell at this level of the mitochondria. So multiscale entropy tells you how much information there is at different scales. And systems that are complex, particularly systems like networks with many, many different parts like you or me or the grid. The power grid, those exhibit a large amount of multiscale information or entropy.
But again, you can construct systems, a fractal system will also have a lot of multiscale information, but fractals, simple fractals, you wouldn’t really call them to be a very complicated system. So having multiscale entropy is mutual information or integrated information is that more symptom rather than a cause.
You expect a complex system to have a lot of multiscale information/entropy just as you expect to have a lot of mutual information, a lot of integrated information, but not all systems with a lot of multiscale information are complex.
Jim: Cool. The final takeaway I’d like, and you let me know if you agree or disagree for the audience is you can’t just say there’s a measure of complexity that applies in all domains. There’s many different ways of measuring complexity, some of which are practical, some of which are impractical. We didn’t need to drill into that. We took people on a little survey of different ways of thinking about complexity and help them perhaps see that the idea of a single measure is probably very unlikely to be true.
Seth: Yeah. And we didn’t even talk about things like social complexity or emotional complexity. Let’s face it, completely legitimate uses of the word complexity or political complexity, like all these kinds of things. So my take on this, there are many different measures of complexity and you should use the one that works for you. That’s what I’d say.
Jim: Yeah. The one I was thinking about adding, but this would’ve been such a rabbit hole is game theory complexity, right?
Seth: Oh, yeah. Yeah. Yeah.
Jim: We’ll leave that one for another day.
Seth: We’ll leave that for another day. I’m feeling too emotionally complex today to actually talk about game theory complexity.
Jim: All right. Well, thanks Seth Lloyd for an extraordinarily interesting tour of the various ways to think about measuring complexity. Thanks.
Seth: Thank you very much, Jim. Great talking with you.
Jim: Yeah. It really was. I really enjoyed this.