EP 192 David Krakauer on Science, Complexity and AI

Jim has a wide-ranging talk with David Krakauer about the ideas in his forthcoming paper “The Structure of Complexity in Machine Learning Science” and how AI may alter the course of science. They discuss data-driven science vs theory-driven science, a bifurcation in science, the protein folding problem, brute force methods, the origin of induction in David Hume, the origin of neural networks in deductive thinking of the ’40s, super-Humean models, crossing the statistical uncanny valley, ultra-high-dimensionality, adaptive computation, why genetic algorithms might come back, Chomsky’s poverty of the stimulus, the lottery ticket hypothesis, neural nets as pre-processors for parsimonious science, how human expertise constrains model-building, GPT-4’s arithmetic problem, cognitive synergy, why LLMs aren’t AGIs, incompressible representations, gravitational lensing, the new sciences LLMs will lead to, encoding adaptive history, Jim’s ScriptWriter software, discovery engines vs libraries vs synthesizers, the history of science as a history of constraint, Occam’s razor & meta-Occam, assembly theory, whether existential risk is a marketing ploy, the Idiocracy risk, using empirical precedent in tech regulation, networks of info agents, the outsourcing of human judgment, and much more.