Algorithmic Information, Induction and Observers in Physics

Remark to last week's participants: This will be a condensed version of last week's talks. I will drop many details (in particular on the relation to quantum theory) and also drop the introductory slides to algorithmic probability (for this, see Marcus Hutter's introductory talk on Tuesday afternoon, April 10). Motivated by the conceptual puzzles of quantum theory and related areas of physics, I describe a rigorous and minimal “proof of principle” theory in which observers are fundamental and in which the physical world is a (provably) emergent phenomenon. This is a reversal of the standard view, which holds that physical theories ought to describe the objective evolution of a unique external world, with observers or agents as derived concepts that play no fundamental role whatsoever. Using insights from algorithmic information theory (AIT), I show that this approach admits to address several foundational puzzles that are difficult to address via standard approaches. This includes the measurement and Boltzmann brain problems, and problems related to the computer simulation of observers. Without assuming the existence of an external world from the outset, the resulting theory actually predicts that there is one as a consequence of AIT — in particular, a world with simple, computable, probabilistic laws on which different observers typically (but not always) agree. This approach represents a consistent but highly unfamiliar picture of the world, leading to a new perspective from which to approach some questions in the foundations of physics.

The causal Markov condition relates statistical dependences to causality. Its relevance is meanwhile widely appreciated in machine learning, statistics, and physics. I describe the *algorithmic* causal Markov condition relating algorithmic dependences to causality, which can be used for inferring causal relations among single objects without referring to statistics. The underlying postulate "no algorithmic dependence without causal relation" extends Reichenbach's Principle to a probability-free setting. I argue that a related postulate called "algorithmic independence of initial state and dynamics" reproduces the non-decrease of entropy according to the thermodynamic arrow of time.

When applied to a physical system, the two main, established notions of information, Shannon Information and Algorithmic Information, explicitly neglect the mechanistic structure of the system under evaluation. Shannon information treats the system as a channel and quantifies correlations between the system’s inputs and outputs, or between its past and future states. Algorithmic information quantifies the length of the shortest program capable of reproducing the system’s outputs or dynamics. The goal in both cases is to predict the system’s behavior from the perspective of an extrinsic investigator. From the intrinsic perspective of the system itself, however, information must be physically instantiated to be causally relevant. For every ‘bit’, there must be some mechanism that is in one of two (or several) possible states, and which state it is in must matter to other mechanisms. In other words, the state must be “a difference that makes a difference” and implementation matters. By examining the informational and causal properties of artificial organisms (“animats”) controlled by small, adaptive neural networks (Markov Brains), I will discuss necessary requirements for intrinsic information, autonomy, and agency.

In this talk, I show how information theoretic concepts can be used to extend the scope of traditional Bayesianism. I will focus on the learning of indicative conditionals (“If A, then B”) and a Bayesian account of argumentation. We will see that there are also interesting connections to research done in the psychology of reasoning. The talk is partly based on the paper “Bayesian Argumentation and the Value of Logical Validity” (with Ben Eva, forthcoming in Psychological Review, http://philsci-archive.pitt.edu/14491/).

Landauer's famous dictum that 'information is physical' has been enthusiastically taken on by a range of communities, with researchers in areas from quantum and unconventional computing to biology, psychology, and economics adopting the language of information processing. However, this rush to make all science about computing runs the risk of collapsing into triviality: if every physical process is computing, then to say that something performs computation gives no meaningful information about it, leaving computational language devoid of content. In this talk I will give an introduction to Abstraction/Representation Theory, a framework for representing both computing and physical science that allows us to draw a meaningful distinction between them. The use of AR theory - with its commuting-diagrammatic framework and associated algebra of representation - allows us to take significant steps towards giving a formal language and framework for the processes of science. I will show how AR theory represents this process (including the potential for automation), and the insights it gives into the usage and limits of computation as a formal process language for, and description of, physical sciences.