Given the successes of energy-based
formalisms in physics, it is natural to want to extend them into other domains
like computation and cognition.

In this vein: My aim here is to sketch what I
think is a workable approach to an energetics of computational processes
(construed very broadly).

By this I mean: I will explain how one can
articulate highly general principles of the dynamics of computational
processes, that take a similar form to physics principles such as the
stationary action principle (which often takes the form of "least
action") and the Second Law of Thermodynamics (the principle of entropy
non-decrease).

Why am I interested in this topic? Two related reasons, actually.

First, I would like to create a "General
Theory of General Intelligence" -- or to be more precise, a general theory
of what kinds of systems can display what levels of general intelligence in
what environments given realistically limited (space, time and energy)
resources. Marcus Hutter's Universal AI
theory is great but it doesn't say much about general intelligence under
realistic resource assumptions, most of its power is limited to the case of AI
systems with unrealistically massive
processing power. I have published some ideas on this before --
e.g. formalizing Cognitive Synergy in terms of category theory,
and articulating the Embodied Communication
Prior in regard to which human-like agents attempt to be intelligent
-- but nothing remotely near fully satisfying.
So I'm searching for new directions.

Second, I would like to come up with a real
scientific theory of psi phenomena. I am
inclined toward what I call "euryphysical" theories -- i.e. theories
that involve embedding our 4D spacetime continuum in a larger space (which
could be a higher dimensional space or could be a non-dimensional topological
space of some sort). However, this begs
the question of what this large space is like -- what rules govern
"dynamics" in this space? In my paper on Euryphysics, I
give some rough ideas in this direction, but again nothing fully satisfying.

It would be nice if mind dynamics -- both in
a traditional AI setting and in a more out-there euryphysical setting -- could
be modeled on dynamical theories in physics, which are based on ideas like
stationary action. After all, if as
Peirce said "matter is just mind hide-bound with habit" then perhaps
the laws of matter are in some way simplifications or specializations of the
laws of mind -- and maybe there are laws
of mind with roughly analogous form to some of the current laws of physics.

**A Few Comments on Friston's Free Energy Ideas**

Friston's "free energy principle"
represents one well-known effort in the direction of modeling cognition using
physics-ish principles. It seems to me
that Friston's ideas have some fundamental shortcomings -- but reviewing these
shortcomings has some value for understanding how to take a more workable
approach.

I should clarify that my own thinking
described in this blog post was not inspired by Friston's thinking to any
degree, but more so by long-ago reading in the systems-theory literature --
i.e. reading stuff like Ilya Prigogine's Order
out of Chaos] and Eric Jantsch's The
Self-Organizing Universe and Hermann
Haken's Synergetics. These authors represented a tradition
within the complex-systems research community, of using far-from-equilibrium
thermodynamics as a guide for thinking about life, the universe and
everything.

Friston's "free energy principle"
seems to have a somewhat similar conceptual orientation, but confusingly to me,
doesn't seem to incorporate the lessons of far-from-equilibrium thermodynamics
that thoroughly, being based more on equilibrium-thermodynamics-ish ideas.

I haven't read everything Friston has
written, but have skimmed various papers of his over the years, and recently
looked at the much-discussed papers The Markov
blankets of life: autonomy, active inference and the free energy principle

My general confusion about Friston's ideas is
largely the same as that expressed by the authors of blog posts such as

As the latter post notes, regarding
perception, Friston basically posits that neural and cognitive systems are
engaged with trying to model the world they live in, and do so by looking for
models with maximum probability conditioned on the data they've observed. This is a useful but not adventurous
perceptive, and one can formulate it in terms of trying to find models
with minimum KL-divergence to reality,
which is one among many ways to describe Bayesian inference ... and which can
be mathematically viewed as attempting to minimize a certain "free
energy" function.

Friston then attempts to extend this
principle to action via a notion of "active inference", and this is
where things get dodgier. As the
above-linked "Markov Blankets" paper puts it,

*"Active inference is a cornerstone of the free energy principle. This principle states that for organisms to maintain their integrity they must minimize variational free energy. Variational free energy bounds surprise because the former can be shown to be either greater than or equal to the latter. It follows that any organism that minimizes free energy thereby reduces surprise—which is the same as saying that such an organism maximizes evidence for its own model, i.e. its own existence*

*...*

*This interpretation means that changing internal states is equivalent to inferring the most probable, hidden causes of sensory signals in terms of expectations about states of the environment*

*...*

*[A] biological system must possess a generative model with temporal depth, which, in turn, implies that it can sample among different options and select the option that has the greatest (expected) evidence or least (expected) free energy. The options sampled from are intuitively probabilistic and future oriented. Hence, living systems are able to ‘free’ themselves from their proximal conditions by making inferences about probabilistic future states and acting so as to minimize the expected surprise (i.e. uncertainty) associated with those possible future states. This capacity connects biological qua homeostatic systems with autonomy, as the latter denotes an organism’s capacity to regulate its internal milieu in the face of an ever-changing environment. This means that if a system is autonomous it must also be adaptive, where adaptivity refers to an ability to operate differentially in certain circumstances.*

*...*

*The key difference between mere and adaptive active inference rests upon selecting among different actions based upon deep (temporal) generative models that minimize the free energy expected under different courses of action.*

*This suggests that living systems can transcend their immediate present state and work towards occupying states with a free energy minimum."*

If you are a math/physics oriented person
and find the above quotes frustratingly vague, unfortunately you will find that
the rest of the paper is equally vague on the confusing points, and Friston's
other papers are also.

What it sounds like to me (doing some
"active inference" myself to try to understand what the paper is
trying to say) is that active inference is being portrayed as a process by
which cognitive systems take actions aimed at putting themselves in situations
that will be minimally surprising, i.e. in which they will have the most
accurate models of reality. If taken
literally this cannot be true, as it would predict that intelligent systems
systematically seek simpler situations they can model better -- which is
obviously not a full description of human motivation, for instance. We do have a motivation to put ourselves in
comprehensible, accurately model-able situations -- but we also have other
motivations, such as the desire to perceive novelty and to challenge ourselves,
which sometimes contradict our will to have a comprehensible environment.

The main thing that jumps out at me
when reading what Friston and colleagues write about active inference is that
it's too much about states and not enough about paths. To model far-from-equilibrium thermodynamics
using energy-based formalisms, one needs to think about paths and path
entropies and such, not just about things like "

*work[ing] towards occupying states with a free energy minimum."*Instead of thinking about ideas like "*selecting among different actions based upon deep (temporal) generative models that minimize the free energy expected under different courses of action." in terms of states with free energy minimum, one needs to be thinking about action selection in terms of stationarity of action functions evaluated along multiple paths.***Energetics for Far-From-Equilibrium Thermodynamics**

It seems clear that equilibrium
thermodynamics isn’t really what we want to use as a guide for cognitive
information processing. Fortunately, the
recent thermodynamics literature contains some quite interesting results
regarding path entropy in far-from-equilibrium thermodynamics.

Abaimov's paper General
formalism of non-equilibrium statistical mechanics, path approach and
Raphael Chetrite and Hugo Touchette's paper Nonequilibrium
Microcanonical and Canonical Ensembles and Their Equivalence each tell part of the story.

David Rogers and Susan Rempe in A First and Second Law for
Nonequilibrium Thermodynamics: Maximum Entropy Derivation of the
Fluctuation-Dissipation Theorem and Entropy Production Functionals"
describe explicitly the far from equilibrium “path
free energy”, but only for the case of processes with short memory, i.e. state
at time i+1 depends on state i but not earlier ones (which is often fine but
not totally general).

The following table from Rogers and Rempe
summarizes some key points concisely.

Conceptually, the key point is that we need
to think not about the entropy of a state, but about the "caliber" of
a path -- a normalization of the number of ways that path can be realized. This then leads to the notion of the free
energy of a certain path.

It follows from this body of work that ideas
like "free energy minimization" need to be re-thought dynamically
rather than statically. One needs to
think about systems as following paths with differential probability based on
the corresponding path free energies.
This is in line with the "Maximum
Caliber principle" which is a
generalization of the Maximum Entropy principle to dynamical systems (both
first proposed in clear form by E.T. Jaynes, though Maximum Entropy has been
more widely developed than Maximum Caliber so far).

Extending these notions further, Diego
Gonzalez outlines a Hamiltonian
formalism that is equivalent to path entropy maximization, building on math
from his earlier paper Inference
of trajectories over a time-dependent phase space distribution.

**Action Selection and Active Inference**

Harking back to Friston for a moment, it
follows that the dynamics of an intelligent system, should be viewed, not as an
attempt by an intelligent system to find a state with minimum free energy or
surprisingness etc., but rather as a process of a system evolving dynamically
along paths chosen probabilistically to have stationary path free energy.

But of
course, this would be just as true for an unintelligent system as for an
intelligent system -- it's not a principle of intelligence but just a
restatement of how physics works (in far from equilibrium cases; in equilibrium
cases one can collapse paths to states).

If we want to say something unique about
intelligent systems in this context, we can look at the goals that an
intelligent system is trying to achieve.
We may say that, along each potential path of the system's evolution,
its various goals will be achieved to a certain degree. The system then has can be viewed to have a
certain utility distribution across paths -- some paths are more desirable to
it than others. A guiding principle of
action selection would then be:

*To take an action A so that, conditioned on action A, the predicted probability distribution across paths is as close as possible to the distribution implied by the system's goals.*
This principle of action selection can be
formalized as KL-divergence minimization if one wishes, and in that sense it
can be formulated as a "free energy minimization" principle. But it's a "free energy" defined
across ensembles of paths, not across states.

A side note is, it's important to understand
that the desirability of a path to an intelligent system need not be
expressible as the expected future utility at all moments of time along that
path. The desirability of a path may be
some more holistic function of everything that happens along that path. Considering only expected utility as a form
of goal leads to various pathologies related to wireheading, as I argued in a
long-ago blog post on ultimate orgasms and such.

**Algorithmic Thermodynamics**

Now let's dig a little deeper. Can we apply these same ideas beyond the
realm of physics, to more general types of processes that change over time?

I am inspired by a general Whiteheadean
notion of procesess as fundamental things.
However, to keep things concrete, for now I'm going to provisionally
assume that the "processes" involved can be formulated as computer
programs, in some standard Turing-equivalent framework, or maybe a
quantum-computing framework. I think
the same ideas actually apply more broadly, but -- one step at a time...

Let us start with Kohtaro Tadaki's truly
beautiful, simple, elegant paper titled A statistical mechanical
interpretation of algorithmic information theory

Section 6 of Tadaki outlines a majorly aesthetic,
obvious-in-hindsight parallel between algorithmic information theory and
equilibrium thermodynamics. There is
seen to be a natural mapping between temperature in thermodynamics and
compression ratio in algorithmic information theory. A natural notion of "algorithmic free
energy" is formulated, as a sort of
weighted program-length over all possible computer programs (where the weights
depend on the temperature).

The following table (drawn from Tadaki's
presentation here) summarizes the key
mappings in Tadaki's theory

To ground the mappings he outlines,
Tadaki gives a simple statistical
mechanical interpretation to algorithmic information theory. He models an optimal computer as decoding
equipment at the receiving end of a noiseless binary communication
channel. In this context, he regards
programs for this computer as codewords (finite binary strings) and regards
computation results (also finite binary strings) as decoded “symbols.” For simplicity he assumes that the infinite
binary string sent through the channel -- constituting a series of codewords in
a prefix-free code is generated by infinitely repeated tosses of a fair
coin. Based on this simple reductive
model, Tadaki formulates computation-theoretic analogues to core constructs of
traditional equilibrium thermodynamics.

Now let's start putting some pieces together.

Perhaps the most useful observation I will
make in this blog post is:

**It seems one could port the path-entropy based treatment of far-from-equilibrium thermodynamics (as seen in the papers I've linked above) to Tadaki's algorithmic-information context, by looking at sources emitting bits that are not independent of each other but rather have some probabilistic dependencies..**
By doing so, one would obtain an “algorithmic
energy” function that measures the energy of an algorithmic process over a
period of time -- without assuming that it’s a memoryless process like Tadaki
does in his paper.

To get this to work, so far as I can limn
without doing all the math (which I don't have time for at the moment, alas), one
needs to assume that the knowledge one has of the dependencies among the bits
produced by the process is given the form of expectations… e.g. that we know the average value of
f_k(x_{i+1}, x_i} for various observables f_k …. Plus one needs to make some other slightly
funny assumptions that are probably replaceable (the paper assumes “the number
of possible transitions does not depend on the starting point”… but I wonder if
this could be replaced by some assumption about causality…)

If I'm not mistaken, this should give us
something like Friston’s free energy principle that actually works and has
meaning…. I.e. we have a rigorous sense
in which complex algorithmic systems are minimizing free energy. The catch is that it’s an algorithmic path
energy -- but hey...

More precisely, relative to an observer S who
is observing a system S1 in a certain
way (by tabulating conditional probabilities of “how often some event of type A
occurs at time T+s, given some event of type B occurred at time T”) … we may
say the evolution of S1 in S’s perspective obeys an energy minimization
principle, where energy is defined algorithmic-informationally (following my
proposed, not-yet-fleshed-out non-equilibrium generalization of Tadaki’s
approach)…

**Into the Quantum Rabbit Hole...**

Now that we've gone this far, we may as well
plunge in a bit deeper, right?

Tadaki deals w/ classical computers but --
gesticulating only moderately wildly -- it seems one could generalize his
approach to quantum computers OK.

Then one is looking at series of qubits
rather than bits, and instead of tabulating conditional probabilities one is
tabulating amplitudes.

The maximum entropy principle is replaced with
the stationary quantropy principle
and one still has the situation that: Relative to S who is observing S1 using
some standard linear quantum observables, S1 may be said to evolve according to
a stationary quantropy trajectory, where quantropy is here defined via
generalizing the non-equilibrium generalization of Tadaki’s
algorithmic-informational entropy via replacing the real values w/ complex
values

So we may well get a kind of free-energy
principle for quantum systems also.

If we want to model cognitive stuff using
bits or qubits, then we have here a physics-ish theory of cognitive
stuff…. Or at least a sketch of the
start of one…

**Out Toward the Eurycosm**

One of the motivations for these
investigations was some discussions on higher-dimensional and more broadly
eurycosmic models of psi. If there are
non-physical dimensions that connect spatiotemporally distant entities, then
what are the dynamical laws in these dimensions? If we can model them as information
dimensions, then maybe the dynamics should be modeled as I’m alluding here…

Physics dynamics should be recoverable as a
special case of algorithmic-information dynamics where one adds special constraints. I.e. the constraints posed by spatial
structure and special relatively etc. should reflect themselves in the
conditional probabilities observed btw various classes of bits or qubits.

Then the linear symmetries of spacetime
structure should mean that when you calculate
maximum-path-algorithmic-information distributions relative to these physics
constraints, you end up getting maximum-Shannon-path-entropy
distributions. Because macrophysics
results from doing computing using ensembles of randomly chosen computer
programs (i.e. chosen subject to given constraints…).

Suppose we want to model a eurycosm that
works according to a principle like Peirce's "tendency to take
habits" aka
Smolin's Precedence Principle aka Sheldrake's morphic resonance? Well then, one can assume that the
probability distribution underlying the emanation of codewords in Tadaki's
model obeys this sort of principle.
I.e., one can assume that the prior probability of a certain subsequence
is higher if that subsequence, or another subsequence with some of the same
patterns in that sequence, have occurred earlier in the overall sequence. Of course there are many ways to modify
Tadaki's precise computational model, and many ways to formalize the notion
that "subsequences with historically more frequent patterns should be more
frequent going forward." But
conceptually this is quite straightforward.

One is forced however to answer the following
question. Suppose we assume that the
probability of pattern P occurring in a subsequence beginning at time T is in
some way proportional to the intensity with which P has occurred as a pattern
in the subsequence prior to time T.
What language of processes are we using to formalize the patterns
P? If -- in line with the framework I
articulate in The
Hidden Pattern and elsewhere -- we formalize a pattern P in X as a process
P that produces X and is simpler than X -- what is the language in which patterns
are expressed? What is the implicit programming language of
our corner of the eurycosm?

For simplicity I have been following Tadaki's conventional Turing machine based computational models here -- with a brief gesture toward quantum computing -- but of course the broad approach outlined here goes beyond these computing paradigms. What if we ported Tadaki's ideas to series of bits emanated by, say, a hypercomputer like the Zeno Machine? Then we don't just get a single infinite bit string as output, but a more complex ordinal construction with infinite bit strings of infinite bit strings etc. -- but the math could be worked. If the size of a Zeno Machine program can be quantified by a single real number, then one can assess Zeno Machine programs as patterns in data, and one can define concepts like compression ratio and algorithmic entropy and energy. The paradigm sketched here is not tied to a Turing Machine model of eurycosmic processes, though TMs are certainly easier for initial sketches and calculations than ZMs or even weirder things.

For simplicity I have been following Tadaki's conventional Turing machine based computational models here -- with a brief gesture toward quantum computing -- but of course the broad approach outlined here goes beyond these computing paradigms. What if we ported Tadaki's ideas to series of bits emanated by, say, a hypercomputer like the Zeno Machine? Then we don't just get a single infinite bit string as output, but a more complex ordinal construction with infinite bit strings of infinite bit strings etc. -- but the math could be worked. If the size of a Zeno Machine program can be quantified by a single real number, then one can assess Zeno Machine programs as patterns in data, and one can define concepts like compression ratio and algorithmic entropy and energy. The paradigm sketched here is not tied to a Turing Machine model of eurycosmic processes, though TMs are certainly easier for initial sketches and calculations than ZMs or even weirder things.

I have definitely raised more questions than
I've answered in this long and winding blog post. My goal has been to indicate a direction for
research and thinking, one that seems not a huge leap from the current state of
research in various fields, but perhaps dramatic in its utility and implications.