Energy as the Universal Organizing Principle

Core Thesis: Energy is the single most powerful organizing concept across science and human experience. Whether it is a particle seeking its ground state, an algorithm learning from data, a protein folding into its functional shape, a brain predicting its next sensation, or a person deciding whether to explore or settle, all are navigating the same fundamental landscape: minimize energy, reduce surprise, increase adaptive fitness.

---

1. Energy in Statistical Physics and Thermodynamics

1.1 What Is Energy?

Energy is the capacity of a system to do work. In statistical physics, it takes on a deeper meaning: energy determines the probability of a system being in a given state.

The Boltzmann Distribution:

$$P(s) = \frac{e^{-E(s)/k_B T}}{Z}$$

where $E(s)$ is the energy of state $s$, $T$ is temperature, $k_B$ is Boltzmann's constant, and $Z$ is the partition function.

Low-energy states are exponentially more probable. Nature gravitates toward low-energy configurations. This is the deepest statement in physics, and the template for everything that follows.

1.2 Thermodynamic Free Energy

Not all energy is usable. Free energy (Helmholtz: $F = U - TS$) is the energy available to do useful work after accounting for entropy:

• Internal energy ($U$): drives systems toward low-energy, ordered states
• Entropy ($S$): drives systems toward high-disorder states
• Free energy: the competition between the two; the system seeks the minimum of $F$, not $U$ alone

1.3 The Energy Landscape

Any system can be visualized as moving on an energy landscape - a surface whose height represents energy over all possible configurations:

• Global minimum: ground state; most stable, most probable
• Local minima: metastable states; trapped but not optimal
• Energy barriers: transitions between states require climbing ridges
• Temperature: controls how readily the system can escape local minima

This landscape of valleys, ridges, and funnels is the conceptual backbone connecting all the domains in this note.

1.4 Key Principles to Carry Forward

• systems evolve to minimize free energy
• probability is weighted by the Boltzmann law
• entropy opposes order, while temperature controls the explore-exploit balance
• phase transitions occur when the energy landscape shifts dramatically
• simulated annealing works by gradually lowering temperature to find global minima

---

2. Energy in Information Theory, ML, and Data Science

2.1 Shannon Entropy and Thermodynamic Entropy

Claude Shannon formalized information using the same mathematics as statistical physics. The Shannon entropy of a distribution is:

$$H(p) = -\sum_i p_i \log p_i$$

This is formally identical to Boltzmann's entropy up to constants and notation.

Information is negative entropy. Receiving information reduces uncertainty. The amount of information in a message is the reduction in entropy it causes.

2.2 Information Gain as Reduction in Posterior Entropy

In Bayesian terms, information gain measures how much a new observation updates beliefs:

$$\text{Information Gain} = H(\text{prior}) - H(\text{posterior}) = D_{KL}[P(\theta \mid \text{data}) \| P(\theta)]$$

Learning is free energy reduction in another language:

• before observation: high-entropy prior; many plausible world states
• after observation: lower-entropy posterior; belief concentrates
• the gain: reduction in surprise, or negative log-probability

This is the information-theoretic analogue of settling into a lower-energy valley.

2.3 Loss Functions as Energy Functions

Every ML loss function can be understood as an energy function:

2.4 Gradient Descent as Physical Dynamics

Training a neural network via SGD simulates physical dynamics on a loss landscape:

• parameters = position of a particle on the surface
• gradient = force pulling the particle downhill
• learning rate = step size
• momentum = inertia in consistent directions
• dropout or noise = thermal fluctuations that help escape local minima
• learning-rate schedules = a form of simulated annealing

2.5 Bayesian Inference as Free Energy Minimization

Bayesian learning minimizes variational free energy:

$$F = \underbrace{D_{KL}[q(\theta) \| p(\theta)]}_{\text{Complexity cost}} - \underbrace{\mathbb{E}_q[\log p(\mathcal{D} \mid \theta)]}_{\text{Accuracy}}$$

• complexity cost: penalty for deviating from the prior
• accuracy: reward for fitting the data

Learning is always balancing fit against complexity.

---

3. Energy-Based Models in Machine Learning

3.1 What Is an Energy-Based Model?

An energy-based model (EBM) defines a scalar energy function $E_\theta(x)$ over inputs and induces probability through:

$$P_\theta(x) = \frac{e^{-E_\theta(x)}}{Z_\theta}$$

The model learns by sculpting the energy landscape:

• push down energy for real data
• push up energy for unreal or negative samples

Core intuition: training an EBM is sculpting a landscape. Real data sits in deep valleys; everything else is pushed onto higher ridges.

3.2 The EBM Landscape of Modern AI

• Hopfield networks - associative memory as energy attractors
• Boltzmann machines - generative EBMs with latent variables
• Contrastive learning - low energy for positive pairs, high energy for negatives
• Score-based diffusion - learning the gradient of log-density
• JEPA - energy-based predictive learning beyond autoregression
• Energy-based reasoning models - low energy corresponds to coherent reasoning chains

3.3 Why EBMs Unify ML

3.4 The Partition Function Challenge

The central difficulty is computing:

$$Z_\theta = \int e^{-E_\theta(x)} dx$$

This is intractable in high dimensions. Common workarounds include:

• MCMC and Langevin dynamics
• contrastive divergence
• score matching
• noise-contrastive estimation

---

4. Energy in Protein Folding

4.1 The Protein Folding Problem

A protein is a chain of amino acids that must fold into a precise 3D structure to function. The conformational space is astronomically large. This is the heart of Levinthal's paradox.

Anfinsen's thermodynamic hypothesis says:

The native structure of a protein is determined solely by its amino acid sequence, and corresponds to the global minimum of its Gibbs free energy. The sequence encodes the landscape; the fold is the basin.

4.2 The Folding Funnel

The protein energy landscape is the same landscape concept from physics:

• wide top: unfolded state; high entropy, many conformations
• funnel shape: free energy decreases as folding progresses
• deep bottom: native state
• local minima: misfolded traps

The forces shaping this landscape include the hydrophobic effect, hydrogen bonding, van der Waals interactions, electrostatics, and solvent entropy.

4.3 Classical Approaches: Explicit Energy Functions

Before modern ML, structure prediction relied on explicit physics-based force fields:

• AMBER, CHARMM - molecular dynamics with explicit atomic interactions
• Rosetta - fragment-based energy function combining physical and statistical terms
• Monte Carlo sampling - exploring conformations via Boltzmann-style acceptance criteria

4.4 AlphaFold as Learned Energy Function

AlphaFold can be understood as implicitly learning an energy function of biological structure:

1. it internalizes what low-energy structures look like from data
2. coevolutionary signals from MSAs act like constraints on the landscape
3. recycling loops progressively refine structure like iterative minimization
4. AlphaFold 3 extends this idea to complexes, nucleic acids, and ligands

4.5 Protein, ML, and Physics in One Table

Protein folding is one of the strongest concrete demonstrations that energy minimization is the language of biology. When AlphaFold learned structure, it learned something like an implicit energy function that evolution has been optimizing for billions of years.

---

5. Energy, the Free Energy Principle, Cognition, and the Explorer's Gene

5.1 Variational Free Energy and the Brain

Karl Friston's Free Energy Principle (FEP) proposes that adaptive biological systems act to minimize variational free energy:

$$F = \underbrace{D_{KL}[q(\theta) \| p(\theta \mid o)]}_{\text{Complexity}} - \underbrace{\ln p(o)}_{\text{Accuracy}}$$

Minimizing $F$ simultaneously:

• updates beliefs
• drives action
• guides learning

The brain can be seen as a physical system on a cognitive energy landscape. Perception, action, and learning all become forms of descent on free energy.

5.2 Expected Free Energy: Why Exploration Is Mathematical

FEP extends into the future through expected free energy:

$$G(\pi) = \underbrace{-\mathbb{E}[\ln p(o \mid \tilde{s}, \pi)]}_{\text{Pragmatic value}} + \underbrace{D_{KL}[q(\tilde{s} \mid \pi) \| p(\tilde{s})]}_{\text{Epistemic value}}$$

• pragmatic value: getting preferred outcomes
• epistemic value: reducing uncertainty through information gain

Exploration is not irrational. Under FEP, curiosity is mathematically justified because information-seeking reduces expected free energy.

5.3 The DRD4-7R Explorer Gene

The DRD4 gene encodes the dopamine D4 receptor. The 7-repeat allele is associated with reduced dopamine binding efficiency, which may bias behavior toward novelty seeking:

• more stimulation is needed for equivalent reward
• higher tolerance for uncertainty and surprise
• elevated prevalence in historically migratory populations

Under an FEP lens, DRD4-7R can be interpreted as tuning a system more strongly toward epistemic value relative to pure exploitation.

5.4 Alex Hutchinson's The Explorer's Gene

Alex Hutchinson's book makes the case that exploration is not an obsolete relic but a deep human drive.

---

6. Energy Landscapes in Personal Growth and Behaviour

6.1 Behaviour as Energy Minimization

Human behaviour can also be understood through energy landscapes:

• habits are low-energy attractors
• change requires crossing an energy barrier
• motivation is the gradient pulling behaviour in a direction
• stress and anxiety can look like being stuck in a high-energy local minimum
• stagnation is being trapped in a local minimum that is comfortable enough to stay in

Every habit is a low-energy valley. Every new skill, relationship, or perspective is another valley separated by an energy barrier. Growth is the work of crossing those barriers.

6.2 Behavioural Energy Landscape

A simple behavioural flow looks like this:

1. Existing habit / comfort zone - a low-energy attractor
2. Curiosity, challenge, or a new environment - a fluctuation that perturbs the system
3. Energy barrier - effort, discomfort, learning cost
4. New attractor - a new habit, skill, or identity basin
5. Adjacent exploration - further growth compounds from the new basin

Psychological safety and play increase the effective "temperature" of the system, making escape from old minima easier. Novelty-seeking tendencies can lower the barrier to transition.

6.3 Principles of Behavioural Energy

1. Activation energy - small habits lower the barrier for new behaviours.

Lally et al. (2010) found that habit automaticity typically plateaus on a timescale of weeks to months, with a mean around 66 days. Missing a single day did not meaningfully derail long-run habit formation. This looks exactly like basin formation: repeated descent gradually carves a stable attractor.

2. Annealing - psychological safety, play, and curiosity raise effective temperature and permit exploration.
3. Deep work as basin formation - deliberate practice carves deep wells for expertise.
4. Identity as landscape shape - changing the self-model reshapes the whole landscape, not just one behaviour.
5. Resilience as multi-basin stability - resilient people can settle into more than one stable attractor.

6.4 Habit Formation as Free Energy Minimisation

Under FEP, habits are what emerge when action-perception loops have been repeated enough times that costly inference is no longer needed.

Stage 1 - Goal-directed

A new behaviour requires active evaluation of policies:

$$G(\pi) = \underbrace{-\mathbb{E}[\ln p(o \mid \tilde{s}, \pi)]}_{\text{Pragmatic value}} + \underbrace{D_{KL}[q(\tilde{s} \mid \pi) \| p(\tilde{s})]}_{\text{Epistemic value}}$$

This stage is metabolically and cognitively expensive.

Stage 2 - Chunking

With repetition, priors over the policy sharpen. The system increasingly selects the behaviour because it has become probable, not because it is being freshly evaluated each time.

Dezfouli and Balleine provide a clean computational picture of this shift: control moves from goal-directed evaluation toward hierarchical habit-like control as computation cost falls.

Stage 3 - Habit

The prior over the policy becomes so concentrated that behaviour runs almost automatically. This is the attractor floor.

Dolan and Dayan frame this in dual-system terms: costly online valuation versus cached policy selection. The brain appears to allocate control toward the lower-cost solution.

Habit Formation as Bandwidth Allocation

The deeper point is that habits are not just rigid routines. They are strategic bandwidth allocation.

By automating predictable contexts, the system frees inferential capacity for uncertainty-rich domains:

Kurzban et al. argue that effort is better modeled as opportunity cost than as fuel depletion. Habits eliminate that cost for predictable domains, leaving more capacity for exploration.

Context as the Trigger

Habits are context-bound because the generative model learns context -> policy associations.

This is why:

• environment design works
• habit stacking works
• radical context change can destabilize entrenched routines

Wood and colleagues show that when contexts change, habits often break even if motivation has not changed. The cue that triggered the policy disappears, so the prior loses its grip.

The Failure Mode

Without stable habits, too much inference budget gets consumed by low-level life management. When everything is cognitively expensive, the system retreats to its strongest pre-existing priors.

This gives a cleaner explanation for cognitive overload: when free energy is high everywhere, the agent falls back to the cheapest available policies.

Structure is not the enemy of creativity. It is often the thermodynamic prerequisite for it. Automate the predictable; explore the uncertain.

6.5 The Explorer Gene in Personal Life

• explorers in research, art, and entrepreneurship often have low activation energy for novelty
• novelty exposure can be understood as behavioural annealing
• richer lives may be the ones that continually reduce expected free energy by building better models of uncertainty

The unifying insight is simple: the optimal strategy is not to stay in the first valley you find. It is to explore the landscape, discover where the deeper valleys are, and repeatedly cross the ridges that separate them.

---

Synthesis: The Universal Energy Landscape

One text version of the synthesis is:

1. Statistical physics gives the Boltzmann view: lower-energy states are more probable.
2. Information theory recasts this as entropy and information gain.
3. Machine learning turns it into loss minimization and variational free energy.
4. EBMs and diffusion make energy landscapes explicit again.
5. Protein folding shows biology navigating the same geometry.
6. AlphaFold demonstrates that learned models can internalize these biological energy constraints.
7. FEP and active inference extend the same logic to brains, perception, action, and uncertainty reduction.
8. Explorer-gene and behaviour arguments show the same principles shaping curiosity, habit, identity, and growth.

All of them converge on one recurring principle:

minimize free energy across scales

---

References

1. Statistical Physics and Thermodynamics

• Jaynes, E.T. (1957). Information Theory and Statistical Mechanics. Physical Review.
• Boltzmann, L. (1877). On the Relationship Between the Second Fundamental Theorem of Mechanical Heat Theory and Probability Calculations.
• Landau, L.D. and Lifshitz, E.M. Statistical Physics. Pergamon Press.

2. Information Theory and ML

• Shannon, C.E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal.
• MacKay, D. (2003). Information Theory, Inference, and Learning Algorithms. Cambridge University Press.
• LeCun, Y. et al. (2006). A Tutorial on Energy-Based Learning.
• LeCun, Y. (2022). A Path Towards Autonomous Machine Intelligence.
• Song, Y. and Ermon, S. (2019). Generative Modeling by Estimating Gradients of the Data Distribution.

3. Energy-Based Models

• Hinton, G. and Sejnowski, T. (1983). Optimal Perceptual Inference.
• Hopfield, J. (1982). Neural networks and physical systems with emergent collective computational abilities.
• Du, Y. and Mordatch, I. (2019). Implicit Generation and Modeling with Energy Based Models.

4. Protein Folding

• Anfinsen, C.B. (1973). Principles that Govern the Folding of Protein Chains. Science.
• Jumper, J. et al. (2021). Highly Accurate Protein Structure Prediction with AlphaFold. Nature.
• Dill, K.A. and MacCallum, J.L. (2012). The Protein-Folding Problem, 50 Years On. Science.

5. Free Energy Principle and Cognition

• Friston, K. (2010). The free-energy principle: a unified brain theory? Nature Reviews Neuroscience.
• Friston, K. et al. (2013). Exploration, novelty, surprise, and free energy minimization. Frontiers in Psychology.
• Parr, T., Pezzulo, G., and Friston, K. (2022). Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. MIT Press.

6. Explorer Gene and Behaviour

• Ebstein, R. et al. (1996). Dopamine D4 receptor (DRD4) exon III polymorphism associated with Novelty Seeking. Nature Genetics.
• Chen, C. et al. (1999). Population migration and the variation of dopamine D4 receptor allele frequencies. Evolution and Human Behavior.
• Matthews, L. and Butler, P. (2011). Novelty-seeking DRD4 polymorphisms associated with human migration distance out-of-Africa. American Journal of Physical Anthropology.
• Hutchinson, A. (2025). The Explorer's Gene. HarperCollins.
• Hutchinson, A. (2018). Endure. William Morrow.
• Dezfouli, A. and Balleine, B.W. (2013). Actions, action sequences and habits: evidence that goal-directed and habitual action control are hierarchically organised. PLOS Computational Biology, 9(12).
• Dolan, R.J. and Dayan, P. (2013). Goals and habits in the brain. Neuron, 80(2), 312-325.
• Kurzban, R., Duckworth, A., Kable, J.W., and Myers, J. (2013). An opportunity cost model of subjective effort and task performance. Behavioral and Brain Sciences, 36(6), 661-679.
• Wood, W., Tam, L., and Witt, M.G. (2005). Changing circumstances, disrupting habits. Journal of Personality and Social Psychology, 88(6), 918-933.
• Friston, K. et al. (2016). Active inference and learning. Neuropharmacology, 108, 52-63.

Settled · 1 Apr 2026