Learning as Theory of Everything

• Computer science is a science of mechanisms. It is a theory of what mechanisms are in computers, biology, psychology, and economics.
• The Church-Turing Thesis asks what is mechanistically computable, and claims that what is mechanistically computable is exactly what is computable by a Turing machine. This is the foundational claim: Turing machines capture the full extent of mechanical and algorithmic computation.
• Leslie Valiant's "A Theory of the Learnable" (1984) asks the analogous learning question. Just as Turing formalized what is computable, Valiant formalized what is learnable.

PAC Learning as the Learnability Boundary

The PAC framework -- Probably Approximately Correct -- says that a concept class is learnable if there exists an algorithm that, given enough examples, can with high probability output a hypothesis that is approximately correct.

In more formal terms:

• a concept class is learnable if, with probability at least $1 - \delta$, the learner outputs a hypothesis whose error is at most $\epsilon$
• crucially, it must do so in polynomial time and with a polynomial number of examples
• this is exactly where the resource-measurement idea from complexity theory enters learning theory

That gives the core parallel:

Why This Starts Looking Like a Theory of Everything

Valiant later argued in Probably Approximately Correct that PAC learning is not just a framework for machine learning, but for understanding how nature learns and evolves.

• evolution can be seen as a PAC-learning process
• organisms "learn" to fit environments with bounded resources and imperfect information
• learning becomes a computational mechanism that operates across computer science, biology, psychology, and economics

This creates a larger arc:

Computability -> Complexity -> Learnability

Each layer asks a deeper question about what mechanisms can achieve under resource constraints.

The Two Preconditions for Learnability

Before asking whether something can be learned, two practical preconditions matter:

1. Teachability -- Is the problem solvable by training a human, organism, or algorithm?
2. Data availability -- Is there some process that generates the training signal?

If both hold, then the real question becomes: can it be learned efficiently?

If Learning Is a Theory of Everything, What Must It Contain?

Any "Theory of Everything" needs:

1. Fundamental entities -- What are the basic objects?
2. Laws or dynamics -- What rules govern how they interact?
3. State space -- What configurations are possible?
4. Conservation or constraints -- What resources are bounded?
5. Emergence -- How does complex behavior arise from simple rules?
6. Universality -- Does it apply across domains?

The learning-theoretic counterparts look like this:

The Five Essential Components of a Complete Theory of Learning

For learning to qualify as a real theory of everything for adaptive systems, it must specify at least five things.

1. What is being learned?

This is the target concept or function class.

• in machine learning: a function $f$ mapping inputs to outputs
• in biology: the fitness landscape
• in economics: the equilibrium strategy
• in psychology: the cognitive model or skill

2. What learns?

This is the learner or hypothesis class.

• in machine learning: neural network, decision tree, kernel method, transformer
• in biology: genome plus phenotype
• in economics: a rational or bounded-rational agent

The learner must be expressive enough, but not too expressive. That is the familiar bias-variance tension in another form.

3. What is the feedback signal?

This is the data or loss.

• in machine learning: labeled examples, rewards, preferences
• in biology: survival and reproduction
• in economics: profit and utility
• in psychology: sensory feedback and reinforcement

This is exactly where the second precondition matters: if no meaningful training signal exists, learning cannot happen.

4. What are the resource constraints?

This is the complexity layer.

• how many samples?
• how much computation?
• how much memory?
• how much time?

This is what separates learnable in principle from learnable in practice. It plays the same role that complexity theory plays for computation.

5. What guarantees can we give?

This is the generalization layer.

• PAC says: with probability at least $1 - \delta$, error is at most $\epsilon$
• this is the analogue of predictive power in physics
• a serious theory has to say something falsifiable about what will happen on unseen situations, not only on past observations

The Deeper Unification

In physics, a theory of everything tries to unify forces. Here, a theory of learning would unify adaptation mechanisms:

• natural selection = learning over evolutionary time
• neural plasticity = learning over a lifetime
• market dynamics = learning over economic time
• SGD on neural networks = learning over training time

What makes this striking is that all of them share the same five components above. The instantiation changes, but the skeleton remains the same.

That is what makes learning a candidate theory of everything for complex adaptive systems.

From Learnable to Educable

Valiant's later work moves beyond PAC learning. In The Importance of Being Educable and in his work on the educability of models, he pushes the question further.

The three levels are:

1. Learnable -- learning from examples and data alone. Passive observation. This is closest to what evolution does.
2. Teachable -- learning from a teacher who curates examples and structures a curriculum. More efficient than raw exposure.
3. Educable -- learning from instruction in a shared language: explanations, definitions, rules, analogies, symbolic reasoning.

This is the deeper ladder:

• Turing -> What is computable?
• Complexity theory -> What is efficiently computable?
• Valiant (1984) -> What is learnable from data?
• Valiant (2024) -> What is educable from language plus reasoning?

Why Educability Matters

Educability may be the real boundary between human intelligence and other forms of adaptation.

• evolution -> learnable
• animals -> teachable
• humans -> educable

Humans do not just absorb examples. They absorb rules through language, integrate them into existing beliefs, and apply them to situations they have never seen before.

That makes educability a frontier question across:

• learning theory
• AI
• linguistics
• cognitive science

and maybe the central question for whether machines can actually reach human-level intelligence.

What Educable Means in Practice

An educable system can do three things:

1. Absorb rules from language

This is not just "here are examples, infer the pattern." It is "here is a principle, understand it." It is closer to reading a textbook than doing blind trial-and-error.

2. Integrate new knowledge with existing beliefs

The new knowledge has to fit with what the system already knows. If it conflicts, the system should recognize the conflict and revise beliefs coherently. This is a belief revision problem, not just a pattern-matching problem.

3. Apply the combined knowledge to novel situations

Once the rule is learned, it should generalize compositionally.

For example:

• a PAC-learnable system sees many examples of cats and dogs and eventually classifies them
• an educable system can be told: "A mammal is warm-blooded. A whale is a mammal. Therefore a whale is warm-blooded."

The system may never have seen a whale directly, yet it can reason about whales by combining a rule with a fact.

That is the difference between learning from data and learning from language plus reasoning.

The Frontier Question for LLMs

Large language models appear educable because they respond to instructions and explanations. But the hard question is whether they truly integrate knowledge the way humans do, or whether they only simulate educability through extremely strong pattern completion.

So the core question becomes:

What is the formal boundary of what can be learned through linguistic instruction and reasoning, rather than only through data?

That is where learning theory, AI, language, and cognition all collapse into the same question.

Why I Keep Returning to This

If computation gave us the theory of mechanisms, and complexity gave us the theory of feasible mechanisms, then learning may give us the theory of adaptive mechanisms.

That is why "learning as theory of everything" feels plausible to me. It is not a claim that learning explains all of physics. It is a claim that for systems that adapt, survive, update, generalize, coordinate, or evolve, learning may be the most general formal language we have.

And if that is true, then the deepest version of the question is no longer only:

What is learnable?

It becomes:

What is educable?

---

References

1. Turing, A. M. (1936). "On Computable Numbers, with an Application to the Entscheidungsproblem." Proceedings of the London Mathematical Society, 2(42), 230-265. Link
2. Valiant, L. G. (1984). "A Theory of the Learnable." Communications of the ACM, 27(11), 1134-1142. Link
3. Valiant, L. G. (2013). Probably Approximately Correct: Nature's Algorithms for Learning and Prospering in a Complex World. Basic Books. Link
4. Valiant, L. G. (2024). The Importance of Being Educable. Princeton University Press. Link

Settled · 30 Mar 2026