The Problem

Predict the motion of a double pendulum over a thousand time steps. A neural network trained on trajectories does well early on — but by step 500, something is wrong. The total energy of the system is growing. The pendulum is gaining energy from nowhere. By step 1000, the predictions are physically impossible.

Neural networks don't know about conservation laws. They learn patterns from trajectories, but they are free to violate the underlying physics that generated those trajectories. Over short horizons, this usually doesn't matter. Over long horizons, the accumulated violations compound and predictions become useless — or worse, misleading.

The Insight

Physics gives us structure. For a large class of physical systems — mechanical, electrical, thermal, fluid — there is a mathematical framework that describes how energy flows, dissipates, and is exchanged between parts. This structure is not something to be learned from data. It is known in advance.

A soft penalty during training says "try not to violate energy conservation." A hard structural constraint built into the architecture says "this network cannot violate energy conservation, no matter what weights it learns." This is the difference between hoping for physics and guaranteeing it.

What Was Built

A neural ODE where the dynamics are governed by three components: one that routes energy between parts of the system, one that controls how energy dissipates, and one that represents the total energy itself. All three are parameterized as neural networks — but with architectural constraints that enforce the right mathematical properties. The energy function is always positive. The routing component is always antisymmetric. The dissipation component always removes energy, never adds it.

The integration scheme is also chosen carefully. Standard numerical integrators accumulate errors in a way that adds or removes energy over time. A symplectic integrator — the kind designed specifically for Hamiltonian systems — preserves the geometric structure of the trajectories. The combination means both the network and the integrator respect the physics.

What Happened

On conservative mechanical systems — pendulum, double pendulum, spring-mass — energy drift over 1000 steps is 0.8% compared to 8.9% for standard neural ODEs: a 91% reduction. Long-horizon predictions stay physically valid where unconstrained models fall apart.

Dissipative systems (where energy is intentionally lost, like a damped oscillator) and port-coupled systems (where energy is exchanged between subsystems) are the harder cases — the architecture handles them, but the training dynamics are less stable. That is the current frontier of this work.

Ongoing. Conservative systems complete. Dissipative and port-coupled extensions in progress.