A repeating four-phase training cycle comparing backprop, predictive coding, and PC-ALM. A narrated comparison of backprop, predictive coding, and PC-ALM. A text transcript follows the figure. Training phase clamp input + target Backprop forward pass backward pass PC + PC-ALM inference weight update Backprop TARGET loss INPUT Predictive coding TARGET INPUT PC-ALM OURS TARGET INPUT loss loss weight credit weight credit weight credit 103010030010005060708090100depth L (layers)MNIST test accuracy (%)Backprop (global)PC-ALM (local)PC (local) prediction error shifted prediction error

Backprop runs in three phases: forward pass, backward pass, and weight update. Neurons in early layers must hold their activations until error signals arrive. But the brain has no known mechanism to enforce this timing across the network.

Predictive Coding replaces forward and backward passes with local dynamics. Layers interact with their neighbors, without separate forward and backward phases. More biologically plausible. But credit decays with depth.

PC-ALM adds dual neurons (Lagrange multipliers) at each layer. Dual neurons accumulate prediction errors, turning each layer into a local PI feedback controller. Credit spreads more evenly throughout the network. Together, these local dynamics recover backprop’s credit signals (exactly at convergence in linear networks).

PC-ALM propagates credit through 1,000 layers using only local dynamics. The plot shows MNIST test accuracy against depth for a deep narrow residual ReLU network of width 32, trained for five epochs, mean and range over three seeds.

The animation uses the original schematic trajectories. Pauses are for explanation, not measures of computation time.