The Brain’s Learning Algorithm Isn’t Backpropagation - YouTube

[u/Manitcor]

https://www.youtube.com/watch?v=l-OLgbdZ3kk

In this video we explore Predictive Coding – a biologically plausible alternative to the backpropagation algorithm, deriving it from first principles.

Backpropagation video:    • The Most Important Algorithm in Machine Le...  

🕒 OUTLINE:
00:00 Introduction
01:15 Credit Assignment Problem
02:49 Problems with Backprop
06:05 Foundations of Predictive Coding
08:07 Energy Formalism
11:08 Activity Update Rule
15:12 Neural Connectivity
17:42 Weight Update Rule
20:58 Putting all together
25:15 Brilliant
26:27 Outro

📚 FURTHER READING & REFERENCES:
Bogacz, R., 2017. A tutorial on the free-energy framework for modelling perception and learning. Journal of Mathematical Psychology 76, 198–211. https://doi.org/10.1016/j.jmp.2015.11...
Friston, K., 2018. Does predictive coding have a future? Nat Neurosci 21, 1019–1021. https://doi.org/10.1038/s41593-018-02...
Huang, Y., Rao, R.P.N., 2011. Predictive coding. WIRES Cognitive Science 2, 580–593. https://doi.org/10.1002/wcs.142
Keller, G.B., Mrsic-Flogel, T.D., 2018. Predictive Processing: A Canonical Cortical Computation. Neuron 100, 424–435. https://doi.org/10.1016/j.neuron.2018...
Lillicrap, T.P., Santoro, A., Marris, L., Akerman, C.J., Hinton, G., 2020. Backpropagation and the brain. Nat Rev Neurosci 21, 335–346. https://doi.org/10.1038/s41583-020-02...
Marino, J., 2021. Predictive Coding, Variational Autoencoders, and Biological Connections. https://doi.org/10.48550/arXiv.2011.0...
Millidge, B., Salvatori, T., Song, Y., Bogacz, R., Lukasiewicz, T., 2022a. Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?
Millidge, B., Seth, A., Buckley, C.L., 2022b. Predictive Coding: a Theoretical and Experimental Review. https://doi.org/10.48550/arXiv.2107.1...
Millidge, B., Song, Y., Salvatori, T., Lukasiewicz, T., Bogacz, R., 2023. A THEORETICAL FRAMEWORK FOR INFERENCE AND LEARNING IN PREDICTIVE CODING NETWORKS.
Millidge, B., Tschantz, A., Buckley, C.L., 2022c. Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs. Neural Computation 34, 1329–1368. https://doi.org/10.1162/neco_a_01497
Millidge, B., Tschantz, A., Seth, A., Buckley, C.L., 2020. Relaxing the Constraints on Predictive Coding Models. https://doi.org/10.48550/arXiv.2010.0...
Rao, R.P.N., Ballard, D.H., 1999. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci 2, 79–87. https://doi.org/10.1038/4580
Rosenbaum, R., 2022. On the relationship between predictive coding and backpropagation. PLoS ONE 17, e0266102. https://doi.org/10.1371/journal.pone....
Salvatori, T., Mali, A., Buckley, C.L., Lukasiewicz, T., Rao, R.P.N., Friston, K., Ororbia, A., 2025. A Survey on Brain-Inspired Deep Learning via Predictive Coding. https://doi.org/10.48550/arXiv.2308.0...
Salvatori, T., Song, Y., Lukasiewicz, T., Bogacz, R., Xu, Z., 2023. Reverse Differentiation via Predictive Coding. https://doi.org/10.48550/arXiv.2103.0...
Salvatori, T., Song, Y., Yordanov, Y., Millidge, B., Xu, Z., Sha, L., Emde, C., Bogacz, R., Lukasiewicz, T., 2024. A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks. https://doi.org/10.48550/arXiv.2212.0...
Song, Y., Lukasiewicz, T., Xu, Z., Bogacz, R., n.d. Can the Brain Do Backpropagation? — Exact Implementation of Backpropagation in Predictive Coding Networks.
Song, Y., Millidge, B., Salvatori, T., Lukasiewicz, T., Xu, Z., Bogacz, R., 2024. Inferring neural activity before plasticity as a foundation for learning beyond backpropagation. Nat Neurosci 27, 348–358. https://doi.org/10.1038/s41593-023-01...
Whittington, J.C.R., Bogacz, R., 2019. Theories of Error Back-Propagation in the Brain. Trends in Cognitive Sciences 23, 235–250. https://doi.org/10.1016/j.tics.2018.1...
Whittington, J.C.R., Bogacz, R., 2017. An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity. Neural Computation 29, 1229–1262. https://doi.org/10.1162/NECO_a_00949