Date and time: Friday 16 October 2026, 08:50-09:50 CEST
Speaker: Prof Pierre Baldi, University of California Irvine
Title: Synaptic Neural Balance
Where: Digital Futures hub, Osquars Backe 5, floor 2 at KTH main campus OR Zoom
Directions: https://www.digitalfutures.kth.se/contact/how-to-get-here/
OR Zoom: https://kth-se.zoom.us/j/69560887455
Host: Alexandre Proutiere alepro@kth.se
Bio: Pierre Baldi earned MS degrees in Mathematics and Psychology from the University of Paris, and a PhD in Mathematics from the California Institute of Technology. He is currently Distinguished Professor in the Department of Computer Science, Founding Director of the AI in Science Institute, and Associate Director of the Center for Machine Learning and Intelligent Systems at the University of California Irvine. The long term focus of his research is on understanding intelligence in brains and machines.
He has made several contributions to the theory of AI and deep learning, and pioneered the application of AI to the natural sciences, to address problems in physics, chemistry, and biomedicine. Examples of application problems include the detection of exotic particles in physics, the prediction of protein structures and of chemical reactions in biochemistry, and the analysis of genomes and images in bio-medicine.
Baldi is currently also studying some of the societal challenges posed by AI, such as the tension between academic and corporate AI research and the quest for AI safety frameworks. He has published ~400 journal articles and five books, including: Deep Learning in Science, Cambridge University Press, 2021. His honors include the 1993 Lew Allen Award at JPL, the 2010 E. R. Caianiello Prize for research in machine learning, the 2023 Dennis Gabor Award of the International Neural Network Society, the 2027 IEEE Neural Network Pioneer Award, and election to Fellow of the AAAS, AAAI, IEEE, ACM, and ISCB.
He serves as Associated Editor for Artificial Intelligence, Neural Networks, and the IEEE/ACM Transactions in Computational Biology and Bioinformatics. He has mentored ~100 graduate students and postdoctoral fellows and co-founded several startup companies.
Abstract: We develop a general theory of synaptic neural balance and how it can emerge or be enforced in neural networks. For a given regularizer, a neuron is said to be in balance if the total cost of its input weights is equal to the total cost of its output weights. The basic example is provided by feedforward networks of ReLU units trained
with L2 regularizers, which exhibit balance after proper training. The theory explains this phenomenon and extends it in several directions.
The first direction is the extension to bilinear and other activation functions. The second direction is the extension to more general regularizers, including all Lp regularizers. The third direction is the extension to non-layered architectures, recurrent architectures, convolutional architectures, as well as architectures with mixed activation functions. Gradient descent on the error function alone does not converge in general to a balanced state, where every neuron is in balance, even when starting from a balanced state. However, gradient descent on the regularized error function ought to converge to a balanced state, and thus network balance can be used to assess learning progress.
The theory is based on two local neuronal operations: scaling which is commutative, and balancing which is not commutative. Given any initial set of weights, when local balancing operations are applied to each neuron in a stochastic manner, global order always emerges through the convergence of the stochastic balancing algorithm to the same unique set of balanced weights. The reason for this is the existence of an underlying strictly convex optimization problem where the relevant variables are constrained to a linear, only architecture-dependent, manifold. Simulations show that balancing neurons prior to learning, or during learning in alternation with gradient descent steps, can improve learning speed and final performance. Finally, we show how synaptic balancing is intimately connected to questions of neural overparameterization, and group symmetries, and discuss the relevance of synaptic neural balance for neurobiological or neuromorphic neural networks.
