How does a neural network learn? A useful way to approach this question is through the idea of compression: learning can be understood as the process of finding a compact representation of data while retaining the information that matters. In this talk, I will explore this perspective from the perspective of model compression, where the model itself is progressively simplified. Our recent work shows that this process is closely connected to the geometry of deep neural-network error landscapes. As a model is compressed, it does not simply lose accuracy gradually; instead, it passes through a hierarchy of phase transitions, each corresponding to the loss of particular features. These transitions are governed by saddle points in the high-dimensional landscape and reveal a surprisingly ordered structure of learned representations. This suggests a broader view of learning: phase transitions marking the points at which new levels of representation emerge.

Joint work with Talha Ersoy, Andrés Cardozo Licha, and Björn Ladewig

Relevant references:

Ersoy, Ibrahim Talha, Andrés Fernando Cardozo Licha, and Karoline Wiesner. “Phase transitions reveal hierarchical structure in deep neural networks.” arXiv preprint arXiv:2512.11866 (2025). — published in the International Conference on Machine Learning (ICML) Workshop: High-dimensional Learning Dynamics. 2025.

Ladewig, Björn, Ibrahim Talha Ersoy, and Karoline Wiesner. “Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networks.” arXiv preprint arXiv:2608.06597 (2026)

Ersoy, Ibrahim Talha, and Karoline Wiesner. “Geometry of learning-L2 phase transitions in deep and shallow neural networks.” arXiv preprint arXiv:2505.06597 (2025) — accepted in Nature Communications AI & Computing (2026)

Ersoy, Ibrahim Talha, and Karoline Wiesner. “Noise-Driven Escape from Metastable Phases explains Grokking in Deep Neural Networks.” arXiv preprint arXiv:2606.17120 (2026). — Published in the International Conference on Machine Learning (ICML) Workshop: High-dimensional Learning Dynamics. 2026.