Speaker
Description
Consistent, abstract descriptions of learning dynamics in neural networks are still uncommon, yet such descriptions appear across many scientific fields. Consequently, predicting how dynamics change with different ML model parameters can fail dramatically, and preventing these failures is challenging. Reliable control therefore demands a deep understanding of mechanisms and conditions that enable learning for specific kinds of datasets. We study autoencoder architectures that succeed when they compress data into representations that capture the data’s underlying physical concepts and then learn an inverse mapping to reconstruct the original physical inputs. Certain physical concepts are acquired in a particular sequence, determined by the representational complexity and the architecture’s theoretical capacity. We evaluate generalization against strict theoretical baselines and analyze the information geometry, stability, and physical interpretability of the latent space throughout training.