Speaker
Description
Deep learning classifiers for high-energy physics have recently advanced along two largely separate tracks: self-supervised pretraining on large unlabelled datasets, and architectures that respect Lorentz symmetry by construction. How these directions combine has not been shown.
To address this, we integrate Lorentz-invariant models into the LeJEPA pretraining framework, in which the network is trained to produce a latent representation that is invariant under predefined augmentations. This naturally aligns with the philosophy of Lorentz-invariant models, where invariance under augmentations consisting purely of a Lorentz transformation is trivially enforced by the network.
We evaluate on JetClass under matched pretraining data, augmentations, and optimization, varying only the backbone. Under this comparison the Lorentz-invariant model achieves substantially superior linear separability and downstream tagging accuracy relative to a non-invariant scalar transformer, showing that exact physical symmetries and self-supervised pretraining are complementary rather than competing formulations.