Speaker
Description
We introduce SPADE (SPlit And Delay Embeddings), which embeds each feature of a token independently and staggers the resulting streams along the sequence with progressively increasing delays. The vocabulary then scales additively rather than multiplicatively, while intra-token correlations are recovered by the ordinary causal self-attention mechanism: each feature is predicted at its own sequence position, conditioned on the features already emitted for the same object. No auxiliary decoder or quantization stage is required.
We demonstrate SPADE on point-cloud shower generation in the highly granular ILD electromagnetic calorimeter. SPADE is competitive with state-of-the-art flow-matching on photon showers and substantially outperforms VQ-VAE-based predecessors. Against a joint-vocabulary baseline at the finest granularity, SPADE uses 74× fewer parameters and converges 6.9× faster in GPU hours, while better reproducing observables sensitive to energy–position correlations.
By eliminating the need for quantized codebooks, SPADE enables direct, LLM-style autoregressive pretraining on multi-feature sensor data across fundamental physics.
Paper: https://arxiv.org/abs/2606.11304