Speaker
Description
The automation of complex experimental systems has been a long-standing goal with numerous real-world applications. While reinforcement learning (RL) has achieved breakthroughs across many domains, prior approaches to physical system control required crafting realistic simulations, complex reward engineering, and extensive training times, limiting practical deployment.
Saha et.al. (2025) introduce a “quantized utility agent” (AQUA), a novel framework that aims to simplify RL for autonomous control in complex physical systems. It declutters the learning process, allowing operational control agents to be pre-trained offline using existing datasets, boosting sample efficiency significantly.
The framework employs generative "world models" that extract meaningful representations from noisy experimental data, encoding inputs and outputs into a compressed feature space that captures process states without explicit temporal ordering. During deployment, AQUA executes efficient searches in this compressed space to imagine outcomes and plan optimal actions through population refinement techniques.
Demonstrated performance across complex optical [1] and quantum systems, AQUA performs optimally out-of-the-box, when deployed on physical systems, adapts to changing conditions, fine-tunes online and surpasses human-level performance.
Emerging quantum computing, communications, sensing and space technologies are carefully assembled, collection of inter-dependent components. As the complexity of these systems grows, maintaining optimal performance manually becomes a major bottleneck, especially for remotely deployed systems. AQUA’s task-agnostic, pre-trainable and adaptable design could effectively streamline these processes, allowing them to scale.
Besides saving hours of human labour, the method’s implied ability to learn features from raw experimental data could, in principle, be used to study the underlying properties of the physical system that are hard to describe in theory, potentially bridging the gap [2].
References:
- A. Saha, et. al., Automating experimental optics with
sample-efficient machine learning methods, Optica 12, 1304-1310
(2025) - S.J. Wetzel, et.al., Interpretable Machine Learning in Physics: A
Review, arXiv:2503.23616 (2025)
