26th Australian Institute of Physics Congress and the Australian and New Zealand Conference on Optics and Photonics

Dynamic optical transfer functions for computational neuromorphic imaging and their application

Not scheduled

20m

Belinda Hutchinson Building (The University of Sydney )

Contributed Oral ANZOS | Photonics and Optics (ANZCOP)

Description

Event cameras report pixel-level log-intensity threshold crossings asynchronously, offering benefits in dynamic range, sensing latency, and data bandwidth. The conventional trade-off between sensing frequency and signal intensity is replaced by balancing contrast thresholds and temporal filtering, opening new possibilities for dynamic applications.
Computational optical imaging has long advanced through physical interpretation of sensor data, solving inverse problems from first principles. The challenge of applying these neuromorphic modalities to generalised "non-imaging" optical systems (wavefront sensing, light-field imaging, spectral imaging, advanced microscopy) becomes apparent when relating events to physics-based information. Unlike conventional pixels, asynchronous events lack a direct photon-flux representation. Neuromorphic engineers, therefore, continue to treat optics, sampling, and computation in isolation.
In this work, we present an optical forward model adapted for dynamic event-based sensors and consider their implementation for computational imaging application, dubbed Computational Neuromorphic Imaging (CNI). We consider event interpolation methods, intensity-derivative inference per event, and spatial-temporal basis transformations (e.g. Fourier transforms on event stream data), to effectively convey scene imaging. We move beyond the model to real-world event-data, and consider the effect of noise events, intensity dependant latency, and other challenges of real-world event pixels.
As a demonstration, we use a liquid lens with focus modulation while imaging a point source. We design the correct time-varying optical transfer function and solve a source localisation problem on the event stream data. The results are generalised to a wider variety of CNI systems, and we discuss the roadmap to get neuromorphic sensing up to speed with contemporary computational imaging methods. CNI opens a range of design possibilities for various non-imaging applications - where broad signal bandwidths, large dynamic ranges, and desire for low SWaP-C solutions are key.