Speaker
Description
Developed by the Science and Technology Facilities Council (STFC) and Cordin Scientific Imaging, presented in this paper will be the Time Pixel Multiplexing (TPM) sensor, a high-speed camera with variable frame depth. Based around the principle first published by Gil Bub [1], TPM is a 1024x1024 CMOS image sensor with the ability to image at speeds up to 10Mfps by changing which pixels are active through time.
In a conventional CMOS image sensor, the array of pixels can either be active all at the same time with global shutter, or in a row-by-row basis as in rolling shutter, but in both the final end goal is to create one coherent snapshot of a moment in time. This means that to achieve high-speed imaging in one device a combination of fast pixel-level readout with exponentially faster full sensor readout [2] or by increasing the pixel size through in-pixel memory with a slow readout [3].
However, by carefully controlling when each pixel is active a subset across the full array can be integrating at the same time whilst another is reading out and a third set are held in reset before integration, as shown in a rough timing diagram in Figure 1. In this way it possible to capture one single image that can then be processed into a sequence of variable frame depth where the only trade-off is the final resolution. Shown below in Figure 2 is one such image where a 4x4 TPM mode has been used to produce a 16-frame video at 256x256 pixel resolution. The TPM mode can be varied between the two extremes as the user desires, from a single 1 Mpixel frame up to over 1 million sequential frames at a single pixel resolution, all at speeds of up to 10Mfps.
This paper expands on initial results published in 2021 [4], and aims to give an overview of the technology, the challenges and developments required within, as well as present results from the first commercially available camera using the STFC-developed TPM sensor from Cordin Scientific Imaging.
Figures:
Figure 1: Simplified readout control graph showing a set of four pixels being held in reset, integrating, being sampled, and then read into the column as each line set is activated.
Figure 2: An example 4x4 video of a bridged wire test: a) The complete 1024x1024 frame; b) separated into 16 frames, with time increasing from top left to bottom right vertically; and c) a single 256x256 image
References:
[1] Bub G., Tecza M., Helmes M., Lee P., Kohl P., “Temporal pixel multiplexing for simultaneous high-speed, high-resolution imaging.” Nat Methods. 2010 Mar;7(3):209-11
[2] Cremers, B., Agarwal, M., Walschap, T., Singh, R., & Geurts, T. (2009, June). “A high speed pipelined snapshot CMOS image sensor with 6.4 Gpixel/s data rate”. Proc. 2009 International Image Sensor Workshop
[3] Crooks, J., B. Marsh, R. Turchetta, K. Taylor, W. Chan, A. Lahav, and Amos Fenigstein. “Kirana: a solid-state megapixel uCMOS image sensor for ultrahigh speed imaging.” Sensors, cameras, and systems for industrial and scientific applications XIV, vol. 8659, pp. 36-49. SPIE, 2013.
[4] Krukauskas, D., Marsh, B., Sedgwick, I., Guerrini, N., & Benhammadi, S. “A Novel Ultra-High-Speed CMOS Image Sensor Implementation with Variable Spatial and Temporal Resolution using Temporal Pixel Multiplexing.”, Proc. 2021 International Image Sensor Workshop