Event-based vision sensors unlock unexpected potential for super-resolution imaging

Cutting-edge research has yielded significant advancements in super-resolution fluorescence imaging through the use of event-based sensors (EBS for event-based sensor). In a collaborative effort led by researchers at the Langevin Institute (ESPCI Paris, CNRS, PSL University), in collaboration with DHNS (INSERM, Paris-Saclay University) and the Vision Institute (CNRS, INSERM, Sorbonne University), these sensors have outperformed traditional scientific cameras in single-molecule localization microscopy (SMLM), pushing the limits of temporal resolution.

Historically, the diffraction limit of light has constrained optical microscopes to a resolution of 200-300 nanometers. However, in the 21st century, intensified efforts by researchers have overcome this barrier, leading to the 2014 Nobel Prize in Chemistry for super-resolution fluorescence microscopy. A key method, SMLM, relies on labeling proteins of interest with fluorescent molecules that blink, allowing them to be localized one by one with nanometric precision. Thus, super-resolution images are reconstructed like a pointillist painting, surpassing the resolution limited by diffraction by an order of magnitude.

Event-based sensors, also known as neuromorphic vision sensors, operate differently than conventional cameras. Inspired by the human eye, these sensors consist of asynchronous pixels that react to changes in brightness with remarkable temporal precision. With their low energy consumption, reduced data volume, fast response times, and cost-effectiveness, event-based sensors have considerable potential in the field of biological imaging, especially for capturing the asynchronous and dynamic behavior of SMLM fluorophores.

Researchers have successfully implemented event-based single-molecule localization microscopy, called Eve-SMLM. This new approach not only produces super-resolution images of quality and resolution comparable to those of traditional scientific cameras but also overcomes the limitations of high-density imaging, where images of individual fluorophores overlap. While conventional cameras fail in such scenarios, Eve-SMLM excels in selectively detecting ON and OFF transitions, extracting crucial information, and performing efficient temporal resampling.

Eve-SMLM marks a paradigm shift for single-molecule microscopy and its applications. This new approach could be the key to studying multi-scale processes in space and time, which are ubiquitous in biology. This study illustrates the transformative potential of event-based detection in super-resolution fluorescence imaging, offering new perspectives in biology, medical applications, materials science, and beyond.

On the left, the principle of event-based detection: response of a conventional camera and an event-based sensor (EBS for event-based sensor). a: simulated profile I(t) of a fluorophore turning on and off; b: signal returned by an ideal camera with an exposure time of 30 ms (corresponding simulated images displayed above the graph); c: signal returned by an EBS, with positive and negative events generated when the molecule turns on and off. On the right, Eve-SMLM super-resolution fluorescence images of fixed COS-7 cells labeled with the AF647 fluorophore against α-tubulin. d-e: 2D Eve-SMLM image obtained with all photons directed towards the event-based sensor. f-g-h: high-density bioimaging obtained with a 50:50 beamsplitter splitting emission photons between the EBS and an EMCCD camera, total acquisition time of 250 s. f: overall view obtained with the EBS; g and h show an enlarged region obtained with the EBS and the EMCCD, respectively, highlighting the superior performance of the EBS in the high-density regime.

Video (.gif) to download here :
https://dl.espci.fr/ticket/406c10be05cb87535dd26b3a15345e51
Blinking (left) and super-resolution reconstruction (right) of the same region of tubulin labeled with the AF647 fluorophore in fixed COS-7 cells. The total acquisition time is 50 seconds, accelerated for reconstruction visualization. The dimensions of the region of interest are 20 µm x 20 µm.

On the left, we have reconstructed blinking information obtained by EBS in the form of images on a 20 ms time basis. For each image i, all events occurring in the time interval [i × 20 ms, (i + 1) × 20 ms[ are displayed. Positive and negative events are color-coded, respectively in green and red. White squares indicate the blinking signals of single molecules recognized by the detection and super-localization algorithm. The sub-pixel coordinates obtained for each detected fluorophore are used in the reconstruction on the right to build a super-resolution image.

PUBLICATION
C. Cabriel, T. Monfort, C. G. Specht, and I. Izeddin. Event-based vision sensor for fast and dense single-molecule localization microscopy. Nat. Photon. (2023). https://doi.org/10.1038/s41566-023-01308-8

CONTACT

· Ignacio Izeddin, associate professor at ESPCI Paris; Institut Langevin, ESPCI Paris, CNRS, PSL University - ignacio.izeddin@espci.fr
· Clément Cabriel, postdoctoral researcher at Institut Langevin, ESPCI Paris, CNRS, PSL University - clement.cabriel@espci.fr