Efficient Multifocal Structured Illumination Microscopy Utilizing a Spatial Light Modulator

Abstract

We demonstrated an efficient system for multifocal structured illumination microscopy (MSIM) utilizing a spatial light modulator (SLM). Nine phase profiles of chessboard phase plates loaded on the SLM in sequence are used to generate nine multifocal arrays on the focal plane. Subsequently, nine raw multifocal images are acquired. Finally, by extracting the parameters of the illumination patterns from the raw images precisely, a final super-resolved image is reconstructed by performing the standard reconstruction procedure of structured illumination microscopy (SIM). Our MSIM system realized nearly a 1.5-fold enhancement in spatial resolution compared with wide-field (WF) microscopy. The feasibility of the present system is validated on experiments and the results show its great performances along with good compatibility.

1. Introduction

The fluorescence microscopy technique has been utilized as a powerful tool for a wide variety of biological applications such as cell biology and neuroscience [1]. However, owing to the wave nature of visible light, the Abbe diffraction limit imposes a critical barrier on the spatial resolution of optical microscopy, meaning that the maximum available resolution in a conventional WF microscope is approximately close to one-half of the emission wavelength of the fluorophores [2], which severely constrains the observation of the detailed structure of biological samples. Hence, high-resolution microscopy is vital for the study of biological structures and accurate diagnosis of medical diseases. To overcome the Abbe limit and achieve super-resolution, impressive developments in the field of super-resolution microscopy have come up in the past two decades [3,4,5,6], enabling observations within spatial regimes that were previously not accessible, such as photo-activated localization microscopy (PALM) [7,8,9], stochastic optical reconstruction microscopy (STORM) [10,11,12], stimulated emission depletion microscopy (STED) [13,14,15] and structured illumination microscopy (SIM) [16,17,18].

As one of the versatile super-resolved microscopy technologies, SIM utilizes spatially patterned illumination to excite the fluorescence of the samples. Linear SIM could double the spatial resolution of the conventional microscopy while nonlinear SIM can improve the performance of that further by exploiting the nonlinear behavior of the sample in response to the illumination light [19,20,21]. Conventional SIM with sinusoidal illumination patterns, does not require sophisticated sample preparation procedures while such a WF illumination even has its own limitation. It is incompatible with the situations that require a high intensity of incident light, which is only practical when focused laser beams are scanned through the focal plane. Furthermore, WF illumination typically limits the imaging depth and only thin samples show great imaging performances [22].

Utilizing point-scanning techniques, the quality of the images can be better maintained, while it might be not adequate for rapid acquisition due to its scanning mechanism and the size of the area to be scanned. Since two-dimensional (2D) SIM was described out [23], its theoretical aspects have been gradually proposed [24] and applied [25] in true super-resolution imaging. As a kind of 2D SIM, multifocal structured illumination microscopy (MSIM), with a sparse lattice of excitation foci, would dramatically increase the acquisition speed and show excellent performance. One way to generate multifocal patterns by a digital micro-mirror device (DMD) proved highly inefficient. Only a small fraction (<1%) of the pixels were in the on state during a given raw profile, wasting the majority of the excitation light [26]. Alternatively, the micro-lens arrays used to generate patterns are even complex to manufacture, of which parameters need to be carefully optimized to ensure the quality of the excitation foci [27]. Meanwhile, the number of the raw images used for reconstruction would affect the efficiency of the reconstruction, and reconstruction utilizing hundreds of raw images might be time-consuming [26,28].

In this paper, we proposed an efficient MSIM system utilizing a spatial light modulator (SLM). Nine phase profiles of chessboard phase plates loaded on the SLM in sequence are used to generate nine multifocal arrays on the focal plane. Then nine raw multifocal images are acquired. Finally, by extracting the parameters of the illumination patterns from the raw images precisely, we can reconstruct the final image. We carry out experiments on several kinds of samples including microspheres and other biological samples and the results show enhanced resolution by a factor of about 1.504 compared with WF microscopy. The proposed system shows its high efficiency and flexibility, which might exploit potential application in the field of super-resolution microscopy.

2. Methods

2.1. Experimental Setup

The schematic of multifocal structured illumination microscopy system configurations is depicted in Figure 1. A single mode laser with the wavelength λ = 532 nm is used as the light source. The laser beam is expanded to illuminate an SLM (LETO-VIS-009, HOLOEYE Photonisc AG, Berlin, Germany). The SLM is a reflective type of pure phase spatial light modulator. It comprises 1920 × 1080 pixels with a pixel size 6.4 × 6.4 μm² and a maximal frame-refresh rate of 60 Hz. The angle of the reflective mirror (RM) is adjusted properly to ensure that the incident beam illuminates on the SLM at an incident angle <6°, as suggested by the manufacturer to obtain desired modulation performances. The polarization of the incident beam is optimized by a half wave plate (HWP, WPZ2220-532-M-25.4, Union Optic, Pylaia, Greece). A linear polarizer (LP, SHP-1025, Union Optic, Wuhan, China) is followed to maintain the polarization of the incident beam horizontal to achieve phase-only modulation by the SLM. By programming the SLM with the phase profiles of the checkerboard phase plates, two sets of diffraction laser beams were generated. The laser beams except for ±1 diffraction orders were blocked by a custom-made spatial filter (SF) and finally focused by a lens (L3, MAG0125, Union Optic, Wuhan, China) and microscope objective (MO2, RMS10X, Olympus, Japan) onto the focal plane.

Fluorescence images were captured with an upright fluorescent microscopy configuration including a dichroic mirror (DM, Semrock Optics, New York, USA), a long-pass filter (LP, Edmund Optics, Barrington, New Jersey, USA) and a CMOS camera (BFS-U3-63S4M-C, FLIR Systems, Oregon, USA). For the purpose of comparison, the conventional WF images were recorded by illuminating the sample only with the uniform illumination. Both the camera and SLM were synchronized and controlled by the program implemented in Labview, while all imaging processing was performed with programs written with MATLAB (MathWorks, Massachusetts, USA).

2.2. Illumination Modality of MSIM

The SLM is programmed with phase profiles of 2D periodic checkerboard phase plates and illuminated by a horizontally polarized laser beam, resulting in two mutually perpendicular sets of diffraction laser beams. Figure 2 illustrates the schematic of the generation of the multifocal arrays. Focused by the MO2, the four remaining laser beams interfere and overlap to form the multifocal illumination patterns on the sample plane as shown in Figure 2b.

A multifocal array is generated by the interference and overlap of two sets of ±1 order diffraction laser beams. The spatial frequency of the multifocal array is confined by the fringes generated by every single set of ±1 order diffraction laser beams. The spatial frequency of the fringe $k_s$, is determined by the intersection angle of the two wave vectors of the laser beams [29]:

$$k_s=2n\times sin\left(\theta \right)/\lambda ,$$

(1)

where $n$ is the refractive index of surrounding medium, $\theta$ is the half intersection angle and $\lambda$ is the wavelength. These laser beams are interfered by a microscope, so Equation (1) can be turned into

$$k_s=2\mathrm{NA}/\lambda ,$$

(2)

where $\mathrm{NA}$ is the numerical aperture of the microscope. It means that the spatial frequency of multifocal arrays is determined by the diffraction limitation composed by the microscope. The above equations show that, to obtain multifocal arrays with high spatial frequency for achieving resolution enhancement as much as possible after reconstruction, the four diffraction incident laser beams should be close enough to the edge of the rear pupil aperture of the microscope.

2.3. Method for the Reconstruction of MSIM Images

In conventional linear SIM, the equation governing an SIM image formation for the detected fluorescence intensity can be approximately described as

$$D\left(r\right)=\left\{I_s\left(r\right)\times S\left(r\right)\right\}\otimes h\left(r\right),$$

(3)

where $I_s\left(r\right)$ is the illumination intensity, $S\left(r\right)$ is the sample fluorescence distribution and $\otimes$ represents the convolution operation, and $h\left(r\right)$ is the detection point spread function. Normally, SIM utilizes sinusoidal striped illumination patterns to elicit the fluorescence. In the frequency space, Equation (1) becomes

$$D\left(k\right)=I_0\times \left\{S\left(k\right)+\frac{m}{2}S\left(k+k_0\right)e^{-i\phi }+\frac{m}{2}S\left(k-k_0\right)e^{i\phi }\right\}\times h\left(k\right),$$

(4)

where $I_0$, $m$, $k_0$ and $\phi$ are the peak illumination intensity, modulation depth, spatial frequency and phase of the illumination patterns, respectively. The first term ($S\left(k\right)\times h\left(k\right)$) in Equation (4) denotes all the spatial frequencies observed by the conventional microscope while the two remaining terms contribute to the spatial resolution enhancement. To disentangle these three overlapping frequency components, one can capture three raw images using illumination patterns with three different phases and then solve a linear function. By recombining these shifted frequency components to their proper locations in the Fourier domain and carrying out inverse Fourier transform, SIM image reconstruction is accomplished.

As for MSIM, the illumination patterns mainly contain the frequency in the X and Y directions. Therefore, the detected images in the Fourier domain can be expressed as

$$D\left(k\right)=I_0\times \left\{\begin{array}{c}S\left(k\right)+\frac{m}{2}S\left(k+k_x\right)e^{-i{\phi }_x}+\frac{m}{2}S\left(k-k_x\right)e^{i{\phi }_x}\\ +\frac{m}{2}S\left(k+k_y\right)e^{-i{\phi }_y}+\frac{m}{2}S\left(k-k_y\right)e^{i{\phi }_y}\end{array}\right\}\times h\left(k\right),$$

(5)

where ${\phi }_{x,or y}$ and $k_{x,or y}$ are the spatial frequency and the phase of the illumination patterns along the X or Y direction, respectively.

To reconstruct the super-resolved images, we perform the reconstruction column by column and row by row separately based on the standard SIM reconstruction procedure. As is shown in Figure 3a, the final super-resolved image is obtained by superimposing the images reconstructed in the three columns and three rows.

Figure 3b shows the schematic of the reconstruction procedure for the MSIM data. To extract the illumination parameters, we place a reflective mirror at the focal plane and nine raw multifocal images, D₁–D₉, are subsequently acquired. Illumination parameters along the Y direction can be extracted out by fitting the data profiles of the same positions (e.g., three red dashed lines in Figure 3a) of D₁–D₃, while illumination parameters along the X direction can be extracted out by fitting the data profiles of the same positions (e.g., three black dashed lines in Figure 3a) of D₁, D₄ and D₇, respectively. $I_0$, $m$ and $k_0$ are the average of the corresponding fitting results while the three different phases (${\phi }_{j,XorY}$, where j = 1, 2, 3) are retained.

During reconstruction in a certain column or row, our method is essentially based on the reconstruction procedure of the standard SIM utilizing the illumination parameters along the Y or X direction. More details of the reconstruction procedure are previously illustrated in our previous work [30]. According to Equation (2) and combined with the extracted illumination parameters, a system of three independent equations can be constructed and solved to separate the three overlapping frequency components. A super-resolved image can be obtained by an inverse Fourier transform of the recombination of these frequency components shifted to their origin locations in the Fourier domain. Once nine raw multifocal images of other samples, D₁–D₉, are acquired under the same conditions, A₁₀–A₁₂ are subsequently reconstructed with Column 1–Column 3 while A₁₃–A₁₅ are reconstructed with Row 1–Row 3. The final super-resolved image A₁₆ is obtained by summing A₁₀–A₁₆.

It should be noted that the spatial frequencies of the illumination patterns are inevitably limited by the optical microscope. Compared with the traditional fluorescence microscopy, MSIM, as well as SIM, could theoretically improve the spatial resolution by a factor of two under the linear condition.

3. Results

3.1. The Multifocal Array Generated by SLM

Before subsequent experiments, it is necessary to check the quality of the multifocal arrays illuminating on the sample plane. Figure 4a is the part phase profile of the checkerboard phase plates loaded on the SLM, and Figure 4b shows the intensity distribution of the multifocal arrays generated by the four laser beams in our experiment. To characterize the amplitude fluctuation of each foci in the multifocal array, we extracted the amplitudes of each single foci and found that the peak intensity of each foci fluctuates within a small range, which would have little effect on the extraction of the illumination parameters. The multifocal array with equivalent periods along the X and Y directions is almost evenly distributed, which is vital to reconstruct an isotropic super-resolved image.

3.2. Characterization of Resolution Enhancement with Microspheres

To validate the spatial resolution improvement of the MSIM system, we imaged 1 um-diameter orange-labeled microspheres (FL-PS-O-01, Big goose Technology, Tianjin, China) with an excitation wavelength of 532 nm and emission wavelength of 581 nm. Having extracted the parameters of the illumination patterns accurately from the nine raw images, the proposed reconstruction procedures were successively performed.

Images of three separate microspheres by WF and MSIM are shown in Figure 5c,b, respectively. Figure 5c shows a decrease in the lateral full width at half maximum (FWHM) from 1.776 ± 0.053 (WF) to 1.181 ± 0.0599 µm (MSIM), which was calculated by repeated fits with the thirteen different microspheres. The MSIM nearly improves the spatial resolution by a factor of nearly 1.504 compared with the conventional WF microscopy under current configurations. According to the period (12 pixels) of the illumination patterns illuminating on the sample and the practical resolution (7 pixels corresponding to the FWHM of WF in Figure 5c), the theoretical spatial resolution of the samples, could be enhanced by a factor of 1.58 at most.

3.3. Resolution Improvement by Imaging Biological Samples

To investigate the performance of this system in biological applications, we began with imaging the fixed sample of Melosira (unicellular organism, usually shown as chain-like clusters). Figure 6a–i is the nine raw multifocal images of this sample. Figure 7 illustrates the MSIM performance compared with the conventional WF image. In comparison with the wide-field image shown in Figure 7a, the sum result of the nine raw multifocal images shown in Figure 7b shows the suppression of the background noise. Furthermore, the MSIM images shown in Figure 7c could distinguish the two chain-like structures which are blurred in the former two images as depicted in Figure 7d,e. To verify the accuracy and reliability of this experiment, the same magnified area of this sample was imaged using an objective with a higher numerical aperture (0.65). The result confirmed that there was a gap between the two chain structures, as shown in Figure 7h and its inset.

In addition, we imaged a fixed transverse section sample (for teaching purposes) of earthworm (Figure 8) and esophagus (Figure 9), respectively. Compared with the uniform illuminated WF microscopy, the finer structures marked by the colored areas in Figure 8c and Figure 9c are resolved by MSIM. For a quantitative analysis, the image modulation contrast $I_{contrast}$ can be introduced by

$$I_{contrast}=\frac{I_{max}-I_{min}}{I_{max}+I_{min}},$$

(6)

where $I_{max}$ and $I_{min}$ are the maxima and minima of certain regions in the images, respectively. Compared with WF, the contrasts of MSIM in Figure 8c and Figure 9c are enhanced by a factor of 1.1319 and 1.2083, respectively. Based on SIM, only the high-frequency components of the sample modulated by MSIM are demodulated out to enhance the resolution of the final images, while the random noise of the background cannot be modulated by the patterned illumination. Once the high-frequency components representing the details of the sample are enhanced, the overall amplitude of the fluorescent signal-to-noise ratio (SNR) will be improved. Therefore, our structured illumination modality and reconstruction approach could achieve great performance with high image contrast.

4. Discussion and Conclusions

In this paper, a novel MSIM system for super-resolution microscopy with good performance is implemented in detail. With our system, a super-resolution image can be recovered just using nine raw multifocal images, realizing nearly a 1.5-fold enhancement in spatial resolution. This system is convenient to build up as we just add only an SLM to conventional WF microscopy, as well as quiet, stable, without moving parts in the illumination path. The utilization efficiency of the incident laser is critical for high-power imaging, especially for nonlinear imaging. The SLM here utilized for generating multifocal illumination patterns proved more efficient than the DMD, for the four diffraction laser beams hold around 42% power of the incident laser which could be reasonably accepted. We believe that this system may find potential application in the field of super-resolution microscopy.

The absolute resolution enhancement reported here is lower than that in theory, which we attribute to two factors. First, achieving the highest possible resolution relies on obtaining diffraction-limited WF performance before MSIM is applied. Although we attempted this, the actual resolution is still not equal to the theoretical prediction (1.417 µm) because of the undersized aberration caused by the optical elements. Second, the frequency of the illumination pattern is not close enough to the cut-off frequency of the WF microscopy owing to the current configurations which might be the main cause. The system reported in Reference [23] utilizes two separate interferometers with an individual delay line to generate and modulate the illumination pattern on the sample plane. From this point of view, our system provides the advantage of more simplicity over it and may have the great applicable potential in fast-scanning imaging.

One of the major speed limitations of our MSIM system is the relatively low frame rate of the SLM (60 Hz). Due to the requirement of nine raw multifocal images for reconstruction, the maximum movement frequency of the sample is limited to 6.7 Hz, which is hard to meet the demand of imaging the moving sample with higher frequency. An alternative approach to alleviate this situation is to change the SLM with a higher frame rate or add 2D scanning mirrors in the optical path. The other limitation is the relatively low frame rate of the camera used. Increasing the camera frame rate invariably results in an increase in read noise and decrease in sensitivity, and the speed of a system is ultimately limited by the signal-to-noise level of the image. A path that we have taken in this system is to optimize its configurations to ensure the camera possesses a considerable sensitivity of detection at high speed.