基于深度学习的可变形镜点扩散函数设计研究

单分子定位显微镜技术 (SMLM) 是突破了传统光学显微镜的衍射极限实现了 纳米级别的成像分辨率，它利用异步激发和稀疏定位来分离接近的荧光分子，与点 扩散函数(PSF)相拟合进行高分辨率图像重建。由于传统的点扩散函数在 z 轴上 信息缺失，科研工作者通过改变相位将深度信息编码到点扩散函数的形状上，从 而显著提高轴向分辨率。

但目前的优化算法得到的点扩散函数无法在复杂的实验条件下表现出优越的 定位性能。近年来，深度学习技术已被广泛应用于单分子定位，在多种复杂场景 下均能表现出优异性能。因此，本研究将深度学习与传统优化算法相结合，选用 特定器件(可变形镜)进行相位调制，优化适应不同实验场景的工程化 PSF。

本文基于可变形镜来调制相位并结合深度学习进行优化，提出能够适用于不 同实验条件，不同轴向范围和不同密度的点扩散函数优化方法，实现不同实验条 件下的高分辨率定位。本文的主要研究内容包括:

1. 构建一个基于可变形镜的矢量点扩散函数快速模拟成像的方法，相较于传 统的模拟方法速度提高了 50 倍。

2.提出了一个高精度的定位网络DilatedLoc，该网络通过单点定位和模拟数据 测试，结果表明在单点情况下能够接近 Cramér-Rao 下界(CRLB)，并在公开测试 集表现出与当前主流定位方法相匹配的性能。

3. 构建了基于可变形镜的矢量点扩散函数作为编码器，定位网络作为解码器 的点扩散函数优化算法。通过网络与编码器的协同训练，最终获得了适用于不同 实验条件下的 PSF。在单点情况下，该算法优化得到的 PSF 具有比常用 PSF 更低 的 CRLB，并且在高密度情况下的优化结果也优于其他优化算法。该方法的可行性 为在复杂实验环境下进行三维单分子定位高分辨成像提供了基础。

Single-molecule localization microscopy (SMLM) technology has broken through the diffraction limit of traditional optical microscopes, achieving nanoscale imaging res- olution. It utilizes asynchronous excitation and sparse localization to separate closely spaced fluorescent molecules, fitting them to the point spread function (PSF) for high- resolution image reconstruction. However, traditional PSFs lack depth information in the z-axis. Researchers have addressed this limitation by encoding depth information into the shape of the PSF through phase modulation, significantly enhancing axial resolution.

Despite these advancements, current optimization algorithms for PSFs fail to demon- strate superior localization performance under complex experimental conditions. In re- cent years, deep learning technology has been widely applied to single-molecule localiza- tion, showing excellent performance in various complex scenarios. Therefore, this study combines deep learning with traditional optimization algorithms, using specific devices (deformable mirrors) for phase modulation, and optimizing engineered PSFs to adapt to different experimental scenarios.

This paper proposes a PSF optimization method suitable for different experimental conditions, axial ranges, and densities, based on phase modulation with deformable mir- rors and combined with deep learning. The main research contents of this paper include:

1. Building a rapid simulation imaging method for vector PSFs based on deformable mirrors, which improves simulation speed by 50 times compared to traditional methods.

2. Proposing a high-precision localization network, DilatedLoc, which, through single-point localization and simulation data testing, shows close approximation to the Cramér-Rao lower bound (CRLB) in single-point scenarios and matches the performance of current mainstream localization methods on publicly available test sets.

3. Constructing a PSF optimization algorithm based on deformable mirrors as en- coders and localization networks as decoders. Through collaborative training between the network and the encoder, PSFs suitable for different experimental conditions are ob- tained. In single-point scenarios, the PSFs optimized by this algorithm have lower CRLBs than common PSFs, and the optimization results under high-density conditions also out- perform other optimization algorithms. This approach lays the foundation for achieving three-dimensional single-molecule localization high-resolution imaging in complex experimental environments.

[1] BETZIG E, PATTERSON G H, SOUGRAT R, et al. Imaging intracellular fluorescent proteins at nanometer resolution[J]. Science, 2006, 313(5793): 1642-1645.
[2] RUST M J, BATES M, ZHUANG X. Sub-diffraction-limit imaging by stochastic optical recon- struction microscopy (STORM)[J]. Nature Methods, 2006, 3(10): 793-796.
[3] WILLIG K I, HARKE B, MEDDA R, et al. STED microscopy with continuous wave beams[J]. Nature Methods, 2007, 4(11): 915-918.
[4] VICIDOMINI G, BIANCHINI P, DIASPRO A. STED super-resolved microscopy[J]. Nature Methods, 2018, 15(3): 173-182.
[5] SHECHTMAN Y, SAHL S J, BACKER A S, et al. Optimal point spread function design for 3D imaging[J]. Physical Review Letters, 2014, 113(13): 133902.
[6] PAVANI S R P, THOMPSON M A, BITEEN J S, et al. Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function [J]. Proceedings of the National Academy of Sciences, 2009, 106(9): 2995-2999.
[7] VON DIEZMANN L, SHECHTMAN Y, MOERNER W. Three-dimensional localization of single molecules for super-resolution imaging and single-particle tracking[J]. Chemical Re- views, 2017, 117(11): 7244-7275.
[8] HUANG B, WANG W, BATES M, et al. Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy[J]. Science, 2008, 319(5864): 810-813.
[9] LEE H L D, SAHL S J, LEW M D, et al. The double-helix microscope super-resolves extended biological structures by localizing single blinking molecules in three dimensions with nanoscale precision[J]. Applied Physics Letters, 2012, 100(15): 153701-1537013.
[10] SHECHTMAN Y, WEISS L E, BACKER A S, et al. Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions[J]. Nano Letters, 2015, 15(6): 4194-4199.
[11] KAO H P, VERKMAN A. Tracking of single fluorescent particles in three dimensions: use of cylindrical optics to encode particle position[J]. Biophysical Journal, 1994, 67(3): 1291-1300.
[12] PETROV P N, SHECHTMAN Y, MOERNER W. Measurement-based estimation of global pupil functions in 3D localization microscopy[J]. Optics Express, 2017, 25(7): 7945-7959.
[13] FU S, LI M, ZHOU L, et al. Deformable mirror based optimal PSF engineering for 3D super- resolution imaging[J]. Optics Letters, 2022, 47(12): 3031-3034.
[14] HERSHKO E, WEISS L E, MICHAELI T, et al. Multicolor localization microscopy and point- spread-function engineering by deep learning[J]. Optics Express, 2019, 27(5): 6158-6183.
[15] NEHME E, FERDMAN B, WEISS L E, et al. Learning Optimal Wavefront Shaping for Multi- Channel Imaging[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(7): 2179-2192.
[16] ZHUANG X. Nano-imaging with STORM[J]. Nature Photonics, 2009, 3(7): 365-367.
[17] ABBE E. Note on the Proper Definition of the Amplifying Power of a Lens or Lens-system[J]. Journal of the Royal Microscopical Society, 1884, 4(3): 348-351.
[18] GUSTAFSSON M G. Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy[J]. Journal of Microscopy, 2000, 198(2): 82-87.
[19] HEINTZMANN R, JOVIN T M, CREMER C. Saturated patterned excitation microscopy—a concept for optical resolution improvement[J]. JOSA A, 2002, 19(8): 1599-1609.
[20] HELL S W, WICHMANN J. Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy[J]. Optics Letters, 1994, 19(11): 780- 782.
[21] NEHME E, WEISS L E, MICHAELI T, et al. Deep-STORM: super-resolution single-molecule microscopy by deep learning[J]. Optica, 2018, 5(4): 458-464.
[22] KIM T, MOON S, XU K. Information-rich localization microscopy through machine learning [J]. Nature Communications, 2019, 10(1): 1-8.
[23] OUYANG W, ARISTOV A, LELEK M, et al. Deep learning massively accelerates super- resolution localization microscopy[J]. Nature Biotechnology, 2018, 36(5): 460-468.
[24] SHROFF H, GALBRAITH C G, GALBRAITH J A, et al. Dual-color superresolution imaging of genetically expressed probes within individual adhesion complexes[J]. Proceedings of the National Academy of Sciences, 2007, 104(51): 20308-20313.
[25] SUBACH F V, PATTERSON G H, MANLEY S, et al. Photoactivatable mCherry for high- resolution two-color fluorescence microscopy[J]. Nature Methods, 2009, 6(2): 153-159.
[26] XU K, ZHONG G, ZHUANG X. Actin, spectrin, and associated proteins form a periodic cy- toskeletal structure in axons[J]. Science, 2013, 339(6118): 452-456.
[27] DEMPSEY G T, VAUGHAN J C, CHEN K H, et al. Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging[J]. Nature Methods, 2011, 8(12): 1027-1036.
[28] ZHANG Z, KENNY S J, HAUSER M, et al. Ultrahigh-throughput single-molecule spec- troscopy and spectrally resolved super-resolution microscopy[J]. Nature Methods, 2015, 12 (10): 935-938.
[29] JAMESON D M, ROSS J A. Fluorescence polarization/anisotropy in diagnostics and imaging [J]. Chemical Reviews, 2010, 110(5): 2685-2708.
[30] BACKLUND M P, LEW M D, BACKER A S, et al. The role of molecular dipole orientation in single-molecule fluorescence microscopy and implications for super-resolution imaging[J]. ChemPhysChem, 2014, 15(4): 587-599.
[31] SHRODER D Y, LIPPERT L G, GOLDMAN Y E. Single molecule optical measurements of orientation and rotations of biological macromolecules[J]. Methods and Applications in Fluorescence, 2016, 4(4): 042004.
[32] MANZO C, GARCIA-PARAJO M F. A review of progress in single particle tracking: from methods to biophysical insights[J]. Reports on Progress in Physics, 2015, 78(12): 124601.
[33] KUSUMI A, TSUNOYAMA T A, HIROSAWA K M, et al. Tracking single molecules at work in living cells[J]. Nature Chemical Biology, 2014, 10(7): 524-532.
[34] BEREZIN M Y, ACHILEFU S. Fluorescence lifetime measurements and biological imaging [J]. Chemical Reviews, 2010, 110(5): 2641-2684.
[35] DATTA R, HEASTER T M, SHARICK J T, et al. Fluorescence lifetime imaging microscopy: fundamentals and advances in instrumentation, analysis, and applications[J]. Journal of Biomedical Optics, 2020, 25(7): 071203-071203.
[36] JUETTE M F, GOULD T J, LESSARD M D, et al. Three-dimensional sub–100 nm resolution fluorescence microscopy of thick samples[J]. Nature Methods, 2008, 5(6): 527-529.
[37] BABCOCK H P. Multiplane and spectrally-resolved single molecule localization microscopy with industrial grade CMOS cameras[J]. Scientific Reports, 2018, 8(1): 1726.
[38] CABRIEL C, BOURG N, JOUCHET P, et al. Combining 3D single molecule localization strategies for reproducible bioimaging[J]. Nature Communications, 2019, 10(1): 1980.
[39] DASGUPTA A, DESCHAMPS J, MATTI U, et al. Direct supercritical angle localization mi- croscopy for nanometer 3D superresolution[J]. Nature Communications, 2021, 12(1): 1180.
[40] SHECHTMAN Y, WEISS L E, BACKER A S, et al. Multicolour localization microscopy by point-spread-function engineering[J]. Nature Photonics, 2016, 10(9): 590-594.
[41] MANLEY S, GILLETTE J M, PATTERSON G H, et al. High-density mapping of single- molecule trajectories with photoactivated localization microscopy[J]. Nature Methods, 2008, 5 (2): 155-157.
[42] SHIM S H, XIA C, ZHONG G, et al. Super-resolution fluorescence imaging of organelles in live cells with photoswitchable membrane probes[J]. Proceedings of the National Academy of Sciences, 2012, 109(35): 13978-13983.
[43] MOHAN N, SOROKINA E M, VERDENY I V, et al. Detyrosinated microtubules spatially constrain lysosomes facilitating lysosome–autophagosome fusion[J]. Journal of Cell Biology, 2019, 218(2): 632-643.
[44] OTTERSTROM J, CASTELLS-GARCIA A, VICARIO C, et al. Super-resolution microscopy reveals how histone tail acetylation affects DNA compaction within nucleosomes in vivo[J]. Nucleic Acids Research, 2019, 47(16): 8470-8484.
[45] BÁLINT Š, VERDENY VILANOVA I, SANDOVAL ÁLVAREZ Á, et al. Correlative live-cell and superresolution microscopy reveals cargo transport dynamics at microtubule intersections [J]. Proceedings of the National Academy of Sciences, 2013, 110(9): 3375-3380.
[46] DOKSANI Y, WU J Y, DE LANGE T, et al. Super-resolution fluorescence imaging of telomeres reveals TRF2-dependent T-loop formation[J]. Cell, 2013, 155(2): 345-356.
[47] PIESTUN R, SCHECHNER Y Y, SHAMIR J. Propagation-invariant wave fields with finite energy[J]. JOSA A, 2000, 17(2): 294-303.
[48] 乔敏达, 白林阁, 王书恒, 等. 计算成像技术中的点扩散函数工程[J]. Journal of Data Acqui- sition and Processing, 2024, 39(2): 271-296.
[49] HUANG B, JONES S A, BRANDENBURG B, et al. Whole-cell 3D STORM reveals interac- tions between cellular structures with nanometer-scale resolution[J]. Nature Methods, 2008, 5 (12): 1047-1052.
[50] SHTENGEL G, GALBRAITH J A, GALBRAITH C G, et al. Interferometric fluorescent super-resolution microscopy resolves 3D cellular ultrastructure[J]. Proceedings of the National Academy of Sciences, 2009, 106(9): 3125-3130.
[51] LEW M D, LEE S F, BADIEIROSTAMI M, et al. Corkscrew point spread function for far-field three-dimensional nanoscale localization of pointlike objects[J]. Optics Letters, 2011, 36(2): 202-204.
[52] BADDELEY D, CANNELL M B, SOELLER C. Three-dimensional sub-100 nm super- resolution imaging of biological samples using a phase ramp in the objective pupil[J]. Nano Research, 2011, 4(6): 589-598.
[53] JIA S, VAUGHAN J C, ZHUANG X. Isotropic three-dimensional super-resolution imaging with a self-bending point spread function[J]. Nature Photonics, 2014, 8(4): 302-306.
[54] XU F, MA D, MACPHERSON K P, et al. Three-dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval[J]. Nature Methods, 2020, 17(5): 531-540.
[55] GAIRE S K, ZHANG Y, LI H, et al. Accelerating multicolor spectroscopic single-molecule localization microscopy using deep learning[J]. Biomedical Optics Express, 2020, 11(5): 2705- 2721.
[56] FU S, SHI W, LUO T, et al. Field-dependent deep learning enables high-throughput whole-cell 3D super-resolution imaging[J]. Nature Methods, 2023, 20(3): 459-468.
[57] SPEISER A, MÜLLER L R, HOESS P, et al. Deep learning enables fast and dense single- molecule localization with high accuracy[J]. Nature Methods, 2021, 18(9): 1082-1090.
[58] NEHME E, FREEDMAN D, GORDON R, et al. DeepSTORM3D: dense 3D localization mi- croscopy and PSF design by deep learning[J]. Nature Methods, 2020, 17(7): 734-740.
[59] ZHANG P, LIU S, CHAURASIA A, et al. Analyzing complex single-molecule emission pat- terns with deep learning[J]. Nature Methods, 2018, 15(11): 913-916.
[60] BOYD N, JONAS E, BABCOCK H, et al. DeepLoco: fast 3D localization microscopy using neural networks[J/OL]. bioRxiv, 2018. https://www.biorxiv.org/content/early/2018/02/16/267 096. DOI: 10.1101/267096.
[61] ZELGER P, KASER K, ROSSBOTH B, et al. Three-dimensional localization microscopy using deep learning[J]. Optics Express, 2018, 26(25): 33166-33179.
[62] NEHME E, FERDMAN B, WEISS L E, et al. Learning optimal wavefront shaping for multi- channel imaging[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43 (7): 2179-2192.
[63] HOLDEN S J, UPHOFF S, KAPANIDIS A N. DAOSTORM: an algorithm for high-density super-resolution microscopy[J]. Nature Methods, 2011, 8(4): 279-280.
[64] OVESNỲ M, KŘÍŽEK P, BORKOVEC J, et al. ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging[J]. Bioinformatics, 2014, 30(16): 2389-2390.
[65] RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedi- cal image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention– MICCAI 2015: 18th International Conference, Munich, Germany. 2015: 234-241.
[66] KAY S M. Fundamentals of statistical signal processing: estimation theory[M]. Prentice-Hall, Inc.Division of Simon and Schuster One Lake Street Upper Saddle River, NJ.United States, 1993: 595.
[67] LI Y, SHI W, LIU S, et al. Global fitting for high-accuracy multi-channel single-molecule localization[J]. Nature Communications, 2022, 13(1): 3133.
[68] LI Y, MUND M, HOESS P, et al. Real-time 3D single-molecule localization using experimental point spread functions[J]. Nature Methods, 2018, 15(5): 367-369.
[69] MÖCKL L, ROY A R, PETROV P N, et al. Accurate and rapid background estimation in single- molecule localization microscopy using the deep neural network BGnet[J]. Proceedings of the National Academy of Sciences, 2020, 117(1): 60-67.
[70] ZHANG Z, ZHANG Y, YING L, et al. Machine-learning based spectral classification for spec- troscopic single-molecule localization microscopy[J]. Optics Letters, 2019, 44(23): 5864-5867.
[71] JEONG D, KIM D. Super-resolution fluorescence microscopy-based single-molecule spec- troscopy[J]. Bulletin of the Korean Chemical Society, 2022, 43(3): 316-327.
[72] MEZOUARI S, HARVEY A R. Validity of Fresnel and Fraunhofer approximations in scalar diffraction[J]. Journal of Optics A: Pure and Applied Optics, 2003, 5(4): S86.
[73] LEUTENEGGER M, RAO R, LEITGEB R A, et al. Fast focus field calculations[J]. Optics Express, 2006, 14(23): 11277-11291.
[74] LIN J, RODRÍGUEZ-HERRERA O, KENNY F, et al. Fast vectorial calculation of the volumet- ric focused field distribution by using a three-dimensional Fourier transform[J]. Optics Express, 2012, 20(2): 1060-1069.
[75] NIE Z Q, LIN H, LIU X F, et al. Three-dimensional super-resolution longitudinal magnetization spot arrays[J]. Light: Science & Applications, 2017, 6(8): e17032-e17032.
[76] BOOTH M J. A basic introduction to adaptive optics for microscopy[M/OL]. Zenodo, 2019. https://doi.org/10.5281/zenodo.3471043.
[77] YU F, KOLTUN V. Multi-Scale Context Aggregation by Dilated Convolutions[C]//BENGIO Y, LECUN Y. 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings. 2016.
[78] LIU R, LEHMAN J, MOLINO P, et al. An intriguing failing of convolutional neural networks and the coordconv solution[C]//Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada. 2018: 9628-9639.
[79] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C/OL]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA, 2016: 770-778. DOI: 10.1109/CVPR.2016.90.
[80] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks [J]. Communications of the ACM, 2020, 63(11): 139-144.
[81] SAGUY A, ALALOUF O, OPATOVSKI N, et al. DBlink: Dynamic localization microscopy in super spatiotemporal resolution via deep learning[J]. Nature Methods, 2023, 20(12): 1939- 1948.
[82] ZHANG S, ZHENG D, HU X, et al. Bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation. Shanghai, China, 2015: 73-78.
[83] JÉGOU S, DROZDZAL M, VAZQUEZ D, et al. The one hundred layers tiramisu: Fully convo- lutional densenets for semantic segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2017, Honolulu, HI, USA. IEEE Com- puter Society, 2017: 1175-1183.
[84] SAGE D, PHAM T A, BABCOCK H, et al. Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software[J]. Nature Methods, 2019, 16(5): 387-395.
[85] WU T, LU J, LEW M D. Dipole-spread-function engineering for simultaneously measuring the 3D orientations and 3D positions of fluorescent molecules[J]. Optica, 2022, 9(5): 505-511.
[86] JOUCHET P, ROY A R, MOERNER W. Combining deep learning approaches and point spread function engineering for simultaneous 3D position and 3D orientation measurements of fluo- rescent single molecules[J]. Optics Communications, 2023: 129589.
[87] VOULODIMOS A, DOULAMIS N, DOULAMIS A, et al. Deep learning for computer vision: A brief review[J]. Computational Intelligence and Neuroscience, 2018, 2018: 1-13.
[88] WIDROW B, LEHR M A. 30 years of adaptive neural networks: perceptron, madaline, and backpropagation[J]. Proceedings of the IEEE, 1990, 78(9): 1415-1442.