From Generative Artificial Intelligence to Statistical Physics and Back: Tackling Non-Equilibrium Gas and Diffusion Process

Generative artificial intelligence (GAI) is the art of harnessing the power of data to model probability distributions. Probability distributions are foundational in various domains, includ- ing computer vision and statistical physics. Their accurate characterization is essential for gen- erating high-quality images and predicting complex fluid dynamics. This thesis delves into the interplay between GAI and statistical physics through probability distributions. We explore how the physics of the diffusion process helps the denoising diffusion probabilistic model (DDPM) create better images and an application of AI in disease diagnosis. Moreover, we investigate the methods, limitations, and challenges of using generative models to model non-equilibrium gas flows.

[1] J. Kaplan et al., Scaling laws for neural language models, 2020 (cit. on p. 1).
[2] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016, http://www.deeplearningbook.org (cit. on p. 1).
[3] M. Kardar, “Statistical fields,” in Statistical Physics of Fields. Cambridge University Press, 2007, pp. 19–34. DOI: 10.1017/CBO9780511815881.003 (cit. on pp. 2, 16, 68).
[4] J. N. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsu- pervised learning using nonequilibrium thermodynamics,” ArXiv, vol. abs/1503.03585, 2015 (cit. on pp. 2, 6, 7).
[5] Y. Song and S. Ermon, “Generative modeling by estimating gradients of the data distri- bution,” in Neural Information Processing Systems, 2019 (cit. on pp. 2, 6, 7).
[6] Y. Song, J. N. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” ArXiv, vol. abs/2011.13456, 2020 (cit. on pp. 2, 6–9, 14, 121, 124, 132, 134, 137).
[7] J. Betker et al., Improving image generation with better captions, 2023 (cit. on pp. 2, 6).
[8] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution im- age synthesis with latent diffusion models,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10 674–10 685, 2021 (cit. on pp. 2, 6, 7, 18, 138).
[9] M. Torrilhon, “Modeling nonequilibrium gas flow based on moment equations,” Annual Review of Fluid Mechanics, vol. 48, pp. 429–458, 2016 (cit. on pp. 2, 30).
[10] H. Schaeffer, “Learning partial differential equations via data discovery and sparse opti- mization,” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2017. DOI: 10.1098/rspa.2016.0446 (cit. on pp. 2, 71, 72, 80).
[11] L. Wasserman, All of statistics : a concise course in statistical inference (Springer Texts in Statistics), eng. New York, NY: Springer New York, 2004 (cit. on pp. 3, 149).
[12] J. McDonald and M. Torrilhon, “Affordable robust moment closures for cfd based on the maximum-entropy hierarchy,” Journal of Computational Physics, vol. 251, pp. 500– 523, 2013 (cit. on pp. 3, 27).
[13] G. W. Alldredge, C. D. Hauck, and A. L. Tits, “High-order entropy-based closures for linear transport in slab geometry ii: A computational study of the optimization problem,” SIAM J. Sci. Comput., vol. 34, 2012 (cit. on pp. 3, 27, 37).
[14] G. W. Alldredge, C. D. Hauck, D. P. O’Leary, and A. L. Tits, “Adaptive change of basis in entropy-based moment closures for linear kinetic equations,” J. Comput. Phys., vol. 258, pp. 489–508, 2013 (cit. on pp. 3, 27, 37–39, 150).
[15] R. P. Schaerer and M. Torrilhon, “The 35-moment system with the maximum-entropy closure for rarefied gas flows,” European Journal of Mechanics B-fluids, vol. 64, pp. 30– 40, 2017 (cit. on pp. 3, 27, 28, 30, 36, 37, 39, 55, 58, 64, 67).
[16] H. Grad, “The profile of a steady plane shock wave,” Communications on Pure and Applied Mathematics, vol. 5, pp. 257–300, 1952 (cit. on pp. 3, 60, 82).
[17] C. Zheng, Y. Wang, and S. Chen, “Data-driven constitutive relation reveals scaling law for hydrodynamic transport coefficients,” Phys. Rev. E, vol. 107, p. 015 104, 1 Jan. 2023. DOI: 10.1103/PhysRevE.107.015104 (cit. on p. 4).
[18] D. Podell et al., Sdxl: Improving latent diffusion models for high-resolution image syn- thesis, 2023 (cit. on pp. 6, 19).
[19] CagliostroLab, Animagine xl 3.0, 2023. [Online]. Available: https://huggingface. co/cagliostrolab/animagine-xl-3.0 (cit. on p. 6).
[20] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” ArXiv, vol. abs/2006.11239, 2020 (cit. on pp. 6–8, 121, 122).
[21] C. Luo, “Understanding diffusion models: A unified perspective,” ArXiv, vol. abs/2208.11970, 2022 (cit. on p. 6).
[22] P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” ArXiv, vol. abs/2105.05233, 2021 (cit. on pp. 6, 7, 9, 124).
[23] J. Ho, “Classifier-free diffusion guidance,” ArXiv, vol. abs/2207.12598, 2022 (cit. on pp. 6, 7, 9, 18, 124).
[24] C. Saharia et al., Photorealistic text-to-image diffusion models with deep language un- derstanding, 2022 (cit. on pp. 7, 21).
[25] C.-H. Lai, Y. Takida, N. Murata, T. Uesaka, Y. Mitsufuji, and S. Ermon, “Fp-diffusion: Improving score-based diffusion models by enforcing the underlying score fokker-planck equation,” in International Conference on Machine Learning, 2022 (cit. on pp. 7, 9, 125).
[26] A. Bansal et al., “Universal guidance for diffusion models,” 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 843–852, 2023 (cit. on p. 7).
[27] J. Yu, Y. Wang, C. Zhao, B. Ghanem, and J. Zhang, “Freedom: Training-free energy- guided conditional diffusion model,” ArXiv, vol. abs/2303.09833, 2023 (cit. on p. 7).
[28] S. Hong, G. Lee, W. Jang, and S. W. Kim, “Improving sample quality of diffusion models using self-attention guidance,” ArXiv, vol. abs/2210.00939, 2022 (cit. on p. 7).
[29] D. Kim, Y. Kim, W. Kang, and I.-C. Moon, “Refining generative process with discrim- inator guidance in score-based diffusion models,” in International Conference on Ma- chine Learning, 2022 (cit. on p. 7).
[30] mcmonkeyprojects, Sd-dynamic-thresholding: A dynamic thresholding method for sta-ble diffusion, 2024. [Online]. Available: https://github.com/mcmonkeyprojects/ sd-dynamic-thresholding (cit. on p. 7).
[31] L. Liu, Y. Ren, Z. Lin, and Z. Zhao, “Pseudo numerical methods for diffusion models on manifolds,” ArXiv, vol. abs/2202.09778, 2022 (cit. on p. 8).
[32] J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” ArXiv, vol. abs/2010.02502, 2020 (cit. on pp. 8, 14, 132, 134, 137).
[33] W. Zhao, L. Bai, Y. Rao, J. Zhou, and J. Lu, “Unipc: A unified predictor-corrector frame- work for fast sampling of diffusion models,” ArXiv, vol. abs/2302.04867, 2023 (cit. on p. 8).
[34] T. Karras, M. Aittala, T. Aila, and S. Laine, “Elucidating the design space of diffusion- based generative models,” ArXiv, vol. abs/2206.00364, 2022 (cit. on p. 8).
[35] C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu, “Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models,” ArXiv, vol. abs/2211.01095, 2022 (cit. on pp. 8, 14, 132, 134, 137).
[36] L. Yang et al., “Diffusion models: A comprehensive survey of methods and applica- tions,” ACM Computing Surveys, 2022 (cit. on pp. 8, 122).
[37] L. C. Evans, Partial differential equations. American Mathematical Society, 2022, vol. 19 (cit. on p. 11).
[38] AUTOMATIC1111, Stable diffusion webui, 2023. [Online]. Available: https : / / github.com/AUTOMATIC1111/stable-diffusion-webui (cit. on p. 19).
[39] Z. Zhang, Z. Zhao, J. Yu, and Q. Tian, “Shiftddpms: Exploring conditional diffusion models by shifting diffusion trajectories,” ArXiv, vol. abs/2302.02373, 2023 (cit. on p. 20).
[40] M. N. Kogan, Rarefied gas dynamics, eng, 1st ed. 1969. New York: Springer Science + Business Media, LLC, 1969 (cit. on pp. 26, 70).
[41] H.-S. Tsien, “Superaerodynamics, mechanics of rarefied gases,” Journal of the Aeronau- tical Sciences, vol. 13, no. 12, pp. 653–664, 1946 (cit. on p. 26).
[42] B. D. Shizgal and D. P. Weaver, Rarefied gas dynamics: space science and engineering. Aiaa, 1994, vol. 160 (cit. on p. 26).
[43] G. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows. Clarendon Press, 1994 (cit. on pp. 26, 60, 70, 150, 153).
[44] J. E. Broadwell, “Study of rarefied shear flow by the discrete velocity method,” Journal ofFluid Mechanics, vol. 19, no. 3, pp. 401–414, 1964. DOI: 10.1017/S0022112064000817 (cit. on p. 26).
[45] “Discrete velocity method,” in Rarefied Gas Dynamics. John Wiley & Sons, Ltd, 2016, ch. 9, pp. 83–96. DOI: https://doi.org/10.1002/9783527685523. ch9 (cit. on pp. 26, 60).
[46] L. Mieussens, “Discrete-velocity models and numerical schemes for the boltzmann-bgk equation in plane and axisymmetric geometries,” Journal of Computational Physics, vol. 162, no. 2, pp. 429–466, 2000 (cit. on pp. 26, 27).
[47] V. V. Aristov, Direct methods for solving the Boltzmann equation and study of nonequi- librium flows. Springer Science & Business Media, 2001, vol. 60 (cit. on p. 26).
[48] R. K. Agarwal, K.-Y. Yun, and R. Balakrishnan, “Beyond navier–stokes: Burnett equa- tions for flows in the continuum–transition regime,” Physics of Fluids, vol. 13, no. 10, pp. 3061–3085, 2001 (cit. on pp. 26, 27, 60, 70, 72).
[49] L. S. García-Colín, R. M. Velasco, and F. J. Uribe, “Beyond the navier-stokes equations: Burnett hydrodynamics,” Physics Reports, vol. 465, pp. 149–189, 2008 (cit. on pp. 26, 27, 60, 70).108
[50] D. Hilbert, “Begründung der kinetischen Gastheorie,” Mathematische Annalen, 1912. DOI: 10.1007/BF01456676 (cit. on pp. 27, 70).
[51] “VI. On the law of distribution of molecular velocities, and on the theory of viscosity and thermal conduction, in a non-uniform simple monatomic gas,” Philosophical Trans- actions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 1916. DOI: 10.1098/rsta.1916.0006 (cit. on pp. 27, 70).
[52] D. Enskog, “Kinetische theorie der vorgänge in mässig verdünnten gasen. i. allgemeiner teil,” Uppsala: Almquist & Wiksells Boktryckeri, 1917 (cit. on pp. 27, 70).
[53] S. Pieraccini and G. Puppo, “Implicit–explicit schemes for bgk kinetic equations,” Jour- nal of Scientific Computing, vol. 32, pp. 1–28, 2007 (cit. on p. 27).
[54] G. Dimarco and L. Pareschi, “Asymptotic preserving implicit-explicit runge–kutta meth- ods for nonlinear kinetic equations,” SIAM Journal on Numerical Analysis, vol. 51, no. 2, pp. 1064–1087, 2013 (cit. on p. 27).
[55] M. Bennoune, M. Lemou, and L. Mieussens, “Uniformly stable numerical schemes for the boltzmann equation preserving the compressible navier–stokes asymptotics,” Jour- nal of Computational Physics, vol. 227, no. 8, pp. 3781–3803, 2008 (cit. on p. 27).
[56] F. Filbet and S. Jin, “A class of asymptotic-preserving schemes for kinetic equations and related problems with stiff sources,” Journal of Computational Physics, vol. 229, no. 20, pp. 7625–7648, 2010 (cit. on p. 27).
[57] K. Xu and J.-C. Huang, “A unified gas-kinetic scheme for continuum and rarefied flows,” Journal of Computational Physics, vol. 229, no. 20, pp. 7747–7764, 2010 (cit. on p. 27).
[58] K. Xu, Direct modeling for computational fluid dynamics: construction and application of unified gas-kinetic schemes. World Scientific, 2014, vol. 4 (cit. on p. 27).
[59] Z. Guo, K. Xu, and R. Wang, “Discrete unified gas kinetic scheme for all knudsen number flows: Low-speed isothermal case,” Physical Review E, vol. 88, no. 3, p. 033 305, 2013 (cit. on p. 27).
[60] Z. Guo, R. Wang, and K. Xu, “Discrete unified gas kinetic scheme for all knudsen number flows. ii. thermal compressible case,” Physical Review E, vol. 91, no. 3, p. 033 313, 2015 (cit. on p. 27).
[61] A. Bobylev, “The chapman-enskog and grad methods for solving the boltzmann equa- tion,” in Akademiia Nauk SSSR Doklady, vol. 262, 1982, pp. 71–75 (cit. on pp. 27, 70).
[62] J. A. McLennan, “Convergence of the chapman‐enskog expansion for the linearized boltzmann equation,” Physics of Fluids, vol. 8, pp. 1580–1584, 1965 (cit. on pp. 27, 70).
[63] P. Wang, M. T. Ho, L. Wu, Z. Guo, and Y. Zhang, “A comparative study of discrete ve- locity methods for low-speed rarefied gas flows,” Computers & Fluids, vol. 161, pp. 33– 46, 2018 (cit. on p. 27).
[64] H. Grad, “On the kinetic theory of rarefied gases,” Communications on Pure and Applied Mathematics, vol. 2, no. 4, pp. 331–407, 1949. DOI: https://doi.org/10. 1002/cpa.3160020403 (cit. on pp. 27, 62, 70).
[65] H. Struchtrup and P. Taheri, “Macroscopic transport models for rarefied gas flows: A brief review,” IMA Journal of Applied Mathematics (Institute of Mathematics and Its Applications), vol. 76, no. 5, pp. 672–697, 2011. DOI: 10.1093/imamat/hxr004 (cit. on pp. 27, 70).
[66] H. M. Mott-Smith, “The solution of the boltzmann equation for a shock wave,” Physical Review, vol. 82, pp. 885–892, 1951 (cit. on pp. 27, 64).
[67] I. Müller and T. Ruggeri, Rational extended thermodynamics. Springer Science & Busi- ness Media, 2013, vol. 37 (cit. on p. 27).
[68] M. Torrilhon, “Hyperbolic moment equations in kinetic gas theory based on multi- variate pearson-iv-distributions,” Communications in Computational Physics, vol. 7, pp. 639–673, 2009 (cit. on pp. 27, 70).
[69] C. D. Levermore, “Moment closure hierarchies for kinetic theories,” Journal of statisti- cal Physics, vol. 83, pp. 1021–1065, 1996 (cit. on pp. 27, 62).
[70] A. DasGupta and A. DasGupta, “The exponential family and statistical applications,” Probability for Statistics and Machine Learning: Fundamentals and Advanced Topics, pp. 583–612, 2011 (cit. on p. 27).
[71] F. Giroux and J. G. McDonald, “An approximation for the twenty-one-moment maximum- entropy model of rarefied gas dynamics,” International Journal of Computational Fluid Dynamics, vol. 35, no. 8, pp. 632–652, 2021 (cit. on p. 27).
[72] R. V. Abramov, “An improved algorithm for the multidimensional moment-constrained maximum entropy problem,” J. Comput. Phys., vol. 226, pp. 621–644, 2007 (cit. on pp. 27, 37, 38).110
[73] G. W. Alldredge, M. Frank, and C. D. Hauck, “A regularized entropy-based moment method for kinetic equations,” SIAM Journal on Applied Mathematics, vol. 79, no. 5, pp. 1627–1653, 2019 (cit. on p. 27).
[74] R. P. Schaerer, P. Bansal, and M. Torrilhon, “Efficient algorithms and implementations of entropy-based moment closures for rarefied gases,” J. Comput. Phys., vol. 340, pp. 138– 159, 2017 (cit. on pp. 27, 28, 30, 31, 36, 37, 55, 58, 60, 64, 67, 148, 150).
[75] P. Blanchard, D. J. Higham, and N. J. Higham, “Accurately computing the log-sum-exp and softmax functions,” IMA Journal of Numerical Analysis, vol. 41, no. 4, pp. 2311– 2330, 2021 (cit. on p. 28).
[76] J. Nocedal, Numerical optimization (Springer series in operations research.), eng, 2nd ed. New York: Springer, 2006, pp. 30–65 (cit. on pp. 28, 38).
[77] C. Cercignani, The Boltzmann equation and its applications (Applied mathematical sci- ences ; v. 67.), eng. New York: Springer-Verlag, 1988 (cit. on pp. 28, 147).
[78] M. Kardar, Statistical physics of particles, eng. Cambridge: Cambridge University Press, 2007 (cit. on p. 29).
[79] M. J. Wainwright and M. I. Jordan, “Graphical models, exponential families, and vari- ational inference,” Foundations and Trends in Machine Learning, vol. 1, no. 1-2, 2008. DOI: 10.1561/2200000001 (cit. on pp. 30, 62).
[80] B. O. Koopman, “On distributions admitting a sufficient statistic,” Transactions of the American Mathematical Society, vol. 39, pp. 399–409, 1936 (cit. on p. 31).
[81] M. Junk, “Maximum entropy for reduced moment problems,” Mathematical Models and Methods in Applied Sciences, vol. 10, pp. 1001–1025, 2000 (cit. on p. 31).
[82] A. N. Gorban and I. V. Karlin, “Hilbert’s 6th problem: Exact and approximate hydrody- namic manifolds for kinetic equations,” Bulletin of the American Mathematical Society, vol. 51, pp. 187–246, 2013 (cit. on p. 33).
[83] J. Bradbury et al., JAX: Composable transformations of Python+NumPy programs, ver- sion 0.2.5, 2018 (cit. on p. 44).
[84] B. R. Crain, “Exponential models, maximum likelihood estimation, and the haar condi- tion,” Journal of the American Statistical Association, vol. 71, pp. 737–740, 1976 (cit. on p. 47).
[85] C. Zheng, Y. Wang, and S. Chen, Phase transition in extended thermodynamics triggers sub-shocks, 2023 (cit. on p. 58).
[86] S. Chen and G. D. Doolen, “Lattice boltzmann method for fluid flows,” Annual Review of Fluid Mechanics, vol. 30, pp. 329–364, 1998 (cit. on pp. 60, 70).
[87] C. Park, Nonequilibrium hypersonic aerothermodynamics. New York: Wiley, 1990 (cit. on p. 60).
[88] R. Blandford and D. Eichler, “Particle acceleration at astrophysical shocks: A theory of cosmic ray origin,” Physics Reports, vol. 154, no. 1, pp. 1–75, 1987 (cit. on p. 60).
[89] S. I. Abarzhi, S. Gauthier, and K. R. Sreenivasan, “Turbulent mixing and beyond: Non- equilibrium processes from atomistic to astrophysical scales,” Philosophical transac- tions of the Royal Society of London. Series A: Mathematical, physical, and engineering sciences, vol. 371, no. 1982, pp. 20 120 435–20 120 435, 2013 (cit. on p. 60).
[90] V. D. Camiola, G. Mascali, and V. Romano, Charge Transport in Low Dimensional Semiconductor Structures: The Maximum Entropy Approach (Mathematics in Industry). Cham: Springer International Publishing AG, 2020, vol. 31 (cit. on p. 60).
[91] C. Cercignani, “The boltzmann equation,” in The Boltzmann Equation and Its Applica- tions. New York, NY: Springer New York, 1988, pp. 40–103. DOI: 10.1007/978- 1-4612-1039-9_2 (cit. on p. 60).
[92] I. MuÌller, Rational extended thermodynamics (Springer Tracts in Natural Philosophy, 37), Second edition. New York, New York State: Springer, 1998 (cit. on pp. 60, 61, 63).
[93] M. Torrilhon, “Modeling nonequilibrium gas flow based on moment equations,” Annual review of fluid mechanics, vol. 48, no. 1, pp. 429–458, 2016 (cit. on p. 60).
[94] J. McDonald and M. Torrilhon, “Affordable robust moment closures for cfd based on the maximum-entropy hierarchy,” Journal of computational physics, vol. 251, pp. 500–523, 2013 (cit. on p. 60).
[95] W. Israel, “Covariant fluid mechanics and thermodynamics: An introduction,” in Rela- tivistic Fluid Dynamics, A. M. Anile and Y. Choquet-Bruhat, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, 1989, pp. 152–210 (cit. on pp. 61, 63).
[96] G. Boillat and T. Ruggeri, “On the shock structure problem for hyperbolic system of balance laws and convex entropy,” Continuum mechanics and thermodynamics, vol. 10, no. 5, pp. 285–292, 1998 (cit. on pp. 61, 63).
[97] T. Ruggeri, “A complete classification of sub-shocks in the shock structure of a binary mixture of eulerian gases with different degrees of freedom,” Physics of fluids (1994), vol. 34, no. 6, pp. 66 116–, 2022 (cit. on pp. 61, 63).112
[98] W. Weiss, “Continuous shock structure in extended thermodynamics,” Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics, vol. 52, no. 6, R5760–R5763, 1995 (cit. on pp. 61, 63, 67).
[99] S. Taniguchi and T. Ruggeri, “On the sub-shock formation in extended thermodynam- ics,” International journal of non-linear mechanics, vol. 99, pp. 69–78, 2018 (cit. on pp. 61, 63).
[100] M. Bisi, F. Conforto, and G. Martalò, “Sub-shock formation in grad 10-moment equa- tions for a binary gas mixture,” Continuum mechanics and thermodynamics, vol. 28, no. 5, pp. 1295–1324, 2016 (cit. on pp. 61, 63).

[101] L. H. Holway, “Existence of kinetic theory solutions to the shock structure problem,” The Physics of Fluids, vol. 7, no. 6, pp. 911–913, 1964. DOI: 10.1063/1.1711307 (cit. on p. 61).

[102] Z. Cai and M. Torrilhon, “On the holway-weiss debate: Convergence of the grad-moment- expansion in kinetic gas theory,” Physics of Fluids, vol. 31, no. 12, p. 126 105, 2019. DOI: 10.1063/1.5127114 (cit. on p. 61).

[103] L. D. ( D. Landau, Fluid mechanics (Course of theoretical physics ; v. 6.), eng, 2nd ed., 2nd English ed., rev. Oxford: Pergamon Press, 1987, p. 335 (cit. on pp. 61, 151).

[104] C. Zheng, W. Yang, and S. Chen, Stabilizing the maximal entropy moment method for rarefied gas dynamics at single-precision, 2023 (cit. on p. 63).

[105] M. Kardar, “Statistical physics of particles,” 2007, ch. 3, pp. 48–50 (cit. on p. 63).

[106] C. Peter and K. Kremer, “Multiscale simulation of soft matter systems – from the atom- istic to the coarse-grained level and back,” Soft Matter, vol. 5, pp. 4357–4366, 2009 (cit. on p. 70).

[107] H. Lin and D. G. Truhlar, “Qm/mm: What have we learned, where are we, and where do we go from here?” Theoretical Chemistry Accounts, vol. 117, pp. 185–199, 2006 (cit. on p. 70).

[108] W. Weiss, “Continuous shock structure in extended thermodynamics,” Physical Review E, vol. 52, no. 6, R5760, 1995 (cit. on p. 70).

[109] J. Hana, C. Ma, Z. Ma, and E. Weinan, “Uniformly accurate machine learning-based hy- drodynamic models for kinetic equations,” Proceedings of the National Academy of Sci- ences of the United States of America, 2019. DOI: 10.1073/pnas.1909854116 (cit. on p. 71).113

[110] J. Zhang and W. Ma, “Data-driven discovery of governing equations for fluid dynamicsbased on molecular simulation,” Journal of Fluid Mechanics, 2020. DOI: 10.1017/jfm.2020.184 (cit. on p. 71).

[111] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Machine learning of linear differen￾tial equations using Gaussian processes,” Journal of Computational Physics, 2017. DOI:10.1016/j.jcp.2017.07.050 (cit. on pp. 71, 72, 80).

[112]S. H. Rudy, S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Data-driven discovery ofpartial differential equations,” Science Advances, 2017. DOI: 10.1126/sciadv.1602614 (cit. on pp. 71, 72, 80).

[113] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: Adeep learning framework for solving forward and inverse problems involving nonlin￾ear partial differential equations,” Journal of Computational Physics, 2019. DOI: 10.1016/j.jcp.2018.10.045 (cit. on pp. 71, 72, 80).

[114] Z. Long, Y. Lu, X. Ma, and B. Dong, “PDE-Net: Learning PDEs from data,” in 6thInternational Conference on Learning Representations, ICLR 2018 - Workshop TrackProceedings, 2018 (cit. on pp. 71, 72, 80).

[115] Z. Long, Y. Lu, and B. Dong, “PDE-Net 2.0: Learning PDEs from data with a numeric￾symbolic hybrid deep network,” Journal of Computational Physics, vol. 399, 2019. DOI:10.1016/j.jcp.2019.108925 (cit. on pp. 71, 72, 80).

[116] Y. LeCun, Y. Bengio, et al., “Convolutional networks for images, speech, and time se￾ries,” The handbook of brain theory and neural networks, vol. 3361, no. 10, p. 1995,1995 (cit. on p. 71).

[117] N. Dal Santo, S. Deparis, and L. Pegolotti, “Data driven approximation of parametrizedPDEs by reduced basis and neural networks,” Journal of Computational Physics, vol. 416,p. 109 550, 2020. DOI: 10.1016/j.jcp.2020.109550 (cit. on p. 71).

[118] A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind, “Automatic differen￾tiation in machine learning: A survey,” Journal of machine learning research, vol. 18,2018 (cit. on p. 71).

[119]S. Yip and M. Nelkin, “Application of a kinetic model to time-dependent density correla￾tions in fluids,” Physical Review, vol. 135, no. 5A, 1964. DOI: 10.1103/PhysRev.135.A1241 (cit. on p. 71).

[120] G. Fiocco and J. DeWolf, “Frequency spectrum of laser echoes from atmospheric con￾stituents and determination of the aerosol content of air,” Journal of Atmospheric Sci￾ences, vol. 25, no. 3, pp. 488–496, 1968 (cit. on p. 71).

[121] L. Wu and X.-J. Gu, “On the accuracy of macroscopic equations for linearized rarefiedgas flows,” Advances in Aerodynamics, vol. 2, no. 1, 2020. DOI: 10.1186/s42774-019-0025-4 (cit. on pp. 71, 81, 169).

[122]S. Chapman and T. G. Cowling, The mathematical theory of non-uniform gases: anaccount of the kinetic theory of viscosity, thermal conduction and diffusion in gases.Cambridge university press, 1990 (cit. on p. 72).

[123] E. M. Lifshitz and L. P. Pitaevskii, Statistical physics: theory of the condensed state.Elsevier, 2013, vol. 9, ch. 88, p. 372 (cit. on pp. 74, 159).

[124] L. D. Landau and E. M. Lifshitz, Course of theoretical physics. Elsevier, 2013, vol. 5,ch. 112, p. 343 (cit. on pp. 74, 160, 168).

[125] V. Michalis, A. Kalarakis, E. Skouras, and V. Burganos, “Rarefaction effects on gas vis￾cosity in the knudsen transition regime,” Microfluidics and Nanofluidics, vol. 9, pp. 847–853, Oct. 2010. DOI: 10.1007/s10404-010-0606-3 (cit. on p. 74).

[126] V. Ghaem-Maghami and A. D. May, “Rayleigh-brillouin spectrum of compressed he,ne, and ar. ii. the hydrodynamic region,” Physical Review A, vol. 22, pp. 698–705, 1980(cit. on p. 75).

[127] I. J. Goodfellow, Y. Bengio, and A. C. Courville, “Deep learning,” Nature, vol. 521,pp. 436–444, 2015 (cit. on p. 75).

[128] R. Pecora, “Doppler shifts in light scattering from pure liquids and polymer solutions,”The Journal of Chemical Physics, vol. 40, no. 6, pp. 1604–1614, 1964. DOI: 10.1063/1.1725368 (cit. on pp. 76, 163).

[129] J. D. Jackson, Classical Electrodynamics, 3rd ed. Wiley, 1998, ch. 10 (cit. on pp. 76,163).

[130] L. D. Landau, J. Bell, M. Kearsley, L. Pitaevskii, E. Lifshitz, and J. Sykes, Electrody￾namics of continuous media. elsevier, 2013, vol. 8, ch. 117 (cit. on pp. 76, 163, 164).

[131] D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” in 3rd In￾ternational Conference on Learning Representations, ICLR 2015 - Conference TrackProceedings, 2015 (cit. on p. 78).

[132] E. Roohi and M. Darbandi, “Extending the navier–stokes solutions to transition regime intwo-dimensional micro- and nanochannel flows using information preservation scheme,”Physics of Fluids, vol. 21, p. 082 001, 2009 (cit. on p. 80).

[133] G. E. Karniadakis, A. Beskok, N. R. Aluru, and C.-M. Ho, “Microflows and nanoflows:Fundamentals and simulation,” 2001 (cit. on p. 80).

[134] C. Cercignani, “Small and large mean free paths,” in The Boltzmann Equation and ItsApplications. New York, NY: Springer New York, 1988, p. 238. DOI: 10.1007/978-1-4612-1039-9_5 (cit. on p. 82).

[135] C. Cercignani, “Small and large mean free paths,” in The Boltzmann Equation and ItsApplications. New York, NY: Springer New York, 1988, p. 248. DOI: 10.1007/978-1-4612-1039-9_5 (cit. on p. 82).

[136] G. Santulli, “Epidemiology of cardiovascular disease in the 21st century: Updated up￾dated numbers and updated facts,” Journal of cardiovascular disease research, vol. 1,2013 (cit. on p. 85).

[137] C. M. Wiener, “Harrison’s principles of internal medicine,” 2013 (cit. on p. 85).

[138] A. Rauf et al., “P897 diagnostic yield of invasive coronary angiography in a uk districtgeneral hospital,” European Heart Journal, vol. 38, no. suppl_1, ehx501–P897, 2017(cit. on p. 86).

[139] M. Tavakol, S. Ashraf, and S. J. Brener, “Risks and complications of coronary angiog￾raphy: A comprehensive review,” Global Journal of Health Science, vol. 4, pp. 65–93,2012 (cit. on p. 86).

[140] J.-P. Collet et al., “2020 esc guidelines for the management of acute coronary syndromesin patients presenting without persistent st-segment elevation.,” European heart journal,2020 (cit. on p. 86).

[141] M. R. Patel et al., “Low diagnostic yield of elective coronary angiography.,” The NewEngland journal of medicine, vol. 362 10, pp. 886–95, 2010 (cit. on p. 86).

[142]S. R. Mehta et al., “Routine vs selective invasive strategies in patients with acute coro￾nary syndromes: A collaborative meta-analysis of randomized trials.,” JAMA, vol. 29323, pp. 2908–2917, 2005 (cit. on p. 86).

[143] L. E. Goldman, E. F. Cook, P. A. Johnson, D. A. Brand, G. W. Rouan, and T. H.Lee, “Prediction of the need for intensive care in patients who come to emergency de￾partments with acute chest pain.,” The New England journal of medicine, vol. 334 23,pp. 1498–1504, 1996 (cit. on p. 86).

[144] E. M. Antman et al., “The timi risk score for unstable angina/non-st elevation mi: Amethod for prognostication and therapeutic decision making.,” JAMA, vol. 284 7, pp. 835–842, 2000 (cit. on p. 86).

[145] J. M. Sanchis et al., “New risk score for patients with acute chest pain, non-st-segmentdeviation, and normal troponin concentrations: A comparison with the timi risk score.,”Journal of the American College of Cardiology, vol. 46 3, pp. 443–449, 2005 (cit. onp. 86).

[146] K. A. A. Fox et al., “Prediction of risk of death and myocardial infarction in the sixmonths after presentation with acute coronary syndrome: Prospective multinational ob￾servational study (grace),” BMJ : British Medical Journal, vol. 333, p. 1091, 2006 (cit.on p. 86).

[147] J.-P. Collet et al., “2020 esc guidelines for the management of acute coronary syndromesin patients presenting without persistent st-segment elevation.,” European heart journal,2020 (cit. on pp. 86, 88).

[148] A. D. Jamthikar et al., “Multiclass machine learning vs. conventional calculators forstroke/cvd risk assessment using carotid plaque predictors with coronary angiographyscores as gold standard: A 500 participants study,” The International Journal of Cardio￾vascular Imaging, vol. 37, pp. 1171–1187, 2020 (cit. on p. 87).

[149] A. D. Jamthikar et al., “Cardiovascular/stroke risk predictive calculators: A compari￾son between statistical and machine learning models.,” Cardiovascular diagnosis andtherapy, vol. 10 4, pp. 919–938, 2020 (cit. on p. 87).

[150] I. Kakadiaris, M. Vrigkas, A. A. Yen, T. Kuznetsova, M. J. Budoff, and M. Naghavi,“Machine learning outperforms acc/aha cvd risk calculator in mesa,” Journal of theAmerican Heart Association: Cardiovascular and Cerebrovascular Disease, vol. 7, 2018(cit. on p. 87).

[151] J. E. Stewart et al., “Applications of machine learning to undifferentiated chest pain in theemergency department: A systematic review,” PLoS ONE, vol. 16, 2021 (cit. on p. 87).

[152] L. Qin, Q. Qi, A. Aikeliyaer, W. Q. Hou, C. X. Zuo, and X. Ma, “Machine learning algo￾rithm can provide assistance for the diagnosis of non-st-segment elevation myocardialinfarction,” Postgraduate Medical Journal, 2022 (cit. on p. 87).

[153] F. Angiulli and C. Pizzuti, “Fast outlier detection in high dimensional spaces,” in PKDD,2002 (cit. on p. 88).

[154] P. Wagner et al., “PTB-XL, a large publicly available electrocardiography dataset,” Sci￾entific Data, vol. 7, no. 1, p. 154, 2020. DOI: 10.1038/s41597-020-0495-6(cit. on p. 90).

[155] P. Wagner, N. Strodthoff, R.-D. Bousseljot, W. Samek, and T. Schaeffter, PTB-XL, alarge publicly available electrocardiography dataset, 2020. DOI: 10.13026/qgmg-0d46 (cit. on p. 90).

[156] A. L. Goldberger et al., “PhysioBank, PhysioToolkit, and PhysioNet,”Circulation, vol. 101,no. 23, e215–e220, 2000. DOI: 10.1161/01.CIR.101.23.e215 (cit. on p. 90).

[157] N. Strodthoff, P. Wagner, T. Schaeffter, and W. Samek, “Deep learning for ecg anal￾ysis: Benchmarks and insights from ptb-xl,” IEEE Journal of Biomedical and HealthInformatics, vol. 25, pp. 1519–1528, 2021 (cit. on pp. 90, 98).

[158] F. J. Ferria, P. Pudilb, M. Hatefc, and J. Kittlerca, “Comparative study of techniques forlarge-scale feature selection,” 2007 (cit. on p. 90).

[159] B. Efron and T. J. Hastie, “Computer age statistical inference: Algorithms, evidence, anddata science,” 2016 (cit. on p. 91).

[160] A. P. Bradley, “The use of the area under the roc curve in the evaluation of machinelearning algorithms,” Pattern Recognit., vol. 30, pp. 1145–1159, 1997 (cit. on p. 92).

[161] C. X. Ling, J. Huang, and H. Zhang, “Auc: A statistically consistent and more discrim￾inating measure than accuracy,” in IJCAI, 2003 (cit. on p. 92).

[162] E. Christodoulou, J. Ma, G. S. Collins, E. W. Steyerberg, J. Y. J. Verbakel, and B. vanCalster, “A systematic review shows no performance benefit of machine learning overlogistic regression for clinical prediction models.,” Journal of clinical epidemiology,vol. 110, pp. 12–22, 2019 (cit. on p. 98).

[163] G. E. Uhlenbeck and L. S. Ornstein, “On the theory of the brownian motion,” PhysicalReview, vol. 36, pp. 823–841, 1930 (cit. on p. 121).

[164] P. Vincent, “A connection between score matching and denoising autoencoders,” NeuralComputation, vol. 23, pp. 1661–1674, 2011 (cit. on p. 122).

[165] B. D. O. Anderson, “Reverse-time diffusion equation models,” Stochastic Processes andtheir Applications, vol. 12, pp. 313–326, 1982 (cit. on p. 123).

[166] H. F. Walker and P. Ni, “Anderson acceleration for fixed-point iterations,” SIAM J. Nu￾mer. Anal., vol. 49, pp. 1715–1735, 2011 (cit. on p. 131).

[167] coderpiaobozhe, Pytorch implementation of classifier-free diffusion guidance, 2023. [On￾line]. Available: https://github.com/coderpiaobozhe/classifier￾free-diffusion-guidance-Pytorch (cit. on p. 137).

[168] P. L. Bhatnagar, E. P. Gross, and M. Krook, “A model for collision processes in gases.i. small amplitude processes in charged and neutral one-component systems,” PhysicalReview, vol. 94, pp. 511–525, 1954 (cit. on p. 147).

[169] G. A. Bird, Molecular gas dynamics and the direct simulation of gas flows (Oxford en￾gineering science series ; 42.), eng. Oxford: Clarendon, 1994 (cit. on p. 148).

[170] M. J. Wainwright and M. I. Jordan, “Graphical models, exponential families, and vari￾ational inference,” Foundations and Trends in Machine Learning, vol. 1, no. 1-2, 2008.DOI: 10.1561/2200000001 (cit. on p. 149).

[171] G. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows. ClarendonPress, 1994, ch. 11.6 (cit. on pp. 152, 171).

[172] G. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows. ClarendonPress, 1994, ch. 2.2, p. 33 (cit. on pp. 154, 171).

[173] D. J. Griffiths and D. F. Schroeter, Introduction to Quantum Mechanics, 3rd ed. Cam￾bridge University Press, 2018, ch. 10.1.1, p. 376. DOI: 10.1017/9781316995433(cit. on pp. 154, 171).

[174] L. Landau and E. Lifshitz, Mechanics: Volume 1. Elsevier Science, 1982, ch. 18, p. 48(cit. on pp. 154, 171).

[175] G. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows. ClarendonPress, 1994, ch. 2.6-2.7 (cit. on pp. 154, 171).

[176] G. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows. ClarendonPress, 1994, ch. 2.4, p. 38 (cit. on p. 155).

[177] G. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows. ClarendonPress, 1994, ch. 4.3, p. 93 (cit. on pp. 155, 172).

[178] G. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows. ClarendonPress, 1994, ch. 3.5, pp. 68–70 (cit. on pp. 155, 172).

[179] M. Kardar, “Statistical physics of particles,” 2007, ch. 3, pp. 78–81 (cit. on p. 157).

[180] K. F. Riley, M. P. Hobson, and S. J. Bence, “Integral transforms,” in Mathematical Meth￾ods for Physics and Engineering: A Comprehensive Guide, 3rd ed. Cambridge UniversityPress, 2006, ch. 13, p. 435. DOI: 10.1017/CBO9780511810763.016 (cit. onp. 158).

[181] E. M. Lifshitz and L. P. Pitaevskii, Statistical physics: theory of the condensed state.Elsevier, 2013, vol. 9, ch. 88, p. 370 (cit. on pp. 160, 168, 170).

[182] J. D. Jackson, Classical Electrodynamics, 3rd ed. Wiley, 1998, ch. 10.2, p. 463 (cit. onp. 163).

[183] D. J. Griffiths and D. F. Schroeter, Introduction to Quantum Mechanics, 3rd ed. Cam￾bridge University Press, 2018, ch. 10.4. DOI: 10.1017/9781316995433 (cit. onp. 164).

[184] K. F. Riley, M. P. Hobson, and S. J. Bence, “Integral transforms,” in Mathematical Meth￾ods for Physics and Engineering: A Comprehensive Guide, 3rd ed. Cambridge UniversityPress, 2006, ch. 13, p. 450. DOI: 10.1017/CBO9780511810763.016 (cit. onp. 164).

[185] E. M. Lifshitz and L. P. Pitaevskii, Statistical physics: theory of the condensed state.Elsevier, 2013, vol. 9, ch. 89, pp. 373–377 (cit. on p. 166).

[186] G. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows. ClarendonPress, 1994, ch. Appendix C, p. 426 (cit. on p. 171).

[187] G. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows. ClarendonPress, 1994, ch. 4.3, p. 93 (cit. on p. 172).