基于机器学习的烟气催化材料性能预测与优化设计

       科技进步推动人类社会不断向前发展，人类发展史实际上就是材料的应用与发展史。现代社会， 人们对于高性能、 低成本材料的需求逐渐增加，但是材料的开发与设计周期漫长， 基于试错法的材料开发无法满足社会的快速发展需求。因此， 亟需寻找更加高效的材料研发方式。 随着计算机技术飞速发展与运算能力显著提升， 使得基于数据驱动的机器学习方法备受关注， 这种基于计算机和数据的探索方式， 标志着材料研究进入了大数据和人工智能时代。
      本研究将数据驱动（ Data-Driven）的机器学习（ Machine Learning） 方法应用于催化材料研发领域，以典型的空气污染物——氮氧化物（ NO_x）的选择性催化还原（ Selective Catalytic Reduction）为例，评估机器学习方法应用于环境催化剂（ Environmental Catalysts） 的筛选和优化方面的适用性，展示了从数据库建立、机器学习模型训练与测试、模型验证和预测、 模型不确定性分析、人机互动实现材料设计与优化等在内的完整流程。 研究结果表明，已有的环境催化剂方面的研究积累的大量实验数据，可被用于探索和优化针对特定应用的环境催化剂，并能够加速这一进程。 此外，该方法对于筛选和设计新的环境催化剂、优化环境催化剂合成和应用条件等均非常有效。 但是， 由于文献报道的数据通常并不完整，限制了数据的应用潜力， 导致模型的模拟结果存在不确定性，尤其表现在未知催化剂的性能预测方面。因此我们提出了通过重复预测（ Repeated Prediction）和整体平均（ Ensemble Average） 的方法来寻找具有应用潜质的环境催化剂，并且耦合特定功能的算法，实现机器学习指导下， 人机互动模式的新型环境催化剂的开发与优化。
 

    The progress of science and technology promotes the development of human society, and the history of human development is actually the history of the application of materials. In modern society, people's demand for high-performance and low-cost materials is gradually increasing. However, the long duration of design and application cycle of materials based on trial-and-error method cannot meet the needs of society. Therefore, it is urgent to find a more efficient way of material research and development. With the rapid improvements in computer technology and computing algorithms, data-driven machine learning methods have attracted much attention. This type of exploration methods based on computers and data marks that materials research has entered the era of big data and artificial intelligence (AI).   

    In this study, a data-driven, machine learning (ML) approach was applied to discover and develop catalytic materials for environmental applications. The selective catalytic reduction (SCR) of nitrogen oxides (NO_x) with ammonia was used as an example in this study to evaluate the applicability of machine learning methods for identifying potential environmental catalysts (ECs). The detail procedures including database assemblage, the training and testing of a machine learning model, the validation and prediction of the model, model uncertainties analysis, and human-machine interaction to achieve material design and optimization are all provided. The results indicated that there is significant amount of data accumulated in environmental catalysts, which can be used to explore and to optimize ECs for specific applications. In addition, this method is very effective for identifying new ECs, and optimizing the synthesis and application conditions of ECs. However, the data reported in the literature are often incomplete, limiting the application potential of the data, leading to uncertainties in the calculation results of the models, especially for the predictions of unknown catalysts. Therefore, an approach of repeated predictions and ensemble averaging were then proposed to find conditions for synthesizing and applying promising ECs. Furthermore, coupled algorithms with specific functions was applied to realize the development and optimization of new ECs under the guidance of machine learning.

[1] HOLZINGER A. Introduction to Machine Learning & Knowledge Extraction (MAKE) [J]. Machine Learning and Knowledge Extraction, 2019, 1(1): 1-20.
[2] BELSKY, A., HELLENBRANDT, M., KAREN, V. L., et al. New Developments in the Inorganic Crystal Structure Database (ICSD): Accessibility in Support of Materials Research and Design [J]. Acta Crystallographica Section B, 2002, 58(3): 364-369.
[3] ALLEN, F. H., The Cambridge Structural Database: A Quarter of A Million Crystal Structures and Rising [J]. Acta Crystallographica Section B, 2002, 58(3): 380-388.
[4] GRAŽULIS S, CHATEIGNER D, DOWNS R T, et al. Crystallography Open Database - An Open-Access Collection of Crystal Structures [J]. Jorunal of Applied Crystallography, 2009, 42(4): 726-729.
[5] 米晓希, 汤爱涛, 朱雨晨, 等. 机器学习技术在材料科学领域中的应用进展 [J]. 材料导报, 2021, 35(15): 15115-15124.
[6] OKOROAFOR E R, SMITH C M, OCHIE K I, et al. Machine Learning in Subsurface Geothermal Energy: Two Decades in Review [J]. Geothermics, 2022, 102: 102401.
[7] 胡越, 罗东阳, 花奎, 等. 关于深度学习的综述与讨论[J]. 智能系统学报, 2019, 14(1): 1-19.
[8] KITCHIN J R. Machine Learning in Catalysis [J]. Nature Catalysis, 2018, 1(4): 230-232.
[9] HUSSAIN S S, RIAZ R, FATIMA T, et al. A Comparative Study of Supervised Machine Learning Techniques for Diagnosing Mode of Delivery in Medical Sciences [J]. International Journal of Advanced Computer Science and Applications, 2019, 10(12): 120-125.
[10] CAIXINHA M, NUNES S. Machine Learning Techniques in Clinical Vision Sciences [J]. Current Eye Research, 2017, 42(1): 1-15.
[11] AWAYSHEH A, WILCKE J, ELVINGER F, et al. Review of Medical Decision Support and Machine-Learning Methods [J]. Veterinary Pathology, 2019, 56(4): 512-525.
[12] VAMATHEVAN J, CLARK D, CZODROWSKI P, et al. Applications of Machine Learning in Drug Discovery and Development [J]. Nature Review Drug Discovery, 2019, 18(6): 463-477.
[13] JIN ML, REN WW, ZHANG WG, et al. Exploring the Underlying Mechanism of Shenyankangfu Tablet in the Treatment of Glomerulonephritis through Network Pharmacology, Machine Learning, Molecular Docking, and Experimental Validation [J]. Drug Design Development and Therapy, 2021, 15: 4585-4601.
[14] IETSWAART R, ARAT S, CHEN A X, et al. Machine Learning Guided Association of Adverse Drug Reactions with in Vitro Target-Based Pharmacology [J]. Ebiomedicine, 2020, 57: 102837.
[15] DARA S, DHAMERCHERLA S, JADAV S S, et al. Machine Learning in Drug Discovery: A Review [J]. Artificial Intelligence Review, 2021, 55(3): 1947-1999.
[16] CHARPENTIER A, ELIE R, REMLINGER C. Reinforcement Learning in Economics and Finance [J]. Computational Economics, 2021, 3(13): 69-84.
[17] Zhu Y, Zhou L, Xie C, et al. Forecasting SMEs' Credit Risk in Supply Chain Finance with An Enhanced Hybrid Ensemble Machine Learning Approach [J]. International Journal of Production Economics, 2019, 2(11): 22-33.
[18] SHUKLA D, KESHMIRI S, BECKAGE N, et al. Imitation Learning for Neural Network Autopilot in Fixed-Wing Unmanned Aerial Systems [Z]. 2020 International Conference on Unmanned Aircraft Systems. 2020: 1508-1517
[19] DELECRAZ S, BECERRA-BONACHE L, FAVRE B, ET AL. Multimodal Machine Learning for Natural Language Processing: Disambiguating Prepositional Phrase Attachments with Images [J]. Neural Processing Letters, 2021, 53(5): 3095-3121.
[20] MORETTI F, PIZZUTI S, PANZIERI S, et al. Urban Traffic Flow Forecasting through Statistical and Neural Network Bagging Ensemble Hybrid Modeling [J]. Neurocomputing, 2015, 167: 3-7.
[21] 赵旭, 奉有泉, 陶原, 等.基于计算流体力学的数据中心机器学习热模型数据增强技术 [J]. 电力信息与通信技术, 2021, 19(04): 18-24.
[22] 王建春, 晋国栋. 机器学习在力学模拟与控制中的应用专题序 [J].力学学报, 2021, 53(10): 2613-2615.
[23] 彭志勇, 龙虎. 人工智能背景下量子机器学习算法的概论 [J]. 信息与电脑, 2020, 32(04): 39-43.
[24] JIANG S. Machine Learning Research in Big Data Environment [Z]. 2018 5th International Conference on Electrical & Electronics Engineering and Computer Science. 2018: 227-231
[25] SHEN LQ, AMATULLI G, SETHI T, et al. Estimating Nitrogen and Phosphorus Concentrations in Streams and Rivers, within A Machine Learning Framework [J]. Scitific Data, 2020, 7(1): 161.
[26] REICHSTEIN M, CAMPS-VALLS G, STEVENS B, et al. Deep Learning and Process Understanding for Data-Driven Earth System Science [J]. Nature, 2019, 566(7743): 195-204.
[27] ZAHRT AF, HENLE JJ, ROSE BT, et al. Prediction of Higher-Selectivity Catalysts by Computer-Driven Workflow and Machine Learning [J]. Science, 2019, 363(6424).
[28] GÜNAY M E, YILDIRIM R. Analysis of Selective CO Oxidation Over Promoted Pt/Al2O3 Catalysts Using Modular Neural Networks: Combining Preparation and Operational Variables [J]. Applied Catalysis A: General, 2010, 377(1-2): 174-180.
[29] Wan X, Feng W, Wang Y, et al. Materials Discovery and Properties Prediction in Thermal Transport Via Materials Informatics: A Mini Review [J]. Nano Letter, 2019, 19(6): 3387-3395.
[30] LI T, XING F, LIU T, et al. Cost, Performance Prediction and Optimization of A Vanadium Flow Battery by Machine-Learning [J]. Energy & Environmental Science, 2020, 13(11): 4353-4361.
[31] SUN W, ZHENG Y, YANG K, et al. Machine Learning–Assisted Molecular Design and Efficiency Prediction for High-Performance Organic Photovoltaic Materials [J]. Science Advances, 5(11): eaay4275.
[32] BARNETT J W, BILCHAK C R, WANG Y, et al. Designing Exceptional Gas-Separation Polymer Membranes Using Machine Learning [J/OL]. Science Advances, 2020, 6(20):eaaz4301
[33] FALIVENE L, CAO Z, PETTA A, et al. Towards the Online Computer-Aided Design of Catalytic Pockets [J]. Nature Chemistry, 2019, 11(10): 872-879.
[34] UMEGAKI T, WATANABE Y, NUKUI N, et al. Optimization of Catalyst for Methanol Synthesis by A Combinatorial Approach Using A Parallel Activity Test and Genetic Algorithm Assisted by A Neural Network [J]. Energy & Fuels, 2003, 17:455-464.
[35] SUZUKI K, TOYAO T, MAENO Z, et al. Statistical Analysis and Discovery of Heterogeneous Catalysts Based on Machine Learning From Diverse Published Data [J]. ChemCatChem, 2019, 11(18): 4537-4547.
[36] LI Z, MA X, XIN H. Feature Engineering of Machine-Learning Chemisorption Models for Catalyst Design [J]. Catalysis Today, 2017, 280: 232-238.
[37] NOH J, KIM J, BACK S, et al. Catalyst Design Using Actively Learned Machine with Non-Ab Initio Input Features Towards CO2 Reduction Reactions [J]. 2017.
[38] YUAN H, SUN S, ABU-REESH I M, et al. Unravelling and Reconstructing the Nexus of Salinity, Electricity, and Microbial Ecology for Bioelectrochemical Desalination [J]. Environment Science & Technology, 2017, 51(21): 12672-12682.
[39] SARAVANAN R, KARTHIKEYAN N, GUPTA V K, et al. ZnO/Ag Nanocomposite: An Efficient Catalyst for Degradation Studies of Textile Effluents Under Visible Light [J]. Materials Science and Engineering: C, 2013, 33(4): 2235-2244.
[40] KUMMER J T. Catalysts for Automobile Emission Control [J]. Progress in Energy and Combustion Science, 1980, 6(2): 177-199.
[41] MARCOTTE N, COQ B, SAVILL-JOVITT C, et al. Multi-Component Zirconia–Titania Mixed Oxides: Catalytic Materials with Unprecedented Performance in the Selective Catalytic Reduction of NOx with NH3 after Harsh Hydrothermal Ageing [J]. Applied Catalysis B: Environmental, 2011, 105(3): 373-376.
[42] CHONG MN, JIN B, CHOW CWK, et al. Recent Developments in Photocatalytic Water Treatment Technology: A Review [J]. Water Research, 2010, 44(10): 2997-3027.
[43] MECAROVA M, MILLER N A, CLARK N C, et al. Selective Catalytic Reduction of NOx with Ammonia on Gallium-Exchanged Ferrierites [J]. Applied Catalysis A-General, 2005, 282(1-2): 267-272.
[44] LIU Y, ZHAO T, JU W, et al. Materials Discovery and Design Using Machine Learning [J]. Journal of Materiomics, 2017, 3(3): 159-177.
[45] RAMPRASAD R, BATRA R, PILANIA G, et al. Machine Learning in Materials Informatics: Recent Applications and Prospects [J]. Npj Computational Materials, 2017, 3(1): 54.
[46] SAAL J E, KIRKLIN S, AYKOL M, et al. Materials Design and Discovery with High-Throughput Density Functional Theory: The Open Quantum Materials Database (OQMD) [J]. The Journal of Materials, 2013, 65(11): 1501-1509.
[47] TAN L S, GE F, LI J, et al. HCEP: A Hybrid Cluster-Based Energy-Efficient Protocol for Wireless Sensor Networks [J]. International Journal of Sensor Networks, 2009, 5(2): 67-78.
[48] BARTLETT P L, LONG P M, LUGOSI G, et al. Benign Overfitting in Linear Regression [J]. PNAS, 2020, 117(48): 30063-30070.
[49] GHIRINGHELLI L M, VYBIRAL J, LEVCHENKO S V, et al. Big Data of Materials Science: Critical Role of the Descriptor [J]. Physic Review Letter, 2015, 114(10): 105503.
[50] KIM B, LEE S, KIM J. Inverse Design of Porous Materials Using Artificial Neural Networks [J]. Science Advances, 2020, 6(1): eaax9324.
[51] BZDOK D, ALTMAN N, KRZYWINSKI M. Statistics Versus Machine Learning [J]. Nature Methods, 2018, 15(4): 233-234.
[52] HASNAOUI H, KREA M, ROIZARD D. Neural Networks for the Prediction of Polymer Permeability to Gases [J]. Journal of Membrane Science, 2017, 541: 541-549.
[53] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: A Simple Way to Prevent Neural Networks from Overfitting [J]. The Journal of Machine Learning Research, 2014, 15(1): 1929–1958.
[54] FORESEE F D, HAGAN M T. Gauss-Newton approximation to Bayesian learning [J]. Proceedings of International Conference on Neural Networks (ICNN'97), 1997, 3(3): 1930-1935.
[55] JAZAYERI K, JAZAYERI M, UYSAL S. Comparative Analysis of Levenberg-Marquardt and Bayesian Regularization Backpropagation Algorithms in Photovoltaic Power Estimation Using Artificial Neural Network [M] Industrial Coference on Data Mining. 2016, 9728: 80-95.
[56] CHO S, KIM M, LYU B, et al. Optimization of An Explosive Waste Incinerator Via An Artificial Neural Network Surrogate Model [J]. Chemical Engineering Journal, 2021, 407: 126659.
[57] MACKAY D J C. Bayesian Interpolation [J]. Neural Computation, 1992, 4(3): 415-47.
[58] KHAN I, RAJA M A Z, SHOAIB M, et al. Design of Neural Network with Levenberg-Marquardt and Bayesian Regularization Backpropagation for Solving Pantograph Delay Differential Equations [J]. IEEE Access, 2020, 8: 137918-137933.
[59] SHI Y, TANG X, YI H, et al. Controlled Synthesis of Spinel-Type Mesoporous Mn–Co Rods for SCR of NOx with NH3 at Low Temperature [J]. Industrial & Engineering Chemistry Research, 2019, 58(9): 3606-3617.
[60] YANG S, GUO Y, CHANG H, et al. Novel Effect of SO2 on the SCR Reaction Over CeO2: Mechanism and Significance [J]. Applied Catalysis B: Environmental, 2013, 136-137: 19-28.
[61] TAN W, LIU A, XIE S, et al. Ce-Si Mixed Oxide: A High Sulfur Resistant Catalyst in the NH3-SCR Reaction through the Mechanism-Enhanced Process [J]. Environment Science & Technology, 2021, 55(6): 4017-4026.
[62] ZHANG D, MA Z, WANG B, et al. VxMn(4-x)Mo3Ce3/Ti catalysts for Selective Catalytic Reduction of NO by NH3 [J]. Journal of Environ Science, 2020, 88: 145-154.
[63] JIANG B, LIN B, LI Z, et al. Mn/TiO2 Catalysts Prepared by Ultrasonic Spray Pyrolysis Method for NOx Removal in Low-Temperature SCR Reaction [J]. Colloids and Surfaces A: Physicochemical and Engineering Aspects, 2020, 586: 124210.
[64] GAO E, SUN G, ZHANG W, et al. Surface Lattice Oxygen Activation Via Zr4+ Cations Substituting on A2+ Sites of MnCr2O4 Forming Zrx-Mn1-xCr2O4 Catalysts for Enhanced NH3-SCR Performance [J]. Chemical Engineering Journal, 2020, 380: 122397.
[65] GAO G, SHI J-W, FAN Z, et al. MnM2O4 Microspheres (M = Co, Cu, Ni) for Selective Catalytic Reduction of NO with NH3: Comparative Study on Catalytic Activity and Reaction Mechanism Via in-Situ Diffuse Reflectance Infrared Fourier Transform Spectroscopy [J]. Chemical Engineering Journal, 2017, 325: 91-100.
[66] GAO C, SHI J-W, FAN Z, et al. Eu-Mn-Ti Mixed Oxides for the SCR of NOx with NH3: The Effects of Eu-Modification on Catalytic Performance and Mechanism [J]. Fuel Processing Technology, 2017, 167: 322-33.
[67] GENG Y, SHAN W, YANG S, et al. W-Modified Mn–Ti Mixed Oxide Catalyst for the Selective Catalytic Reduction of NO with NH3 [J]. Industrial & Engineering Chemistry Research, 2018, 57(28): 9112-9119.
[68] WANG C, YU F, ZHU M, et al. Highly Selective Catalytic Reduction of NOx by MnOx-CeO2-Al2O3 Catalysts Prepared by Self-Propagating High-Temperature Synthesis [J]. Journal of Environment Science, 2019, 75: 124-35.
[69] GROSSALE A, NOVA I, TRONCONI E, et al. The Chemistry of the NO/NO2–NH3 “Fast” SCR Reaction Over Fe-ZSM5 Investigated by Transient Reaction Analysis [J]. Journal of Catalysis, 2008, 256(2): 312-322.
[70] CHEN Z, YANG Q, LI H, et al. Cr–MnOx Mixed-Oxide Catalysts for Selective Catalytic Reduction of NOx with NH3 at Low Temperature [J]. Journal of Catalysis, 2010, 276(1): 56-65.
[71] KIRKLIN S, SAAL J E, MEREDIG B, et al. The Open Quantum Materials Database (OQMD): Assessing the Accuracy of DFT Formation Energies [J]. Npj Computational Materials, 2015, 1(1): 15010.
[72] ZAVYALOVA U, HOLENA M, SCHLÖGL R, et al. Statistical Analysis of Past Catalytic Data on Oxidative Methane Coupling for New Insights Into the Composition of High-Performance Catalysts [J]. ChemCatChem, 2011, 3(12): 1935-1947.
[73] RACCUGLIA P, ELBERT K C, ADLER P D, et al. Machine-Learning-Assisted Materials Discovery Using Failed Experiments [J]. Nature, 2016, 533(7601): 73-76.
[74] LI H, ZHANG Z, LIU Z. Application of Artificial Neural Networks for Catalysis: A Review [J]. Catalysts, 2017, 7(10): 306.
[75] CHEN Y, LI R, SUO H, et al. Evaluation of A Data-Driven, Machine Learning Approach for Identifying Potential Candidates for Environmental Catalysts: From Database Development to Prediction [J]. ACS ES&T Engineering, 2021, 1(8): 1246-1257.
[76] LABOUDI Z, CHIKHI S. Comparison of Genetic Algorithm and Quantum Genetic Algorithm [J]. International Arab Journal of Information Technology, 2012, 9(3): 243-249.
[77] GU R. Research on Matching Mechanism of Resource Based on Genetic Algorithm [Z]. Proceedings of the 2018 International Conference on Computer Modeling, Simulation and Algorithm (CMSA 2018). 2018: 123-126