基于深度学习的有机反应分类模型研究

如何快速有效地对有机反应进行分类一直是研究者感兴趣的话题，对有机化学的研究具有重要的意义。对有机反应进行分类可以帮助构建反应数据库、方便反应检索和知识发现、揭示反应之间的内在联系和规律、发现新的反应等。传统的反应分类通常基于人工经验和规则，而深度学习作
为一种数据驱动的方法，可以通过学习大规模反应数据来实现快速而准确 的反应类别预测，并且不需要人工定义复杂的规则或特征。有机反应可以看成一种自然语言，利用深度学习语言模型可以很好地学习到反应的规律并做出较准确的预测。本研究使用的数据集源于开源数据集 USPTO 和商业数据集 Pistachio，并从断键和成键的角度，抽取化学反应模板来定义反应类别。本文采用基于 RNN 架构的 GRU 和 LSTM 模型以及基于 transformer编码器的 BERT 模型预测反应类别，并使用准确率和马修斯相关系数等指 标评价模型的性能。深度学习模型在训练集上训练后，在测试集上都取得了较高的预测准确率和马修斯相关系数，结果表明了深度学习模型在类别不平衡的反应数据上具有良好的学习能力和泛化能力，能够捕捉到不同化学反应类别之间的细微差别，然后较准确地预测反应的类别。本文证明了基于深度学习的反应类型预测模型的有效性和可靠性，三个模型的都达到了比较高的预测性能，这些结果为进一步开发更精准和高效的化学反应预 测方法提供了有价值的参考。

[1] PASZKE A, GROSS S, MASSA F, et al. Pytorch: An imperative style, high performance deep learning library[J]. Advances in Neural Information Processing Systems, 2019, 32
[2] TOWNSHEND R J, EISMANN S, WATKINS A M, et al. Geometric deep learning of RNA structure[J]. Science, 2021, 373(6558): 1047-1051.
[3] JUMPER J, EVANS R, PRITZEL A, et al. Highly accurate protein structure prediction with AlphaFold[J]. Nature, 2021, 596(7873): 583-589.
[4] TUNYASUVUNAKOOL K, ADLER J, WU Z, et al. Highly accurate protein structure prediction for the human proteome[J]. Nature, 2021, 596(7873): 590-596.
[5] CALDEIRA J, WU W K, NORD B, et al. DeepCMB: Lensing reconstruction of the cosmic microwave background with deep neural networks[J]. Astronomy and Computing, 2019, 28: 100307.
[6] WAHL C B, AYKOL M, SWISHER J H, et al. Machine learning–accelerated design and synthesis of polyelemental heterostructures[J]. Science Advances, 2021, 7(52): eabj5505.
[7] SUN W B, ZHENG Y J, YANG K, et al. Machine learning-assisted molecular design and efficiency prediction for high-performance organic photovoltaic materials[J]. Science Advances, 2019, 5(11)
[8] KONG S, RICCI F, GUEVARRA D, et al. Density of states prediction for materials discovery via contrastive learning from probabilistic embeddings[J]. Nature Communications, 2022, 13(1): 949.
[9] LIU C, FUJITA E, KATSURA Y, et al. Machine learning to predict quasicrystals from chemical compositions[J]. Advanced Materials, 2021, 33(36): 2102507.
[10] DIJKSTRA M, LUIJTEN E. From predictive modelling to machine learning and reverse engineering of colloidal self-assembly[J]. Nature Materials, 2021, 20(6): 762-773.
[11] AHNEMAN D T, ESTRADA J G, LIN S, et al. Predicting reaction performance in C–N cross-coupling using machine learning[J]. Science, 2018, 360(6385): 186-190.
[12] REID J P, SIGMAN M S. Holistic prediction of enantioselectivity in asymmetric catalysis[J]. Nature, 2019, 571(7765): 343-348.
[13] ZAHRT A F, HENLE J J, ROSE B T, et al. Prediction of higher-selectivity catalysts by computer-driven workflow and machine learning[J]. Science, 2019, 363(6424): eaau5631.
[14] DE ALMEIDA A F, MOREIRA R, RODRIGUES T. Synthetic organic driven by artificial intelligence[J]. Nature Reviews Chemistry, 2019, 3(10): 589-604.
[15] SCHWALLER P, VAUCHER A C, LAINO T, et al. Prediction of chemical reaction yields using deep learning[J]. Machine Learning: Science and Technology, 2021, 2(1): 015016.
[16] SINGH S, PAREEK M, CHANGOTRA A, et al. A unified machine-learning protocol for asymmetric catalysis as a proof of concept demonstration using asymmetric hydrogenation[J]. Proceedings of the National Academy of Sciences, 2020, 117(3): 1339-1345.
[17] GUAN Y, COLEY C W, WU H, et al. Regio-selectivity prediction with a machine learned reaction representation and on-the-fly quantum mechanical descriptors[J]. Chemical Science, 2021, 12(6): 2198-2208.
[18] GAO H, STRUBLE T J, COLEY C W, et al. Using machine learning to predict suitable conditions for organic reactions[J]. ACS Central Science, 2018, 4(11): 1465-1476.
[19] SHIELDS B J, STEVENS J, LI J, et al. Bayesian reaction optimization as a tool for chemical synthesis[J]. Nature, 2021, 590(7844): 89-96.
[20] LIU B, RAMSUNDAR B, KAWTHEKAR P, et al. Retrosynthetic reaction prediction using neural sequence-to-sequence models[J]. ACS Central Science, 2017, 3(10): 1103-1113.
[21] ZHANG P, EUN J, ELKIN M, et al. A neural network model informs the total synthesis of clovane sesquiterpenoids[J]. Nature Synthesis, 2023: 1-8.
[22] BROCKHERDE F, VOGT L, LI L, et al. Bypassing the Kohn-Sham equations with machine learning[J]. Nature Communications, 2017, 8(1): 872.
[23] PFAU D, SPENCER J S, MATTHEWS A G, et al. Ab initio solution of the many electron Schrödinger equation with deep neural networks[J]. Physical Review Research, 2020, 2(3): 033429.
[24] KIRKPATRICK J, MCMORROW B, TURBAN D H, et al. Pushing the frontiers of density functionals by solving the fractional electron problem[J]. Science, 2021, 374(6573): 1385-1389.
[25] PEDERSON R, KALITA B, BURKE K. Machine learning and density functional theory[J]. Nature Reviews Physics, 2022, 4(6): 357-358.
[26] SANDFORT F, STRIETH-KALTHOFF F, KUHNEMUND M, et al. A Structure-Based Platform for Predicting Chemical Reactivity[J]. Chem, 2020, 6(6): 1379-1390.
[27] MARYASIN B, MARQUETAND P, MAULIDE N. Machine Learning for Organic Synthesis: Are Robots Replacing Chemists?[J]. Angewandte Chemie-International Edition, 2018, 57(24): 6978-6980.
[28] SANTANILLA A B, REGALADO E L, PEREIRA T, et al. Nanomole-scale high throughput chemistry for the synthesis of complex molecules[J]. Science, 2015, 347(6217): 49-53.
[29] SELEKMAN J A, QIU J, TRAN K, et al. High-Throughput Automation in Process Development[J]. Annual Review of Chemical and Biomolecular Engineering, Vol 8, 2017, 8: 525-547.
[30] WARR W A. A short review of chemical reaction database systems, computer‐aided synthesis design, reaction prediction and synthetic feasibility[J]. Molecular Informatics, 2014, 33(6‐7): 469-476.
[31] KRAUT H, EIBLMAIER J, GRETHE G, et al. Algorithm for reaction classification[J]. Journal of Chemical Information and Modeling, 2013, 53(11): 2884-2895.
[32] CHEN L R, GASTEIGER J. Knowledge discovery in reaction databases: Landscaping organic reactions by a self-organizing neural network[J]. Journal of the American Chemical Society, 1997, 119(17): 4033-4042.
[33] ENGKVIST O, NORRBY P-O, SELMI N, et al. Computational prediction of chemical reactions: current status and outlook[J]. Drug Discovery Today, 2018, 23(6): 1203-1218.
[34] LOWE D M. Extraction of chemical structures and reactions from the literature[D]. University of Cambridge, 2012.
[35] DUGUNDJI J, UGI I. An algebraic model of constitutional chemistry as a basis for chemical computer programs[M]. Computers in Chemistry. Springer. 2006: 19-64.
[36] SCHNEIDER N, LOWE D M, SAYLE R A, et al. Development of a Novel Fingerprint for Chemical Reactions and Its Application to Large-Scale Reaction Classification and Similarity[J]. Journal of Chemical Information and Modeling, 2015, 55(1): 39-53.
[37] CHEN L, GASTEIGER J. Organic reactions classified by neural networks: Michael additions, Friedel-Crafts alkylations by alkenes, and related reactions[J]. Angewandte Chemie-International Edition in English, 1996, 35(7): 763-765.
[38] SATOH H, SACHER O, NAKATA T, et al. Classification of organic reactions: Similarity of reactions based on changes in the electronic features of oxygen atoms at the reaction sites[J]. Journal of Chemical Information and Computer Sciences, 1998, 38(2): 210-219.
[39] ÖZTüRK H, ÖZGüR A, SCHWALLER P, et al. Exploring chemical space using natural language processing methodologies for drug discovery[J]. Drug Discovery Today, 2020, 25(4): 689-705.
[40] SCHWALLER P, PETRAGLIA R, ZULLO V, et al. Predicting retrosynthetic pathways using transformer-based models and a hyper-graph exploration strategy[J]. Chemical Science, 2020, 11(12): 3316-3325.
[41] JORNER K, BRINCK T, NORRBY P-O, et al. Machine learning meets mechanistic modelling for accurate prediction of experimental activation energies[J]. Chemical Science, 2021, 12(3): 1163-1175.
[42] AGRAWAL A, CHOUDHARY A. Perspective: Materials informatics and big data: Realization of the “fourth paradigm” of science in materials science[J]. APL Materials, 2016, 4(5): 053208
[43] VON LILIENFELD O A, BURKE K. Retrospective on a decade of machine learning for chemical discovery[J]. Nature Communications, 2020, 11(1): 4895.
[44] KUNTZ D, WILSON A K. Machine learning, artificial intelligence, and chemistry: how smart algorithms are reshaping simulation and the laboratory[J]. Pure and Applied Chemistry, 2022, 94(8): 1019-1054.
[45] BAUM Z J, YU X, AYALA P Y, et al. Artificial Intelligence in Chemistry: Current Trends and Future Directions[J]. Journal of Chemical Information and Modeling, 2021, 61(7): 3197-3212.
[46] MA J S, SHERIDAN R P, LIAW A, et al. Deep Neural Nets as a Method for Quantitative Structure-Activity Relationships[J]. Journal of Chemical Information and Modeling, 2015, 55(2): 263-274.
[47] BéDARD A-C, ADAMO A, AROH K C, et al. Reconfigurable system for automated optimization of diverse chemical reactions[J]. Science, 2018, 361(6408): 1220-1225.
[48] PERERA D, TUCKER J W, BRAHMBHATT S, et al. A platform for automated nanomole-scale reaction screening and micromole-scale synthesis in flow[J]. Science, 2018, 359(6374): 429-434.
[49] SCHWALLER P, LAINO T, GAUDIN T, et al. Molecular transformer: a model for uncertainty-calibrated chemical reaction prediction[J]. ACS Central Science, 2019, 5(9): 1572-1583.
[50] SCHWALLER P, HOOVER B, REYMOND J-L, et al. Extraction of organic chemistry grammar from unsupervised learning of chemical reactions[J]. Science Advances, 2021, 7(15): eabe4166.
[51] FLAM-SHEPHERD D, ZHU K, ASPURU-GUZIK A. Language models can learn complex molecular distributions[J]. Nature Communications, 2022, 13(1): 3293.
[52] SCHWALLER P, PROBST D, VAUCHER A C, et al. Mapping the space of chemical reactions using attention-based neural networks[J]. Nature Machine Intelligence, 2021, 3(2): 144-152.
[53] PEREIRA F, XIAO K X, LATINO D, et al. Machine Learning Methods to Predict Density Functional Theory B3LYP Energies of HOMO and LUMO Orbitals[J]. Journal of Chemical Information and Modeling, 2017, 57(1): 11-21.
[54] MULLIN R. The lab of the future is now[J]. Chem Eng News, 2021, 99(11): 28.
[55] CORTéS-BORDA D, WIMMER E, GOUILLEUX B, et al. An autonomous self optimizing flow reactor for the synthesis of natural product carpanone[J]. The Journal of Organic Chemistry, 2018, 83(23): 14286-14299.
[56] GRANDA J M, DONINA L, DRAGONE V, et al. Controlling an organic synthesis robot with machine learning to search for new reactivity[J]. Nature, 2018, 559(7714): 377-381.
[57] COLEY C W, THOMAS III D A, LUMMISS J A, et al. A robotic platform for flow synthesis of organic compounds informed by AI planning[J]. Science, 2019, 365(6453)eaax1566.
[58] ZHAO H, CHEN W, HUANG H, et al. A robotic platform for the synthesis of colloidal nanocrystals[J]. Nature Synthesis, 2023: 1-10.
[59] SILVER D, HUANG A, MADDISON C J, et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484-489.
[60] HAND D J, YU K. Idiot's Bayes—not so stupid after all?[J]. International Statistical Review, 2001, 69(3): 385-398.
[61] VASWANI A, SHAZEER N, PARMAR N, et al. Attention Is All You Need[J]. Advances in Neural Information Processing Systems 30 (Nips 2017), 2017, 30
[62] DEVLIN J, CHANG M-W, LEE K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:181004805, 2018
[63] LAN Z, CHEN M, GOODMAN S, et al. Albert: A lite bert for self-supervised learning of language representations[J]. arXiv preprint arXiv:190911942, 2019
[64] HELLER S, MCNAUGHT A, STEIN S, et al. InChI-the worldwide chemical structure identifier standard[J]. Journal of Cheminformatics, 2013, 5(1): 1-9.
[65] HELLER S R, MCNAUGHT A, PLETNEV I, et al. InChI, the IUPAC international chemical identifier[J]. Journal of Cheminformatics, 2015, 7(1): 1-34.
[66] WEININGER D. SMILES, A CHEMICAL LANGUAGE AND INFORMATION SYSTEM .1. INTRODUCTION TO METHODOLOGY AND ENCODING RULES[J]. Journal of Chemical Information and Computer Sciences, 1988, 28(1): 31-36.
[67] WEININGER D, WEININGER A, WEININGER J L. SMILES .2. ALGORITHM FOR GENERATION OF UNIQUE SMILES NOTATION[J]. Journal of Chemical Information and Computer Sciences, 1989, 29(2): 97-101.
[68] JELIAZKOVA N, KOCHEV N. AMBIT‐SMARTS: Efficient Searching of Chemical Structures and Fragments[J]. Molecular Informatics, 2011, 30(8): 707-720.
[69] ROGERS D, HAHN M. Extended-Connectivity Fingerprints[J]. Journal of Chemical Information and Modeling, 2010, 50(5): 742-754.
[70] WIGH D S, GOODMAN J M, LAPKIN A A. A review of molecular representation in the age of machine learning[J]. Wiley Interdisciplinary Reviews: Computational Molecular Science, 2022, 12(5): e1603.
[71] XU Z, WANG S, ZHU F Y, et al. Seq2seq Fingerprint: An Unsupervised Deep Molecular Embedding for Drug Discovery[J]. Acm-Bcb' 2017: Proceedings of the 8th Acm International Conference on Bioinformatics, Computational Biology,and Health Informatics, 2017: 285-294.
[72] WINTER R, MONTANARI F, NOE F, et al. Learning continuous and data-driven molecular descriptors by translating equivalent chemical representations[J]. Chemical Science, 2019, 10(6): 1692-1701.
[73] WANG S, GUO Y Z, WANG Y H, et al. SMILES-BERT: Large Scale Pre-Training for Molecular Property Prediction[J]. Acm-Bcb'19: Proceedings of the 10th Acm International Conference on Bioinformatics, Computational Biology and Health Informatics, 2019: 429-436.
[74] COLEY C W, GREEN W H, JENSEN K F. Machine learning in computer-aided synthesis planning[J]. Accounts of Chemical Research, 2018, 51(5): 1281-1289.
[75] COLEY C W, GREEN W H, JENSEN K F. RDChiral: An RDKit Wrapper for Handling Stereochemistry in Retrosynthetic Template Extraction and Application[J]. Journal of Chemical Information and Modeling, 2019, 59(6): 2529-2537.
[76] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[77] CHO K, VAN MERRIëNBOER B, BAHDANAU D, et al. On the properties of neural machine translation: Encoder-decoder approaches[J]. arXiv preprint arXiv:14091259, 2014
[78] CHO K, VAN MERRIëNBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J]. arXiv preprint arXiv:14061078, 2014
[79] CHUNG J, GULCEHRE C, CHO K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. arXiv preprint arXiv:14123555, 2014