基于多任务学习的化学反应预测模型研究

  寻找一个反应的化学空间是有机合成的一个重要内容，传统的研究策略一般是基于现有经验的有限穷举；整个条件筛选过程依赖于合成实验，耗时耗力。

  为此，本研究将高通量合成技术和人工智能技术结合起来，搭建了基于图编码的多任务反应预测模型，共同探索合成反应的化学空间。主要完成了以下几部分的工作：单组数据集共享反应机理和反应分子骨架的关联性构建多任务模型；共享不同反应类型中相近的反应机理的联系构建多任务模型；从反应的不同目标（如产率、立体选择性、反应分类、转化率）出发构建多任务模型。
  本研究探索了人工智能在化学领域的应用前景。利用多任务反应模型，可以快速构建多个反应预测模型，并实现各任务之间的信息共享，从而提高同类高通量反应数据的预测精度。此外，研究还发现，针对化学反应的不同目标采用多任务方法，可以增强反应模型对隐含反应关联性的捕捉能力。与其他模型相比，本研究验证了该模型的稳定性和可靠性。实验还从数据分布的角度对算法模型进行了分析，并利用机器学习方法对反应过程进行了解释性分析，探索了化学反应空间的特征，表明算法模型能够在一定程度上学习到反应发生的内在机制。

[1] SCHWALLER P, PROBST D, VAUCHER A C, et al. Mapping the space of chemical reactions using attention-based neural networks[J]. Nature machine intelligence, 2021, 3(2): 144-152.
[2] JORDAN M I, MITCHELL T M. Machine learning: trends, perspectives, and prospects[J].Science, 2015, 349(6245): 255-260.
[3] LAVECCHIA A. Machine-learning approaches in drug discovery: methods and applications[J]. Drug discovery today, 2015, 20(3): 318-331.
[4] MA J, SHERIDAN R P, LIAW A, et al. Deep neural nets as a method for quantitative structure–activity relationships[J]. Journal of chemical information and modeling, 2015, 55(2): 263-274.
[5] SVETNIK V, LIAW A, TONG C, et al. Random forest: a classification and regression tool forcompound classification and QSAR modeling[J]. Journal of chemical information and computersciences, 2003, 43(6): 1947-1958.
[6] STRUBLE T J, ALVAREZ J C, BROWN S P, et al. Current and future roles of artificial in-telligence in medicinal chemistry synthesis[J]. Journal of medicinal chemistry, 2020, 63(16):8667-8682.
[7] SCHMINK J R, BELLOMO A, BERRITT S. Scientist-led high-throughput experimentation(HTE) and its utility in academia and industry[J]. Aldrichimica acta, 2013, 46(3): 71-80.
[8] SHEVLIN M. Practical high-throughput experimentation for chemists[J]. Acs medicinal chem-istry letters, 2017, 8(6): 601-607.
[9] BUITRAGO SANTANILLA A, REGALADO E L, PEREIRA T, et al. Nanomole-scale high-throughput chemistry for the synthesis of complex molecules[J]. Science, 2015, 347(6217):49-53.
[10] THUNG K H, WEE C Y. A brief review on multi-task learning[J]. Multimedia tools andapplications, 2018, 77: 29705-29725.
[11] COLLINS K D, GENSCH T, GLORIUS F. Contemporary screening approaches to reaction discovery and development[J]. Nature chemistry, 2014, 6(10): 859-871.
[12] CARUANA R. Multitask learning[J]. Machine learning, 1997, 28: 41-75.
[13] STRUBLE T J, COLEY C W, JENSEN K F. Multitask prediction of site selectivity in aromatic C–H functionalization reactions[J]. Reaction chemistry & engineering, 2020, 5(5): 896-902.
[14] WISWESSER W J. How the WLN began in 1949 and how it might be in 1999[J]. Journal of chemical information and computer sciences, 1982, 22(2): 88-93.
[15] LU J, ZHANG Y. Unified deep learning model for multitask reaction predictions with explana-tion[J]. Journal of chemical information and modeling, 2022, 62(6): 1376-1387.
[16] WEININGER D. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules[J]. Journal of chemical information and computer sciences,1988, 28(1): 31-36.
[17] TORRES J A G, LAU S H, ANCHURI P, et al. A multi-objective active learning platform and web app for reaction optimization[J]. Journal of the american chemical society, 2022, 144(43):19999-20007.
[18] MEHR S H M, CRAVEN M, LEONOV A I, et al. A universal system for digitization and automatic execution of the chemical synthesis literature[J]. Science, 2020, 370(6512): 101-108.
[19] ZHANG Y, YANG Q. A survey on multi-task learning[J]. IEEE transactions on knowledge and data engineering, 2021, 34(12): 5586-5609.
[20] DE ALMEIDA A F, MOREIRA R, RODRIGUES T. Synthetic organic chemistry driven by artificial intelligence[J]. Nature reviews chemistry, 2019, 3(10): 589-604.
[21] AICHEMECO. AUTOMATION PROCESS SNAPSHOT[EB/OL]. 2021. https://www.aichem eco.com/.
[22] VANDENHENDE S, GEORGOULIS S, VAN GANSBEKE W, et al. Multi-task learning for dense prediction tasks: A survey[J]. IEEE transactions on pattern analysis and machine intelli-gence, 2021, 44(7): 3614-3633.
[23] MISRA I, SHRIVASTAVA A, GUPTA A, et al. Cross-stitch networks for multi-task learning[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016:3994-4003.
[24] CHEN Z, BADRINARAYANAN V, LEE C Y, et al. Gradnorm: gradient normalization for adaptive loss balancing in deep multitask networks[C]//International conference on machine learning. PMLR, 2018: 794-803.
[25] KENDALL A, GAL Y, CIPOLLA R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 7482-7491.
[26] LIU S, JOHNS E, DAVISON A J. End-to-end multi-task learning with attention[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019:1871-1880.
[27] SENER O, KOLTUN V. Multi-task learning as multi-objective optimization[J]. Advances in neural information processing systems, 2018, 31.
[28] WANG Z, TSVETKOV Y, FIRAT O, et al. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models[A]. 2020.
[29] LIU B, LIU X, JIN X, et al. Conflict-averse gradient descent for multi-task learning[J]. Advances in neural information processing systems, 2021, 34: 18878-18890.
[30] CHEN Z, NGIAM J, HUANG Y, et al. Just pick a sign: Optimizing deep multitask models with gradient sign dropout[J]. Advances in neural information processing systems, 2020, 33:2039-2050.
[31] BAGAYOKO D. Understanding density functional theory (DFT) and completing it in practice[J]. AIP advances, 2014, 4(12): 127104.
[32] KAYALA M A, AZENCOTT C A, CHEN J H, et al. Learning to predict chemical reactions[J].Journal of chemical information and modeling, 2011, 51(9): 2209-2222.
[33] SALIM N, HOLLIDAY J, WILLETT P. Combination of fingerprint-based similarity coefficientsusing data fusion[J]. Journal of chemical information and computer sciences, 2003, 43(2): 435-442.
[34] MORGAN H L. The generation of a unique machine description for chemical structures-a technique developed at chemical abstracts service.[J]. Journal of chemical documentation, 1965,5(2): 107-113.
[35] ROGERS D, HAHN M. Extended-connectivity fingerprints[J]. Journal of chemical information and modeling, 2010, 50(5): 742-754.
[36] KEARNES S, MCCLOSKEY K, BERNDL M, et al. Molecular graph convolutions: moving beyond fingerprints[J]. Journal of computer-aided molecular design, 2016, 30: 595-608.
[37] FORTIN S. The graph isomorphism problem[Z]. 1996.
[38] GILMER J, SCHOENHOLZ S S, RILEY P F, et al. Neural message passing for quantum chemistry[C]//International conference on machine learning. PMLR, 2017: 1263-1272.
[39] XIONG G, WU Z, YI J, et al. ADMETlab 2.0: an integrated online platform for accurate and comprehensive predictions of ADMET properties[J]. Nucleic acids research, 2021, 49(W1):W5-W14.
[40] SOSNIN S, VASHURINA M, WITHNALL M, et al. A survey of multi-task learning methods in chemoinformatics[J]. Molecular informatics, 2019, 38(4): 1800108.
[41] RUDER S. An overview of multi-task learning in deep neural networks[A]. 2017.
[42] ZHANG Y, YANG Q. An overview of multi-task learning[J]. National science review, 2018, 5(1): 30-43.
[43] LIN B, ZHANG Y. LibMTL: A python library for multi-task learning[A]. 2022.
[44] YANG K, SWANSON K, JIN W, et al. Analyzing learned molecular representations for property prediction[J]. Journal of chemical information and modeling, 2019, 59(8): 3370-3388.
[45] HEID E, GREEN W H. Machine learning of reaction properties via learned representations of the condensed graph of reaction[J]. Journal of chemical information and modeling, 2021, 62(9): 2101-2110.
[46] LIN B, FEIYANG Y, ZHANG Y, et al. Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning[J]. Transactions on machine learning research, 2022.
[47] LI B, SU S, ZHU C, et al. GraphRXN: a novel representation for reaction prediction[Z]. 2022.
[48] AHNEMAN D T, ESTRADA J G, LIN S, et al. Predicting reaction performance in C–N cross-coupling using machine learning[J]. Science, 2018, 360(6385): 186-190.
[49] PERERA D, TUCKER J W, BRAHMBHATT S, et al. A platform for automated nanomole-scale reaction screening and micromole-scale synthesis in flow[J]. Science, 2018, 359(6374):429-434.
[50] ZAHRT A F, HENLE J J, ROSE B T, et al. Prediction of higher-selectivity catalysts by computer-driven workflow and machine learning[J]. Science, 2019, 363(6424): eaau5631.
[51] PING Y, KONG W. Ni-catalyzed reductive difunctionalization of alkenes[J]. Synthesis, 2020,52(07): 979-992.
[52] BIEWALD L. Experiment tracking with weights and biases, 2020[J]. Software available from wandb. com, 2020, 2(5).
[53] SCHWALLER P, VAUCHER A C, LAINO T, et al. Prediction of chemical reaction yields using deep learning[J]. Machine learning: science and technology, 2021, 2(1): 015016.
[54] GONG Y, XUE D, CHUAI G, et al. DeepReac+: deep active learning for quantitative modeling of organic chemical reactions[J]. Chemical science, 2021, 12(43): 14459-14472.
[55] HALKO N, MARTINSSON P G, TROPP J A. Finding structure with randomness: Stochas-tic algorithms for constructing approximate matrix decompositions[M]. California Institute of Technology, 2009.
[56] PARSA A B, MOVAHEDI A, TAGHIPOUR H, et al. Toward safer highways, application of XGBoost and SHAP for real-time accident detection and feature analysis[J]. Accident analysis & prevention, 2020, 136: 105405.
[57] LI X, LI Z, WU X, et al. Deep learning enhancing kinome-wide polypharmacology profiling: model construction and experiment validation[J]. Journal of medicinal chemistry, 2019, 63(16):8723-8737.
[58] GOU J, YU B, MAYBANK S J, et al. Knowledge distillation: A survey[J]. International journalof computer vision, 2021, 129: 1789-1819.