基于迁移学习的多策略动态多目标优化算法

动态多目标优化问题（Dynamic Multi-objective Optimization Problems，DMOPs）是实际生活中常见的优化问题，此类问题包含多个相互冲突的目标，并且问题所在的环境是随时间而变化的，例如问题的目标函数、约束条件和问题的参数等。DMOPs通常具有变化的Pareto最优解集（Pareto Optimal Set，POS）或Pareto最优前沿（Pareto Optimal Front，POF），因此解决DMOPs不仅要求算法快速、准确地搜索到最优解集，还需要在环境变化时快速追踪移动的POS。

进化算法是求解DMOPs的关键方法。在已有的动态多目标进化优化算法中，基于预测的动态响应机制是常见的处理环境变化的方法，此类方法通常利用历史信息预测新环境中的最优解，以快速响应环境变化。基于迁移学习的预测方法最近在动态多目标优化领域引起了研究者的关注，迁移学习相较于传统预测模型释放了独立同分布的假设，允许训练和测试用的数据分布不同但相关，这使基于迁移学习的动态多目标算法在解决DMOPs上有更好的表现。然而，现有的基于迁移学习的方法没有考虑动态问题的特征，直接应用于所有测试问题上，这将导致算法缺乏通用性。针对此问题，本文分别从迁移什么、何时迁移以及如何迁移三个方面分别提出以下三种基于迁移学习的多策略动态多目标优化算法：

（1）提出一种基于动态类型估计的动态多目标进化迁移优化算法，算法利用前两个环境中的最优解估计当前环境中POS和POF的动态变化模式，然后根据变化模式自适应地选择是否在决策空间和目标空间进行解的迁移。所提出的算法在不同变化类型的DMOPs上进行了测试，实验结果表明其能够适应不同类型的问题。

（2）提出一种基于域相似性估计的动态多目标进化迁移优化算法，算法针对负迁移现象提出一种迁移决策策略，该策略通过环境变化前后解集分布的差异估计环境间的相似性，然后根据相似性决定是否进行迁移学习。为验证该策略的有效性将其与现有的方法相结合，实验结果表明算法能有效减少负迁移现象。

（3）提出一种多迁移方法集成的动态多目标进化迁移优化算法，算法使用了一个基于贡献度的迁移方法选择策略，根据不同迁移学习方法对不同DMOPs的求解能力，自适应地调整不同方法的选取概率，使算法选择适合解决当前问题的迁移策略。实验结果表明集成方法相对于单一方法在总体性能上有较大的提升。

[1] DEB K, SINDHYA K, HAKANEN J. Multi-objective optimization[M]//Decision sciences. CRC Press, 2016: 161-200.
[2] RAQUEL C, YAO X. Dynamic multi-objective optimization: a survey of the state-of-the-art [M]//Evolutionary computation for dynamic optimization problems. Springer, 2013: 85-106.
[3] NGUYEN S, ZHANG M, JOHNSTON M, et al. Automatic design of scheduling policies for dynamic multi-objective job shop scheduling via cooperative coevolution genetic programming [J]. IEEE Transactions on Evolutionary Computation, 2013, 18(2): 193-208.
[4] 柴天佑, 丁进良, 王宏, 等. 复杂工业过程运行的混合智能优化控制方法[J]. 自动化学报, 2008: 505-515.
[5] GHANNADPOUR S F, NOORI S, TAVAKKOLI-MOGHADDAM R, et al. A multi-objective dynamic vehicle routing problem with fuzzy time windows: Model, solution and application [J]. Applied Soft Computing, 2014, 14: 504-527.
[6] HUTZSCHENREUTER A K, BOSMAN P A, LA POUTRÉ H. Evolutionary multiobjective optimization for dynamic hospital resource management[C]//International conference on evo- lutionary multi-criterion optimization. Springer, 2009: 320-334.
[7] YU X, GEN M. Introduction to evolutionary algorithms[M]. Springer Science & Business Media, 2010.
[8] AZZOUZ R, BECHIKH S, SAID LB. A dynamic multi-objective evolutionary algorithm using a change severity-based adaptive population management strategy[J]. Soft Computing, 2017, 21: 885-906.
[9] COELLO CC, LECHUGA M S. MOPSO: A proposal for multiple objective particle swarm op- timization[C]//Proceedings of the 2002 Congress on Evolutionary Computation. CEC’02 (Cat. No. 02TH8600): volume 2. IEEE, 2002: 1051-1056.
[10] DEB K, PRATAP A, AGARWAL S, et al. A fast and elitist multiobjective genetic algorithm: NSGA-II[J]. IEEE transactions on evolutionary computation, 2002, 6(2): 182-197.
[11] ZHANG Q, ZHOU A, JIN Y. RM-MEDA: A regularity model-based multiobjective estimation of distribution algorithm[J]. IEEE Transactions on Evolutionary Computation, 2008, 12(1): 41-63.
[12] ZHANG Q, LI H. MOEA/D: A multiobjective evolutionary algorithm based on decomposition [J]. IEEE Transactions on evolutionary computation, 2007, 11(6): 712-731.
[13] COBB HG. An investigation into the use of hypermutation as an adaptive operator in genetical- gorithms having continuous, time-dependent nonstationary environments[M]. Naval Research Laboratory, Navy Center for Applied Research in Artificial …, 1990.
[14] DEB K, RAO NUB, KARTHIK S. Dynamic multi-objective optimization and decision-making using modified NSGA-II: A case study on hydro-thermal power scheduling[C]//International conference on evolutionary multi-criterion optimization. Springer, 2007: 803-817.
[15] CHEN H, LIM, CHEN X. Using diversity as an additional-objective in dynamic multi-objective optimization algorithms[C]//2009 Second International Symposium on Electronic Commerce and Security: volume 1. IEEE, 2009: 484-487.
[16] JIANG S, YANG S. A steady-state and generational evolutionary algorithm for dynamic multi- objective optimization[J]. IEEE Transactions on evolutionary Computation, 2016, 21(1): 65-82.
[17] GOH C K, TAN K C. A competitive-cooperative coevolutionary paradigm for dynamic multi- objective optimization[J]. IEEE Transactions on Evolutionary Computation, 2008, 13(1): 103- 127.
[18] CHEN R, LI K, YAO X. Dynamic multiobjectives optimization with a changing number of objectives[J]. IEEE Transactions on Evolutionary Computation, 2017, 22(1): 157-171.
[19] MA X, YANG J, SUN H, et al. Multiregional co-evolutionary algorithm for dynamic multiob- jective optimization[J]. Information Sciences, 2021, 545: 1-24.
[20] WANG Y, LI B. Investigation of memory-based multi-objective optimization evolutionary algorithm in dynamic environment[C]//2009 IEEE Congress on Evolutionary Computation. IEEE, 2009: 630-637.
[21] WANG Y, LI B. Multi-strategy ensemble evolutionary algorithm for dynamic multi-objective optimization[J]. Memetic Computing, 2010, 2: 3-24.
[22] LIANG Z, ZHENG S, ZHU Z, et al. Hybrid of memory and prediction strategies for dynamic multiobjective optimization[J]. Information Sciences, 2019, 485: 200-218.
[23] YANG S. Memory-based immigrants for genetic algorithms in dynamic environments[C]// Proceedings of the 7th annual conference on Genetic and evolutionary computation. 2005: 1115-1122.
[24] FARINA M, DEB K, AMATO P. Dynamic multiobjective optimization problems: test cases, approximations, and applications[J]. IEEE Transactions on evolutionary computation, 2004, 8(5): 425-442.
[25] HATZAKIS I, WALLACE D. Dynamic multi-objective optimization with evolutionary algo- rithms: a forward-looking approach[C]//Proceedings of the 8th annual conference on Genetic and evolutionary computation. 2006: 1201-1208.
[26] ZHOU A, JIN Y, ZHANG Q. A population prediction strategy for evolutionary dynamic mul- tiobjective optimization[J]. IEEE transactions on cybernetics, 2013, 44(1): 40-53.
[27] MURUGANANTHAM A, TANK C, VADAKKEPAT P. Evolutionary dynamic multiobjective optimization via Kalman filter prediction[J]. IEEE transactions on cybernetics, 2015, 46(12): 2862-2873.
[28] CAO L, XU L, GOODMAN E D, et al. Evolutionary dynamic multiobjective optimization assisted by a support vector regression predictor[J]. IEEE Transactions on Evolutionary Com- putation, 2019, 24(2): 305-319.
[29] RONG M, GONG D, ZHANG Y, et al. Multidirectional prediction approach for dynamic multi- objective optimization problems[J]. IEEE transactions on cybernetics, 2018, 49(9): 3362-3374.
[30] RONG M, GONG D, PEDRYCZW, et al. A multimodel prediction method for dynamic multi- objective evolutionary optimization[J]. IEEE Transactions on Evolutionary Computation, 2019, 24(2): 290-304.
[31] JIANG M, HUANG Z, QIUL, et al. Transfer learning-based dynamic multiobjective optimiza- tion algorithms[J]. IEEE Transactions on Evolutionary Computation, 2017, 22(4): 501-514.
[32] PAN S J, YANG Q. A survey on transfer learning[J]. IEEE Transactions on knowledge and data engineering, 2009, 22(10): 1345-1359.
[33] TAN K C, FENG L, JIANG M. Evolutionary transfer optimization-a new frontier in evolution- ary computation research[J]. IEEE Computational Intelligence Magazine, 2021, 16(1): 22-33.
[34] PAN S J, TSANG I W, KWOK J T, et al. Domain adaptation via transfer component analysis [J]. IEEE transactions on neural networks, 2010, 22(2): 199-210.
[35] GRETTON A, BORGWARDT K M, RASCH M J, et al. A kernel two-sample test[J]. The Journal of Machine Learning Research, 2012, 13(1): 723-773.
[36] JIANG M, WANG Z, HONG H, et al. Knee point-based imbalanced transfer learning for dy- namic multiobjective optimization[J]. IEEE Transactions on Evolutionary Computation, 2020, 25(1): 117-129.
[37] LI E, MAX. Dynamic Multi-objective Optimization Algorithm based on Transfer Learning for Environmental Protection.[J]. Ekoloji Dergisi, 2019(107).
[38] LIU Z, WANG H. Improved population prediction strategy for dynamic multi-objective opti- mization algorithms using transfer learning[C]//2021 IEEE Congress on Evolutionary Compu- tation (CEC). IEEE, 2021: 103-110.
[39] JIANG M, WANG Z, QIU L, et al. A fast dynamic evolutionary multiobjective algorithm via manifold transfer learning[J]. IEEE Transactions on Cybernetics, 2020, 51(7): 3417-3428.
[40] JIANG M, WANG Z, GUO S, et al. Individual-based transfer learning for dynamic multiobjec- tive optimization[J]. IEEE Transactions on Cybernetics, 2020, 51(10): 4968-4981.
[41] DAI W, YANG Q, XUE GR, et al. Boosting for Transfer Learning[C/OL]//ICML ’07: Proceed- ings of the 24th International Conference on Machine Learning. New York, NY, USA: Associ- ation for Computing Machinery, 2007: 193-200. https://doi.org/10.1145/1273496.1273521.
[42] FENG L, ZHOU W, LIU W, et al. Solving dynamic multiobjective problem via autoencoding evolutionary search[J]. IEEE Transactions on Cybernetics, 2020, 52(5): 2649-2662.
[43] CHEN G, GUO Y, HUANG M, et al. A domain adaptation learning strategy for dynamic mul- tiobjective optimization[J]. Information Sciences, 2022, 606: 328-349.
[44] SUN B, SAENKO K. Subspace Distribution Alignment for Unsupervised Domain Adaptation [C/OL]//British Machine Vision Conference: volume 4. 2015: 24-31. https://doi.org/10.48550 /arXiv.1409.5241.
[45] ZHOU W, FENG L, TAN K C, et al. Evolutionary Search With Multiview Prediction for Dy- namic Multiobjective Optimization[J]. IEEE Transactions on Evolutionary Computation, 2021, 26(5): 911-925.
[46] ZHANG W, DENG L, ZHANG L, et al. A survey on negative transfer[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 10(2): 305-329.
[47] KULLBACKS, LEIBLER RA. On information and sufficiency[J]. The annals of mathematical statistics, 1951, 22(1): 79-86.
[48] LONG M, WANG J, DING G, et al. Dual transfer learning[C]//Proceedings of the 2012 SIAM International Conference on Data Mining. SIAM, 2012: 540-551.
[49] LI J, SUN T, LIN Q, et al. Reducing negative transfer learning via clustering for dynamic multiobjective optimization[J]. IEEE Transactions on Evolutionary Computation, 2022, 26(5): 1102-1116.
[50] YE Y, LIN Q, MA L, et al. Multiple source transfer learning for dynamic multiobjective opti- mization[J]. Information Sciences, 2022, 607: 739-757.
[51] DESBORDES JK, ZHANG K, XUE X, et al. Dynamic production optimization based on trans- fer learning algorithms[J]. Journal of Petroleum Science and Engineering, 2022, 208: 109278.
[52] 王晋东, 陈益强. 迁移学习导论[M]. 电子工业出版社, 2021.
[53] ZHU Y, CHEN Y, LU Z, et al. Heterogeneous transfer learning for image classification[C]// Proceedings of the AAAI Conference on Artificial Intelligence: volume 25. 2011: 1304-1309.
[54] RUDER S, PETERS M E, SWAYAMDIPTA S, et al. Transfer learning in natural language processing[C]//Proceedings of the 2019 conference of the North American chapter of the asso- ciation for computational linguistics: Tutorials. 2019: 15-18.
[55] JAYARAM V, ALAMGIR M, ALTUN Y, et al. Transfer learning in brain-computer interfaces [J]. IEEE Computational Intelligence Magazine, 2016, 11(1): 20-31.
[56] RAGHU M, ZHANG C, KLEINBERG J, et al. Transfusion: Understanding transfer learning for medical imaging[J]. Advances in neural information processing systems, 2019, 32.
[57] YAO Y, DORETTO G. Boosting for transfer learning with multiple sources[C]//2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, 2010: 1855- 1862.
[58] HUANG J, GRETTON A, BORGWARDT K, et al. Correcting sample selection bias by unla- beled data[J]. Advances in neural information processing systems, 2006, 19.
[59] LONGM, WANG J, DING G, et al. Transfer feature learning with joint distribution adaptation [C]//Proceedings of the IEEE international conference on computer vision. 2013: 2200-2207.
[60] LI F, PAN S J, JIN O, et al. Cross-domain co-extraction of sentiment and topic lexicons[C]// Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers). 2012: 410-419.
[61] MIHALKOVA L, HUYNH T, MOONEY R J. Mapping and revising markov logic networks for transfer learning[C]//Aaai: volume 7. 2007: 608-614.
[62] YANG C, DING J, JIN Y, et al. Multitasking multiobjective evolutionary operational indices optimization of beneficiation processes[J]. IEEE Transactions on Automation Science and En- gineering, 2018, 16(3): 1046-1057.
[63] GUPTA A, ONG Y S, FENG L. Multifactorial evolution: Toward evolutionary multitasking [J]. IEEE Transactions on Evolutionary Computation, 2015, 20(3): 343-357.
[64] FENG L, HUANG Y, ZHOU L, et al. Explicit evolutionary multitasking for combinatorial optimization: A case study on capacitated vehicle routing problem[J]. IEEE transactions on cybernetics, 2020, 51(6): 3143-3156.
[65] MIN A T W, ONG Y S, GUPTA A, et al. Multiproblem surrogates: Transfer evolutionary multiobjective optimization of computationally expensive problems[J]. IEEE Transactions on Evolutionary Computation, 2017, 23(1): 15-28.
[66] MIKA S, RATSCH G, WESTON J, et al. Fisher discriminant analysis with kernels[C]//Neural networks for signal processing IX: Proceedings of the 1999 IEEE signal processing society workshop (cat. no. 98th8468). Ieee, 1999: 41-48.
[67] BENGIO Y, LAMBLIN P, POPOVICI D, et al. Greedy layer-wise training of deep networks [J]. Advances in neural information processing systems, 2006, 19: 153-160.
[68] VINCENT P, LAROCHELLE H, BENGIO Y, et al. Extracting and composing robust features with denoising autoencoders[C]//Proceedings of the 25th international conference on Machine learning. 2008: 1096-1103.
[69] KANDASWAMY C, SILVA L M, ALEXANDRE L A, et al. Improving transfer learning ac- curacy by reusing stacked denoising autoencoders[C]//2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2014: 1380-1387.
[70] DEV K, ASHRAF Z, MUHURI P K, et al. Deep autoencoder based domain adaptation for transfer learning[J]. Multimedia Tools and Applications, 2022, 81(16): 22379-22405.
[71] FENG L, ONG Y S, JIANG S, et al. Autoencoding evolutionary search with learning across heterogeneous problems[J]. IEEE Transactions on Evolutionary Computation, 2017, 21(5): 760-772.
[72] ZHOU L, FENG L, GUPTA A, et al. Learnable evolutionary search across heterogeneous problems via kernelized autoencoding[J]. IEEE Transactions on Evolutionary Computation, 2021, 25(3): 567-581.
[73] HEX, NIYOGI P. Locality preserving projections[J]. Advances in neural information process- ing systems, 2003, 16: 153-160.
[74] GOPALAN R, LI R, CHELLAPPA R. Domain adaptation for object recognition: An unsuper- vised approach[C]//2011 international conference on computer vision. IEEE, 2011: 999-1006.
[75] HELBIG M, ENGELBRECHT A P. Benchmarks for dynamic multi-objective optimisation algorithms[J]. ACM Computing Surveys (CSUR), 2014, 46(3): 1-39.
[76] GUO Y, CHEN G, JIANG M, et al. A knowledge guided transfer strategy for evolutionary dy- namic multiobjective optimization[J]. IEEE Transactions on Evolutionary Computation, 2022.
[77] JIANG S, YANG S, YAO X, et al. Benchmark Functions for the CEC’2018 Competition on Dynamic Multiobjective Optimization[R]. Newcastle University, 2018.
[78] KOO W T, GOH CK, TANK C. A predictive gradient strategy for multiobjective evolutionary algorithms in a fast changing environment[J]. Memetic Computing, 2010, 2: 87-110.
[79] JIANG S, YANG S. Evolutionary dynamic multiobjective optimization: Benchmarks and al- gorithm comparisons[J]. IEEE transactions on cybernetics, 2016, 47(1): 198-211.
[80] SCHOLKOPF B, SUNG K K, BURGES C J, et al. Comparing support vector machines with Gaussian kernels to radial basis function classifiers[J]. IEEE transactions on Signal Processing, 1997, 45(11): 2758-2765.
[81] WILCOXON F. Individual Comparisons by Ranking Methods[J]. Biometrics, 1945, 1(6): 80- 83.
[82] MURTAGH F, CONTRERAS P. Algorithms for hierarchical clustering: an overview[J]. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2012, 2(1): 86-97.
[83] ZITZLER E, THIELE L. Multiobjective optimization using evolutionary algorithms—a com- parative case study[C]//International conference on parallel problem solving from nature. Springer, 1998: 292-301.
[84] WHILE L, HINGSTON P, BARONE L, et al. A faster algorithm for calculating hypervolume [J]. IEEE transactions on evolutionary computation, 2006, 10(1): 29-38.
[85] LIN W, LIN Q, FENG L, et al. Ensemble of Domain Adaptation-Based Knowledge Transfer for Evolutionary Multitasking[J/OL]. IEEE Transactions on Evolutionary Computation, 2023: 1-1. DOI: 10.1109/TEVC.2023.3259067.