基于特征归因解释的机器学习公平性与可解释性研究

随着人工智能的快速发展和广泛应用，其带来的风险和挑战日渐显著。人工智能系统在决策过程中往往缺乏透明性和可解释性，这一问题已经逐渐成为当前人工智能发展与治理中的重要议题之一。为此，可解释人工智能应运而生，它旨在开发技术方法，使人工智能系统能够向利益相关者解释决策背后的原因和推理过程，以增强对其决策的信任，并协助验证其决策的合理性。其中，特征归因解释方法被誉为最具活力、使用最广泛、研究最深入的可解释人工智能技术。该方法通过揭示每个输入特征对模型预测结果的贡献程度来解释机器学习模型的决策。

本文首先对可解释人工智能领域进行了全面调研，然后聚焦于特征归因解释的应用研究和方法设计。一方面，本文探索了特征归因解释在机器学习公平性方面的相关应用，包括利用特征归因解释来评估、协助实现公平的机器学习模型，以及分析特征归因解释方法本身的公平性。在应用过程中，本文发现现有的特征归因解释方法在设计过程中大多仅考虑单一维度的解释质量。因此，另一方面，本文提出了一种新的多目标特征归因解释方法，该方法同时考虑特征归因解释质量的多个维度。本学位论文的工作内容和创新点可分为以下三个方面：  

(1) 基于特征归因解释的过程公平性评估与实现：本文首先为机器学习模型提供了更清晰、更全面的过程公平性的定义，并基于特征归因解释方法提出了一种评估机器学习模型群体过程公平性的定量指标。此外，通过在机器学习模型的训练过程中考虑所提出的定量指标，实现了机器学习模型的过程公平性。  

(2) 特征归因解释公平性和模型过程公平性的关联分析：本文揭示了特征归因解释中不公平现象的根源，指出其本质上源于机器学习模型决策过程的不公平。此外，提高机器学习模型的过程公平性能够显著提高特征归因解释的公平性。

(3) 多目标特征归因解释方法：本文通过多目标学习同时考虑解释质量的多个维度，从而获得了一组在多个特征归因解释质量指标上有较好表现和权衡的解集。

综上所述，本论文聚焦于特征归因解释方法，深入探讨了其与机器学习公平性的相互作用，并设计了更加理想的特征归因解释方法。本研究有助于构建更透明、更公平的机器学习模型，从而促进可信人工智能的发展。

[1] QIU H, GONG D, LI Z, et al. End2End occluded face recognition by masking corrupted features [J/OL]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021. DOI: 10.110 9/TPAMI.2021.3098962.
[2] KLEPPE A, SKREDE O J, DE RAEDT S, et al. Designing deep learning studies in cancer diagnostics[J]. Nature Reviews Cancer, 2021, 21(3): 199-211.
[3] ZHANG H. Optimization of risk control in financial markets based on particle swarm optimization algorithm[J]. Journal of Computational and Applied Mathematics, 2020, 368: 112530.
[4] HUANG C, ZHANG Z, MAO B, et al. An overview of artificial intelligence ethics[J]. IEEE Transactions on Artificial Intelligence, 2022: 1-21.
[5] ARRIETA A B, DÍAZ-RODRÍGUEZ N, DEL SER J, et al. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI[J]. Information Fusion, 2020, 58: 82-115.
[6] UNION E. General data protection regulation (GDPR)[EB/OL]. 2018. https://gdpr-info.eu/.
[7] COMMITTEE C N G A G E. Next generation AI governance principles - developing responsible AI[Z]. 2019.
[8] INTELLIGENCE E C H L E G A. Ethics guidelines for trustworthy AI[M]. Publications Office, 2019.
[9] FOOD, ADMINISTRATION D, et al. Artificial intelligence and machine learning (AI/ML) software as a medical device action plan[EB/OL]. 2021. https://www.fda.gov/medical-devic es/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-med ical-device.
[10] SIMONYAN K, VEDALDI A, ZISSERMAN A. Deep inside convolutional networks: Visualising image classification models and saliency maps[C]//Workshop at International Conference on Learning Representations. Citeseer, 2014.
[11] BHATT U, XIANG A, SHARMA S, et al. Explainable machine learning in deployment[C]// Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 2020: 648- 657.
[12] DAVE D, NAIK H, SINGHAL S, et al. Explainable AI meets healthcare: A study on heart disease dataset[A]. 2020. arXiv preprint arXiv:2011.03195.
[13] SHAIKH T A, MIR W A, SOFI S. Decrypting the black boxing of artificial intelligence using explainable artificial intelligence in smart healthcare[M]//Connected e-Health. Springer, 2022: 53-82.
[14] OHANA J J, OHANA S, BENHAMOU E, et al. Explainable AI (XAI) models applied to the multi-agent environment of financial markets[C]//International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems. Springer, 2021: 189-207.
[15] JAEGER M, KRÜGEL S, MARINELLI D, et al. Interpretable machine learning for diversified portfolio construction[J]. The Journal of Financial Data Science, 2021, 3(3): 31-51.
[16] GÓRSKI Ł, RAMAKRISHNA S. Explainable artificial intelligence, lawyer’s perspective[C]// Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law. 2021: 60-68.
[17] RIBEIRO M T, SINGH S, GUESTRIN C. “Why should I trust you?” Explaining the predictions of any classifier[C]//Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016: 1135-1144.
[18] LUNDBERG S M, LEE S I. A unified approach to interpreting model predictions[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 4768–4777.
[19] SUNDARARAJAN M, TALY A, YAN Q. Axiomatic attribution for deep networks[C]// International Conference on Machine Learning. 2017: 3319-3328.
[20] SMILKOV D, THORAT N, KIM B, et al. Smoothgrad: removing noise by adding noise[A]. 2017. arXiv preprint arXiv:1706.03825.
[21] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-cam: Visual explanations from deep networks via gradient-based localization[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017: 618-626.
[22] BENTO V, KOHLER M, DIAZ P, et al. Improving deep learning performance by using Explainable Artificial Intelligence (XAI) approaches[J]. Discover Artificial Intelligence, 2021, 1 (1): 1-11.
[23] ZAERI-AMIRANI M, AFGHAH F, MOUSAVI S. A feature selection method based on shapley value to false alarm reduction in ICUs a genetic-algorithm approach[C]//2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2018: 319-323.
[24] TRIPATHI S, HEMACHANDRA N, TRIVEDI P. Interpretable feature subset selection: A Shapley value based approach[C]//2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020: 5463-5472.
[25] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]// European Conference on Computer Vision. Springer, 2014: 818-833.
[26] MOSTAFA S, MONDAL D, BECK M, et al. Visualizing feature maps for model selection in convolutional neural networks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 1362-1371.
[27] WONGSUPHASAWAT K, SMILKOV D, WEXLER J, et al. Visualizing dataflow graphs of deep learning models in tensorflow[J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 24(1): 1-12.
[28] GUILLOT SUAREZ C. Human-in-the-loop hyperparameter tuning of deep nets to improve explainability of classifications[D/OL]. Aalto University. School of Electrical Engineering, 2022: 47. http://urn.fi/URN:NBN:fi:aalto-202205223354.
[29] DONG Y, SU H, ZHU J, et al. Improving interpretability of deep neural networks with semantic information[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 4306-4314.
[30] BARNARD P, MACALUSO I, MARCHETTI N, et al. Resource reservation in sliced networks: An explainable artificial intelligence (XAI) approach[C]//ICC 2022-IEEE International Conference on Communications. IEEE, 2022: 1530-1535.
[31] FRIEDRICH F, STAMMER W, SCHRAMOWSKI P, et al. A typology to explore and guide explanatory interactive machine learning[A]. 2022. arXiv preprint arXiv:2203.03668.
[32] BOJARSKI M, YERES P, CHOROMANSKA A, et al. Explaining how a deep neural network trained with end-to-end learning steers a car[A]. 2017. arXiv preprint arXiv:1704.07911.
[33] MING Y, QU H, BERTINI E. Rulematrix: Visualizing and understanding classifiers with rules [J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 25(1): 342-352.
[34] RAD J, TENNANKORE K K, VINSON A, et al. Extracting surrogate decision trees from blackbox models to explain the temporal importance of clinical features in predicting kidney graft survival[C]//International Conference on Artificial Intelligence in Medicine. Springer, 2022: 88-98.
[35] BALKIR E, KIRITCHENKO S, NEJADGHOLI I, et al. Challenges in applying explainability methods to improve the fairness of NLP models[A]. 2022. arXiv preprint arXiv:2206.03945.
[36] ABDOLLAHI B, NASRAOUI O. Transparency in fair machine learning: The case of explainable recommender systems[J]. Human and Machine Learning: Visible, Explainable, Trustworthy and Transparent, 2018: 21-35.
[37] ZHAO Y, WANG Y, DERR T. Fairness and explainability: Bridging the gap towards fair model explanations[A]. 2022. arXiv preprint arXiv:2212.03840.
[38] DAI J, UPADHYAY S, AIVODJI U, et al. Fairness via explanation quality: Evaluating disparities in the quality of post hoc explanations[C]//Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 2022: 203-214.
[39] MOHSENI S, ZAREI N, RAGAN E D. A multidisciplinary survey and framework for design and evaluation of explainable AI systems[J]. ACM Transactions on Interactive Intelligent Systems (TiiS), 2021, 11(3-4): 1-45.
[40] ROSENFELD A, RICHARDSON A. Explainability in human–agent systems[J]. Autonomous Agents and Multi-Agent Systems, 2019, 33(6): 673-705.
[41] GERLINGS J, JENSEN M S, SHOLLO A. Explainable AI, but explainable to whom? An exploratory case study of XAI in healthcare[M]//Handbook of Artificial Intelligence in Healthcare. Springer, 2022: 169-198.
[42] TOMSETT R, BRAINES D, HARBORNE D, et al. Interpretable to whom? A role-based model for analyzing interpretable machine learning systems[A]. 2018. arXiv preprint arXiv:1806.07552.
[43] GUIDOTTI R, MONREALE A, RUGGIERI S, et al. A survey of methods for explaining black box models[J]. ACM Computing Surveys (CSUR), 2018, 51(5): 1-42.
[44] LINARDATOS P, PAPASTEFANOPOULOS V, KOTSIANTIS S. Explainable AI: A review of machine learning interpretability methods[J]. Entropy, 2020, 23(1): 18.
[45] GILPIN L H, BAU D, YUAN B Z, et al. Explaining explanations: An overview of interpretability of machine learning[C]//2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 2018: 80-89.
[46] WIRTH R, HIPP J. CRISP-DM: Towards a standard process model for data mining[C]// Proceedings of the 4th International Conference on the Practical Applications of Knowledge Discovery and Data Mining: Vol. 1. Manchester, 2000: 29-39.
[47] ERICSON G, ROHM W A, MARTENS J, et al. Team data science process documentation[J]. Retrieved April, 2017, 11: 2019.
[48] AMERSHI S, BEGEL A, BIRD C, et al. Software engineering for machine learning: A case study[C]//2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 2019: 291-300.
[49] HAAKMAN M, CRUZ L, HUIJGENS H, et al. AI lifecycle models need to be revised[J]. Empirical Software Engineering, 2021, 26(5): 1-29.
[50] PREECE A, HARBORNE D, BRAINES D, et al. Stakeholders in explainable AI[A]. 2018. arXiv preprint arXiv:1810.00184.
[51] LIM B Y, DEY A K, AVRAHAMI D. Why and why not explanations improve the intelligibility of context-aware intelligent systems[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2009: 2119-2128.
[52] LIAO Q V, GRUEN D, MILLER S. Questioning the AI: informing design practices for explainable AI user experiences[C]//Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 2020: 1-15.
[53] DAS A, RAD P. Opportunities and challenges in explainable artificial intelligence (XAI): A survey[A]. 2020. arXiv preprint arXiv:2006.11371.
[54] ZHANG Y, TIŇO P, LEONARDIS A, et al. A survey on neural network interpretability[J]. IEEE Transactions on Emerging Topics in Computational Intelligence, 2021, 5(5): 726-742.
[55] THAMPI A. Interpretable AI[M]. Simon and Schuster, 2022.
[56] KAMATH U, LIU J. Explainable artificial intelligence: An introduction to interpretable machine learning[M]. Springer, 2021.
[57] LI B, LI J, TANG K, et al. Many-objective evolutionary algorithms: A survey[J]. ACM Comput. Surv., 2015, 48(1): 1-35.
[58] LOU Y, CARUANA R, GEHRKE J, et al. Accurate intelligible models with pairwise interactions[C]//Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2013: 623-631.
[59] HENELIUS A, PUOLAMÄKI K, BOSTRÖM H, et al. A peek into the black box: Exploring classifiers by randomization[J]. Data Mining and Knowledge Discovery, 2014, 28(5): 1503- 1529.
[60] VIDOVIC M M C, GÖRNITZ N, MÜLLER K R, et al. Feature importance measure for nonlinear learning algorithms[A]. 2016. arXiv preprint arXiv:1611.07567.
[61] TOLOMEI G, SILVESTRI F, HAINES A, et al. Interpretable predictions of tree-based ensembles via actionable feature tweaking[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2017: 465-474.
[62] BHATT U, WELLER A, MOURA J M. Evaluating and aggregating feature-based model explanations[C]//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. 2021: 3016-3022.
[63] SHRIKUMAR A, GREENSIDE P, KUNDAJE A. Learning important features through propagating activation differences[C]//Proceedings of the 34th International Conference on Machine Learning. PMLR, 2017: 3145-3153.
[64] CRAVEN M, SHAVLIK J. Extracting tree-structured representations of trained networks[C]// Advances in Neural Information Processing Systems: Vol. 8. 1995: 24-30.
[65] KRISHNAN R, SIVAKUMAR G, BHATTACHARYA P. Extracting decision trees from trained neural networks[J]. Pattern Recognition, 1999, 32(12): 1999-2009.
[66] BOZ O. Extracting decision trees from trained neural networks[C]//Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2002: 456-461.
[67] JOHANSSON U, NIKLASSON L. Evolving decision trees using oracle guides[C]//2009 IEEE Symposium on Computational Intelligence and Data Mining. IEEE, 2009: 238-244.
[68] TOWELL G G, SHAVLIK J W. Extracting refined rules from knowledge-based neural networks [J]. Machine Learning, 1993, 13(1): 71-101.
[69] CRAVEN M W, SHAVLIK J W. Using sampling and queries to extract rules from trained neural networks[C]//Proceedings of the Eleventh International Conference on International Conference on Machine Learning. 1994: 37-45.
[70] JOHANSSON U, KÖNIG R, NIKLASSON L. Rule extraction from trained neural networks using genetic programming[C]//13th International Conference on Artificial Neural Networks. 2003: 13-16.
[71] SETIONO R, LIU H. Understanding neural networks via rule extraction[C]//Proceedings of the 14th International Joint Conference on Artificial Intelligence: Vol. 1. 1995: 480-485.
[72] ZHANG Q, YANG Y, MA H, et al. Interpreting CNNs via decision trees[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 6261-6270.
[73] RIBEIRO M T, SINGH S, GUESTRIN C. Anchors: High-precision model-agnostic explanations[C]//Proceedings of the AAAI Conference on Artificial Intelligence: Vol. 32. 2018: 1527-1535.
[74] WANG T. Gaining free or low-cost interpretability with interpretable partial substitute[C]// International Conference on Machine Learning. PMLR, 2019: 6505-6514.
[75] FRIEDMAN J H. Greedy function approximation: A gradient boosting machine[J]. Annals of Statistics, 2001: 1189-1232.
[76] APLEY D W, ZHU J. Visualizing the effects of predictor variables in black box supervised learning models[J]. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2020, 82(4): 1059-1086.
[77] GOLDSTEIN A, KAPELNER A, BLEICH J, et al. Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation[J]. Journal of Computational and Graphical Statistics, 2015, 24(1): 44-65.
[78] BIECEK P. Ceteris paribus plots (what-if plots) for explanations of a single observation[J/OL]. GitHub repository, 2019. https://github.com/pbiecek/ceterisParibus.
[79] WACHTER S, MITTELSTADT B, RUSSELL C. Counterfactual explanations without opening the black box: Automated decisions and the GDPR[J]. Harv. JL & Tech., 2017, 31: 841.
[80] MOLNAR C. Interpretable machine learning[M]. Lulu. com, 2020.
[81] DHURANDHAR A, CHEN P Y, LUSS R, et al. Explanations based on the missing: Towards contrastive explanations with pertinent negatives[C]//Advances in Neural Information Processing Systems: Vol. 31. 2018: 592-603.
[82] VERMA S, DICKERSON J, HINES K. Counterfactual explanations for machine learning: A review[A]. 2020. arXiv preprint arXiv:2010.10596.
[83] LAUGEL T, LESOT M J, MARSALA C, et al. Inverse classification for comparison-based interpretability in machine learning[A]. 2017. arXiv preprint arXiv:1712.08443.
[84] MOTHILAL R K, SHARMA A, TAN C. Explaining machine learning classifiers through diverse counterfactual explanations[C]//Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 2020: 607-617.
[85] GURUMOORTHY K S, DHURANDHAR A, CECCHI G, et al. Efficient data representation by selecting prototypes with importance weights[C]//2019 IEEE International Conference on Data Mining (ICDM). IEEE, 2019: 260-269.
[86] LI O, LIU H, CHEN C, et al. Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions[C]//Proceedings of the AAAI Conference on Artificial Intelligence: Vol. 32. 2018: 3530-3537.
[87] RAS G, XIE N, VAN GERVEN M, et al. Explainable deep learning: A field guide for the uninitiated[J]. Journal of Artificial Intelligence Research, 2022, 73: 329-397.
[88] LEI T, BARZILAY R, JAAKKOLA T. Rationalizing neural predictions[A]. 2016. arXiv preprint arXiv:1606.04155.
[89] TUFFÉRY S. Data mining and statistics for decision making[M]. John Wiley & Sons, 2011.
[90] SURESH H, GOMEZ S R, NAM K K, et al. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs[C]//Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 2021: 1-16.
[91] DHANORKAR S, WOLF C T, QIAN K, et al. Who needs to know what, when?: Broadening the Explainable AI (XAI) Design Space by Looking at Explanations Across the AI Lifecycle [C]//Designing Interactive Systems Conference 2021. 2021: 1591-1602.
[92] YOSINSKI J, CLUNE J, NGUYEN A, et al. Understanding neural networks through deep visualization[A]. 2015. arXiv preprint arXiv:1506.06579.
[93] ERHAN D, BENGIO Y, COURVILLE A, et al. Visualizing higher-layer features of a deep network[J]. University of Montreal, 2009, 1341(3): 1-13.
[94] ERHAN D, COURVILLE A, BENGIO Y. Understanding representations learned in deep architectures[M]. Technical Report 1355, Université de Montréal/DIRO, 2010.
[95] SPRINGENBERG J T, DOSOVITSKIY A, BROX T, et al. Striving for simplicity: The all convolutional net[A]. 2014. arXiv preprint arXiv:1412.6806.
[96] ZHANG J, BARGAL S A, LIN Z, et al. Top-down neural attention by excitation backprop[J]. International Journal of Computer Vision, 2018, 126(10): 1084-1102.
[97] ZHANG Q, CAO R, SHI F, et al. Interpreting CNN knowledge via an explanatory graph[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018: 4454-4463.
[98] ZHANG Q, WANG X, CAO R, et al. Extraction of an explanatory graph to interpret a CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(11): 3863-3877.
[99] ZHANG Q, CAO R, WU Y N, et al. Growing interpretable part graphs on convnets via multishot learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence: Vol. 31. 2017.
[100] ZHANG Q, REN J, HUANG G, et al. Mining interpretable AOG representations from convolutional networks via active question answering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(11): 3949-3963.
[101] ZHANG Q, CAO R, NIAN WU Y, et al. Mining object parts from cnns via active questionanswering[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 346-355.
[102] ZHANG Q, WU Y N, ZHU S C. Interpretable convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 8827-8836.
[103] BAU D, ZHOU B, KHOSLA A, et al. Network dissection: Quantifying interpretability of deep visual representations[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 6541-6549.
[104] HOHMAN F, PARK H, ROBINSON C, et al. S ummit: Scaling deep learning interpretability by visualizing activation and attribution summarizations[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(1): 1096-1106.
[105] KAHNG M, ANDREWS P Y, KALRO A, et al. A cti v is: Visual exploration of industry-scale deep neural network models[J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 24(1): 88-97.
[106] OLAH C, SATYANARAYAN A, JOHNSON I, et al. The building blocks of interpretability[J]. Distill, 2018, 3(3): e10.
[107] PEZZOTTI N, HÖLLT T, VAN GEMERT J, et al. Deepeyes: Progressive visual analytics for designing deep neural networks[J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 24(1): 98-108.
[108] FANG F, VENTRE C, LI L, et al. Better model selection with a new definition of feature importance[A]. 2020. arXiv preprint arXiv:2009.07708.
[109] NARKAR S, ZHANG Y, LIAO Q V, et al. Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML[C]//26th International Conference on Intelligent User Interfaces. 2021: 170-174.
[110] OUEDRAOGO D N. Interpretable machine learning model selection for breast cancer diagnosis based on k-means clustering[J]. Applied Medical Informatics, 2021, 43(3): 91-102.
[111] KANDEEL A A, ABBAS H M, HASSANEIN H S. Explainable model selection of a convolutional neural network for driver’s facial emotion identification[C]//International Conference on Pattern Recognition. Springer, 2021: 699-713.
[112] MENG L, VAN DEN ELZEN S, VILANOVA A. ModelWise: Interactive model comparison for model diagnosis, improvement and selection[C]//Computer Graphics Forum: Vol. 41. 2022: 97-108.
[113] DSOUZA J, et al. Using exploratory data analysis for generating inferences on the correlation of COVID-19 cases[C]//2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT). IEEE, 2020: 1-6.
[114] SCHMIDT B. A public exploratory data analysis of gender bias in teaching evaluations[C]// Proceedings of Workshop on Visualization for the Digital Humanities (Vis4DH 16). 2016.
[115] JOTHI N, HUSAIN W, et al. Predicting generalized anxiety disorder among women using shapley value[J]. Journal of Infection and Public Health, 2021, 14(1): 103-108.
[116] MARCÍLIO W E, ELER D M. From explanations to feature selection: Assessing SHAP values as feature selection mechanism[C]//2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). Ieee, 2020: 340-347.
[117] KHAMPARIA A, GUPTA D, KHANNA A, et al. Biomedical data analysis and processing using explainable (XAI) and responsive artificial intelligence (RAI)[M]. Springer, 2022.
[118] MORENO-SANCHEZ P A. An automated feature selection and classification pipeline to improve explainability of clinical prediction models[C]//2021 IEEE 9th International Conference on Healthcare Informatics (ICHI). IEEE, 2021: 527-534.
[119] TAO J, XIONG Y, ZHAO S, et al. XAI-driven explainable multi-view game cheating detection [C]//2020 IEEE Conference on Games (CoG). IEEE, 2020: 144-151.
[120] SOLLICH P. Bayesian methods for support vector machines: Evidence and predictive class probabilities[J]. Machine Learning, 2002, 46(1): 21-52.
[121] LIU M, SHI J, CAO K, et al. Analyzing the training processes of deep generative models[J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 24(1): 77-87.
[122] KRAUSE J, PERER A, NG K. Interacting with predictions: Visual inspection of black-box machine learning models[C]//Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 2016: 5686-5697.
[123] BENNETOT A, LAURENT J L, CHATILA R, et al. Towards explainable neural-symbolic visual reasoning[A]. 2019. arXiv preprint arXiv:1909.09065.
[124] CHEN C J, CHEN L W, YANG C H, et al. Improving CNN-based pest recognition with a post-hoc explanation of XAI[J]. Research Square, 2021.
[125] GRABOWICZ P A, PERELLO N, MISHRA A. Marrying fairness and explainability in supervised learning[C]//2022 ACM Conference on Fairness, Accountability, and Transparency. 2022: 1905-1916.
[126] VIRTANEN K. Using XAI tools to detect harmful bias in ML models[Z]. 2022.
[127] PALATNIK DE SOUSA I, VELLASCO M M, COSTA DA SILVA E. Explainable artificial intelligence for bias detection in covid CT-Scan classifiers[J]. Sensors, 2021, 21(16): 5657.
[128] SCHRAMOWSKI P, STAMMER W, TESO S, et al. Making deep neural networks right for the right scientific reasons by interacting with their explanations[J]. Nature Machine Intelligence, 2020, 2(8): 476-486.
[129] KULESZA T, BURNETT M, WONG W K, et al. Principles of explanatory debugging to personalize interactive machine learning[C]//Proceedings of the 20th International Conference on Intelligent User Interfaces. 2015: 126-137.
[130] TESO S, KERSTING K. Explanatory interactive machine learning[C]//Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 2019: 239-245.
[131] STAMMER W, SCHRAMOWSKI P, KERSTING K. Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 3619-3629.
[132] AHN Y, LIN Y R. Fairsight: Visual analytics for fairness in decision making[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(1): 1086-1095.
[133] PRADHAN R, ZHU J, GLAVIC B, et al. Interpretable data-based explanations for fairness debugging[A]. 2021. arXiv preprint arXiv:2112.09745.
[134] RIEGER L, SINGH C, MURDOCH W, et al. Interpretations are useful: penalizing explanations to align neural networks with prior knowledge[C]//International Conference on Machine Learning. PMLR, 2020: 8116-8126.
[135] BRANDAO M, MAGAZZENI D. Explaining plans at scale: scalable path planning explanations in navigation meshes using inverse optimization[C]//Workshop on XAI, IJCAI-PRICAI. 2020.
[136] CASTLE S, SCHWARZENBERG R, POURVALI M. Detecting covariate drift with explanations[C]//CCF International Conference on Natural Language Processing and Chinese Computing. Springer, 2021: 317-322.
[137] MASEGOSA A R, MARTÍNEZ A M, RAMOS-LÓPEZ D, et al. Analyzing concept drift: A case study in the financial sector[J]. Intelligent Data Analysis, 2020, 24(3): 665-688.
[138] KOMOROWSKI M, MARSHALL D C, SALCICCIOLI J D, et al. Exploratory data analysis [J]. Secondary Analysis of Electronic Health Records, 2016: 185-203.
[139] CHRISTIANTO H, LEE G K K, JAIR Z W, et al. Smart interpretable model (SIM) enabling subject matter experts in rule generation[J]. Expert Systems with Applications, 2022: 117945.
[140] LÖTSCH J, KRINGEL D, ULTSCH A. Explainable artificial intelligence (XAI) in biomedicine: Making AI decisions trustworthy for physicians and patients[J]. BioMedInformatics, 2021, 2(1): 1-17.
[141] DEPERLIOGLU O, KOSE U, GUPTA D, et al. Explainable framework for Glaucoma diagnosis by image processing and convolutional neural network synergy: Analysis with doctor evaluation[J]. Future Generation Computer Systems, 2022, 129: 152-169.
[142] KRÖLL J P, EICKHOFF S B, HOFFSTAEDTER F, et al. Evolving complex yet interpretable representations: Application to Alzheimer’s diagnosis and prognosis[C]//2020 IEEE Congress on Evolutionary Computation (CEC). IEEE, 2020: 1-8.
[143] KOLYSHKINA I, SIMOFF S. Interpretability of machine learning solutions in industrial decision engineering[C]//Australasian Conference on Data Mining. Springer, 2019: 156-170.
[144] POLZER A, FLEISS J, EBNER T, et al. Validation of AI-based Information Systems for Sensitive Use Cases: Using an XAI Approach in Pharmaceutical Engineering[Z]. 2022.
[145] MOSCA E, WICH M, GROH G. Understanding and interpreting the impact of user context in hate speech detection[C]//Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media. 2021: 91-102.
[146] WICH M, BAUER J, GROH G. Impact of politically biased data on hate speech classification [C]//Proceedings of the Fourth Workshop on Online Abuse and Harms. 2020: 54-64.
[147] PATEL A R, CHANDRASEKARAN J, LEI Y, et al. A combinatorial approach to fairness testing of machine learning models[C]//2022 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 2022: 94-101.
[148] NGUYEN T N, CHOO R. Human-in-the-loop XAI-enabled vulnerability detection, investigation, and mitigation[C]//2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2021: 1210-1212.
[149] ZHANG C A, CHO S, VASARHELYI M. Explainable artificial intelligence (XAI) in auditing [J]. International Journal of Accounting Information Systems, 2022: 100572.
[150] ZHANG K, ZHANG J, XU P D, et al. Explainable AI in deep reinforcement learning models for power system emergency control[J]. IEEE Transactions on Computational Social Systems, 2021, 9(2): 419-427.
[151] MALTBIE N, NIU N, VAN DOREN M, et al. XAI tools in the public sector: A case study on predicting combined sewer overflows[C]//Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2021: 1032-1044.
[152] RAJABI E, ETMINANI K. Towards a knowledge graph-based explainable decision support hystem in Healthcare[Z]. 2021.
[153] LUNDBERG S M, NAIR B, VAVILALA M S, et al. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery[J]. Nature Biomedical Engineering, 2018, 2 (10): 749-760.
[154] YE Q, XIA J, YANG G. Explainable AI for COVID-19 CT classifiers: An initial comparison study[C]//2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS). IEEE, 2021: 521-526.
[155] XIE Y, CHEN M, KAO D, et al. CheXplain: enabling physicians to explore and understand data-driven, AI-enabled medical imaging analysis[C]//Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 2020: 1-13.
[156] ABOU EL HOUDA Z, BRIK B, KHOUKHI L. ‘Why should I trust your IDS?’: An explainable deep learning framework for intrusion detection systems in internet of things networks[J]. IEEE Open Journal of the Communications Society, 2022, 3: 1164-1176.
[157] EL-SAPPAGH S, ALONSO J M, ISLAM S, et al. A multilayer multimodal detection and prediction model based on explainable artificial intelligence for Alzheimer’s disease[J]. Scientific Reports, 2021, 11(1): 1-26.
[158] PENG J, ZOU K, ZHOU M, et al. An explainable artificial intelligence framework for the deterioration risk prediction of hepatitis patients[J]. Journal of Medical Systems, 2021, 45(5): 1-9.
[159] SARP S, KUZLU M, WILSON E, et al. The enlightening role of explainable artificial intelligence in chronic wound classification[J]. Electronics, 2021, 10(12): 1406.
[160] SABOL P, SINČÁK P, HARTONO P, et al. Explainable classifier for improving the accountability in decision-making for colorectal cancer diagnosis from histopathological images[J]. Journal of Biomedical Informatics, 2020, 109: 103523.
[161] DA CRUZ H F, PFAHRINGER B, MARTENSEN T, et al. Using interpretability approaches to update“black-box”clinical prediction models: An external validation study in nephrology [J]. Artificial Intelligence in Medicine, 2021, 111: 101982.
[162] WANG M, ZHENG K, YANG Y, et al. An explainable machine learning framework for intrusion detection systems[J]. IEEE Access, 2020, 8: 73127-73141.
[163] SUH J, YOO S, PARK J, et al. Development and validation of an explainable artificial intelligence-based decision-supporting tool for prostate biopsy[J]. BJU International, 2020, 126(6): 694-703.
[164] MONTEATH I, SHEH R. Assisted and incremental medical diagnosis using explainable artificial intelligence[C]//Proceedings of the 2nd Workshop on Explainable Artificial Intelligence. 2018: 104-108.
[165] KAVYA R, CHRISTOPHER J, PANDA S, et al. Machine learning and XAI approaches for allergy diagnosis[J]. Biomedical Signal Processing and Control, 2021, 69: 102681.
[166] FERREIRA J J, MONTEIRO M. The human-AI relationship in decision-making: AI explanation to support people on justifying their decisions[A]. 2021. arXiv preprint arXiv:2102.05460.
[167] CHEN D, ZHAO H, HE J, et al. An causal XAI diagnostic model for breast cancer based on mammography reports[C]//2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). 2021: 3341-3349.
[168] WALI S, KHAN I. Explainable AI and random forest based reliable intrusion detection system [J]. TechRxiv Preprint, 2021.
[169] ZHANG J, BAREINBOIM E. Fairness in decision-making—the causal explanation formula [C]//Proceedings of the AAAI Conference on Artificial Intelligence: Vol. 32. 2018.
[170] ADLER P, FALK C, FRIEDLER S A, et al. Auditing black-box models for indirect influence [J]. Knowledge and Information Systems, 2018, 54(1): 95-122.
[171] ARYA V, BELLAMY R K, CHEN P Y, et al. AI Explainability 360: An Extensible Toolkit for Understanding Data and Machine Learning Models.[J]. J. Mach. Learn. Res., 2020, 21(130): 1-6.
[172] MALHI A, KNAPIC S, FRÄMLING K. Explainable agents for less bias in human-agent decision making[C]//International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems. Springer, 2020: 129-146.
[173] WIEGAND G, SCHMIDMAIER M, WEBER T, et al. I drive-you trust: Explaining driving behavior of autonomous cars[C]//Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 2019: 1-6.
[174] VAN DER WAA J, SCHOONDERWOERD T, VAN DIGGELEN J, et al. Interpretable confidence measures for decision support systems[J]. International Journal of Human-Computer Studies, 2020, 144: 102493.
[175] PU P, CHEN L. Trust-inspiring explanation interfaces for recommender systems[J]. Knowledge-Based Systems, 2007, 20(6): 542-556.
[176] BOJARSKI M, CHOROMANSKA A, CHOROMANSKI K, et al. Visualbackprop: visualizing cnns for autonomous driving: Vol. 2[A]. 2016. arXiv preprint arXiv:1611.05418.
[177] DRUCE J, HARRADON M, TITTLE J. Explainable artificial intelligence (XAI) for increasing user trust in deep reinforcement learning driven autonomous systems[A]. 2021. arXiv preprint arXiv:2106.03775.
[178] HEIMERL A, WEITZ K, BAUR T, et al. Unraveling ML models of emotion with nova: multilevel explainable ai for non-experts[J]. IEEE Transactions on Affective Computing, 2020.
[179] WEITZ K, SCHILLER D, SCHLAGOWSKI R, et al. ‘Do you trust me?’ Increasing user-trust by integrating virtual agents in explainable AI interaction design[C]//Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents. 2019: 7-9.
[180] AGARWAL S, JABBARI S, AGARWAL C, et al. Towards the unification and robustness of perturbation and gradient based explanations[C]//International Conference on Machine Learning. PMLR, 2021: 110-119.
[181] FEL T, VIGOUROUX D, CADÈNE R, et al. How good is your explanation? Algorithmic stability measures to assess the quality of explanations for deep neural networks[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2022: 720-730.
[182] NGUYEN A P, MARTÍNEZ M R. On quantitative aspects of model interpretability[A]. 2020. arXiv preprint arXiv:2007.07584.
[183] LIAO Q V, VARSHNEY K R. Human-centered explainable AI (XAI): From algorithms to user experiences[A]. 2021. arXiv preprint arXiv:2110.10790.
[184] MEHRABI N, MORSTATTER F, SAXENA N, et al. A survey on bias and fairness in machine learning[J]. ACM Computing Surveys (CSUR), 2021, 54(6): 1-35.
[185] GREENBERG J. A taxonomy of organizational justice theories[J]. Academy of Management Review, 1987, 12(1): 9-22.
[186] MORSE L, TEODORESCU M H M, AWWAD Y, et al. Do the ends justify the means? Variation in the distributive and procedural fairness of machine learning algorithms[J]. Journal of Business Ethics, 2022, 181(4): 1083-1095.
[187] GRGIĆ-HLAČA N, ZAFAR M B, GUMMADI K P, et al. Beyond distributive fairness in algorithmic decision making: Feature selection for procedurally fair learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence: Vol. 32. 2018: 51-60.
[188] BEGLEY T, SCHWEDES T, FRYE C, et al. Explainability for fair machine learning[A]. 2020. arXiv preprint arXiv:2010.07389.
[189] PAN W, CUI S, BIAN J, et al. Explaining algorithmic fairness through fairness-aware causal path decomposition[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021: 1287-1297.
[190] DIMANOV B, BHATT U, JAMNIK M, et al. You shouldn’t trust me: Learning models which conceal unfairness from multiple explanation methods.[C]//European Conference on Artificial Intelligence. 2020: 2473-2480.
[191] BALAGOPALAN A, ZHANG H, HAMIDIEH K, et al. The road to explainability is paved with bias: Measuring the fairness of explanations[C]//Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 2022: 1194-1206.
[192] ZHAO Y, WANG Y, DERR T. Fairness and explainability: Bridging the gap towards fair model explanations[C]//Proceedings of the AAAI Conference on Artificial Intelligence: Vol. 37. 2023: 11363-11371.
[193] LE QUY T, ROY A, IOSIFIDIS V, et al. A survey on datasets for fairness-aware machine learning[J]. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2022, 12(3): e1452.
[194] JONES G P, HICKEY J M, DI STEFANO P G, et al. Metrics and methods for a systematic comparison of fairness-aware machine learning algorithms[A]. 2020. arXiv preprint arXiv:2010.03986.
[195] BELLAMY R K, DEY K, HIND M, et al. AI Fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias[A]. 2018. arXiv preprint arXiv:1810.01943.
[196] STEVENS A, DERUYCK P, VAN VELDHOVEN Z, et al. Explainability and fairness in machine learning: Improve fair end-to-end lending for kiva[C]//2020 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 2020: 1241-1248.
[197] AMBROSE M L, ARNAUD A. Are procedural justice and distributive justice conceptually distinct?[M]//Handbook of Organizational Justice. Psychology Press, 2013: 59-84.
[198] DUA D, GRAFF C. UCI machine learning repository[EB/OL]. University of California, Irvine, School of Information and Computer Sciences, 2017. http://archive.ics.uci.edu/ml.
[199] ANGWIN J, LARSON J, MATTU S, et al. Machine bias[M]//Ethics of Data and Analytics. Auerbach Publications, 2016: 254-264.
[200] WIGHTMAN L F. LSAC national longitudinal bar passage study. LSAC research report series. [M]. ERIC, 1998.
[201] ZAFAR M B, VALERA I, GOMEZ-RODRIGUEZ M, et al. Fairness constraints: A flexible approach for fair classification[J]. The Journal of Machine Learning Research, 2019, 20(1): 2737-2778.
[202] ZEMEL R, WU Y, SWERSKY K, et al. Learning fair representations[C]//International Conference on Machine Learning. PMLR, 2013: 325-333.
[203] KAMISHIMA T, AKAHO S, ASOH H, et al. Fairness-aware classifier with prejudice remover regularizer[C]//Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD. 2012: 35-50.
[204] AGARWAL A, BEYGELZIMER A, DUDÍK M, et al. A reductions approach to fair classification[C]//International Conference on Machine Learning. PMLR, 2018: 60-69.
[205] ZHANG B H, LEMOINE B, MITCHELL M. Mitigating unwanted biases with adversarial learning[C]//Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. 2018: 335-340.
[206] BIRD S, DUDÍK M, EDGAR R, et al. Fairlearn: A toolkit for assessing and improving fairness in AI[J]. Microsoft, Tech. Rep. MSR-TR-2020-32, 2020.
[207] PEDREGOSA F, VAROQUAUX G, GRAMFORT A, et al. Scikit-learn: Machine learning in Python[J]. Journal of Machine Learning Research, 2011, 12: 2825-2830.
[208] CHANDRA A, YAO X. Ensemble learning using multi-objective evolutionary algorithms[J]. Journal of Mathematical Modelling and Algorithms, 2006, 5(4): 417-445.
[209] TIAN Y, SI L, ZHANG X, et al. Evolutionary large-scale multi-objective optimization: A survey[J]. ACM Computing Surveys (CSUR), 2021, 54(8): 1-34.
[210] DEB K, JAIN H. An evolutionary many-objective optimization algorithm using referencepoint-based nondominated sorting approach, part I: solving problems with box constraints[J]. IEEE Transactions on Evolutionary Computation, 2013, 18(4): 577-601.
[211] SHRIKUMAR A, GREENSIDE P, SHCHERBINA A, et al. Not just a black box: Learning important features through propagating activation differences[A]. 2016. arXiv preprint arXiv:1605.01713.
[212] ZHANG Q, LI H. MOEA/D: A multiobjective evolutionary algorithm based on decomposition [J]. IEEE Transactions on Evolutionary Computation, 2007, 11(6): 712-731.
[213] ZITZLER E, THIELE L. Multiobjective optimization using evolutionary algorithms—a comparative case study[C]//International Conference on Parallel Problem Solving from Nature. Springer, 1998: 292-301.
[214] ZITZLER E, DEB K, THIELE L. Comparison of multiobjective evolutionary algorithms: Empirical results[J]. Evolutionary Computation, 2000, 8(2): 173-195.
[215] SOLOW A, POLASKY S, BROADUS J. On the measurement of biological diversity[J]. Journal of Environmental Economics and Management, 1993, 24(1): 60-68.
[216] LI M, YAO X. Quality evaluation of solution sets in multiobjective optimisation: A survey[J]. ACM Computing Surveys, 2019, 52(2): 1-38.
[217] LI B, TANG K, LI J, et al. Stochastic ranking algorithm for many-objective optimization based on multiple indicators[J]. IEEE Transactions on Evolutionary Computation, 2016, 20(6): 924- 938.
[218] FLEISHER W. Understanding, idealization, and explainable AI[J]. Episteme, 2022, 19(4): 534-560.
[219] ZHANG X, TIAN Y, JIN Y. A knee point-driven evolutionary algorithm for many-objective optimization[J]. IEEE Transactions on Evolutionary Computation, 2014, 19(6): 761-776.