Prioritizing Causation in Decision Trees: A Framework for Interpretable Modeling

Zhang，Songming1; Chen，Xiaofeng1; Ran，Xuming2; Li，Zhongshan3; Cao，Wenming1

2024-07-01

10.1016/j.engappai.2024.108224

Engineering Applications of Artificial Intelligence   影响因子和分区

0952-1976

As a popular machine learning model, decision trees classify and generalize well, but face challenges in engineering applications: 1) Sensitivity to perturbations and lack of interpretability due to correlation reliance. 2) Manual setting of stopping criterion which is unrelated to correlation strength and easily leads to over-partitioning. To address these two challenges, we first theoretically analyze what leads to sub-optimal decision trees. By incorporating causal discovery, this limitation can be attributed to the fact that trees grown with spurious correlations often fall into sub-optimal that lead to overfitting and unfair behaviors. Neglecting causality motivates us to develop a ‘better’ tree with low Kolmogorov complexity and high generalization capability. Then we propose a causality decision tree framework, CausalDT, based on our theoretical expectation, where Hilbert-Schmidt independence criterion serves as a baseline. Unlike previous approaches that prioritize relevance, our framework determines branch nodes based on causation between features, with the significance level determining whether the tree should be expanded further. Experimental results demonstrate that our model maintains performance while reducing average tree depth by 35% on various datasets. Furthermore, our model enhances decision fairness and interpretability.

Causal discovery Decision tree Interpretability Spurious correlation

SCI ; EI

英语

其他

20241215769494

Computational complexity

Computer Theory, Includes Formal Logic, Automata Theory, Switching Theory, Programming Theory:721.1 ; Combinatorial Mathematics, Includes Graph Theory, Set Theory:921.4 ; Systems Science:961

ENGINEERING

2-s2.0-85187808154

Scopus

1.Department of Mathematics,Chongqing Jiaotong University,Chongqing,400074,China
2.Department of Biomedical Engineering,Southern University of Science and Technology,Shenzhen,518055,China
3.Department of Mathematics and Statistics,Georgia State University,Atlanta,30302,United States

Zhang，Songming,Chen，Xiaofeng,Ran，Xuming,et al. Prioritizing Causation in Decision Trees: A Framework for Interpretable Modeling[J]. Engineering Applications of Artificial Intelligence,2024,133.

Zhang，Songming,Chen，Xiaofeng,Ran，Xuming,Li，Zhongshan,&Cao，Wenming.(2024).Prioritizing Causation in Decision Trees: A Framework for Interpretable Modeling.Engineering Applications of Artificial Intelligence,133.

Zhang，Songming,et al."Prioritizing Causation in Decision Trees: A Framework for Interpretable Modeling".Engineering Applications of Artificial Intelligence 133(2024).