Demystifying "bad" Error Messages in Data Science Libraries

Tao，Yida1; Chen，Zhihui1; Liu，Yepang2; Xuan，Jifeng3; Xu，Zhiwu1; Qin，Shengchao4

10.1145/3468264.3468560

2021-08-20

ESEC/FSE 2021 - Proceedings of the 29th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering

818-829

Error messages are critical starting points for debugging. Unfortunately, they seem to be notoriously cryptic, confusing, and uninformative. Yet, it still remains a mystery why error messages receive such bad reputations, especially given that they are merely very short pieces of natural language text. In this paper, we empirically demystify the causes and fixes of "bad"error messages, by qualitatively studying 201 Stack Overflow threads and 335 GitHub issues. We specifically focus on error messages encountered in data science development, which is an increasingly important but not well studied domain. We found that the causes of "bad"error messages are far more complicated than poor phrasing or flawed articulation of error message content. Many error messages are inherently and inevitably misleading or uninformative, since libraries do not know user intentions and cannot "see"external errors. Fixes to error-message-related issues mostly involve source code changes, while exclusive message content updates only take up a small portion. In addition, whether an error message is informative or helpful is not always clear-cut; even error messages that clearly pinpoint faults and resolutions can still cause confusion for certain users. These findings thus call for a more in-depth investigation on how error messages should be evaluated and improved in the future.

data science debugging aid empirical study Error message

其他

英语

EI

20214010982555

Errors ; Libraries

Information Sources and Analysis:903.1 ; Libraries:903.4.1

2-s2.0-85116296548

Scopus

1.Shenzhen University,China
2.Southern University of Science and Technology,China
3.Wuhan University,China
4.Teesside University,United Kingdom

Tao，Yida,Chen，Zhihui,Liu，Yepang,et al. Demystifying "bad" Error Messages in Data Science Libraries[C],2021:818-829.