How Can Deep Neural Networks Aid Visualization Perception Research? Three Studies on Correlation Judgments in Scatterplots

Yang，Fumeng1; Ma，Yuxin2; Harrison，Lane3; Tompkin，James4; Laidlaw，David H.4

10.1145/3544548.3581111

2023-04-19

CHI conference on Human Factors in Computing Systems (CHI)

Conference on Human Factors in Computing Systems - Proceedings

APR 23-28, 2023

null,Hamburg,GERMANY

1601 Broadway, 10th Floor, NEW YORK, NY, UNITED STATES

ASSOC COMPUTING MACHINERY

How deep neural networks can aid visualization perception research is a wide-open question. This paper provides insights from three perspectives - prediction, generalization, and interpretation - via training and analyzing deep convolutional neural networks on human correlation judgments in scatterplots across three studies. The first study assesses the accuracy of twenty-nine neural network architectures in predicting human judgments, finding that a subset of the architectures (e.g., VGG-19) has comparable accuracy to the best-performing regression analyses in prior research. The second study shows that the resulting models from the first study display better generalizability than prior models on two other judgment datasets for different scatterplot designs. The third study interprets visual features learned by a convolutional neural network model, providing insights about how the model makes predictions, and identifies potential features that could be investigated in human correlation perception studies. Together, this paper suggests that deep neural networks can serve as a tool for visualization perception researchers in devising potential empirical study designs and hypothesizing about perpetual judgments. The preprint, data, code, and training logs are available at https://doi.org/10.17605/osf.io/exa8m.

deep neural networks perception predictive modeling scatterplots visualization features

其他

英语

CPCI-S ; EI

NSF["2127309","IIS-2107409","IIS-1815587"]

Computer Science ; Robotics

Computer Science, Information Systems ; Computer Science, Theory & Methods ; Robotics

WOS:001037809507032

20232214151193

Convolution ; Forecasting ; Network architecture ; Neural network models ; Regression analysis ; Visualization

Ergonomics and Human Factors Engineering:461.4 ; Information Theory and Signal Processing:716.1 ; Artificial Intelligence:723.4 ; Mathematical Statistics:922.2

2-s2.0-85160004667

Scopus

1.Northwestern University,Evanston,United States
2.Southern University of Science and Technology,Shenzhen,China
3.Worcester Polytechnic Institute,Worcester,United States
4.Brown University,Providence,United States

Yang，Fumeng,Ma，Yuxin,Harrison，Lane,et al. How Can Deep Neural Networks Aid Visualization Perception Research? Three Studies on Correlation Judgments in Scatterplots[C]. 1601 Broadway, 10th Floor, NEW YORK, NY, UNITED STATES:ASSOC COMPUTING MACHINERY,2023.