交互式时空视觉叙事生成系统

大数据时代，海量的时空数据被收集、存储，逐渐成为人们决策的重要依据， 并在医疗、气象、交通等领域具有重要研究价值。时空数据可视化以计算机强大 的存储和计算能力为基础，通过设计有效且直观的视觉呈现方式，将时空数据的 特征传递给决策者，以帮助决策者快速、准确理解数据。时空视觉叙事是时空数 据可视化中的一个重要分支，旨在以视觉故事的方式，组织、编排时空数据中有意 义的内容，并以合适的视觉形式呈现。这种方式的优点是使得数据易于理解的同 时具有较高的趣味性，在各行业中被广泛使用。 以传统方法创作视觉故事时，通常要求创作人员同时具备数据分析、故事讲 述及视觉设计等多项技能，创作过程复杂且周期较长。为此，数据驱动的视觉故事 创作工具应运而生。这类工具可以自动从数据中获取有意义的故事内容，形成视 觉故事进行展示，并提供相应的交互方法，帮助创作者对故事内容进行调整。然 而，到目前为止，相关领域中针对时空视觉叙事工具的研究仍然较为缺乏。本文 将基于该研究，设计并实现一个针对时空数据的交互式故事生成系统。 本文主要贡献如下：（1）广泛调研时空视觉叙事样例，总结提出时空视觉叙事 生成系统的设计空间，指导交互式时空视觉叙事生成系统的设计和实现；（2）基 于当前工作，介绍故事内容相关定义，设计并实现故事内容挖掘算法，为系统的实 现提供理论基础；（3）总结时空故事的叙事逻辑，对已有故事内容扩展，并确定生 成故事的步骤，为系统生成故事提供丰富素材；（4）实现交互式时空视觉叙事生 成系统，确保故事内容的视觉呈现效果。 

[1] 王家耀, 武芳, 郭建忠, 等. 时空大数据面临的挑战与机遇[J]. 测绘科学, 2017, 42(7): 1-7.
[2] LEE C, KIM Y, JIN S, et al. A visual analytics system for exploring, monitoring, and forecasting road traffic congestion[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(11): 3133-3146.
[3] LIU H, GAO Y, LU L, et al. Visual analysis of route diversity[C]//2011 IEEE Conference on Visual Analytics Science and Technology (VAST). Providence: IEEE Computer Society, 2011: 171-180.
[4] ZHANG J, YANLI E, MA J, et al. Visual analysis of public utility service problems in a metropolis[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(12): 1843-1852.
[5] ZENG W, SHEN Q, JIANG Y, et al. Route-aware edge bundling for visualizing origindestination trails in urban traffic[J]. Computer Graphics Forum, 2019, 38(3): 581-593.
[6] ANDRIENKO G, ANDRIENKO N, DYKES J, et al. Geovisualization of dynamics, movement and change: Key issues and developing approaches in visualization research[J]. Information visualization, 2008, 7(3-4): 173-180.
[7] YANG L, MA Z, ZHU L, et al. Research on the visualization of spatio-temporal data[J]. IOP Conference Series: Earth and Environmental Science, 2019, 234(1): 012013.
[8] 周志光, 石晨, 史林松, 等. 地理空间数据可视分析综述[J]. 计算机辅助设计与图形学学报, 2018, 20(5): 747-763.
[9] CHEN W, GUO F, WANG F. A survey of traffic data visualization[J]. IEEE Transactions on Intelligent Transportation Systems, 2015, 16(6): 2970-2984.
[10] 蒲剑苏, 屈华民, 倪明选, 等. 移动轨迹数据的可视化[J]. 计算机辅助设计与图形学学报,2012, 24(10): 1273-1282.
[11] LIEN N H, CHEN Y L. Narrative ads: The effect of argument strength and story format[J]. Journal of Business Research, 2013, 66(4): 516-522.
[12] MINARD C J. Des tableaux graphiques et des cartes figuratives[M]. Pairs: Thunot, 1862.
[13] SNOW J. On the mode of communication of cholera[M]. London: John Churchill, 1855: 44-45.
[14] WANG Y, SUN Z, ZHANG H, et al. Datashot: Automatic generation of fact sheets from tabular data[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(1): 895-905.
[15] SHI D, XU X, SUN F, et al. Calliope: Automatic visual data story generation from a spreadsheet[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 27(2): 453-463.
[16] ANDRIENKO N, ANDRIENKO G. Visual analytics of movement: An overview of methods, tools and procedures[J]. Information Visualization, 2013, 12(1): 3-24.
[17] 涂乐, 陈彬捷, 周志光. OD 数据可视分析综述[J]. 计算机辅助设计与图形学学报, 2021, 33(8): 1160-1171.
[18] 王祖超, 袁晓如. 轨迹数据可视分析研究[J]. 计算机辅助设计与图形学学报, 2015, 27(1):9-25.
[19] THOM D, BOSCH H, KOCH S, et al. Spatiotemporal anomaly detection through visual analysis of geolocated twitter messages[C]//2012 IEEE Pacific Visualization Symposium (PacificVis). Songdo: IEEE Computer Society, 2012: 41-48.
[20] CHEN S, YUAN X, WANG Z, et al. Interactive visual discovering of movement patterns from sparsely sampled geo-tagged social media data[J]. IEEE Transactions on Visualization and Computer Graphics, 2015, 22(1): 270-279.
[21] ZHENG Y, WU W, ZENG H, et al. Telcoflow: Visual exploration of collective behaviors based on telco data[C]//2016 IEEE International Conference on Big Data (Big Data). Washington DC: IEEE Computer Society, 2016: 843-852.
[22] GUO D, ZHU X. Origin-destination flow data smoothing and mapping[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(12): 2043-2052.
[23] ZHENG Y, WU W, QU H, et al. Visual analysis of bi-directional movement behavior[C]//2015 IEEE International Conference on Big Data (Big Data). Santa Clara: IEEE Computer Society, 2015: 581-590.
[24] ZHOU Z, MENG L, TANG C, et al. Visual abstraction of large scale geospatial origindestination movement data[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 25(1): 43-53.
[25] VON LANDESBERGER T, BRODKORB F, ROSKOSCH P, et al. Mobilitygraphs: Visualanalysis of mass mobility dynamics via spatio-temporal graphs and clustering[J]. IEEE Transactions on Visualization and Computer Graphics, 2015, 22(1): 11-20.
[26] LYU Y, LIU X, CHEN H, et al. Od morphing: Balancing simplicity with faithfulness for od bundling[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(1): 811-821.
[27] ZENG W, FU C W, MÜLLER ARISONA S, et al. Visualizing waypoints-constrained origindestination patterns for massive transportation data[J]. Computer Graphics Forum, 2016, 35(8): 95-107.
[28] LU M, WANG Z, LIANG J, et al. Od-wheel: Visual design to explore od patterns of a central region[C]//2015 IEEE Pacific Visualization Symposium (PacificVis). Hang Zhou: IEEE Computer Society, 2015: 87-91.
[29] SUN G, LIU Y, WU W, et al. Embedding temporal display into maps for occlusion-free visualization of spatio-temporal data[C]//2014 IEEE Pacific Visualization Symposium (PacificVis).Yokohama: IEEE Computer Society, 2014: 185-192.
[30] SUN G, LIANG R, QU H, et al. Embedding spatio-temporal information into maps by routezooming[J]. IEEE Transactions on Visualization and Computer Graphics, 2016, 23(5): 1506-1519.
[31] HUANG X, ZHAO Y, MA C, et al. Trajgraph: A graph-based visual analytics approach to studying urban network centralities using taxi trajectory data[J]. IEEE Transactions on Visualization and Computer Graphics, 2015, 22(1): 160-169
[32] KRÜGER R, THOM D, WÖRNER M, et al. Trajectorylenses - a set-based filtering and exploration technique for long-term trajectory data[J]. Computer Graphics Forum, 2013, 32(3): 451-460.
[33] ANDRIENKO N, ANDRIENKO G, BARRETT L, et al. Space transformation for understanding group movement[J]. IEEE Transactions on Visualization and Computer Graphics, 2013, 19(12): 2169-2178.
[34] ANDRIENKO G, ANDRIENKO N, HURTER C, et al. From movement tracks through events to places: Extracting and characterizing significant places from mobility data[C]//2011 IEEE Conference on Visual Analytics Science and Technology (VAST). Providence: IEEE Computer Society, 2011: 161-170.
[35] WAGNER FILHO J A, STUERZLINGER W, NEDEL L. Evaluating an immersive space-time cube geovisualization for intuitive trajectory data exploration[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(1): 514-524.
[36] TOMINSKI C, SCHUMANN H, ANDRIENKO G, et al. Stacking-based visualization of trajectory attribute data[J]. IEEE Transactions on Visualization and Computer Graphics, 2012, 18 (12): 2565-2574.
[37] VROTSOU K, JANETZKO H, NAVARRA C, et al. Simplifly: A methodology for simplification and thematic enhancement of trajectories[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 21(1): 107-121.
[38] WU W, ZHENG Y, CAO N, et al. Mobiseg: Interactive region segmentation using heterogeneous mobility data[C]//2017 IEEE Pacific visualization symposium (PacificVis). Seoul: IEEE Computer Society, 2017: 91-100.
[39] WANG Z, YE T, LU M, et al. Visual exploration of sparse traffic trajectory data[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(12): 1813-1822.
[40] PARK Y, CAFARELLA M, BARZAN M. Visualization-aware sampling for very large databases [C]//2016 IEEE 32nd International Conference on Data Engineering (ICDE). Helsinki: IEEE Computer Society, 2016: 755-766.
[41] ZHANG D, DING M, YANG D, et al. Trajectory simplification: An experimental study and quality analysis[J]. Proceedings of the VLDB Endowment, 2018, 11(9): 934-946.
[42] PI M, YEON H, SON H, et al. Visual cause analytics for traffic congestion[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 27(3): 2186-2201.
[43] HÄB K, MIDDEL A, RUDDELL B L, et al. Travis - a visualization framework for mobile transect data sets in an urban microclimate context[C]//2015 IEEE Pacific Visualization Symposium (PacificVis). Hang Zhou: IEEE Computer Society, 2015: 167-174.
[44] LIU S, PU J, LUO Q, et al. Vait: A visual analytics system for metropolitan transportation[J]. IEEE Transactions on Intelligent Transportation Systems, 2013, 14(4): 1586-1596.
[45] WANG Q, LU M, LI Q. Interactive, multiscale urban-traffic pattern exploration leveraging massive gps trajectories[J]. Sensors, 2020, 20(4): 1084.
[46] GUO H, WANG Z, YU B, et al. Tripvista: Triple perspective visual trajectory analytics and its application on microscopic traffic data at a road intersection[C]//2011 IEEE Pacific Visualization Symposium (PacificVis). Hong Kong: IEEE Computer Society, 2011: 163-170.
[47] LIU H, JIN S, YAN Y, et al. Visual analytics of taxi trajectory data via topical sub-trajectories [J]. Visual Informatics, 2019, 3(3): 140-149.
[48] 姜晓睿, 郑春益, 蒋莉, 等. 大规模出租车起止点数据可视分析[J]. 计算机辅助设计与图形学学报, 2015, 27(10): 1907-1917.
[49] LU M, LIANG J, WANG Z, et al. Exploring od patterns of interested region based on taxi trajectories[J]. Journal of Visualization, 2016, 19(4): 811-821.
[50] 何贤国, 孙国道, 高家全, 等. 出租车 GPS 大数据的道路行车可视分析[J]. 计算机辅助设计与图形学学报, 2014, 26(12): 2163-2172.
[51] LU M, WANG Z, YUAN X. Trajrank: Exploring travel behaviour on a route by trajectory ranking[C]//2015 IEEE Pacific Visualization Symposium (PacificVis). Hang Zhou: IEEE Computer Society, 2015: 311-318.
[52] LU M, LAI C, YE T, et al. Visual analysis of multiple route choices based on general gps trajectories[J]. IEEE Transactions on Big Data, 2017, 3(2): 234-247.
[53] KRUEGER R, HAN Q, IVANOV N, et al. Bird’s-eye-large-scale visual analytics of city dynamics using social location data[J]. Computer Graphics Forum, 2019, 38(3): 595-607.
[54] FERREIRA N, POCO J, VO H T, et al. Visual exploration of big spatio-temporal urban data: A study of new york city taxi trips[J]. IEEE Transactions on Visualization and Computer Graphics, 2013, 19(12): 2149-2158.
[55] 吴向平, 徐懂事, 吴向阳, 等. 面向出租车出行规律的预测式可视分析方法[J]. 计算机辅助设计与图形学学报, 2020, 32(4): 520-530.
[56] AL-DOHUKI S, WU Y, KAMW F, et al. Semantictraj: A new approach to interacting with massive taxi trajectories[J]. IEEE Transactions on Visualization and Computer Graphics, 2016, 23(1): 11-20.
[57] LIU D, WENG D, LI Y, et al. Smartadp: Visual analytics of large-scale taxi trajectories for selecting billboard locations[J]. IEEE Transactions on Visualization and Computer Graphics, 2016, 23(1): 1-10.
[58] WANG F, CHEN W, WU F, et al. A visual reasoning approach for data-driven transport assessment on urban roads[C]//2014 IEEE Conference on Visual Analytics Science and Technology (VAST). Pairs: IEEE Computer Society, 2014: 103-112.
[59] ADRIENKO N, ADRIENKO G. Spatial generalization and aggregation of massive movement data[J]. IEEE Transactions on Visualization and Computer Graphics, 2010, 17(2): 205-219.
[60] JIANG X, ZHENG C, TIAN Y, et al. Large-scale taxi o/d visual analytics for understanding metropolitan human movement patterns[J]. Journal of Visualization, 2015, 18(2): 185-200.
[61] KALAMARAS I, ZAMICHOS A, SALAMANIS A, et al. An interactive visual analytics platform for smart intelligent transportation systems management[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 19(2): 487-496.
[62] WENG D, ZHENG C, DENG Z, et al. Towards better bus networks: A visual analytics approach [J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 27(2): 817-827.
[63] POCO J, DORAISWAMY H, VO H T, et al. Exploring traffic dynamics in urban environments using vector-valued functions[J]. Computer Graphics Forum, 2015, 34(3): 161-170.
[64] CHEN Y C, WANG Y S, LIN W C, et al. Interactive visual analysis for vehicle detector data [J]. Computer Graphics Forum, 2015, 34(3): 171-180.
[65] CAO N, LIN C, ZHU Q, et al. Voila: Visual anomaly detection and monitoring with streaming spatiotemporal data[J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 24 (1): 23-33.
[66] LIU D, XU P, REN L. Tpflow: Progressive partition and multidimensional pattern extraction for large-scale spatio-temporal data analysis[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 25(1): 1-11.
[67] WU W, XU J, ZENG H, et al. Telcovis: Visual exploration of co-occurrence in urban human mobility based on telco data[J]. IEEE Transactions on Visualization and Computer Graphics, 2015, 22(1): 935-944.
[68] CHU D, SHEETS D A, ZHAO Y, et al. Visualizing hidden themes of taxi movement with semantic transformation[C]//2014 IEEE Pacific Visualization Symposium (PacificVis). Yokohama: IEEE Computer Society, 2014: 137-144.
[69] WU T, XIN D, MEI Q, et al. Promotion analysis in multi-dimensional space[J]. Proceedings of the VLDB Endowment, 2009, 2(1): 109-120.
[70] VARTAK M, RAHMAN S, MADDEN S, et al. Seedb: Efficient data-driven visualization recommendations to support visual analytics[J]. Proceedings of the VLDB Endowment, 2015, 8 (13): 2182-2193.
[71] PALPANAS T, KOUDAS N. Entropy based approximate querying and exploration of datacubes [C]//Proceedings Thirteenth International Conference on Scientific and Statistical Database Management(SSDBM). George Mason University: IEEE Computer Society, 2001: 81-90.
[72] CHEN Y, DONG G, HAN J, et al. Multi-dimensional regression analysis of time-series data streams[C]//Proceedings of the 28th International Conference on Very Large Databases(VLDB). Hong Kong: Morgan Kaufmann, 2002: 323-334.
[73] TANG B, HAN S, YIU M L, et al. Extracting top-k insights from multi-dimensional data[C]// Proceedings of the 2017 ACM International Conference on Management of Data. Chicago: ACM, 2017: 1509-1524.
[74] DING R, HAN S, XU Y, et al. Quickinsights: Quick and automatic discovery of insights from，multi-dimensional data[C]//Proceedings of the 2019 International Conference on Management of Data. Amsterdam: ACM, 2019: 317-332.
[75] LIN Q, KE W, LOU J G, et al. Bigin4: Instant, interactive insight identification for multidimensional big data[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. London: ACM, 2018: 547-555.
[76] MA P, DING R, HAN S, et al. Metainsight: Automatic discovery of structured knowledge for exploratory data analysis[C]//Proceedings of the 2021 International Conference on Management of Data. Virtual Event, China: ACM, 2021: 1262-1274.
[77] WU T, SUN Y, LI C, et al. Region-based online promotion analysis[C]//Proceedings of the 13th International Conference on Extending Database Technology. Lausanne: ACM, 2010: 63-74.
[78] TONG C, ROBERTS R, BORGO R, et al. Storytelling and visualization: An extended survey [J]. Information, 2018, 9(3): 65.
[79] LEE B, RICHE N H, ISENBERG P, et al. More than telling a story: Transforming data into visually shared stories[J]. IEEE Computer Graphics and Applications, 2015, 35(5): 84-90.
[80] SEGEL E, HEER J. Narrative visualization: Telling stories with data[J]. IEEE Transactions on Visualization and Computer Graphics, 2010, 16(6): 1139-1148.
[81] MCKENNA S, HENRY RICHE N, LEE B, et al. Visual narrative flow: Exploring factors shaping data visualization story reading experiences[J]. Computer Graphics Forum, 2017, 36 (3): 377-387.
[82] STOLPER C D, LEE B, RICHE N H, et al. Emerging and recurring data-driven storytelling techniques: Analysis of a curated collection of recent stories[EB/OL]. 2016
[2016-04-03]. http: //research.microsoft.com/apps/pubs/default.aspx?id=264484.
[83] BACH B, STEFANER M, BOY J, et al. Narrative design patterns for data-driven storytelling [M]//Data-driven storytelling. A K Peters/CRC Press, 2018: 107-133.
[84] HULLMAN J, DRUCKER S, RICHE N H, et al. A deeper understanding of sequence in narrative visualization[J]. IEEE Transactions on Visualization and Computer Graphics, 2013, 19 (12): 2406-2415.
[85] AMINI F, RICHE N H, LEE B, et al. Authoring data-driven videos with dataclips[J]. IEEE Transactions on Visualization and Computer Graphics, 2016, 23(1): 501-510.
[86] REN D, BREHMER M, LEE B, et al. Chartaccent: Annotation for data-driven storytelling[C]// 2017 IEEE Pacific Visualization Symposium (PacificVis). Seoul: IEEE Computer Society, 2017: 230-239.
[87] GAO T, HULLMAN J R, ADAR E, et al. Newsviews: An automated pipeline for creating custom geovisualizations for news[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Toronto: ACM, 2014: 3005-3014.
[88] HULLMAN J, DIAKOPOULOS N, ADAR E. Contextifier: Automatic generation of annotated stock visualizations[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Pairs: ACM, 2013: 2707-2716.
[89] SHI D, SUN F, XU X, et al. Autoclips: An automatic approach to video generation from data facts[J]. Computer Graphics Forum, 2021, 40(3): 495-505.
[90] FREYTAG G. Technique of the drama: An exposition of dramatic composition and art[M]. S. Griggs, 1895.
[91] XU X, YANG L, YIP D, et al. From “wow” to “why”: Guidelines for creating the opening of a data video with cinematic styles[EB/OL]. 2022
[2022-02-06]. https://doi.org/10.48550/arXiv.2 202.02709.
[92] WOLF F, GIBSON E. Representing discourse coherence: A corpus-based study[J]. Computational Linguistics, 2005, 31(2): 249-287.
[93] BREWER C A. Cartography: Thematic map design[J]. Cartographic Perspectives, 1994(17): 26-27.
[94] NUSRAT S, KOBOUROV S. The state of the art in cartograms[J]. Computer Graphics Forum, 2016, 35(3): 619-642.
[95] NUSRAT S, ALAM M J, KOBOUROV S. Evaluating cartogram effectiveness[J]. IEEE Transactions on Visualization and Computer Graphics, 2016, 24(2): 1077-1090.
[96] BOGUCKA E P, CONSTANTINIDES M, AIELLO L M, et al. Cartographic design of cultural maps[J]. IEEE Computer Graphics and Applications, 2020, 40(6): 12-20.
[97] JAVED W, ELMQVIST N. Exploring the design space of composite visualization[C]//2012 IEEE Pacific Visualization Symposium (PacificVis). Songdo: IEEE Computer Society, 2012: 1-8.
[98] SPECKMANN B, VERBEEK K. Necklace maps.[J]. IEEE Transactions on Visualization and Computer Graphics, 2010, 16(6): 881-889.
[99] LATIF S, CHEN S, BECK F. A deeper understanding of visualization-text interplay in geographic data-driven stories[J]. Computer Graphics Forum, 2021, 40(3): 311-322.
[100] MAYR E, WINDHAGER F. Once upon a spacetime: Visual storytelling in cognitive and geotemporal information spaces[J]. ISPRS International Journal of Geo-Information, 2018, 7(3): 96.
[101] ECCLES R, KAPLER T, HARPER R, et al. Stories in geotime[J]. Information Visualization, 2008, 7(1): 3-17.
[102] CAQUARD S, CARTWRIGHT W. Narrative cartography: From mapping stories to the narrative of maps and mapping[J]. The Cartographic Journal, 2014, 51(2): 101-106.
[103] PEÑA-ARAYA V, PIETRIGA E, BEZERIANOS A. A comparison of visualizations for identifying correlation over space and time[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(1): 375-385.
[104] DUNCAN I K, TINGSHENG S, PERRAULT S T, et al. Task-based effectiveness of interactive contiguous area cartograms[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 27(3): 2136-2152.
[105] LI J, CHEN S, CHEN W, et al. Semantics-space-time cube: A conceptual framework for systematic analysis of texts in space and time[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 26(4): 1789-1806.