Towards Adaptive, Scalable and Interpretable Spatial-Temporal Forecasting

In recent years, the application of deep learning models in the realm of spatial-temporal forecasting has significantly outpaced traditional statistical learning methods such as ARMA, VAR, and STL. However, a notable drawback to these models is their 'black box' nature, leaving their internal workings largely uninterpretable. Furthermore, recent studies have exposed the limitations of deep learning models, particularly those within the Transformer family, which often fall short when compared to carefully crafted linear models in various situations. These revelations invite reconsideration of the indispensability of deep learning models in spatial-temporal forecasting.

This thesis seeks to explore the complexities of modeling spatial-temporal data, striving to devise an interpretable, adaptive, and scalable spatial-temporal forecasting model that can compete with current state-of-the-art methodologies. By accomplishing this, it is anticipated that a deeper understanding of the underlying processes will be achieved, potentially paving the way for advancements in the field of spatial-temporal forecasting. This research encapsulates three key contributions: 1. The development of an attention-based method to probe correlations within diverse spatial-temporal contexts, addressing both adaptability and interpretability challenges; 2. The invention of a normalization-based method to take advantage of the characteristics of spatial and temporal data distribution, grappling with adaptability, interpretability, and scalability issues; 3. The introduction of a novel framework designed to emulate the dynamic behavior of underlying components in time series data, tackling adaptability, interpretability, and scalability concerns.

[1] Alon, U. and Yahav, E. (2020). On the bottleneck of graph neural networks and its practical implications. arXiv preprint arXiv:2006.05205.
[2] BAI, L., Yao, L., Li, C., Wang, X., and Wang, C. (2020). Adaptive graph convolutional recurrent network for traffic forecasting. Advances in Neural Information Processing Systems, 33.
[3] Bai, S., Kolter, J. Z., and Koltun, V. (2018). An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271.
[4] Bandara, K., Bergmeir, C., and Hewamalage, H. (2021). Lstm-msnet: Leveraging forecasts on sets of related time series with multiple seasonal patterns. IEEE Transactions on Neural Networks and Learning Systems, 32(4):1586–1599.
[5] Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
[6] Cao, D., Wang, Y., Duan, J., Zhang, C., Zhu, X., Huang, C., Tong, Y., Xu, B., Bai, Y., Tong, J., et al. (2020). Spectral temporal graph neural network for multivariate time-series forecasting. Proceedings of the NeurIPS 2020.
[7] Chai, D., Wang, L., and Yang, Q. (2018). Bike flow prediction with multigraph convolutional networks. In Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pages 397–400.
[8] Challu, C., Olivares, K. G., Oreshkin, B. N., Ramirez, F. G., Canseco, M. M., and Dubrawski, A. (2023). Nhits: Neural hierarchical interpolation for time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 6989–6997.
[9] Chen, C., Li, K., Teo, S. G., Zou, X., Li, K., and Zeng, Z. (2020). Citywide traffic flow prediction based on multiple gated spatio-temporal convolutional neural networks. ACM Transactions on Knowledge Discovery from Data (TKDD), 14(4):1–23.
[10] Chen, C., Li, K., Teo, S. G., Zou, X., Wang, K., Wang, J., and Zeng, Z. (2019). Gated residual recurrent graph neural networks for traffic prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 485–492.
[11] Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using rnn encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724–1734.
[12] Choi, J., Choi, H., Hwang, J., and Park, N. (2022). Graph neural controlled differential equations for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence.
[13] Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555.
[14] Cirstea, R.-G., Yang, B., Guo, C., Kieu, T., and Pan, S. (2022). Towards spatiotemporal aware traffic time series forecasting. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 2900–2913.
[15] Cook, R. D. and Weisberg, S. (1983). Diagnostics for heteroscedasticity in regression. Biometrika, 70(1):1–10.
[16] Das, A., Kong, W., Leach, A., Sen, R., and Yu, R. (2023). Long-term forecasting with tide: Time-series dense encoder. arXiv preprint arXiv:2304.08424.
[17] Deng, J., Chen, X., Fan, Z., Jiang, R., Song, X., and Tsang, I. W. (2021a). The pulse of urban transport: exploring the co-evolving pattern for spatio-temporal forecasting. ACM Transactions on Knowledge Discovery from Data (TKDD), 15(6):1–25.
[18] Deng, J., Chen, X., Jiang, R., Song, X., and Tsang, I. W. (2021b). St-norm: Spatial and temporal normalization for multi-variate time series forecasting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 269–278.
[19] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee.
[20] Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pretraining of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
[21] Ding, D., Zhang, M., Pan, X., Yang, M., and He, X. (2019). Modeling extreme events in time series prediction. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1114–1122.
[22] Engle, R. (2002). Dynamic conditional correlation: A simple class of multivariate generalized autoregressive conditional heteroskedasticity models. Journal of Business & Economic Statistics, 20(3):339–350.
[23] Fan, W., Wang, P., Wang, D., Wang, D., Zhou, Y., and Fu, Y. (2023). Dish-ts: A general paradigm for alleviating distribution shift in time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7522–7529.
[24] Fan, W., Zheng, S., Yi, X., Cao, W., Fu, Y., Bian, J., and Liu, T.-Y. (2022). DEPTS: Deep expansion learning for periodic time series forecasting. In International Conference on Learning Representations.
[25] Fang, S., Zhang, Q., Meng, G., Xiang, S., and Pan, C. (2019). Gstnet: Global spatial temporal network for traffic flow prediction. In Proceedings of the Twenty Eighth International Joint Conference on Artificial Intelligence, IJCAI, pages 10–16.
[26] Feng, J., Li, Y., Lin, Z., Rong, C., Sun, F., Guo, D., and Jin, D. (2021). Context-aware spatial-temporal neural network for citywide crowd flow prediction via modeling long-range spatial dependency. ACM Transactions on Knowledge Discovery from Data (TKDD), 16(3):1–21.
[27] Gama, J., éliobait˙e, I., Bifet, A., Pechenizkiy, M., and Bouchachia, A. (2014). A survey on concept drift adaptation. ACM computing surveys (CSUR), 46(4):1–37.
[28] Geng, X., Li, Y., Wang, L., Zhang, L., Yang, Q., Ye, J., and Liu, Y. (2019). Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 3656–3663.
[29] Gong, Y., Li, Z., Zhang, J., Liu, W., Zheng, Y., and Kirsch, C. (2018). Networkwide crowd flow prediction of sydney trains via customized online non-negative matrix factorization. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pages 1243–1252.
[30] Guo, K., Hu, Y., Sun, Y., Qian, S., Gao, J., and Yin, B. (2021a). Hierarchical graph convolution networks for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 151–159.
[31] Guo, S., Lin, Y., Feng, N., Song, C., and Wan, H. (2019a). Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 922–929.
[32] Guo, S., Lin, Y., Li, S., Chen, Z., and Wan, H. (2019b). Deep spatial–temporal 3d convolutional neural networks for traffic data forecasting. IEEE Transactions on Intelligent Transportation Systems, 20(10):3913–3926.
[33] Guo, S., Lin, Y., Li, S., Chen, Z., and Wan, H. (2019c). Deep spatial–temporal 3d convolutional neural networks for traffic data forecasting. IEEE Transactions on Intelligent Transportation Systems, 20(10):3913–3926.
[34] Guo, S., Lin, Y., Wan, H., Li, X., and Cong, G. (2021b). Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting. IEEE Transactions on Knowledge and Data Engineering, 34(11):5415–5428.
[35] Han, L., Ye, H.-J., and Zhan, D.-C. (2023). The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting.
[36] Harvey, A. C. (1976). Estimating regression models with multiplicative heteroscedasticity. Econometrica: Journal of the Econometric Society, pages 461–465.
[37] He, S. and Shin, K. G. (2020). Towards fine-grained flow forecasting: A graph attention approach for bike sharing systems. In Proceedings of The Web Conference 2020, pages 88–98.
[38] Hillmer, S. C. and Tiao, G. C. (1982). An arima-model-based approach to seasonal adjustment. Journal of the American Statistical Association, 77(377):63–70.
[39] Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8):1735–1780.
[40] Holtz-Eakin, D., Newey, W., and Rosen, H. S. (1988). Estimating vector autoregressions with panel data. Econometrica: Journal of the econometric society, pages 1371–1395.
[41] Huang, C., Zhang, C., Zhao, J., Wu, X., Yin, D., and Chawla, N. (2019). Mist: A multiview and multimodal spatial-temporal learning framework for citywide abnormal event forecasting. In The World Wide Web Conference, WWW ’19, page 717–728, New York, NY, USA. Association for Computing Machinery.
[42] Jiang, R., Song, X., Huang, D., Song, X., Xia, T., Cai, Z., Wang, Z., Kim, K.-S., and Shibasaki, R. (2019). Deepurbanevent: A system for predicting citywide crowd dynamics at big events. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2114–2122. ACM.
[43] Jiang, R., Wang, Z., Yong, J., Jeph, P., Chen, Q., Kobayashi, Y., Song, X., Fukushima, S., and Suzumura, T. (2023). Spatio-temporal meta-graph learning for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8078–8086.
[44] Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., and Choo, J. (2021). Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations.
[45] Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., and Choo, J. (2022). Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations.
[46] Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
[47] Kingma, D. P. and Ba, J. (2015). Adam: A method for stochastic optimization. In ICLR (Poster).
[48] Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2021). Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, pages 5637–5664. PMLR.
[49] Lai, G., Chang, W.-C., Yang, Y., and Liu, H. (2018). Modeling long-and short-term temporal patterns with deep neural networks. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 95–104.
[50] Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Pritzel, A., Ravuri, S., Ewalds, T., Alet, F., Eaton-Rosen, Z., et al. (2022). Graphcast: Learning skillful medium-range global weather forecasting. arXiv preprint arXiv:2212.12794.
[51] Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L. (2019). Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461.
[52] Li, L., Pagnucco, M., and Song, Y. (2022). Graph-based spatial transformer with memory replay for multi-future pedestrian trajectory prediction. In Proceed ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2231–2241.
[53] Li, M., Qin, Z., Jiao, Y., Yang, Y., Wang, J., Wang, C., Wu, G., and Ye, J. (2019a). Efficient ridesharing order dispatching with mean field multi-agent reinforcement learning. In The World Wide Web Conference, pages 983–994.
[54] Li, Q., Han, Z., and Wu, X.-M. (2018a). Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence, volume 32.
[55] Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W.,Wang, Y.-X., and Yan, X. (2019b). Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In Advances in Neural Information Processing Systems, pages 5243–5253.
[56] Li, Y., Yu, R., Shahabi, C., and Liu, Y. (2018b). Diffusion convolutional recurrentneural network: Data-driven traffic forecasting. In International Conferenceon Learning Representations.
[57] Li, Y., Zhu, Z., Kong, D., Xu, M., and Zhao, Y. (2019c). Learning heterogeneous spatial-temporal representation for bike-sharing demand prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1004–1011.
[58] Liang, Y., Ke, S., Zhang, J., Yi, X., and Zheng, Y. (2018). Geoman: Multilevel attention networks for geo-sensory time series prediction. In IJCAI, pages 3428–3434.
[59] Lin, S., Lin, W., Wu, W., Wang, S., and Wang, Y. (2023). Petformer: Longterm time series forecasting via placeholder-enhanced transformer.
[60] Lin, Z., Feng, J., Lu, Z., Li, Y., and Jin, D. (2019). Deepstn+: Context-aware spatial-temporal neural network for crowd flow prediction in metropolis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1020–1027.
[61] Liu, M., Zeng, A., Chen, M., Xu, Z., Lai, Q., Ma, L., and Xu, Q. (2022a). Scinet: Time series modeling and forecasting with sample convolution and interaction. Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS), 2022.
[62] Liu, S., Yu, H., Liao, C., Li, J., Lin, W., Liu, A. X., and Dustdar, S. (2022b).Pyraformer: Low-complexity pyramidal attention for long-range time series modelingand forecasting. In International Conference on Learning Representations.
[63] Liu, Y., Wu, H., Wang, J., and Long, M. (2022c). Non-stationary transformers: Rethinking the stationarity in time series forecasting. arXiv preprint arXiv:2205.14415.
[64] Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., and Zhang, G. (2018). Learning under concept drift: A review. IEEE transactions on knowledge and data engineering, 31(12):2346–2363.
[65] Ma, X., Zhou, C., Kong, X., He, J., Gui, L., Neubig, G., May, J., and Zettlemoyer, L. (2023). Mega: Moving average equipped gated attention. In The Eleventh International Conference on Learning Representations.
[66] Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119.
[67] Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. (2023). A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations.
[68] Oreshkin, B. N., Carpov, D., Chapados, N., and Bengio, Y. (2020). N-beats: Neural basis expansion analysis for interpretable time series forecasting. In International Conference on Learning Representations.
[69] Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744.
[70] Pan, B., Demiryurek, U., and Shahabi, C. (2012). Utilizing real-world transportation data for accurate traffic prediction. In 2012 IEEE 12th International Conference on Data Mining, pages 595–604. IEEE.
[71] Pan, Z., Liang, Y., Wang, W., Yu, Y., Zheng, Y., and Zhang, J. (2019). Urban traffic prediction from spatio-temporal data using deep meta learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1720–1730.
[72] Pan, Z., Zhang, W., Liang, Y., Zhang, W., Yu, Y., Zhang, J., and Zheng, Y. (2020). Spatio-temporal meta learning for urban traffic prediction. IEEE Transactions on Knowledge and Data Engineering.
[73] Passalis, N., Tefas, A., Kanniainen, J., Gabbouj, M., and Iosifidis, A. (2020). Deep adaptive input normalization for time series forecasting. IEEE Transactions on Neural Networks and Learning Systems, 31(9):3760–3765.
[74] Patton, A. J. (2012). A review of copula models for economic time series. Journal of Multivariate Analysis, 110:4–18.
[75] Rangapuram, S. S., Seeger, M. W., Gasthaus, J., Stella, L., Wang, Y., and Januschowski, T. (2018). Deep state space models for time series forecasting. In Advances in neural information processing systems, pages 7785–7794.
[76] Rumelhart, D. E., Hinton, G. E., Williams, R. J., et al. (1985). Learning internal representations by error propagation.
[77] Salinas, D., Flunkert, V., Gasthaus, J., and Januschowski, T. (2019). Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting.
[78] Salinas, D., Flunkert, V., Gasthaus, J., and Januschowski, T. (2020). Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181–1191.
[79] Shabani, M. A., Abdi, A. H., Meng, L., and Sylvain, T. (2023). Scaleformer: Iterative multi-scale refining transformers for time series forecasting. In The Eleventh International Conference on Learning Representations.
[80] Shang, C., Chen, J., and Bi, J. (2021). Discrete graph structure learning for forecasting multiple time series. In International Conference on Learning Representations.
[81] Shao, Z., Zhang, Z., Wang, F., Wei, W., and Xu, Y. (2022). Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 4454–4458.
[82] Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., and Woo, W.-c. (2015). Convolutional lstm network: A machine learning approach for precipitation nowcasting. Advances in neural information processing systems, 28.
[83] Shumway, R. H., Stoffer, D. S., and Stoffer, D. S. (2000). Time series analysis and its applications, volume 3. Springer.
[84] Sims, C. A. (1980). Macroeconomics and reality. Econometrica: journal of the Econometric Society, pages 1–48.
[85] Tobler, W. R. (1970). A computer movie simulating urban growth in the detroit region. Economic geography, 46(sup1):234–240.
[86] Tong, Y., Chen, Y., Zhou, Z., Chen, L., Wang, J., Yang, Q., Ye, J., and Lv, W. (2017). The simpler the better: a unified approach to predicting original taxi demands based on large scale online platforms. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1653–1662.
[87] Ulyanov, D., Vedaldi, A., and Lempitsky, V. S. (2016). Instance normalization: The missing ingredient for fast stylization. CoRR, abs/1607.08022.[van den Oord et al.] van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. Wavenet: A generative model for raw audio. In 9th ISCA Speech Synthesis Workshop, pages 125–125.
[89] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, £., and Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems, pages 5998–6008.
[90] VelickoviÊ, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. (2018). Graph Attention Networks. International Conference on Learning Representations. accepted as poster.
[91] Wang, H., Peng, J., Huang, F., Wang, J., Chen, J., and Xiao, Y. (2023a). MICN: Multi-scale local and global context modeling for long-term series forecasting. In The Eleventh International Conference on Learning Representations.
[92] Wang, Y., Gao, Z., Long, M., Wang, J., and Philip, S. Y. (2018). Predrnn++: Towards a resolution of the deep-in-time dilemma in spatiotemporal predictive learning. In International Conference on Machine Learning, pages 5123–5132. PMLR.
[93] Wang, Y., Long, M., Wang, J., Gao, Z., and Yu, P. S. (2017). Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms. Advances in neural information processing systems, 30.
[94] Wang, Y., Smola, A., Maddix, D., Gasthaus, J., Foster, D., and Januschowski, T. (2019a). Deep factors for forecasting. In International Conference on Machine Learning, pages 6607–6617. PMLR.
[95] Wang, Y., Wu, H., Zhang, J., Gao, Z., Wang, J., Philip, S. Y., and Long, M. (2022a). Predrnn: A recurrent neural network for spatiotemporal predictive learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):2208–2225.
[96] Wang, Y., Yin, H., Chen, H., Wo, T., Xu, J., and Zheng, K. (2019b). Origin-destination matrix prediction via graph convolution: a new perspective of passenger demand modeling. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1227–1235.
[97] Wang, Z., Nie, Y., Sun, P., Nguyen, N. H., Mulvey, J., and Poor, H. V. (2023b). St-mlp: A cascaded spatio-temporal linear framework with channel-independence strategy for traffic forecasting. arXiv preprint arXiv:2308.07496.
[98] Wang, Z., Xu, X., Trajcevski, G., Zhang, W., Zhong, T., and Zhou, F. (2022b). Learning latent seasonal-trend representations for time series forecasting. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K., editors, Advances in Neural Information Processing Systems.
[99] Woo, G., Liu, C., Sahoo, D., Kumar, A., and Hoi, S. (2021). Cost: Contrastive learning of disentangled seasonal-trend representations for time series forecasting. In International Conference on Learning Representations.
[100] Woo, G., Liu, C., Sahoo, D., Kumar, A., and Hoi, S. (2022a). Deeptime: Deep time index meta-learning for non-stationary time-series forecasting. arXiv preprint arXiv:2207.06046.
[101] Woo, G., Liu, C., Sahoo, D., Kumar, A., and Hoi, S. (2022b). Etsformer: Exponential smoothing transformers for time-series forecasting.
[102] Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., and Long, M. (2023). Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations.
[103] Wu, H., Xu, J., Wang, J., and Long, M. (2021). Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Thirty-Fifth Conference on Neural Information Processing Systems.
[104] Wu, Z., Pan, S., Long, G., Jiang, J., Chang, X., and Zhang, C. (2020). Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 753–763.
[105] Wu, Z., Pan, S., Long, G., Jiang, J., and Zhang, C. (2019). Graph wavenet for deep spatial-temporal graph modeling. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 1907–1913. AAAI Press.
[106] Xu, M., Dai, W., Liu, C., Gao, X., Lin, W., Qi, G.-J., and Xiong, H. (2020). Spatial-temporal transformer networks for traffic flow forecasting. arXiv preprint arXiv:2001.02908.
[107] Xu, Z., Li, Z., Guan, Q., Zhang, D., Li, Q., Nan, J., Liu, C., Bian, W., and Ye, J. (2018). Large-scale order dispatch in on-demand ride-hailing platforms: A learning and planning approach. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 905–913.
[108] Xue, W., Zhou, T., Wen, Q., Gao, J., Ding, B., and Jin, R. (2023). Make transformer great again for time series forecasting: Channel aligned robust dual transformer.
[109] Yang, S., Liu, J., and Zhao, K. (2021). Space meets time: Local spacetime neural network for traffic flow forecasting. In 2021 IEEE International Conference on Data Mining (ICDM), pages 817–826. IEEE.
[110] Yao, H., Liu, Y., Wei, Y., Tang, X., and Li, Z. (2019a). Learning from multiple cities: A meta-learning approach for spatial-temporal prediction. In The World Wide Web Conference, pages 2181–2191.
[111] Yao, H., Tang, X., Wei, H., Zheng, G., and Li, Z. (2019b). Revisiting spatial-temporal similarity: A deep learning framework for traffic prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 5668–5675.
[112] Yao, H., Wu, F., Ke, J., Tang, X., Jia, Y., Lu, S., Gong, P., Ye, J., and Li, Z. (2018). Deep multi-view spatial-temporal network for taxi demand prediction. In Thirty-Second AAAI Conference on Artificial Intelligence.
[113] Ye, J., Sun, L., Du, B., Fu, Y., Tong, X., and Xiong, H. (2019). Co-prediction of multiple transportation demands based on deep spatio-temporal neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 305–313. ACM.
[114] Yu, B., Yin, H., and Zhu, Z. (2018). Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 3634–3640.
[115] Yu, C., Wang, F., Shao, Z., Sun, T., Wu, L., and Xu, Y. (2023). Dsformer: A double sampling transformer for multivariate time series long-term prediction.
[116] Zeng, A., Chen, M., Zhang, L., and Xu, Q. (2022). Are transformers effective for time series forecasting? arXiv preprint arXiv:2205.13504.
[117] Zeng, A., Chen, M., Zhang, L., and Xu, Q. (2023). Are transformers effective for time series forecasting?
[118] Zhang, J., Zheng, Y., and Qi, D. (2017). Deep spatio-temporal residual networks for citywide crowd flows prediction. In Thirty-First AAAI Conference on Artificial Intelligence.
[119] Zhang, J., Zheng, Y., Qi, D., Li, R., and Yi, X. (2016). Dnn-based prediction model for spatio-temporal data. In Proceedings of the 24th ACM SIGSPATIAL international conference on advances in geographic information systems, pages 1–4.
[120] Zhang, J., Zheng, Y., Sun, J., and Qi, D. (2019). Flow prediction in spatiotemporal networks based on multitask deep learning. IEEE Transactions on Knowledge and Data Engineering, 32(3):468–478.
[121] Zhang, W., Liu, H., Liu, Y., Zhou, J., Xu, T., and Xiong, H. (2020). Semisupervised city-wide parking availability prediction via hierarchical recurrent graph neural network. IEEE Transactions on Knowledge and Data Engineering, 34(8):3984–3996.
[122] Zhang, X., Huang, C., Xu, Y., Xia, L., Dai, P., Bo, L., Zhang, J., and Zheng, Y. (2021). Traffic flow forecasting with spatial-temporal graph diffusion network. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 15008–15015.
[123] Zhang, X., Jin, X., Gopalswamy, K., Gupta, G., Park, Y., Shi, X., Wang, H., Maddix, D. C., and Wang, B. (2022). First de-trend then attend: Rethinking attention for time-series forecasting. In NeurIPS’22 Workshop on All Things Attention: Bridging Di erent Perspectives on Attention.
[124] Zhang, Y. and Yan, J. (2023). Crossformer: Transformer utilizing crossdimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations.
[125] Zhang, Z., Wang, X., and Gu, Y. (2023). Sageformer: Series-aware graphenhanced transformers for multivariate time series forecasting.
[126] Zhao, J., Huang, F., Lv, J., Duan, Y., Qin, Z., Li, G., and Tian, G. (2020). Do rnn and lstm have long memory? In International Conference on Machine Learning, pages 11365–11375. PMLR.
[127] Zhao, L., Song, Y., Zhang, C., Liu, Y., Wang, P., Lin, T., Deng, M., and Li, H. (2019). T-gcn: A temporal graph convolutional network for traffic prediction. IEEE Transactions on Intelligent Transportation Systems, 21(9):3848–3858.
[128] Zheng, C., Fan, X., Wang, C., and Qi, J. (2020a). Gman: A graph multiattention network for traffic prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1234–1241.
[129] Zheng, H., Lin, F., Feng, X., and Chen, Y. (2020b). A hybrid deep learning model with attention-based conv-lstm networks for short-term traffic flow prediction. IEEE Transactions on Intelligent Transportation Systems, 22(11):6910–6920.
[130] Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. (2021). Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of AAAI.
[131] Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., and Jin, R. (2022). Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning, pages 27268–27286. PMLR.
[132] Zhou, X., Shen, Y., Zhu, Y., and Huang, L. (2018). Predicting multi-step citywide passenger demands using attention-based neural networks. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pages 736–744.
[133] Zhou, Z.-H. (2022). Open-environment machine learning. National Science Review, 9(8):nwac123.
[134] éliobait˙e, I., Pechenizkiy, M., and Gama, J. (2016). An overview of concept drift applications. Big data analysis: new algorithms for a new society, pages 91–114.
[135] Zonoozi, A., Kim, J.-j., Li, X.-L., and Cong, G. (2018). Periodic-crn: A convolutional recurrent model for crowd density prediction with recurring periodic patterns. In IJCAI, pages 3732–3738.