基于域适应的迁移学习在交通预测问题中的研究

在智慧城市研究中，如何进行准确和稳健的交通流量预测是一个关键性的问题。近几年来，许多基于大数据的时空预测方法在交通预测任务上表现优越。然而，大数据驱动的模型需要通过大量的历史数据训练才能保证较好的泛化性能，而城市之间的发展存在较大差异，中小城市往往缺乏足量的数据训练交通预测模型。迁移学习作为一种尝试通过知识迁移来提升模型在目标域学习性能的方法，被认为是解决这种数据匮乏的一种有效方法。也因此，如何在城市之间有效进行知识迁移成为一个值得研究的课题。在本文中，我们将研究如何在城市的交通数据之间进行知识迁移。建立一个具有这种在城市道路网络之间传递知识的能力的模型非常具有挑战性。例如，以往的研究主要集中在利用从单个城市收集的数据建立预测模型，很少考虑如何跨城市来进行数据的传递和知识的分享；因此对于交通特征差异较大的城市，如果我们直接将源城市的训练模型迁移到目标域城市，可能会导致强烈的负面迁移效果。为了解决这些问题，我们提出了一种基于域适应的跨城市交通预测迁移学习模型。本文提出的模型通过使用共享的时空编码器网络结构来实现多个城市之间的知识传递。为了避免训练过程中跨域学习导致的冲突，本文提出跨城市预测模型通过利用多重梯度下降的方法来动态调整损失的权重以在达到全局最优的同时避免过度的负迁移效果，同时使用训练好的跨城市预测模型针对单个需要预测的交通节点进行调参来进一步提升模型效果。此外，本文还提出了一种基于弱监督的学习方法用来辅助域适应迁移学习的训练。我们利用六个城市数据集进行了实验，通过实验结果体现该模型的性能优势，以及迁移学习方法对于解决交通预测任务的有效性。

[1] SILVA B N, KHAN M, HAN K. Towards Sustainable Smart Cities: A Review Of Trends, Architectures, Components, And Open Challenges In Smart Cities[J]. Sustainable Cities And Society, 2018, 38: 697-713.
[2] ZHU L, YU F R, WANG Y, et al. Big Data Analytics In Intelligent Transportation Systems: A Survey[J]. IEEE Transactions On Intelligent Transportation Systems, 2018, 20(1): 383-398.
[3] FERDOWSI A, CHALLITA U, SAAD W. Deep Learning For Reliable Mobile Edge Analytics In Intelligent Transportation Systems: An Overview[J]. IEEE Vehicular Technology Magazine, 2019, 14(1): 62-70.
[4] 余辰, 张丽娟, 金海. 大数据驱动的智能交通系统研究进展与趋势[J]. 物联网学报, 2018, 2(1): 56-63.
[5] 李国勇. “十四五”城市轨道交通的发展与安全[J]. 城市轨道交通, 2021.
[6] WILSON G, COOK D J. A Survey Of Unsupervised Deep Domain Adaptation[J]. ACM Transactions On Intelligent Systems And Technology (TIST), 2020, 11(5): 1-46.
[7] 任远. 城市病和高密度城市的精细化管理[J]. 社会科学, 2018(5): 76-82.
[8] 沈洁, 张可云. 中国大城市病典型症状诱发因素的实证分析[J]. 地理科学进展, 2020, 39(1): 1-12.
[9] 高柯夫, 孙宏彬, 王楠, 等. “互联网 +”智能交通发展战略研究[J]. 中国工程科学, 2020,22(4): 101-105.
[10] 徐凤. 城市交通运行效率指标体系的构建与应用[J]. 交通科技与经济, 2018, 20(2): 18-22.
[11] 拉萨主干道通行时长降低 26.%，科技打造高原城市“治堵范本”[EB/OL]. 2021.
[12] SMITH B L, DEMETSKY M J. Traffic Flow Forecasting: Comparison Of Modeling Approaches[J]. journal Of Transportation Engineering, 1997, 123(4): 261-266.
[13] ZHAO L, SONG Y, ZHANG C, et al. T-gcn: A Temporal Graph Convolutional Network For Traffic Prediction[J]. IEEE Transactions On Intelligent Transportation Systems, 2019.
[14] YAO H, LIU Y, WEI Y, et al. Learning From Multiple Cities: A Meta-learning Approach For Spatial-temporal Prediction[C]//The World Wide Web Conference. 2019: 2181-2191.
[15] WANG L, GENG X, MA X, et al. Cross-city Transfer Learning For Deep Spatio-temporal Prediction[C]//Proceedings Of The 28th International Joint Conference On Artificial Intelligence. AAAI Press, 2019: 1893-1899.
[16] MALLICK T, BALAPRAKASH P, RASK E, et al. Transfer Learning With Graph Neural Networks For Short-Term Highway Traffic Forecasting[J]. arXiv:2004.08038, 2020.
[17] YU B, LEE Y, SOHN K. Forecasting Road Traffic Speeds by Considering Area-wide Spatiotemporal Dependencies based On A Graph Convolutional Neural Network (GCN)[J]. Transportation Research Part C: Emerging Technologies, 2020, 114: 189-204.
[18] HOCHREITER S, SCHMIDHUBER J. Long Short-term Memory[J]. Neural Computation,1997, 9(8): 1735-1780.
[19] CHUNG J, GULCEHRE C, CHO K, et al. Empirical Evaluation Of Gated Recurrent Neural Networks On Sequence Modeling[C]//NIPS 2014 Workshop On Deep Learning, December2014. 2014.
[20] 张溪. 大数据下智能交通系统的发展综述[J]. 信息与电脑, 2019(1): 17-19.
[21] ABDULHAFEDH A. Road Traffic Crash Data: An Overview On Sources, Problems, AndCollection Methods[J]. journal Of Transportation Technologies, 2017, 7(2): 206-219.
[22] SOH J W, CHO S, CHO N I. Meta-transfer Learning For Zero-shot Super-resolution[C]// Proceedings Of The IEEE/CVF Conference On Computer Vision And Pattern Recognition. 2020: 3516-3525.
[23] LI X, GRANDVALET Y, DAVOINE F, et al. Transfer Learning In Computer Vision Tasks: Remember Where You Come From[J]. Image And Vision Computing, 2020, 93: 103853.
[24] KIM Y, SOH J W, PARK G Y, et al. Transfer Learning From Synthetic To Real-noise Denoising With Adaptive Instance Normalization[C]//Proceedings Of The IEEE/CVF Conference On Computer Vision And Pattern Recognition. 2020: 3482-3492.
[25] BLLOSHMI R, TRIPODI R, NAVIGLI R. Enabling Cross-Lingual AMR Parsing With Transfer Learning Techniques[C]//Proceedings Of The 2020 Conference On Empirical Methods In Natural Language Processing (EMNLP). 2020: 2487-2500.
[26] HOULSBY N, GIURGIU A, JASTRZEBSKI S, et al. Parameter-efficient Transfer Learning For NLP[C]//International Conference On Machine Learning. PMLR, 2019: 2790-2799.
[27] RUDER S, PETERS M E, SWAYAMDIPTA S, et al. Transfer Learning In Natural Language Processing[C]//Proceedings Of The 2019 Conference Of The North American Chapter Of The Association For Computational Linguistics: Tutorials. 2019: 15-18.
[28] WANG J, HUANG P, ZHAO H, et al. Billion-scale Commodity Embedding For E-commerce Recommendation In Alibaba[C]//Proceedings Of The 24th ACM SIGKDD International Conference On Knowledge Discovery & Data Mining. 2018: 839-848.
[29] YE J, CHENG H, ZHU Z, et al. Predicting Positive And Negative Links In Signed Social Networks by Transfer Learning[C]//Proceedings Of The 22nd International Conference On World Wide Web. 2013: 1477-1488.
[30] 范苍宁, 刘鹏, 肖婷, 等. 深度域适应综述: 一般情况与复杂情况[J]. 自动化学报, 2021, 47(3): 515-548.
[31] NAGY A M, SIMON V. Survey On Traffic Prediction In Smart Cities[J]. Pervasive And MobileComputing, 2018, 50: 148-163.
[32] MANIBARDO E L, LAÑA I, DEL SER J. Deep Learning For Road Traffic Forecasting: Does It Make A Difference?[J]. IEEE Transactions On Intelligent Transportation Systems, 2021.
[33] CUI Z, HENRICKSON K, KE R, et al. Traffic Graph Convolutional Recurrent Neural Network: A Deep Learning Framework For Network-scale Traffic Learning And Forecasting[J]. IEEE Transactions On Intelligent Transportation Systems, 2019, 21(11): 4883-4894
[34] ORESHKIN B N, CARPOV D, CHAPADOS N, et al. N-BEATS: Neural basis Expansion Analysis For Interpretable Time Series Forecasting[C]//International Conference On Learning Representations. 2020.
[35] ZHOU T, HAN G, XU X, et al. 𝛿-agree AdaBoost Stacked Autoencoder For Short-term Traffic Flow Forecasting[J]. Neurocomputing, 2017, 247: 31-38.
[36] LI Y, YU R, SHAHABI C, et al. Diffusion Convolutional Recurrent Neural Network: Data Driven Traffic Forecasting[C]//International Conference On Learning Representations. 2018.
[37] XIE Y, ZHANG Y, YE Z. Short-term Traffic volume Forecasting Using Kalman Filter With Discrete Wavelet Decomposition[J]. Computer-Aided Civil And Infrastructure Engineering, 2007, 22(5): 326-334.
[38] CHEN Z, WEN J, GENG Y. Predicting Future Traffic Using Hidden Markov Models[C]//2016 IEEE 24th International Conference On Network Protocols (ICNP). IEEE, 2016: 1-6.
[39] ZHANG C, YU J J, LIU Y. Spatial-temporal Graph Attention Networks: A Deep Learning Approach For Traffic Forecasting[J]. IEEE Access, 2019, 7: 166246-166256.
[40] YOU K, KOU Z, LONG M, et al. Co-tuning For Transfer Learning[J]. Advances In Neural Information Processing Systems, 2020, 33.
[41] SAITO K, WATANABE K, USHIKU Y, et al. Maximum Classifier Discrepancy For Unsupervised Domain Adaptation[C]//Proceedings Of The IEEE/CVF Conference On Computer Vision And Pattern Recognition. 2018: 3723-3732.
[42] TZENG E, HOFFMAN J, SAENKO K, et al. Adversarial Discriminative Domain Adaptation [C]//Proceedings Of The IEEE/CVF Conference On Computer Vision And Pattern Recognition. 2017: 7167-7176.
[43] LONG M, CAO Y, WANG J, et al. Learning Transferable Features With Deep Adaptation Networks[C]//International Conference On Machine Learning. PMLR, 2015: 97-105.
[44] SUN B, SAENKO K. Deep Coral: Correlation Alignment For Deep Domain Adaptation[C]// European Conference On Computer Vision. Springer, 2016: 443-450.
[45] LI C L, CHANG W C, CHENG Y, et al. Mmd Gan: Towards Deeper Understanding Of Moment Matching Network[C]//Advances In Neural Information Processing Systems: volume 30. 2017.
[46] LI X, HU Y, ZHENG J, et al. Central Moment Discrepancy based Domain Adaptation For Intelligent bearing Fault Diagnosis[J]. Neurocomputing, 2021, 429: 12-24.
[47] PURUSHOTHAM S, CARVALHO W, NILANON T, et al. Variational Recurrent Adversarial Deep Domain Adaptation[C]//International Conference On Learning Representations 2017. 2017.
[48] WILSON G, DOPPA J R, COOK D J. Multi-source Deep Domain Adaptation With Weak Supervision For Time-series Sensor Data[C]//Proceedings Of The 26th ACM SIGKDD International Conference On Knowledge Discovery & Data Mining. 2020: 1768-1778.
[49] ZOU X, ZHANG S, ZHANG C, et al. Long-term Origin-Destination Demand Prediction With Graph Deep Learning[J]. IEEE Transactions On Big Data, 2021.
[50] YU J J. Citywide Traffic Speed Prediction: A Geometric Deep Learning Approach[J]. Knowledge-Based Systems, 2021, 212: 106592.
[51] CHEN H, GRANT-MULLER S, MUSSONE L, et al. A Study Of Hybrid Neural NetworkApproaches And The Effects Of Missing Data On Traffic Forecasting[J]. Neural Computing & Applications, 2001, 10(3): 277-286.
[52] WU P, XU L, HUANG Z. Imputation Methods Used In Missing Traffic Data: A Literature Review[C]//International Symposium On Intelligence Computation And Applications. Springer, 2019: 662-677.
[53] REDMAN T C. If Your Data Is bad, Your Machine Learning Tools Are Useless[J]. Harvard Business Review, 2018, 2.
[54] LAÑA I, OLABARRIETA I I, VÉLEZ M, et al. On The Imputation Of Missing Data For Road Traffic Forecasting: New Insights And Novel Techniques[J]. Transportation Research Part C: Emerging Technologies, 2018, 90: 18-33.
[55] VLAHOGIANNI E I, KARLAFTIS M G, GOLIAS J C. Short-term Traffic Forecasting: Where We Are And Where We’re Going[J]. Transportation Research Part C: Emerging Technologies, 2014, 43: 3-19.
[56] YU R, LI Y, SHAHABI C, et al. Deep Learning: A Generic Approach For Extreme Condition Traffic Forecasting[C]//Proceedings Of The 2017 SIAM International Conference On Data Mining. SIAM, 2017: 777-785.
[57] CHEN X, HE Z, SUN L. A Bayesian Tensor Decomposition Approach For Spatiotemporal Traffic Data Imputation[J]. Transportation Research Part C: Emerging Technologies, 2019, 98: 73-84.
[58] Yanjie Duan, Yisheng Lv, Wenwen Kang, et al. A Deep Learning based Approach For Traffic Data Imputation[C]//17th International IEEE Conference On Intelligent Transportation Systems (ITSC). 2014: 912-917.
[59] SMITH B L, SCHERER W T, CONKLIN J H. Exploring Imputation Techniques For Missing Data In Transportation Management Systems[J]. Transportation Research Record, 2003, 1836 (1): 132-142.
[60] GUO Q, QIU X, LIU P, et al. Star-Transformer[C]//Proceedings Of The 2019 Conference Of The North American Chapter Of The Association For Computational Linguistics: Human Language Technologies, volume 1 (Long And Short Papers). 2019: 1315-1325.
[61] DAI Z, YANG Z, YANG Y, et al. Transformer-Xl: Attentive Language Models Beyond A Fixed-Length Context[J]. The 57th Annual Meeting Of The Association For Computational Linguistics, 2019: 2978-2988.
[62] WU Z, LIU Z, LIN J, et al. Lite Transformer With Long-Short Range Attention[C]//International Conference On Learning Representations. 2019.
[63] YU Y, SI X, HU C, et al. A Review Of Recurrent Neural Networks: LSTM Cells And Network Architectures[J]. Neural Computation, 2019, 31(7): 1235-1270.
[64] CHEN Z, HUANG Y, LI J, et al. Improving Mask Learning Based Speech Enhancement System With Restoration Layers And Residual Connection.[C]//INTERSPEECH. 2017: 3632-3636.
[65] XU J, SUN X, ZHANG Z, et al. Understanding And Improving Layer Normalization[C]// Advances In Neural Information Processing Systems: volume 32. 2019
[66] TAKASE S, OKAZAKI N. Positional Encoding To Control Output Sequence Length[C]//Proceedings Of The 2019 Conference Of The North American Chapter Of The Association For Computational Linguistics: Human Language Technologies, volume 1 (Long And Short Papers). 2019: 3999-4004.
[67] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training Of Deep Bidirectional Transformers For Language Understanding[C]//Proceedings Of The 2019 Conference Of The North American Chapter Of The Association For Computational Linguistics: Human Language Technologies, volume 1 (Long And Short Papers). 2019: 4171-4186.
[68] GAL Y, GHAHRAMANI Z. Dropout As A bayesian Approximation: Representing Model Uncertainty In Deep Learning[C]//International Conference On Machine Learning. PMLR, 2016: 1050-1059.
[69] WU R, ZHANG A, ILYAS I, et al. Attention-based Learning For Missing Data Imputation In HoloClean[J]. Proceedings Of Machine Learning And Systems, 2020, 2: 307-325.
[70] VIG J, BELINKOV Y. Analyzing The Structure Of Attention In A Transformer Language Model[C]//Proceedings Of The 2019 ACL Workshop BlackboxNLP: Analyzing And Interpreting Neural Networks For NLP. 2019: 63-76.
[71] VASWANI A, SHAZEER N, PARMAR N, et al. Attention Is All You Need[C]//Advances In Neural Information Processing Systems: volume 30. 2017.
[72] KINGMA D P, BA J. Adam: A Method For Stochastic Optimization[J]. arXiv Preprint ArXiv:1412.6980, 2014.
[73] 刘静, 关伟. 交通流预测方法综述[J]. 公路交通科技, 2004, 21(3): 82-85.
[74] SALAKHUTDINOV R, MNIH A. Bayesian Probabilistic Matrix Factorization Using MarkovChain Monte Carlo[C]//Proceedings Of The 25th International Conference On Machine Learning. 2008: 880-887.
[75] YOON J, JORDON J, SCHAAR M. Gain: Missing Data Imputation Using Generative Adversarial Nets[C]//International Conference On Machine Learning. PMLR, 2018: 5689-5698.
[76] XIAO Z, FU X, ZHANG L, et al. Traffic Pattern Mining And Forecasting Technologies In Maritime Traffic Service Networks: A Comprehensive Survey[J]. IEEE Transactions On Intelligent Transportation Systems, 2019, 21(5): 1796-1825.
[77] YAO H, TANG X, WEI H, et al. Revisiting Spatial-temporal Similarity: A Deep Learning Framework For Traffic Prediction[C]//Proceedings Of The AAAI Conference On Artificial Intelligence: volume 33. 2019: 5668-5675.
[78] LIU X, LIU X, WANG Y, et al. Detecting Anomaly In Traffic Flow From Road Similarity Analysis[C]//International Conference On Web-Age Information Management. Springer, 2016:92-104.
[79] LIU Q, CAI Y, JIANG H, et al. Traffic State Spatial-temporal Characteristic Analysis And Short-term Forecasting based On Manifold Similarity[J]. IEEE Access, 2018, 6: 9690-9702.
[80] ERMAGUN A, LEVINSON D. Spatiotemporal Traffic Forecasting: Review And Proposed Directions[J]. Transport Reviews, 2018, 38(6): 786-814
[81] WU Z, PAN S, LONG G, et al. Connecting The Dots: Multivariate Time Series Forecasting WithGraph Neural Networks[C]//Proceedings Of The 26th ACM SIGKDD International Conference On Knowledge Discovery & Data Mining. 2020: 753-763.
[82] SHANG C, CHEN J, BI J. Discrete Graph Structure Learning For Forecasting Multiple Time Series[C]//International Conference On Learning Representations. 2021.
[83] WU Z, PAN S, LONG G, et al. Graph WaveNet For Deep Spatial-Temporal Graph Modeling.[C]//Proceedings Of The 28th International Joint Conference On Artificial Intelligence. 2019.
[84] JIANG W, LUO J. Graph Neural Network For Traffic Forecasting: A Survey[J]. arXiv:2101.11174, 2021.
[85] PEROZZI B, AL-RFOU R, SKIENA S. Deepwalk: Online Learning Of Social Representations [C]//Proceedings Of The 20th ACM SIGKDD International Conference On Knowledge Discov ery And Data Mining. 2014: 701-710.
[86] SUTSKEVER I, VINYALS O, LE Q V. Sequence To Sequence Learning With Neural Networks [C]//Advances In Neural Information Processing Systems: volume 27. 2014: 3104-3112.
[87] YU B, YIN H, ZHU Z. Spatio-temporal Graph Convolutional Networks: A Deep Learning Framework For Traffic Forecasting[C]//Proceedings Of The 27th International Joint Conference On Artificial Intelligence. 2018: 3634-3640.
[88] WU Z, PAN S, LONG G, et al. Graph WaveNet For Deep Spatial-Temporal Graph Modeling [C]//Proceedings Of The 28th International Joint Conference On Artificial Intelligence. 2019:1907-1913.
[89] XU M, DAI W, LIU C, et al. Spatial-temporal Transformer Networks For Traffic Flow Forecasting[J]. arXiv Preprint ArXiv:2001.02908, 2020.
[90] ZHANG J, ZHENG Y, QI D, et al. DNN-based Prediction Model For Spatio-Temporal Data [C]//Proceedings Of The 24th ACM SIGSPATIAL International Conference On Advances In Geographic Information Systems. 2016: 1-4.
[91] PEETA S, ZILIASKOPOULOS A K. Foundations Of Dynamic Traffic Assignment: The Past, The Present And The Future[J]. Networks And Spatial Economics, 2001, 1(3): 233-265.
[92] LUCAS B, PELLETIER C, SCHMIDT D, et al. A Bayesian-inspired, Deep Learning-based, Semi-supervised Domain Adaptation Technique For Land Cover Mapping[J]. Machine Learn ing, 2021: 1-33.
[93] ARIEF-ANG I B, SALIM F D, HAMILTON M. DA-HOC: Semi-supervised Domain Adap tation For Room Occupancy Prediction Using CO2 Sensor Data[C]//Proceedings Of The 4th ACM International Conference On Systems For Energy-Efficient Built Environments. 2017: 1-10.
[94] DENG J, ZHANG Z, EYBEN F, et al. Autoencoder-based Unsupervised Domain Adaptation For Speech Emotion Recognition[J]. IEEE Signal Processing Letters, 2014, 21(9): 1068-1072.
[95] HUANG S W, LIN C T, CHEN S P, et al. Auggan: Cross Domain Adaptation With Gan based Data Augmentation[C]//Proceedings Of The European Conference On Computer Vision (ECCV). 2018: 718-731.
[96] ABDELWAHAB O, ELMAGHRABY A. Deep Learning-based Vs. Markov Chain-based Text Generation For Cross Domain Adaptation For Sentiment Classification[C]//2018 IEEE International Conference On Information Reuse And Integration (IRI). IEEE, 2018: 252-255.
[97] RIETZLER A, STABINGER S, OPITZ P, et al. Adapt Or Get Left Behind: Domain Adaptation Through BERT Language Model Finetuning For Aspect-Target Sentiment Classification[C]// Proceedings Of The 12th Language Resources And Evaluation Conference. 2020: 4933-4941.
[98] GANIN Y, USTINOVA E, AJAKAN H, et al. Domain-adversarial Training Of Neural Networks [J]. The journal Of Machine Learning Research, 2016, 17(1): 2096-2030.
[99] MACLAURIN D, DUVENAUD D, ADAMS R. Gradient-based Hyperparameter Optimiza tion Through Reversible Learning[C]//International Conference On Machine Learning. PMLR,2015: 2113-2122.
[100] SENER O, KOLTUN V. Multi-task Learning As Multi-objective Optimization[C]//Proceedings Of The 32nd International Conference On Neural Information Processing Systems. 2018: 525- 536.
[101] LUO Z Q, ZHANG S. On Extensions Of The Frank-Wolfe Theorems[J]. Computational Opti mization And Applications, 1999, 13(1): 87-110.
[102] YANG C, GIDOFALVI G. Fast Map Matching, An Algorithm Integrating Hidden Markov Model With Precomputation[J]. International journal Of Geographical Information Science, 2018, 32(3): 547 - 570.
[103] WANG L, GENG X, MA X, et al. Cross-city Transfer Learning For Deep Spatio-temporal Prediction[J]. arXiv Preprint ArXiv:1802.00386, 2018.
[104] CHEN D, LIN Y, LI W, et al. Measuring And Relieving The Over-smoothing Problem For Graph Neural Networks From The Topological View[C]//Proceedings Of The AAAI Conference On Artificial Intelligence: volume 34. 2020: 3438-3445.