Data Engineering for Smart City Traffic Prediction
Keywords:
Traffic flow prediction,Smart cities,Data engineering,Spatio-temporal data,Intelligent transportation systems (ITS),Big data analytics,Internet of Things (IoT),Real-time data processing,Data pipelines,Feature engineering,Graph neural networks (GNN),Deep learning,Time series forecasting,Data fusion (multi-source integration),Edge computing.Abstract
Traffic flow prediction is a crucial component of developing intelligent transportation systems in smart cities. The core goal is to investigate the data engineering approaches and methodologies that support traffic flow prediction. Structured but diverse traffic flow data sets generated by sensor networks and other sources are ingested, processed, and stored in data platforms that enable operational real-time or near-real-time predictions and alternative exploratory offline analyses. Effective data engineering process solutions for traffic flow prediction require the careful design of all stages—from data sources to model application—because the dynamic and still little-understood nature of traffic flow data makes dedicated traffic prediction models sensitive to a variety of factors, including data quality, time frame, spatial resolution, external data, feature representation, and algorithm selection. Consideration of these factors can yield appropriate solutions across data scenario alternatives: data from a limited number of senors, real-time prediction with external data to better represent special events, online learning for concept drift adaptation, and feature engineering with new perspectives or resources.
Traffic flow prediction represents an operational application in the complex and multidisciplinary scenarios of a smart city. Intelligent transport systems rely on timely and accurate real-time predictions to optimize vehicle distribution, reduce waiting times, increase passenger satisfaction, and enable vehicle tracking. Such predictions also support external decision-making processes that require support from an intelligent system or subsystem. Usually classified as time-series forecasting, traffic flow prediction aims to inferring future values of a time-ordered series generated by one or more object through a range sensor, such as microwave, loop, infrared, or video camera sensor. The tasks for traffic flow prediction comprises sensor networks and the incoming traffic flow data streams, as well as multiple connected external data source that contribute to broaden the representation of predicted-related phenomena—e.g. weather conditions, official event schedule—and support data fusion.
References
1. Jia, T., & Yan, P. (2021). Predicting citywide road traffic flow using deep spatiotemporal neural networks. IEEE Transactions on Intelligent Transportation Systems, 22(5), 3101–3111.
2. Li, G., Knoop, V. L., & van Lint, H. (2021). Multistep traffic forecasting by dynamic graph convolution: Interpretations of real-time spatial correlations. Transportation Research Part C: Emerging Technologies, 128, 103185.
3. Li, J., Guo, F., Sivakumar, A., Dong, Y., & Krishnan, R. (2021). Transferability improvement in short-term traffic prediction using stacked LSTM network. Transportation Research Part C: Emerging Technologies, 124, 102977.
4. Inala, R. (2023). Big Data Architectures for Modernizing Customer Master Systems in Group Insurance and Retirement Planning. Educational Administration: Theory and Practice, 29(4), 5493-5505.
5. Manibardo, E. L., Laña, I., & del Ser, J. (2022). Deep learning for road traffic forecasting: Does it make a difference? IEEE Transactions on Intelligent Transportation Systems, 23(7), 6164–6188.
6. Wu, S. (2022). Spatiotemporal dynamic forecasting and analysis of regional traffic flow in urban road networks using deep learning convolutional neural network. IEEE Transactions on Intelligent Transportation Systems, 23(2), 1607–1615.
7. Ali, A., Zhu, Y., & Zakarya, M. (2022). Exploiting dynamic spatio-temporal graph convolutional neural networks for citywide traffic flows prediction. Neural Networks, 145, 233–247.
8. Lee, K., & Rhee, W. (2022). DDP-GCN: Multi-graph convolutional network for spatiotemporal traffic forecasting. Transportation Research Part C: Emerging Technologies, 134, 103466.
9. Afrin, T., & Yodo, N. (2022). A long short-term memory-based correlated traffic data prediction framework. Knowledge-Based Systems, 237, 107755.
10. Ren, Y., Jiang, H., Ji, N., & Yu, H. (2022). TBSM: A traffic burst-sensitive model for short-term prediction under special events. Knowledge-Based Systems, 240, 108120.
11. Wang, Y., Jing, C., & Guo, T. (2022). Attention based spatiotemporal graph attention networks for traffic flow forecasting. Information Sciences, 607, 869–883.
12. Wang, Q., Jiang, H., Qiu, M., & others. (2022). TGAE: Temporal graph autoencoder for travel forecasting. IEEE Transactions on Intelligent Transportation Systems.
13. Xue, R., Zhao, S., & Han, F. (2022). An embedding-driven multi-hop spatio-temporal attention network for traffic prediction. IEEE Transactions on Intelligent Transportation Systems.
14. Guo, C., Chen, C.-H., Hwang, F.-J., & others. (2022). Fast spatiotemporal learning framework for traffic flow forecasting. IEEE Transactions on Intelligent Transportation Systems.
15. Fang, Y., Zhao, F., Qin, Y., Luo, H., & Wang, C. (2022). Learning all dynamics: Traffic forecasting via locality-aware spatio-temporal joint Transformer. IEEE Transactions on Intelligent Transportation Systems, 23(12), 23433–23446.
16. Li, M., Tang, Y., & Ma, W. (2023). Few-sample traffic prediction with graph networks using locale as relational inductive biases. IEEE Transactions on Intelligent Transportation Systems, 24(2), 1894–1908.
17. Chen, Y., Li, K., Yeo, C. K., & Li, K. (2023). Traffic forecasting with graph spatial–temporal position recurrent network. Neural Networks, 162, 340–349.
18. Huang, Y., Song, X., Zhu, Y., Zhang, S., & others. (2023). Traffic prediction with transfer learning: A mutual information-based approach. IEEE Transactions on Intelligent Transportation Systems, 24(8), 8236–8252.
19. Aitha, A. R. (2024). Generative AI-Powered Fraud Detection in Workers' Compensation: A DevOps-Based Multi-Cloud Architecture Leveraging, Deep Learning, and Explainable AI. Computer Fraud and Security.
20. Zheng, C., Fan, X., Pan, S., Jin, H., & others. (2023). Spatio-temporal joint graph convolutional networks for traffic forecasting. IEEE Transactions on Knowledge and Data Engineering, 36(1), 372–385.
21. Wang, C., Zuo, K., Zhang, S., Lei, H., & others. (2023). PFNet: Large-scale traffic forecasting with progressive spatio-temporal fusion. IEEE Transactions on Intelligent Transportation Systems, 24(12), 14580–14597.
22. Jiang, W., & Luo, J. (2023). Graph neural network for traffic forecasting: A survey. Expert Systems with Applications, 207, 117921.
