ST-LaneNet: Lane Line Detection Method Based on Swin Transformer and LaneNet
Tóm tắt
The advancement of autonomous driving heavily relies on the ability to accurate lane lines detection. As deep learning and computer vision technologies evolve, a variety of deep learning-based methods for lane line detection have been proposed by researchers in the field. However, owing to the simple appearance of lane lines and the lack of distinctive features, it is easy for other objects with similar local appearances to interfere with the process of detecting lane lines. The precision of lane line detection is limited by the unpredictable quantity and diversity of lane lines. To address the aforementioned challenges, we propose a novel deep learning approach for lane line detection. This method leverages the Swin Transformer in conjunction with LaneNet (called ST-LaneNet). The experience results showed that the true positive detection rate can reach 97.53% for easy lanes and 96.83% for difficult lanes (such as scenes with severe occlusion and extreme lighting conditions), which can better accomplish the objective of detecting lane lines. In 1000 detection samples, the average detection accuracy can reach 97.83%, the average inference time per image can reach 17.8 ms, and the average number of frames per second can reach 64.8 Hz. The programming scripts and associated models for this project can be accessed openly at the following GitHub repository:
https://github.com/Duane711/Lane-line-detection-ST-LaneNet
.
Từ khóa
Tài liệu tham khảo
Y Wang, J Hu, F Wang, et al. Tire road friction coefficient estimation: Review and research perspectives. Chinese Journal of Mechanical Engineering, 2022, 35: 6. https://doi.org/10.1186/s10033-021-00675-z.
X Pan, J Shi, P Luo, et al. Spatial as deep: Spatial CNN for traffic scene understanding. 32nd AAAI Conference on Artificial Intelligence, 2018: 7276-7283.
D Yang, W Bao, K Zheng. Lane detection of smart car based on deep learning. Journal of Physics: Conference Series, 2021, 1873 (1): 012068. https://doi.org/10.1088/1742-6596/1873/1/012068.
H Li, X Li. Flexible lane detection using CNNs. Proceedings-2021 International Conference on Computer Technology and Media Convergence Design, 2021: 235–238. https://doi.org/10.1109/CTMCD53128.2021.00057.
Z Zhang. Z-Net: A novel way of lane detection. Journal of Physics: Conference Series, 2020, 1682 (1): 012013. https://doi.org/10.1088/1742-6596/1682/1/012013.
J Hur, S N Kang, S W Seo. Multi-lane detection in urban driving environments using conditional random fields. IEEE Intelligent Vehicles Symposium, 2013: 1297–1302. https://doi.org/10.1109/IVS.2013.6629645.
T Chen, H Zhang, D Chen, et al. Lane detection based on high priority pixels and tracking by kalman filter. Qiche Gongcheng/Automotive Engineering, 2016, 38 (2): 200-205.
H Wang, Y F Cai, G Y Lin, et al. Lane-line detection method based on orientation variance haar feature and hyperbolic model. Journal Traffic Transportation Engineering, 2014, 14 (5): 119–126.
H Y Wu, X M Zhao. Multi-interference lane recognition based on IPM and edge image filtering. China Journal Highway Transportation, 2020, 33 (5): 153–164. https://doi.org/10.19721/j.cnki.1001-7372.2020.05.014.
D Neven, B De Brabandere, S Georgoulis, et al. Towards end-to-end lane detection: An instance segmentation approach. IEEE Intelligent Vehicles Symposium, Proceedings, 2018: 286–291. https://doi.org/10.1109/IVS.2018.8500547.
B He, R Ai, Y Yan, et al. Accurate and robust lane detection based on dual-view convolutional neutral network. IEEE Intelligent Vehicles Symposium, Proceedings, 2016: 1041–1046. https://doi.org/10.1109/IVS.2016.7535517.
J Zhang, T Deng, F Yan, et al. Lane detection model based on spatio-temporal network with double convolutional gated recurrent units. IEEE Transactions on Intelligent Transportation Systems, 2022, 23 (7): 6666–6678. https://doi.org/10.1109/TITS.2021.3060258.
S Lee, J Kim, J Yoon, et al. VPGNet: Vanishing point guided network for lane and road marking detection and recognition. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, October 29, 2017: 1965–1973. https://doi.org/10.1109/ICCV.2017.215.
DC Andrade, F Bueno, F Franco, et al. A novel strategy for road lane detection and tracking based on a vehicle’s forward monocular camera. IEEE Transactions on Intelligent Transportation Systems, 2019, 20 (4): 1497–1507. https://doi.org/10.1109/TITS.2018.2856361.
Q Zou, H Jiang, Q Dai, et al. Robust lane detection from continuous driving scenes using deep neural networks. IEEE Transactions on Vehicular Technology, 2020, 69 (1): 41–54. https://doi.org/10.1109/TVT.2019.2949603.
Y Zhang, Z Lu, D Ma, et al. Ripple-GAN: Lane line detection with ripple lane line detection network and Wasserstein GAN. IEEE Transactions on Intelligent Transportation Systems, 2021, 22 (3): 1532-1542. https://doi.org/10.1109/TITS.2020.2971728.
S Chen, L Huang, H Chen, et al. Multi-lane detection and tracking using temporal-spatial model and particle filtering. IEEE Transactions on Intelligent Transportation Systems, 2022, 23 (3): 2227–2245. https://doi.org/10.1109/TITS.2020.3035614.
E Grave, A Joulin, M Cissé, et al. Efficient softmax approximation for GPUs. The 34th International Conference on Machine Learning, 2017, 3: 2111–2119.
M Bertozzi, A Broggi, A Fascioli. Stereo inverse perspective mapping: theory and applications. Image and Vision Computing, 1998, 16 (8): 585–590. https://doi.org/10.1016/s0262-8856(97)00093-0.
J Long, E Shelhamer, T Darrell. Fully convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA July 12, 2015: 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965.
V Badrinarayanan, A Kendall, R Cipolla. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (12): 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615.
Z Wang, W Ren, Q Qiu. LaneNet: Real-time lane detection networks for autonomous driving. IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Utah, USA, June 22, 2018: 1–9. Available: http://arxiv.org/abs/1807.01726.
K Cho, B Merrienboer, C Gulcehre, et al. Learning phrase representations using RNN encoder–decoder for statistical machine translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014: 1724–1734. https://doi.org/10.3115/v1/D14-1179.
F Chollet. Deep learning with depthwise separable convolutions. IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, January 25, 2017: 1800–1807. https://doi.org/10.1109/CVPR.2017.195.
F Yu, V Koltun. Multi-scale context aggregation by dilated convolutions. 4th International Conference on Learning Representations, San Juan, Puerto Rico, May 2-4, 2016: 1–13. arXiv:1511.07122, 2015.
J Wang, K Chen, R Xu, et al. CARAFE: Content-aware reassembly of features. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea, November 3, 2019: 3007–3016. https://doi.org/10.1109/ICCV.2019.00310.
K He, X Zhang, S Ren, et al. Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, June 30, 2016: 770–778. https://doi.org/10.1109/CVPR.2016.90.
Z Liu, Y Lin, Y Cao, et al. Swin Transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE International Conference on Computer Vision, 2021: 9992–10002. https://doi.org/10.1109/ICCV48922.2021.00986.
P Netrapalli. Stochastic gradient descent and its variants in machine learning. Journal of the Indian Institute of Science, 99 (2): 201–213. https://doi.org/10.1007/s41745-019-0098-4.
T Y Lin, P Goyal, R Girshick, et al. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42 (2): 318–327. https://doi.org/10.1109/TPAMI.2018.2858826.
M Ghafoorian, C Nugteren, N Baka, et al. EL-GAN: embedding loss driven generative adversarial networks for lane detection. IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, California, 2019: 256–272.
L Bottou. Large-scale machine learning with stochastic gradient descent. Proceedings of COMPSTAT 2010, 2010: 177–186. https://doi.org/10.1007/978-3-7908-2604-3_16.
A Paszke, A Chaurasia, S Kim, et al. ENet: A deep neural network architecture for real-time semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, June 30, 2016: 1–10