Communications - Scientific Letters of the University of Zilina 2022, 24(1):C1-C17 | DOI: 10.26552/com.C.2022.1.C1-C17

Redundant Multi-Object Detection for Autonomous Vehicles in Structured Environments

Stefano Feraco ORCID...1, Angelo Bonfitto ORCID...1, Nicola Amati ORCID...1, Andrea Tonoli ORCID...1
Politecnico di Torino, Torino, Italy

This paper presents a redundant multi-object detection method for autonomous driving, exploiting a combination of Light Detection and Ranging (LiDAR) and stereocamera sensors to detect different obstacles. These sensors are used for distinct perception pipelines considering a custom hardware/software architecture deployed on a self-driving electric racing vehicle. Consequently, the creation of a local map with respect to the vehicle position enables development of further local trajectory planning algorithms. The LiDAR-based algorithm exploits segmentation of point clouds for the ground filtering and obstacle detection. The stereocamera-based perception pipeline is based on a Single Shot Detector using a deep learning neural network. The presented algorithm is experimentally validated on the instrumented vehicle during different driving maneuvers.

Keywords: perception, autonomous driving, obstacle detection, point-cloud segmentation, single shot detector, LiDAR (Light Detection and Ranging)
Grants and funding:

This research work was developed in the framework of the activities of the Interdepartmental Center for Automotive Research and Sustainable Mobility (CARS) at Politecnico di Torino (www.cars.polito.it).

Received: April 29, 2021; Accepted: June 4, 2021; Prepublished online: September 30, 2021; Published: January 1, 2022  Show citation

ACS AIP APA ASA Harvard Chicago Chicago Notes IEEE ISO690 MLA NLM Turabian Vancouver
Feraco, S., Bonfitto, A., Amati, N., & Tonoli, A. (2022). Redundant Multi-Object Detection for Autonomous Vehicles in Structured Environments. Communications - Scientific Letters of the University of Zilina, 24(1), C1-17. doi: 10.26552/com.C.2022.1.C1-C17
Share...
Download citation
  • BibTeX (.bib)
  • Bookends (.ris)
  • EasyBib (.ris)
  • EndNote (.enw)
  • EndNote 8 (.xml)
  • ISI WoS (.isi)
  • Medlars (.medlars)
  • Mendeley (.ris)
  • MODS (.xml)
  • Papers (.ris)
  • RefWorks (.txt)
  • RefManager (.ris)
  • RIS (.ris)
  • MS Word (.xml)
  • Zotero (.ris)
    • Open full article

    References

    1. SILBERG, G., MANASSA, M., EVERHART, K., SUBRAMANIAN, D., CORLEY, M., FRASER, H., SINHA, V. Self-driving cars: are we ready? [online]. Netherlands: KPMG LLp, 2013. Available from: https://assets.kpmg/content/dam/kpmg/pdf/2013/10/self-driving-cars-are-we-ready.pdf
    2. FRAEDRICH, E., LENZ, B. Automated driving: individual and societal aspects. Transportation Research Record: Journal of the Transportation Research Board [online]. 2014, 2416(1), p. 64-72. ISSN 0361-1981, eISSN 2169-4052. Available from: https://doi.org/10.3141/2416-08 Go to original source...
    3. ZIEBINSKI, A., CUPEK, R., GRZECHCA, D., CHRUSZCZYK, L. Review of advanced driver assistance systems (ADAS). AIP Conference Proceedings [online]. 2017, 1906(1), 120002. ISSN 0094-243X, eISSN 1551-7616. Available from: https://doi.org/10.1063/1.5012394 Go to original source...
    4. KAUR, K., RAMPERSAD, G. Trust in driverless cars: Investigating key factors influencing the adoption of driverless cars. Journal of Engineering and Technology Management [online]. 2018, 48, p. 87-96. ISSN 0923-4748. Available from: https://doi.org/10.1016/j.jengtecman.2018.04.006 Go to original source...
    5. HORL, S., CIARI, F., AXHAUSEN, K. W. Recent perspectives on the impact of autonomous vehicles [online]. Working paper 10XX. Zurich: Institute for Transport Planning and Systems, 2016. Available from: https://doi.org/10.3929/ethz-b-000121359 Go to original source...
    6. BONFITTO, A., FERACO, S., TONOLI, A., AMATI, N. Combined regression and classification artificial neural networks for sideslip angle estimation and road condition identification. Vehicle System Dynamics [online]. 2020, 58(11), p. 1766-1787. ISSN 0042-3114, eISSN 1744-5159. Available from: https://doi.org/10.1080/00423114.2019.1645860 Go to original source...
    7. BUEHLER, M., IAGNEMMA, K., SINGH, S. The 2005 DARPA grand challenge: the great robot race [online]. Vol.36. Berlin Heidelberg: Springer, 2007. ISBN 978-3-540-73429-1, eISBN 978-3-540-73429-1. Available from: https://doi.org/10.1007/978-3-540-73429-1 Go to original source...
    8. THRUN, S., MONTEMERLO, M., DAHLKAMP, H., STAVENS, D., ARON, A., DIEBEL, J., FONG, P., GALE, J., HALPENNY, M., HOFFMANN, G., LAU, K., OAKLEY, C., PALATUCCI, M., PRATT, V., STANG, P., STROHBAND, S., DUPONT, C., JENDROSSEK, L. - E., KOELEN, CH., MARKEY, CH., RUMMEL, C., VAN NIEKERK, J., JENSEN, E., ALESSANDRINI, P., BRADSKI, G., DAVIES, B., ETTINGER, S., KAEHLER, A., NEFIAN, A., MAHONEY, P. Stanley: The robot that won the DARPA Grand Challenge [online]. Journal of Field Robotics. 2006, 23(9), p. 661-692. eISSN 1556-4967. Available from: https://doi.org/10.1002/rob.20147 Go to original source...
    9. LITMAN, T. Autonomous vehicle implementation predictions: implications for transport planning. In: Transportation Research Board 94th Annual Meeting: proceedings. 2015. No. 15-3326.
    10. SINGH, S. Critical reasons for crashes investigated in the national motor vehicle crash causation survey. Washington, DC: National Center for Statistics and Analysis, 2015. No. DOT HS 812 115.
    11. YURTSEVER, E., LAMBERT, J., CARBALLO, A., TAKEDA, K. A survey of autonomous driving: common practices and emerging technologies. IEEE Access [online]. 2020, 8, p. 58443-58469. eISSN 2169-3536. Available from: https://doi.org/10.1109/ACCESS.2020.2983149 Go to original source...
    12. PENDLETON, S. D., ANDERSEN, H., DU, X., SHEN, X., MEGHJANI, M., ENG, Y. H., RUS, D., ANG, M. H. Perception, planning, control and coordination for autonomous vehicles. Machines [online]. 2017, 5(1), 6. eISSN 2075-1702. Available from: https://doi.org/10.3390/machines5010006 Go to original source...
    13. MARTI, E., DE MIGUEL, M. A., GARCIA, F., PEREZ, J. A review of sensor technologies for perception in automated driving. IEEE Intelligent Transportation Systems Magazine [online]. 2019, 11(4), p. 94-108. ISSN 1939-1390, eISSN 1941-1197. Available from: https://doi.org/10.1109/MITS.2019.2907630 Go to original source...
    14. KEMSARAM, N., DAS, A., DUBBELMAN, G. A stereo perception framework for autonomous vehicles. In: 2020 IEEE 91st Vehicular Technology Conference VTC2020-Spring: proceedings [online]. IEEE. 2020. eISBN 978-1-7281-5207-3, p. 1-6. Available from: https://doi.org/10.1109/VTC2020-Spring48590.2020.9128899 Go to original source...
    15. FERACO, S., BONFITTO, A., KHAN, I., AMATI, N., TONOLI, A. Optimal trajectory generation using an improved probabilistic road map algorithm for autonomous driving. In: 22nd International Design Engineering Technical Conferences and Computers and Information in Engineering Conference: proceedings [online]. American Society of Mechanical Engineers. Vol. 83938. 2020. ISBN 978-0-7918-8393-8, V004T04A006. Available from: https://doi.org/10.1115/DETC2020-22311 Go to original source...
    16. FERACO, S., BONFITTO, A., AMATI, N., & TONOLI, A. A LIDAR-based clustering technique for obstacles and lane boundaries detection in assisted and autonomous driving. In: 22nd International Design Engineering Technical Conferences and Computers and Information in Engineering Conference: proceedings [online]. American Society of Mechanical Engineers. Vol. 83938. 2020. ISBN 978-0-7918-8393-8, V004T04A007. Available from: https://doi.org/10.1115/DETC2020-22339 Go to original source...
    17. WANG, M., LIU, W. - Q., LU, Y. - H., ZHAO, X. - S., SONG, B. - CH., ZHANG, Y. - J., WANG, Y. - P., LIAN, C. - H., CHEN, J., CHENG, Y., LIU, J. - G., WEI, Q. - N. Study on the measurement of the atmospheric extinction of fog and rain by forward-scattering near infrared spectroscopy. Spectroscopy and Spectral Analysis. 2008, 28(8), p. 1776-1780.
    18. PHILLIPS, T. G., GUENTHER, N., MCAREE, P. R. When the dust settles: the four behaviors of LIDAR in the presence of fine airborne particulates. Journal of Field Robotics [online]. 2017, 34(5), p. 985-1009. eISSN 1556-4967. Available from: https://doi.org/10.1002/rob.21701 Go to original source...
    19. GOSALA, N., BUHLER; A., PRAJAPAT, M., EHMKE, C., GUPTA, M., SIVANESAN, R., GAWEL, A., PFEIFFER, M., BURKI, M., SA, I., DUBE, R., SIEGWART, R. Redundant perception and state estimation for reliable autonomous racing. In: 2019 International Conference on Robotics and Automation ICRA 2019: proceedings [online]. IEEE. 2019. Available from: https://doi.org/10.1109/ICRA.2019.8794155 Go to original source...
    20. KUMAR, G. A., LEE, J. H., HWANG, J., PARK, J., YOUN, S. H., KWON, S. LiDAR and camera fusion approach for object distance estimation in self-driving vehicles. Symmetry [online]. 2020, 12(2), 324. eISSN 2073-8994. Available from: https://doi.org/10.3390/sym12020324 Go to original source...
    21. ROSIQUE, F., NAVARRO, P. J., FERNANDEZ, C., PADILLA, A. A systematic review of perception system and simulators for autonomous vehicles research. Sensors [online]. 2019, 19(3), 648. eISSN 1424-8220. Available from: https://doi.org/10.3390/s19030648 Go to original source...
    22. LEE, K. B., HAN, M. H. Lane-following method for high speed autonomous vehicles. International Journal of Automotive Technology [online]. 2008, 9(5), p. 607-613. ISSN 1229-9138. eISSN 1976-3832. Available from: https://doi.org/10.1007/s12239-008-0072-z Go to original source...
    23. CHU, K., KIM, J., JO, K., SUNWOO, M. Real-time path planning of autonomous vehicles for unstructured road navigation. International Journal of Automotive Technology [online]. 2015, 16(4), p. 653-668. ISSN 1229-9138. eISSN 1976-3832. Available from: https://doi.org/10.1007/s12239-015-0067-5 Go to original source...
    24. HU, J., XIONG, S., ZHA, J., FU, C. Lane detection and trajectory tracking control of autonomous vehicle based on model predictive control. International Journal of Automotive Technology [online]. 2020, 21(2), p. 285-295. ISSN 1229-9138. eISSN 1976-3832. Available from: https://doi.org/10.1007/s12239-020-0027-6 Go to original source...
    25. KHAN, I., FERACO, S., BONFITTO, A., AMATI, N. A model predictive control strategy for lateral and longitudinal dynamics in autonomous driving. In: International Design Engineering Technical Conferences and Computers and Information in Engineering Conference ASME 2020: proceedings. American Society of Mechanical Engineers Digital Collection. 2020. Go to original source...
    26. HIMMELSBACH, M., HUNDELSHAUSEN, F. V., WUENSCHE, H. J. Fast segmentation of 3D point clouds for ground vehicles. In: 2010 IEEE Intelligent Vehicles Symposium: proceedings. IEEE. 2010. ISBN 978-1-4244-7866-8, p. 560-565. Go to original source...
    27. ZERMAS, D., IZZAT I., PAPANIKOLOPOULOS, N. Fast segmentation of 3D point clouds: a paradigm on LiDAR data for autonomous vehicle applications. In: 2017 IEEE International Conference on Robotics and Automation ICRA 2017: proceedings [online]. IEEE, 2017. eISBN 9781509046331, ISSN 1050-4729. Available from: https://doi.org/10.1109/ICRA.2017.7989591 Go to original source...
    28. NYGREN, P., JASINSKI, M. A comparative study of segmentation and classification methods for 3D point clouds [online]. MS thesis. Gothenburg, Sweden: Chalmers University of Technology, University of Gothenburg, 2016. Available from: https://hdl.handle.net/20.500.12380/238602
    29. ZHOU, Y., TUZEL, O. Voxelnet: End-to-end learning for point cloud based 3D object detection. In: IEEE Conference on Computer Vision and Pattern Recognition CVPR 2018: proceedings [online]. 2018. p. 4490-4499. Available from: https://doi.org/10.1109/CVPR.2018.00472 Go to original source...
    30. LIU, W., SUN, J., LI, W., HU, T., WANG, P. Deep learning on point clouds and its application: a survey. Sensors [online], 2019, 19(19), 4188. eISSN 1424-8220. Available from: https://doi.org/10.3390/s19194188 Go to original source...
    31. GRILLI, E., MENNA, F., REMONDINO, F. A review of point clouds segmentation and classification algorithms. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences [online]. 2017, XLII, p. 339-344. eISSN 2194-9034. Available from: https://doi.org/10.5194/isprs-archives-XLII-2-W3-339-2017 Go to original source...
    32. GUO, Y., WANG, H., HU, Q., LIU, H., LIU, L., BENNAMOUN, M. Deep learning for 3D point clouds: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence [online]. 2020, early access. ISSN 0162-8828, eISSN 1939-3539. Available from: https://doi.org/10.1109/TPAMI.2020.3005434 Go to original source...
    33. LIU, W., ANGUELOV, D., ERHAN, D., SZEGEDY, C., REED, S., FU, C. Y., BERG, A. C. SSD: Single shot multibox detector. In: European Conference on Computer Vision: proceedings [online]. Springer, Cham. 2016. ISBN 978-3-319-46447-3, eISBN 978-3-319-46448-0, p. 21-37. Available from: https://doi.org/10.1007/978-3-319-46448-0_2 Go to original source...
    34. ZHAO, Z. Q., ZHENG, P., XU, S. T., WU, X. Object detection with deep learning: A review. IEEE Transactions on Neural Networks and Learning Systems [online]. 2019, 30(11), p. 3212-3232. ISSN 2162-237X, eISSN 2162-2388. Available from: https://doi.org/10.1109/TNNLS.2018.2876865 Go to original source...
    35. JIAO, L., ZHANG, F., LIU, F., YANG, S., LI, L., FENG, Z., QU, R. A survey of deep learning-based object detection. IEEE Access [online]. 2019, 7, p. 128837-128868. eISSN 2169-3536. Available from: https://doi.org/10.1109/ACCESS.2019.2939201 Go to original source...
    36. KABZAN, J., VALLS, M. I., REIJGWART, V. J., HENDRIKX, H. F., EHMKE, C., PRAJAPAT, M., BUHLER, A., GOSALA, N., GUPTA, M., SIVANESAN, R., DHALL, A., CHISARI, E., KARNCHANACHARI, N., BRITS, S., DANGEL, M., SA, I., DUBE, R., GAWEL, A., PFEIFFER, M.,LINIGER, A., LYGEROS, J., SIEGWART, R. AMZ driverless: The full autonomous racing system. Journal of Field Robotics [online]. 2020, 37(7), p. 1267-1294. eISSN 1556-4967. Available from: https://doi.org/10.1002/rob.21977 Go to original source...
    37. GLENNIE, C. L., KUSARI, A., FACCHIN, A. Calibration and stability analysis of the VLP-16 laser scanner. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences [online]. 2016, XL, p. 55-60. eISSN 2194-9034. Available from: https://doi.org/10.5194/isprs-archives-XL-3-W4-55-2016 Go to original source...
    38. ORTIZ, L. E., CABRERA, E. V., GONÇALVES, L. M. Depth data error modeling of the ZED 3D vision sensor from stereolabs. ELCVIA: Electronic Letters on Computer Vision And Image Analysis [online]. 2018, 17(1), p. 1-15. eISSN 1577-5097. Available from: https://doi.org/10.5565/rev/elcvia.1084 Go to original source...
    39. DITTY, M., KARANDIKAR, A., REED, D. Nvidia's xavier SoC. I: Hot Chips: A Symposium on High Performance Chips: proceedings. 2018.
    40. FSG competition handbook 2019 - Formula Student Germany [online]. 2019. Available from: https://www.formulastudent.de/fileadmin/user_upload/all/2019/rules/FSG19_Competition_Handbook_v1.0.pdf
    41. WANG, S., WU, T., VOROBEYCHIK, Y. Towards robust sensor fusion in visual perception [online]. 2020. Available from: arXiv:2006.13192
    42. URMSON, C., ANHALT, J., BAGNELL, D., BAKER, C., BITTNER, R., CLARK, M. N., DOLAN, J., DUGGINS, D., GALATALI, T., GEYER, CH., GITTLEMAN, M., HARBAUGH, S., HEBERT, M., HOWARD, T. M., KOLSKI, S., KELLY, A., LIKHACHEV, M., MCNAUGHTON, M., MILLER, N., PETERSON, K., PILNICK, B., RAJKUMAR, R., RYBSKI, P., SALESKY, B., SEO, Y. - W., SINGH, S., SNIDER, J., STENTZ, A., WHITTAKER, W. "R.", WOLKOWICKI, Z., ZIGLAR, J., BAE, H., BROWN, T., DEMITRISH, D., LITKOUHI, B., NICKOLAOU, J., SADEKAR, V., ZHANG, W., STRUBLE, J., TAYLOR, M., DARMS, M., FERGUSON, D. Autonomous driving in urban environments: boss and the urban challenge. Journal of Field Robotics [online]. 2008, 25(8), p. 425-466. eISSN 1556-4967. Available from: https://doi.org/10.1002/rob.20255 Go to original source...
    43. KAMMEL, S., ZIEGLER, J., PITZER, B., WERLING, M., GINDELE, T., JAGZENT, D., SCHRODER, J., THUY, M., GOEBL, M., VON HUNDELSHAUSEN, F., PINK, O., FRESE, CH., STILLER, CH. Team AnnieWAY's autonomous system for the 2007 DARPA urban challenge. Journal of Field Robotics [online]. 2008, 25(9), p. 615-639. eISSN 1556-4967. Available from: https://doi.org/10.1002/rob.20252 Go to original source...
    44. MONTEMERLO, M., BECKER, J., BHAT, S., DAHLKAMP, H., DOLGOV, D., ETTINGER, S., HAEHNEL, D., HILDEN, T., HOFFMANN, G., HUHNKE, B., JOHNSTON, D., KLUMPP, S., LANGER, D., LEVANDOWSKI, A., LEVINSON, J., MARCIL, J., ORENSTEIN, D., PAEFGEN, J., PENNY, I., PETROVSKAYA, A., PFLUEGER, M., STANEK, G., STAVENS, D., VOGT, A., THRUN, S. Junior: the Stanford entry in the urban challenge. Journal of Field Robotics [online]. 2008, 25(9), p. 569-597. eISSN 1556-4967. Available from: https://doi.org/10.1002/rob.20258 Go to original source...
    45. MOOSMANN, F., PINK, O., STILLER, C. Segmentation of 3D LiDAR data in non-flat urban environments using a local convexity criterion. In: IEEE Intelligent Vehicles Symposium IV 09: proceedings [online]. 2009. ISSN 1931-0587. Available from: https://doi.org/10.1109/IVS.2009.5164280 Go to original source...
    46. KLASING, K., WOLLHERR, D., BUSS, M. Realtime segmentation of range data using continuous nearest neighbors. In: 2009 IEEE International Conference on Robotics and Automation ICRA: proceedings 2009. Go to original source...
    47. KLASING, K., WOLLHERR, D., BUSS, M. A clustering method for efficient segmentation of 3D laser data. In: IEEE International Conference on Robotics and Automation ICRA 2008: proceedings. 2008. p. 4043-4048. Go to original source...
    48. STEINHAUSER, D., RUEPP, O., BURSCHKA, D. Motion segmentation and scene classification from 3D LiDAR data. In: IEEE Intelligent Vehicles Symposium: proceedings. 2008. p. 398-403. Go to original source...
    49. ANGUELOV, D., TASKAR, B., CHATALBASHEV, V., KOLLER, D., GUPTA, D., HEITZ, G., NG, A. Discriminative learning of Markov random fields for segmentation of 3D scan data. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR'05: proceedings. Vol. 2. 2005. p. 169-176.
    50. RUSU, R. B. Semantic 3D object maps for everyday manipulation in human living environments. KI-Kunstliche Intelligenz [online]. 2010, 24(4), p. 345-348. ISSN 0933-1875, eISSN 1610-1987. Available from: https://doi.org/10.1007/s13218-010-0059-6 Go to original source...
    51. RUSU, R. B., COUSINS, S. 3D is here: Point cloud library (PCL). In: 2011 IEEE International Conference on Robotics and Automation: proceedings. 2011. p. 1-4. Go to original source...
    52. YAN, Z., DUCKETT, T., BELLOTTO, N. Online learning for 3D LiDAR-based human detection: experimental analysis of point cloud clustering and classification methods. Autonomous Robots [online]. 2020, 44(2), p. 147-164. ISSN 0929-5593, eISSN 1573-7527. Available from: https://doi.org/10.1007/s10514-019-09883-y Go to original source...
    53. BENTLEY, J. L. Multidimensional binary search trees used for associative searching. Communications of the ACM [online]. 1975, 18(9), p. 509-517. ISSN 0001-0782, eISSN 1557-7317. Available from: https://doi.org/10.1145/361002.361007 Go to original source...
    54. MACQUEEN, J. B. Some methods for classification and analysis of multivariate observations. In: 5-th Berkeley Symposium on Mathematical Statistics and Probability: proceedings. Berkeley, University of California Press. Vol. 1. 1967. p. 281- 297.
    55. HOWARD, A. G., ZHU, M., CHEN, B., KALENICHENKO, D., WANG, W., WEYAND, T., ANDREETTO, M., ADAM, H. Mobile nets: efficient convolutional neural networks for mobile vision applications [online]. 2017. Available from: http://arxiv.org/abs/1704.04861
    56. GALVEZ, R. L., BANDALA, A. A., DADIOS, E. P., VICERRA, R. R. P., MANINGO, J. M. Z. Object detection using convolutional neural networks. In: IEEE Region 10 Conference TENCON 2018: proceedings. 2018. p. 2023-2027. Go to original source...
    57. ZHUANG, F., QI, Z., DUAN, K., XI, D., ZHU, Y., ZHU, H., XIONG, H., HE, Q. A comprehensive survey on transfer learning. Proceedings of the IEEE [online]. 2020, 109(1), p. 43-76. Available from: https://doi.org/10.1109/JPROC.2020.3004555 Go to original source...
    58. R. SZELISKY, Computer vision: algorithms and applications [online]. London: Springer Link, 2012. ISBN 978-1-84882-934-3, eISBN 978-1-84882-935-0. Available from: https://doi.org/10.1007/978-1-84882-935-0 Go to original source...
    59. ADI, K., WIDODO, C. E. Distance measurement with a stereo camera. International Journal of Innovative Research in Advanced Engineering [online]. 2017, 4(11), p. 24-27. ISSN 2349-2163. Available from: https://doi.org/10.26562/IJIRAE.2017.NVAE10087 Go to original source...
    60. ZAARANE, A., SLIMANI, I., AL OKAISHI, W., ATOUF, I., HAMDOUN, A. Distance measurement system for autonomous vehicles using stereo camera. Array [online]. 2020, 5, 100016. ISSN 2590-0056. Available from: https://doi.org/10.1016/j.array.2020.100016 Go to original source...

    This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits use, distribution, and reproduction in any medium, provided the original publication is properly cited. No use, distribution or reproduction is permitted which does not comply with these terms.


    Return to the content