Automated decision making by the onboard unmanned aerial vehicle system while road traffic monitoring

Аuthors

Kim N. V.^*, Bodunkov N. E.^**, Mikhailov N. A.^***

Moscow Aviation Institute (National Research University), 4, Volokolamskoe shosse, Moscow, А-80, GSP-3, 125993, Russia

*e-mail: nkim2011@list.ru
**e-mail: boduncov63@yandex.ru
***e-mail: mikhailov.mai@gmail.com

Abstract

The article presents the developed method for efficiency increase of the operator, performing traffic surveillance by an unmanned aerial vehicle (UAV) with the built-in computer vision system. Analyzing video information, received via the radio channel in real time mode by the human-controlled point, is associated with errors in decision-making. These errors are stipulated by the vast volume of information, which overburdens the operator, and, as a rule, by the so-called human factor. Productivity of such system can be increased significantly through addition of autonomous road situation estimation system. The UAVs equipped with surveillance systems, such as video cameras, receive images onboard (video sequences), and are able to extract from them the objects of interest: roads and transport means.

Estimation and analysis in this article are ensured by the road incidents consequences severity classification. The work employs the classification consisted of five classes. Each situation class is described by attributes' dictionary, which separates the attribute space into non-crossing areas, corresponding to the selected classes.

In addition, the article describes the developed hierarchical structure of “Description of the Scene Being Surveyed”. This structure relates to the so-called semantic descriptions, is rather universal, and ensures the possibility to describe various road traffic situations.

The article presents the technique for traffic situations classification over the images. It demonstrates the example of the situation classification based on the real image of the road accident.

Keywords:

situations classification based on the semantic descriptions, of computer vision system of autonomous drone, traffic situation attributes recognition

References

1. Karasulu B. Review and Evaluation of well-known Methods for Moving Object Detection and Tracking in videos. Journal of Aeronautics and Space Technologies, July, 2010, vol. 4, no. 4, pp. 11-22.

2. Kim N.V., Kuznetsov A.G., Krylov I.G. Vestnik Moskovskogo aviatsionnogo instituta, 2010, vol. 17. no. 3, pp. 42-49.

3. Bokovoi A.V. Materialy IV Vserossiiskogo nauchno-prakticheskogo seminara “Bespilotnye transportnye sredstva s elementami iskusstvennogo intellekta” (BTS-II-2017). Kazanskii (Privolzhskii) federal'nyi universitet, pp. 36-46.

4. Buival A.K., Gavrilenkov M.A. Materialy IV Vserossiiskogo nauchno-prakticheskogo seminara “Bespilotnye transportnye sredstva s elementami iskusstvennogo intellekta” (BTS-II-2017). Kazanskii (Privolzhskii) federal'nyi universitet, pp. 14-25.

5. Lin Feng, Lum Kai-Yew, Chen Ben M., Lee Tong H. Development of a vision-based ground target detection and tracking system for a small unmanned helicopter. Science in Chine Series F: Information Sciences, 2009, vol. 52, pp. 2201-2215. DOI: 10.1007/s11432-009-0187-5

6. Lin Yuping, Yu Qian, Medioni Gerard. Efficient detection and tracking of moving objects in geo-coordinates. Machine Vision & Applications, 2011, vol. 22, no. 3, pp. 505–520.

7. Qadir Ashraf, Semke William, Neubert Jeremiah. Implementation of an Onboard Visual Tracking System with Small Unmanned Aerial Vehicle (UAV). International Journal of Innovative Technology & Creative Engineering, 2011, vol. 1, no. 10, 9 p.

8. Fan Jialue, Wu Ying, and Dai Shengyang. Discriminative Spatial Attention for Robust Tracking. 11th European Conference on Computer Vision (ECCV), 2010, vol. 6311 LNCS, pp. 480-493. DOI: 10.1007/978-3-642-15549-9_35

9. Tikhonov K.M., Tishkov V.V. Vestnik Moskovskogo aviatsionnogo instituta, 2011, vol. 18, no. 6, pp. 68-77.

10. Li Liang, Jiang Shuqiang, Huang Qingming. Learning Hierarchical Semantic Description Via Mixed-Norm Regularization for Image Understanding. IEEE Transactions on Multimedia, 2012, vol. 14, issue 5, pp. 1401–1413. DOI: 10.1109/TMM.2012.2194993

11. Yan Xu, Wu Xuqing, Kakadiaris Ioannis A., Shah Shishir K. To Track or To Detect? An Ensemble Framework for Optimal Selection. 12th European Conference on Computer Vision (ECCV), Florence, Italy, October 7-13, 2012, Proceedings. Part V, pp. 594-607. DOI: 10.1007/978-3-642-33715-4_43

12. Kim N., Chervonenkis M. Situational control of unmanned aerial vehicles for road traffic monitoring. Modern Applied Science, 2015, vol. 9, no. 5 Special Issue, 13 p. DOI: 10.5539/mas.v9n5p1

13. Matas J., Chum O., Urban M., Pajdla T. Robust wide baseline stereo from maximally stable extremal regions. Proceedings of the British Machine Vision Conference (BMVC), 2002, pp. 384-393. DOI: 10.1016/j.imavis.2004.02.006

14. Lienhart R. and Maydt J. An extended set of Haar-like features for rapid object detection. Proceedings. International Conference on Image Proceeding, 2002, vol. 1, pp. 900-903. DOI: 10.1109/ICIP.2002.1038171

15. Farneback G.Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field. Proceedings Eighth IEEE International Conference on Computer Vision. Vancouver, Canada, 2001, vol. I, pp. 171-177. DOI: 10.1109/ICCV.2001.937514

16. Vazaev A.V., Noskov V.P. Materialy VII nauchno-tekhnicheskoi konferentsii “Tekhnicheskoe zrenie v sistemakh upravleniya-2016”, Moscow, IKI RAN, 2016, pp. 11-12.

17. Forsyth D.A., Ponce Jean. Computer Vision: a Modern Approach. Pearson Education, Prеntiсe Hall, 2012, 793 p.

18. Jahne BDigital Image Processing, Springer, 2005, 654 р.

19. Turmer S., Leitloff J., Reinartz P., Stilla U.Evaluation of selected features for car detection in aerial images. ISPRS Hannover Workshop, 2011, vol. 14, pp. 341-346.

20. Kim N., Bodunkov N. Computer Vision in Advanced Control Systems: Innovations in Practice. Editors M. Favorskaya, L.C. Jain. Springer, 2014, vol. 2– 295 p.

21. Lucas B.D., Kanade T. An iterative image registration technique with an application to stereo vision. Proceedings of Imaging Understanding Workshop, 1981, pp. 121-130.

22. Zhang J., Liu L., Wang B., Chen X., Wang Q., Zheng T. High speed automatic power line detection and tracking for a UAV-based inspection. International Conference on Industrial Control and Electronics Engineering (ICICEE), Xi'an, China, 2012, pp. 266-269.

23. Kelly Alonzo. A 3D state space formulation of a navigation Kalman filter for autonomous vehicles. Carnegie Mellon University. Technical Report CMU-RI-TR-94-19-REV 2.0, 1994, 105 p.

24. Zhong S., Ghosh J. A unified framework for model-based clustering. Journal of Machine Learning Research, 2003, vol. 4, pp. 1001-1037.