An object bounding box refinement algorithm while the tracking process initialization from the uav

Аuthors

Aglyamutdinova D. B.^*, Sidyakin S. V.^**

State Institute of Aviation Systems, 7, Victorenko str., Moscow, 125319, Russia

*e-mail: aglyamutdinova.d@yandex.ru
**e-mail: sersid@bk.ru

Abstract

The presented article deals with the problem of semi-automatic initialization of the selected object tracking by unmanned averial vehicles (UAVs) or drones. Here, we proposed an algorithm of the position and sizes refining of the boundary rectangle of the tracked object at the start time (on the first frame) based on saliency detection algorithm, which simulates the map of human attention. The advantage of the proposed approach is that it applies the principles used by the human visual system: the color contrast, the main attention is centered on the central objects. The first stage of the proposed approach consists in preliminary image processing (noise removal) by the Gaussian filter and converting the image into the CIE LAB color space. The next stage is segmenting the image into homogeneous areas (superpixels) by simple linear iterative clustering (SLIC) algorithm. Undirected graph is employed as a container for information on segments storage. Based on information from the resulting graph, measures of identity, which assign superpixels to the background or an object, are computed. The resulting saliency measure is computed for each superpixel by optimizing the target cost function, which combines the measures of identity to the background, an the object and the smoothing component. The obtained saliency map of the image superpixels is binarized by the Otsu method. After that, the pixels belonging to the shadow can be determined. At the final stage, the operations of morphological filtering were applied to reduce fragmentation of objects and an algorithm for allocating coherent components, assigning the final dimensions and position of the object of interest for tracking initialization.

The algorithm was used to initialize a number of fast and effective methods of object tracking: DCF_CA, MOSSE_CA, SAMF, DCF, DSST, MOSSE, SRDCF.At the same time, the quality of the tracking was tested on the largest and most complex database of video clips, shot from an unmanned aerial vehicle – UAV 123.

The results of experimental testing allow conclude that the best tracking quality as a result of initialization by the proposed algorithm is achieved by tracking algorithms “SRDCF” and “MOSSE_CA”. In assessing the performance, you can notice that “MOSSE_CA” tracking algorithm is noticeably superior to the other algorithms. In this way, the most suitable algorithm for tracking objects by UAV, along with the proposed initialization algorithm, is “MOSSE_CA”, due to its least sensitivity to the of initial initialization accuracy and fastness among competitors.

The proposed algorithm does not require special hardware and can work in real-time. It is implemented in C ++. The average time required refining the object, occupying 40% of the image size of 256 × 256 pixels, equals 60 milliseconds on the Intel^® CoreTM i5-3470 CPU @ 3.20GHz.

Keywords:

objects tracking, tracking algorithms initialization, unmanned aerial vehicle (UAV)

References

1. Achanta R., Shaji A., Smith K., Lucchi A., Fua P., Sьsstrunk S. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, vol. 34, no. 11, pp. 2274-2282. DOI: 10.1109/TPAMI.2012.120

2. Zhu W., Liang S., Wei Y., Sun J. Saliency Optimization from Robust Background Detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 2814-2821. DOI: 10.1109/CVPR.2014.360

3. Hornung A., Pritch Y., Krahenbuhl P., Perazzi F. Saliency filters: Contrast based filtering for salient region detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2012, pp. 733–740. DOI: 10.1109/CVPR.2012.6247743

4. Blajovici C., Kiss P.J., Bonus Z., Varga L. Shadow detection and removal from a single image. 19th Summer School on Image Processing (SSIP), 2011, Szeged, Hungary, 6 p.

5. Borji A., Cheng M.-M., Jiang H., Hou Q., Jiang H., Li J. Salient object detection: A survey. Computer Vision and Pattern Recognition, 2014, 21 p. DOI: arXiv:1411.5878

6. Kul'chin Yu.N., Notkin B.S., Sedov V.A. Komp'yuternaya optika, 2009, vol. 33, no. 4, pp. 446-455.

7. Dovzhenko A.Yu., Krashakov S.A. II Mezhdunarodnyi simpozium “Komp'yuternoe obespechenie khimicheskikh issledovanii” (Moskva, 22-23 May 2001) i III Vseros. shkoly-konferensii po kvantovoi i vychisl. khimii im. V.A. Foka, pp. 52-53.

8. Fatica M. CUDA for High Performance Computing. Materials of HPC-NA Workshop 3 (January 2009), 40 p.

9. Belgian researchers develop desktop supercomputer, http://fastra.ua.ac.be/en/index.html

10. Wu Y., Lim J., Yang M.-H. Online Object Tracking: A Benchmark. IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 2411–2418. DOI: 10.1109/CVPR.2013.312

11. Harel J., Koch C., Perona P. Graph-based visual saliency. 19th International Conference on Neural Information Processing Systems, 2006, pp. 545–552.

12. Achanta R., Shaji A., Smith K., Lucchi A., Fua P., Susstrunk S. Slic superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, vol. 34, no. 11, pp. 2274–2281. DOI: 10.1109/TPAMI.2012.120

13. Achanta R., Hemami S., Estrada F., Susstrunk S. Frequency-tuned salient region detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009, 8 p. DOI: 10.1109/CVPR.2009.5206596

14. Frintrop S., Rome E., Henrik I.C. Computational visual attention systems and their cognitive foundations: A survey. ACM Transactions on Applied Perception, 2010, vol. 7, no. 1, 46 p. DOI: 10.1145/1658349.1658355

15. Itti L., Koch C., Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, vol. 20, no. 11, pp. 1254–1259. DOI: 10.1109/34.730558

16. Hou X., Zhang L. Saliency Detection: A Spectral Residual Approach. IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN, USA, 2007. DOI: 10.1109/CVPR.2007.383267

17. Guo C., Ma Q., Zhang L. Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform. IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA, 2008, 8 p. DOI: 10.1109/CVPR.2008.4587715

18. Li J., Levine M.D., An X., Xu X., He H. Visual Saliency Based on Scale-Space Analysis in the Frequency Domain. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, vol. 35, no. 4, pp. 996–1010. DOI: 10.1109/TPAMI.2012.147

19. Pyt'ev Yu.P. Matematicheskie metody issledovaniya prirodnykh resursov Zemli iz kosmosa. Sbornik statei. Moscow, Nauka, 1984, pp. 41–83.

20. Vizil'ter Yu.V., Zheltov S.Yu., Bondarenko A.V., Ososkov M.V., Morzhin A.V. Obrabotka i analiz izobrazhenii v zadachakh mashinnogo zreniya (Processing and analysis of images in computer vision), Moscow, Fizmatkniga, 2010, 672 p.

21. Mueller M., Smith N., Ghanem B. A Benchmark and Simulator for UAV Tracking. European Conference on Computer Vision (ECCV 2016), pp. 445-461. DOI: 10.1007/978-3-319-46448-0_27

22. Mirkes E.M. K-means and K-medoids applet. University of Leicester, 2011.

23. Grady L. Random Walks for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, pp. 1768-1783. DOI: 10.1109/TPAMI.2006.233

24. Otsu N. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 1979, vol. 9, no. 1, pp. 62-66. DOI: 10.1109/TSMC.1979.4310076

25. Sidyakin S.V., Egorov A.I., Malin I.K. Vestnik Moskovskogo aviatsionnogo instituta, 2014, vol. 21, no. 5, pp. 157-166.

26. Mirzoyan A.S., Malyshev O.V., Khmarov I.M., Kanivets V.Yu. Vestnik Moskovskogo aviatsionnogo instituta, 2014, vol. 21, no. 5, pp. 145-156.