Журнал

About Editorial Board Information for authors and reviewers Publication ethics Contacts User Account: send / review a manuscript
Archive
Volume 21, 2024
Number 1 Number 2 Number 3 Number 4 Number 5
Volume 20, 2023
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 19, 2022
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 18, 2021
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 17, 2020
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6 Number 7
Volume 16, 2019
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 15, 2018
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6 Number 7
Volume 14, 2017
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6 Number 7
Volume 13, 2016
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 12, 2015
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 11, 2014
Number 1 Number 2 Number 3 Number 4
Volume 10, 2013
Number 1 Number 2 Number 3 Number 4
Volume 9, 2012
Number 1 Number 2 Number 3 Number 4 Number 5
Volume 8, 2011
Number 1 Number 2 Number 3 Number 4
Volume 7, 2010
Number 1 Number 2 Number 3 Number 4
Issue 6, 2009
Volume 1 Volume 2
Issue 5, 2008
Volume 1 Volume 2
Issue 4, 2007
Volume 1 Volume 2
Issue 3, 2006
Volume 1 Volume 2
Issue 2, 2005
Volume 1 Volume 2
Issue 1, 2004
Volume 1
Search
Find:
русский
ISSN 2070-7401 (Print), ISSN 2411-0280 (Online)
Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa
CURRENT PROBLEMS IN REMOTE SENSING OF THE EARTH FROM SPACE

  

Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2022, Vol. 19, No. 1, pp. 65-77

U-Net models for semantic segmentation of damaged Pinus sibirica trees in UAV imagery

N.G. Markov 1 , К.А. Maslov 1 , I.A. Kerchev 2 , O.S. Tokareva 1 
1 Tomsk Politechnical University, Tomsk, Russia
2 Institute of Monitoring of Climatic and Ecological Systems SB RAS, Tomsk, Russia
Accepted: 13.01.2022
DOI: 10.21046/2070-7401-2022-19-1-65-77
Since 2019, there have been rapid increases in mortality rates of Pinus sibirica Du Tour stands caused by a new alien bark beetle — Ips amitinus Eichh. — in three regions of Western Siberia: Tomsk, Kemerovo and Novosibirsk regions. The success of pest management directly depends on the timeliness of identifying the colonised trees. Dried out treetops, which are hardly noticeable during ground surveys, is a distinct feature of the damaged trees. The use of unmanned aerial vehicles (UAVs) ensures time efficiency of observations and provides an ultra-high spatial resolution of tree crown images. This paper is devoted to the development of U-Net models and their testing when solving the problem of semantic segmentation of Pinus sibiricatrees damaged by the pest in UAV imagery. To assess the condition of the trees, experts identify five classes of them: healthy, recently colonised, with a dried out treetop, current year deadwood and old deadwood. Trees of other species and the remaining objects in the images belong to the background. Images acquired in July 2019 with the DJI Phantom 3 Standard drone were used to train, validate and test the models. To solve the addressed problem several fully convolutional networks were proposed: a minor modification of U-Net and two major modifications — multihead-U-Net (MH-U-Net) and multihead-residual-U-Net (MH-Res-U-Net). MH-U-Net is an ensemble of three U-Nets of different depths. The models in the ensemble share part of their weights and simultaneously analyse an image at three different scales. MH-Res-U-Net has all the properties of MH-U-Net and additionally introduces residual blocks. The research has shown that all the models successfully classify pixels of five classes out of six: U-Net and MH-Res-U-Net successfully classify all the classes except the recently colonised trees, and MH-U-Net — all the classes except the current year deadwood trees. Intermediate classes of the tree condition represent the main difficulty for segmentation. However, MH-U-Net copes with the segmentation of the intermediate class of the recently colonised trees, U-Net — of the trees with a dried out treetop, and MH-Res-U-Net — of the current year deadwood trees.
Keywords: unmanned aerial vehicle, deep learning, fully convolutional network, U-Net, semantic segmentation, Pinus sibiricaDu Tour, Ips amitinus Eichh.
Full text

References:

  1. Kerchev I. A., Krivets S. A., Bisirova E. M., Smirnov N. A. (2021a), Distribution of the small spruce bark beetle Ips amitinus (Eichhoff, 1872) in Western Siberia, Rossiiskii zhurnal biologicheskikh invazii, 2021, No. 4 (in Russian), DOI: 10.35885/1996-1499-2021-14-4.
  2. Kerchev I. A., Maslov K. A., Markov N. G., Tokareva O. S. (2021a), Semantic Segmentation of Damaged Fir Trees in Unmanned Aerial Vehicle Images, Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2021, Vol. 18, No. 1, pp. 116–126 (in Russian), DOI: 10.21046/2070-7401-2021-18-1-116-126.
  3. Bertels J., Eelbode T., Berman M., Vandermeulen D., Maes F., Bisschops R., Blaschko M., Optimizing the Dice Score and Jaccard Index for Medical Image Segmentation: Theory and Practice, arXiv preprint, arXiv:1911.01685, 2019, 9 p., available at: https://arxiv.org/abs/1911.01685 (accessed 18.11.2021).
  4. Bradshaw C. J. A., Leroy B., Bellard C., Roiz D., Albert C., Fournier A., Barbet-Massin M., Salles J.-M., Simard F., Courchamp F., Massive yet grossly underestimated global costs of invasive insects, Nature Communications, 2016, Vol. 7, Art. No. 12986, DOI: 10.1038/ncomms12986.
  5. Chenari A., Erfanifard Y., Dehghani M., Pourghasemi H. R., Woodland Mapping at Single-Tree Levels Using Object-Oriented Classification of UAV Images, The Intern. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, Vol. XLII-4/W4, pp. 43–49, DOI: 10.5194/isprs-archives-XLII-4-W4-43-2017.
  6. Gini R., Sona G., Ronchetti G., Passoni D., Pinto L., Improving Tree Species Classification Using UAS Multispectral Images and Texture Measures, Intern. J. Geo-Informatics, 2018, Vol. 7, Issue 8, Art. No. 325, DOI: 10.3390/ijgi7080315.
  7. He K., Zhang X., Ren S., Sun J., Deep Residual Learning for Image Recognition, arXiv preprint, arXiv:1512.03385, 2015, 12 p., available at: https://arxiv.org/abs/1512.03385 (accessed 21.11.2021).
  8. Ioffe S., Szegedy C., Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, arXiv preprint, arXiv:1502.03167, 2015, 11 p., available at: https://arxiv.org/abs/1502.03167 (accessed 07.11.2021).
  9. Kerchev I. A., Mandelshtam M. Yu, Krivets S. A., Ilinsky Yu. Yu., Small spruce bark beetle Ips amitinus (Eichhoff, 1872) (Coleoptera, Curculionidae: Scolytinae): a new alien species in West Siberia, Entomological Review, 2019, Vol. 99, No. 5, pp. 639–644.
  10. Kingma D. P., Ba J., Adam: A Method for Stochastic Optimization, arXiv preprint, arXiv:1412.6980, 2014, 15 p., available at: https://arxiv.org/abs/1412.6980 (accessed 07.11.2021).
  11. Lee S., Park S., Baek G., Kim H., Lee C., Detection of Damaged Pine Tree by the Pine Wilt Disease Using UAV Image, Korean J. Remote Sensing, 2019, Vol. 35. No. 3, pp. 359–373, DOI: 10.7780/kjrs.2019.35.3.2.
  12. Liu C., Diagne C., Angulo E., Banerjee A.-K., Chen Y., Cuthbert R. N., Haubrock P. J., Kirichenko N., Pattison Z., Watari Y., Xiong W., Courchamp F., Economic costs of biological invasions in Asia, In: The economic costs of biological invasions around the world, Zenni R. D., McDermott S., García-Berthou E., Essl F. (eds), NeoBiota, 2021, Vol. 67, pp. 53–78, available at: https://neobiota.pensoft.net/article/58147/.
  13. McInnes L., Healy J., Melville J., UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction, arXiv preprint, arXiv:1802.03426, 2018, 51 p., available at: https://arxiv.org/abs/1802.03426 (accessed 01.11.2021).
  14. Økland B., Flø D., Schroeder M., Zach P., Cocos D., Martikainen P., Siitonen J., Mandelshtam M. Y., Musolin D. L., Neuvonen S., Vakula J., Nikolov C., Lindelöw Å., Voolma K., Range expansion of the small spruce bark beetle Ips amitinus: a newcomer in northern Europe, Agricultural and Forest Entomology, 2019, Vol. 21, No. 3, pp. 286–298.
  15. Onishi M., Ise T., Automatic Classification of Trees Using a UAV Onboard Camera and Deep Learning, arXiv preprint, arXiv:1804.10390, 2018, 9 p., available at: https://arxiv.org/abs/1804.10390 (accessed 01.11.2021).
  16. Paszke A., Chaurasia A., Kim S., Culurciello E., ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation, arXiv preprint, arXiv:1606.02147, 2016, 10 p., available at: https://arxiv.org/abs/1606.02147 (accessed 13.11.2021).
  17. Rahman M. A., Wang Y., Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation, In: Advances in Visual Computing: Intern. Symp. Visual Computing, Cham: Springer, 2016, Vol. 10072, pp. 234–244, DOI: 10.1007/978-3-319-50835-1_22.
  18. Ronneberger O., Fischer P., Brox T., U-Net: Convolutional Networks for Biomedical Image Segmentation, arXiv preprint, arXiv:1505.04597, 2015, 8 p., available at: https://arxiv.org/abs/1505.04597 (accessed 12.11.2021).
  19. Rousseeuw P. J., Silhouettes: A graphical aid to the interpretation and validation of cluster analysis, J. Computational and Applied Mathematics, 1987, Vol. 30, pp. 33–65, DOI: 10.1016/0377-0427(87)90125-7.
  20. Safonova A., Tabik S., Alcaraz-Segura D., Rubtsov A., Maglinets Y., Herrera F., Detection of Fir Trees (Abies sibirica) Damaged by the Bark Beetle in Unmanned Aerial Vehicle Images with Deep Learning, Remote Sensing, 2019, Vol. 11, No. 6, Art. No. 643, DOI: 10.3390/rs11060643.
  21. Simard P. Y., Steinkraus D., Platt J. C., Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis, Proc. Intern. Conf. Document Analysis and Recognition, 2003, pp. 958–963.
  22. Srivatsava N., Hinton J., Krizhevsky A., Sutskever I., Salakhutdinov R., Dropout: A Simple Way to Prevent Neural Networks from Overfitting, J. Machine Learning Research, 2014, Vol. 15, pp. 1929–1958.