Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2021, Vol. 18, No. 6, pp. 35-45
Experimental assessment of the accuracy of multiclass segmentation of objects from satellite images based on a modified convolutional neural network U-net
S.M. Bagaev
1 , E.V. Medvedeva
1 1 Vyatka State University, Kirov, Russia
Accepted: 09.11.2021
DOI: 10.21046/2070-7401-2021-18-6-35-45
A methodology for studying the accuracy of segmentation of target objects in satellite images belonging to different classes based on a modified convolutional neural network U-net is presented. The main stages of segmentation are considered: preparation of input data; modification of the structure of the convolutional neural network (CNN) taking into account the parameters of the studied images; training of the CNN on the formed training sample; segmentation of test images. Multispectral images obtained from the WorldView-3 satellite were used to train and test the modified U-net network. Experimental studies were conducted to improve the accuracy of target object segmentation. Two algorithms are used to implement the multi-class segmentation method. The first algorithm is implemented on a single multiclass CNN. The second algorithm is based on separately trained CNN to work with each specific class and then combine the results of their work. The average segmentation accuracy by the first algorithm according to the Jaccard metric was 76 %, according to the F-measure — 0.66. The second algorithm increased the segmentation accuracy of objects of similar and rare classes by 23 % according to the Jaccard metric and by 26 % according to the F-measure metric with a limited training sample of images and computational resources.
Keywords: multiclass segmentation, satellite imagery, convolutional neural network, segmentation accuracy assessment
Full textReferences:
- Gonzalez R., Woods R., Digital Image Processing: Intern. Version 3 rd Edition, Prentice Hall, 2008, 976 p.
- Zakhvatkina N. Yu., Bychkova I. A., Smirnov V. G., Digital processing of Sentinel-1 data for automated detection of old ice edge, Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2020, Vol. 17, No. 5, pp. 23–34 (in Russian), DOI: 10.21046/2070-7401-2020-17-5-23-34.
- Medvedeva E. V., Kurbatova E. E., Okulova A. A., Texture segmentation of Earth’s surface noisy images, Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2017, Vol. 14, No. 7, pp. 20–38 (in Russian), DOI: 10.21046/2070-7401-2017-14-7-20-28.
- Rodionova N. V., One channel texture based segmentation: application examples, Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2012, Vol. 9, No. 3, pp. 65–69 (in Russian).
- Shapiro L. G., Stockman G. C., Computer Vision, Prentice Hall, 2001, 580 p.
- Schowengerdt R. A., Remote Sensing, Models and Methods for Image Processing , 3 rd Edition, Amsterdam: Elsevier; Burlington: Academic Press, 2007, 515 p.
- Borne F., Viennois G., Texture-based classification for characterizing regions on remote sensing images, J. Applied Remote Sensing, 2017, Vol. 11(3), DOI: 10.1117/1.JRS.11.036028.
- Fichtel L., Fruhwald A. M., Hosch L., Schreibmann V., Bachmeir C., Bohlander F., Tree Localization and Monitoring on Autonomous Drones employing Deep Learning, Proc. 29 th Conf. Open Innovations Association (FRUCT), 2021, pp. 132–140, DOI: 10.23919/FRUCT52173.2021.9435549.
- Gudzius P., Kurasova O., Filatovas E., Optimal U-Net Architecture for Object Recognition Problems in Multispectral Satellite Imagery, Proc. 2019 IEEE/ACS 16 th Intern. Conf. Computer Systems and Applications (AICCSA), 2019, Art. No. 19454678, 2 p., DOI: 10.1109/AICCSA47632.2019.9035305.
- Li M., Zang S., Zhang B., Li S., Wu C., A Review of Remote Sensing Image Classification Techniques: the Role of Spatio-contextual Information, European J. Remote Sensing, 2014, Vol. 47, pp. 389–411, DOI: 10.5721/EuJRS20144723.
- Medvedeva E., Evdokimova A., Detection of Texture Objects on Multichannel Images, Proc. 24 th Conf. Open Innovations Association (FRUCT), 2019, pp. 249–254, DOI: 10.23919/FRUCT.2019.8711986.
- Medvedeva E. V., Kurbatova E. E., Image Segmentation Based on Two-Dimensional Markov Chains, In: Computer Vision in Control Systems-2 , Innovations in Practice, Switzerland: Springer Intern. Publishing, 2015, pp. 277–295, DOI: 10.1007/978-3-319-11430-9_11.
- Overton T., Tucker A., DO-U-Net for Segmentation and Counting: Applications to Satellite and Medical Images, In: Advances in Intelligent Data Analysis XVIII, 2020, pp. 391–403, DOI: 10.1007/978-3-030-44584-3_31.
- Persello C., Tolpekin V. A., Bergado J. R., de By R. A., Delineation of agricultural fields in smallholder farms from satellite images using fully convolutional networks and combinatorial grouping, Remote Sensing of Environment, 2019, Vol. 231, Art. No. 111253, DOI: 10.1016/j.rse.2019.111253.
- Ronneberger O., Fischer P., Brox T., U-net: Convolutional networks for biomedical image segmentation, Proc. Intern. Conf. Medical Image Computing and Computer-Assisted Intervention, Springer Intern. Publishing, 2015, pp. 234–241, DOI: 10.1007/978-3-319-24574-4_28.
- Yuan J., Wang D. L., Remote sensing image segmentation by combining spectral and texture features, IEEE Trans. Geoscience and Remote Sensing, 2014, Vol. 52(1), pp. 16–24, DOI: 10.1109/TGRS.2012.2234755.