Журнал

About Editorial Board Information for authors and reviewers Publication ethics Contacts User Account: send / review a manuscript
Archive
Volume 21, 2024
Number 1 Number 2 Number 3 Number 4 Number 5
Volume 20, 2023
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 19, 2022
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 18, 2021
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 17, 2020
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6 Number 7
Volume 16, 2019
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 15, 2018
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6 Number 7
Volume 14, 2017
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6 Number 7
Volume 13, 2016
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 12, 2015
Number 1 Number 2 Number 3 Number 4 Number 5 Number 6
Volume 11, 2014
Number 1 Number 2 Number 3 Number 4
Volume 10, 2013
Number 1 Number 2 Number 3 Number 4
Volume 9, 2012
Number 1 Number 2 Number 3 Number 4 Number 5
Volume 8, 2011
Number 1 Number 2 Number 3 Number 4
Volume 7, 2010
Number 1 Number 2 Number 3 Number 4
Issue 6, 2009
Volume 1 Volume 2
Issue 5, 2008
Volume 1 Volume 2
Issue 4, 2007
Volume 1 Volume 2
Issue 3, 2006
Volume 1 Volume 2
Issue 2, 2005
Volume 1 Volume 2
Issue 1, 2004
Volume 1
Search
Find:
русский
ISSN 2070-7401 (Print), ISSN 2411-0280 (Online)
Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa
CURRENT PROBLEMS IN REMOTE SENSING OF THE EARTH FROM SPACE

  

Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2020, Vol. 17, No. 7, pp. 79-93

Technology of satellite data dynamic block provision to distributed processing systems

A.A. Proshin 1 , E.A. Loupian 1 , I.V. Balashov 1 , A.V. Kashnitskii 1 , A.M. Matveev 1 , B.P. Rutkevych 1 
1 Space Research Institute RAS, Moscow, Russia
Accepted: 10.12.2020
DOI: 10.21046/2070-7401-2020-17-7-79-93
In many tasks related to the processing and visualization of spatial data, a significant benefit is provided by the use of a block (tile) approach, within which data are used, presented in the form of blocks, into which the entire area of interest is evenly divided. This makes it possible to effectively organize parallel data processing on various computational nodes, use the caching mechanism and reduce the load on centralized resources. In this case, as a rule, already prepared data archives with fixed spatial partitions are used. At the same time, in many cases, first of all, when it is necessary to jointly process various types of satellite information, the use of prepared data archives may be impractical. This is due to the fact that various types of satellite data differ significantly both in spatial resolution and in the organization of their storage. As a result, it is almost impossible to choose one spatial partitioning that would be optimal for organizing work with all types of data. Therefore, it is required to develop such approaches to organizing satellite data processing that could maximize the advantages of the block approach and optimize the storage scheme for each specific type of data. The article presents such an approach to organizing the processing of satellite data using dynamic block access to heterogeneous archives of satellite data, within which the data necessary for processing are generated upon request in such a spatial division and with such characteristics that will be optimal for a specific type of processing. The paper provides a brief description of the developed technology, and also considers a methodology for evaluating the optimal partitioning parameters for a specific processing and existing infrastructure.
Keywords: parallel processing of satellite data, satellite data archives, big data, IKI-Monitoring center
Full text

References:

  1. Balashov I. V., Efremov V. Yu., Mazurov A. A., Mamaev A. S., Matveev A. M., Proshin A. A., Osobennosti organizatsii kontrolya i upravleniya raspredelennykh sistem distantsionnogo monitoringa (Features of remote monitoring distributed systems control and monitoring), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2011, Vol. 8, No. 3, pp. 161–166.
  2. Egorov V. A., Il’in V. O., Loupian E. A., Mazurov A. A., Flitman E. V., Vozmozhnosti postroeniya avtomatizirovannykh sistem obrabotki sputnikovykh dannykh na osnove programmnogo kompleksa XV_SAT (Possibilities of developing automated satellite data processing systems on the basis of XV_SAT software package), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2004, No. 1, pp. 431–436.
  3. Kobets D. A., Matveev A. M., Mazurov A. A., Proshin A. A., Organizatsiya avtomatizirovannoi mnogopotokovoi obrabotki sputnikovoi informatsii v sistemakh distantsionnogo monitoringa (Organization of automated multithreaded processing of satellite information in remote monitoring systems), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2015, Vol. 12, No. 1, pp. 145–155.
  4. Kobets D. A., Matveev A. M., Proshin A. A., Mazurov A. A., Kontrol’ rabotosposobnosti i upravlenie raspredelennymi kompleksami avtomaticheskoi potokovoi obrabotki sputnikovykh dannykh (Operation control and management of distributed complexes of automatic streaming processing of satellite data), Voprosy elektromekhaniki. Trudy VNIIEM. Prilozhenie za 2017 g. Materialy 5-i mezhdunarodnoi nauchno-tekhnicheskoi konferentsii “Aktual’nye problemy sozdaniya kosmicheskikh sistem distantsionnogo zondirovaniya Zemli” (Materials of the 5th international scientific and technical conf. “Actual problems of creation of space remote sensing systems of the Earth”), Electromechanical matters, VNIIEM studies, 2017, pp. 225–234, available at: http://www.vniiem.ru/ru/uploads/files/conferences/170525/materialy_2017.pdf.
  5. Loupian E. A., Proshin A. A., Burtsev M. A., Balashov I. V., Bartalev S. A., Efremov V. Yu., Kashnitskii A. V., Mazurov A. A., Matveev A. M., Sudneva O. A., Sychugov I. G., Tolpin V. A., Uvarov I. A., Tsentr kollektivnogo pol’zovaniya sistemami arkhivatsii, obrabotki i analiza sputnikovykh dannykh IKI RAN dlya resheniya zadach izucheniya i monitoringa okruzhayushchei sredy (IKI center for collective use of satellite data archiving, processing and analysis systems aimed at solving the problems of environmental study and monitoring), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2015, Vol. 12, No. 5, pp. 263–284.
  6. Loupian E. A., Proshin A. A., Burtsev M. A., Kashnitskii A. V., Balashov I. V., Bartalev S. A., Konstantinova A. M., Kobets D. A., Mazurov A. A., Marchenkov V. V., Matveev A. M., Radchenko M. V., Sychugov I. G., Tolpin V. A., Uvarov I. A., Opyt ekspluatatsii i razvitiya tsentra kollektivnogo pol’zovaniya sistemami arkhivatsii, obrabotki i analiza sputnikovykh dannykh (TsKP “IKI-Monitoring:) (Experience of development and operation of the IKI-Monitoring center for collective use of systems for archiving, processing and analyzing satellite data), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2019, Vol. 16, No. 3, pp. 151–170, DOI: 10.21046/2070-7401-2019-16-3-151-170.
  7. Proshin A. A., Loupian E. A., Balashov I. V., Kashnitskii A. V., Burtsev M. A., Sozdanie unifitsirovannoi sistemy vedeniya arkhivov sputnikovykh dannykh, prednaznachennoi dlya postroeniya sovremennykh sistem distantsionnogo monitoringa (Unified satellite data archive management platform for remote monitoring systems development), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2016, Vol. 13, No. 3, pp. 9–27, DOI: 10.21046/2070-7401-2016-13-3-9-27.
  8. Gorelick N., Hancher M., Dixon M., Ilyushchenko S., Thau D., Moore R., Google Earth Engine: Planetaryscale geospatial analysis for everyone, Remote Sensing of Environment, 2017, Vol. 202, pp. 18–27.
  9. Peterson M. P., Web Mapping Services, In: Intern. Encyclopedia of Geography: People, the Earth, Environment and Technology, 2016, pp. 1–19.