Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2015, Vol. 12, No. 1, pp. 156-170
Development of software tools for satellite data remote processing in contemporary information systems
A.V. Kashnitskiy
1 , I.V. Balashov
1 , E.A. Loupian
1 , V.A. Tolpin
1 , I.A. Uvarov
1
1 Space Research Institute RAS, Moscow, Russia
The problem of development of new technologies for satellite observation data processing capable for the efficient extra large continuously updated archive management has become more and more urgent in recent years. “Traditional” approaches propose selecting matching input datasets from the archives, making their local copy for the user or researcher at the first step, and performing data processing and analysis after that. Using of this kind of approach is limited due to a number of factors. The most significant ones are the explosive growth of the data volume acquired from Earth observation satellite systems, the need for selection, transmission and storage of huge data volumes, the existence of powerful datacenter-based computational resources, which typically do not belong to end users. This paper presents the technology of distributed satellite data processing management development in Space Research Institute of the Russian Academy of Sciences (IKI RAS). The technology is aimed at extension of functionality from simple selection and displaying of satellite based information to providing the means of data processing and analysis of data products with use of computational resources located in data archiving and representation centers. The technology enables the researchers to use the distributed information and computational resources as easily as if they were local. It provides capabilities of efficient access to extra large distributed satellite imagery and data products archives. The paper describes the proposed distributed data processing systems architecture and its principal functional capabilities. One can find examples of data processing procedures implementation and use in a number of remote monitoring systems for arrangement of satellite data processing and analysis, including hyperspectral data.
Keywords: remote sensing, information systems, distributed data processing, satellite data processing, web interfaces, extra large data archives, satellite data
Full textReferences:
- Antonov A.V., Bourtsev M.A., Efremov V.Yu., Kalashnikov A.V., Kramareva L.S., Krasheninnikova Yu.S., Loupian E.A., Matveev A.M., Proshin A.A., Flitman E.V. Postroenie ob"edinennogo kataloga raspredelennykh arkhivov sputnikovykh dannykh razlichnykh tsentrov (Design of the union catalogue of distributed satellite data archives of various centers), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2010, Vol. 7, No. 2, pp. 84-89.
- Babyak P.V., Nedoluzhko I.V., Fomin E.V. Podkhod k predostavleniyu uslug po obrabotke sputnikovykh dannykh v Tsentre kollektivnogo pol'zovaniya regional'nogo sputnikovogo monitoringa okruzhayushchei sredy DVO RAN (Approach to providing the satellite data processing service in framework of regional environmental monitoring Collective Users' Center of the RAS Far East branch), Internet i sovremennoe obshchestvo (Internet and the modern society), Proc. Conf., 2014, pp. 27-32.
- Balashov I.V., Bourtsev M.A., Galeev A.A., Efremov V.Yu., Krasheninnikova Yu.S., Loupian E.A., Mazurov A.A., Moskovskii A.A., Proshin A.A., Sergeeva K.O., Flitman E.V. Razrabotka effektivnykh vysokoproizvoditel'nykh reshenii dlya sozdaniya sistem dinamicheskoi obrabotki sputnikovykh dannykh i rezul'tatov ikh analiza (Development of effective high performance solutions for making of satellite imagery and data products dynamic processing systems), 3rd Int. Sci. Conf. Supercomputer systems and applications (SSA' 2010), May 25-27, 2010, Minsk, Proc. Conf., 2010, Vol. 1, pp. 30-34.
- Balashov I.V., Bourtsev M.A., Efremov V.Yu., Loupian E.A., Proshin A.A., Tolpin V.A. Postroenie arkhivov rezul'tatov obrabotki sputnikovykh dannykh dlya sistem dinamicheskogo formirovaniya proizvodnykh informatsionnykh produktov (Development of satellite data processing products archives for dynamic derived information products production systems), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2008, Issue 5, No. 1, pp. 26-32.
- Balashov I. V., Efremov V.Yu., Loupian E.A., Proshin A.A., Tolpin V.A. Postroenie sistem, obespechivayushchikh dinamicheskoe formirovanie kompleksnykh informatsionnykh produktov na osnove dannykh distantsionnogo zondirovaniya (Development of Remote Sensing Complex Information Products Dynamic Creation Systems), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2009, Issue 6, No. 2, pp. 513-520.
- Balashov I.V., Khalikova O.A., Bourtsev M.A., Loupian E.A., Matveev A.M. Organizatsiya avtomaticheskogo polucheniya naborov informatsionnykh produktov iz tsentrov arkhivatsii i rasprostraneniya sputnikovykh i meteodannykh (Organization of automatic data acquisition from satellite and meteorological data and disrtibution centers), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2013, Vol. 10, No. 3, pp. 9-20.
- Bartalev S.A., Egorov V.A., Ershov D.V., Isaev A.S., Loupian E.A., Plotnikov D.E., Uvarov I.A. Sputnikovoe kartografirovanie rastitel'nogo pokrova Rossii po dannym spektroradiometra MODIS (Maping of Russia's vegetation cover using MODIS satellite spectroradiometer data), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2011, Vol.8, No. 4, pp. 285-302.
- Bartalev S.A., Ershov D.V., Korovin G.N., Kotel'nikov R.V., Loupian E.A., Shchetinskii V.E. Osnovnye vozmozhnosti i struktura informatsionnoi sistemy distantsionnogo monitoringa lesnykh pozharov Federal'nogo agentstva lesnogo khozyaistva (ISDM Rosleskhoz) (The Main Functionalities and Structure of the Forest Fire Satellite Monitoring Information System of Russian Federal Forestry Agency (ISDM-Rosleshoz)), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2010, Vol. 7, No. 2, pp. 97-105.
- Bourtsev M.A., Antonov V.N., Efremov V.Yu., Kashnitskiy A.V., Kramareva L.S., Loupian E.A., Mazurov A.A., Matveev A.M., Milekhin O.E., Proshin A.A., Solov'ev V.I. Sistema raboty s raspredelennymi arkhivami rezul'tatov obrabotki sputnikovykh dannykh tsentrov priema NITs "Planeta" (Distributed satellite data processing products archives operation system in the SRC Planeta centres), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2012, Vol. 9, No. 5, pp. 55-76.
- Bourtsev M.A., Efremov V.Yu., Balashov I.V., Mazurov A.A., Proshin A.A., Loupian E.A., Milekhin O.E. Sistema dostupa k dannym Evropeiskogo, Sibirskogo i Dal'nevostochnogo tsentrov priema Rosgidrometa (Data handling system for the European, Siberian and Far Eastern Centres of Roshydromet), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2011, Vol. 8, No. 3, pp. 113-119.
- Efremov V.Yu., Balashov I.V., Kotel'nikov R.V., Loupian E.A., Mazurov A.A., Proshin A.A., Tolpin V.A., Uvarov I.A., Flitman E.V. Ob"edinennyi kartograficheskii interfeis dlya raboty s dannymi ISDM Rosleskhoz (Integrated mapping interface for operations with ISDM-Rosleskhoz data), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2011, Vol. 8, No. 3, pp. 129-139.
- Efremov V.Yu., Girina O.A., Kramareva L.S., Loupian E.A., Manevich A.G., Mel'nikov D.V., Matveev A.M., Proshin A.A., Sorokin A.A., Flitman E.V. Sozdanie informatsionnogo servisa "Distantsionnyi monitoring aktivnosti vulkanov Kamchatki i Kuril" (Creating the information service “Remote Monitoring of Active Volcanoes of Kamchatka and the Kuril Islands”), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2012, Vol. 9, No. 5, pp. 155-170.
- Kussul' N.N., Shelestov A.Yu. Grid-sistemy dlya zadach issledovaniya Zemli (Grid systems in the Earth observation area), Kyiv, Naukova dumka, 2008, 452 p.
- Loupian E.A., Mazurov A.A., Nazirov R.R., Proshin A.A., Flitman E.V., Krasheninnikova Yu.S. Tekhnologii postroeniya informatsionnykh sistem distantsionnogo monitoringa (Technologies for Building Remote Monitoring Information Systems), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2011, Vol. 8, No. 1, pp. 26-43.
- Loupian E.A., Matveev A.A., Uvarov I.A., Bocharova T.Yu., Lavrova O.Yu., Mityagina M.I. Sputnikovyi servis See the Sea – instrument dlya izucheniya razlichnykh yavlenii na poverkhnosti okeana (The Satellite Service See the Sea - a tool for the study of oceanic phenomena and processes), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2012, Vol. 9, No. 2, pp. 251-262.
- Loupian E.A., Savin I.Yu., Bartalev S.A., Tolpin V.A., Balashov I.V., Plotnikov D.E. Sputnikovyi servis monitoringa sostoyaniya rastitel'nosti VEGA (Satellite service for vegetation monitoring VEGA), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2011, Vol. 8, No. 1, pp. 190-198.
- Loupian E.A., Savorskii V.P., Shokin Yu.I., Aleksanin A.I., Nazirov R.R., Nedoluzhko I.V., Panova O.Yu. Sovremennye podkhody i tekhnologii organizatsii raboty s dannymi distantsionnogo zondirovaniya Zemli dlya resheniya nauchnykh zadach (Up-to-date approaches and technology arrangement of Earth observation data applications aimed to solve scientific tasks), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2012, Vol. 9, No. 5, pp. 21-44.
- Miklashevich S.E., Balashov I.V., Bourtsev M.A., Efremov V.Yu., Mazurov A.A., Matveev A.M., Proshin A.A., Radchenko M.V., Flitman E.V. Programmno-apparatnyi kompleks dlya sbora, obrabotki, arkhivatsii i rasprostraneniya sputnikovykh dannykh i produktov ikh tematicheskoi obrabotki (Complex system for the receiving, processing, archiving and distribution of satellite data and product of thematic processing), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2012, Vol. 9, No. 4, pp. 47-56.
- Tolpin V.A., Balashov I.V., Efremov V.Yu., Loupian E.A., Proshin A.A., Uvarov I.A., Flitman E.V. Sozdanie interfeisov dlya raboty s dannymi sovremennykh sistem distantsionnogo monitoringa (sistema GEOSMIS) (GEOSMIS: developing interfaces to operate data in modern remote monitoring systems), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2011, Vol. 8, No. 3, pp. 93-108.
- Shokin Yu.I., Antonov V.N., Dobretsov N.N., Kikhtenko V.A., Lagutin A.A., Smirnov V.V., Chubarov D.L., Chubarov L.B. Raspredelennaya sistema priema i obrabotki sputnikovykh dannykh Sibiri i Dal'nego Vostoka. Tekushchee sostoyanie i perspektivy razvitiya (Development and status of the distributed system for satellite data acquisition and processing for Siberia and the Far East), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2012, Vol. 9, No. 5, pp. 45-54.
- Acker J. G., Leptoukh G. Online analysis enhances use of NASA earth science data, Eos, Transactions American Geophysical Union, 2007, Vol. 88, No. 2, pp. 14-17.
- Bouman C. A., Shapiro M. A multiscale random field model for Bayesian image segmentation, IEEE Trans. Image Process., 1994, Vol.3, No. 2, pp. 162-177.
- Moore R. T., Hansen M. C. Google Earth Engine: a new cloud-computing platform for global-scale Earth observation data and analysis, AGU Fall Meeting, Abstracts, 2011, Vol.1, pp. 2.
- Neteler, M.; Bowman, M.H.; Landa, M.; Metz, M. GRASS GIS: A multi-purpose open source GIS, Environmental Modelling & Software, 2012, Vol. 31, pp. 124-130.
- Warmerdam F. The geospatial data abstraction library, In: Open Source Approaches in Spatial Data Handling, Springer Berlin Heidelberg, 2008, pp. 87-104.