Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2015, Vol. 12, No. 1, pp. 145-155
Organization of automated multithreaded processing of satellite information in remote monitoring systems
D.A. Kobets
1 , A.M. Matveev
1 , A.A. Mazurov
1 , A.A. Proshin
1
1 Space Research Institute RAS, Moscow, Russia
In the present work, the possibilities of the process control library commonbat.pm realized in PERL are examined. This program product is intended for constructing efficient distributed systems of processing information and control over their functioning. The basic operational objects and concepts of the library are discussed: session, flow and task. The prerequisites for creation of the library and its basic tasks are described in detail; the infrastructure for operation of the library is also described. The structure of the database for statistical information is given. The basic possibilities of the library are briefly discussed, such as parallel starting of applications in different regions of disk space, timeout and emergency termination mechanism of operations, tracking of return codes of started procedures and conducting the detailed logs of all processes with the possibility of their subsequent analysis. Examples and references to different information systems employing the developed methods of satellite data processing are given, such as ISDM of the Russian Federal Forest Service, Vega service, etc. It should be noted that the realized methods and algorithms can also be used in other systems of information processing, including those working with other types of data.
Keywords: data processing, parallel computing, automatic data processing, processing of remote sensing data
Full textReferences:
- Andreyev M.V., Galeyev A.A., Efremov V.J., Ilyin V.O., Krasheninnikova J.S., Lupyan E.A., Mazurov A.A., Nazirov R.R., Proshin A.A., Flitman E.V., Postroenie avtomatizirovannykh sistem sbora, khraneniya, obrabotki i predstavleniya sputnikovykh dannykh dlya resheniya zadach monitoringa okruzhayushchei sredy (Construction of automated systems for collection, storage, processing and representation of satellite data for solving problems of environmental monitoring), Solar-Terrestrial Physics, 2004, Issue 5, pp. 8-11.
- Balashov I.V., Bourtsev M.A., Efremov V.Yu., Loupian E.A., Proshin A.A., Tolpin V.A., Postroenie arkhivov rezul'tatov obrabotki sputnikovykh dannykh dlya sistem dinamicheskogo formirovaniya proizvodnykh informatsionnykh produktov (Development of satellite data processing products archives for dynamic derived information products production systems), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2008, Vol. 5, No. 1, pp. 26-31.
- Bartalev S.A., Bourtsev M.A., Egorov V.A., Efremov V.Yu., Kotelnikov R.V., Loupian E.A., Mazurov A.A., Matveev A.M., Shchetinsky V. E., Organizatsiya raboty so sputnikovymi dannymi vysokogo prostranstvennogo razresheniya v informatsionnoi sisteme distantsionnogo monitoringa lesnykh pozharov Federal'nogo agentstva lesnogo khozyaistva RF (ISDM Rosleskhoz) (The organization of work with high resolution satellite data in the information system of remote monitoring of forest fires of Federal Forestry Agency of Russian Federation (FFA ISDM)), Pozharovzryvobezopasnost' (Fire and explosion safety), Izd. Pozhnauka, 2009, No. 8, pp. 50-56.
- Bartalev S.A., Ershov D.V., Korovin G.N., Kotelnikov R.V., Loupian E.A., Shchetinsky V. E. Osnovnye vozmozhnosti i struktura informatsionnoi sistemy distantsionnogo monitoringa lesnykh pozharov Federal'nogo agentstva lesnogo khozyaistva (ISDM Rosleskhoz) (The main functionalities and structure of the Forest Fire Satellite Monitoring Information System of Russian Federal Forestry Agency (ISDM-Rosleshoz)), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2010, Vol. 7, No. 2, pp. 97-105.
- Bourtsev M.A., Antonov V.N., Efremov V.Yu., Kashnitsky A.V., Kramareva L.S., Loupian E.A., Mazurov A.A., Matveev A.M., Milexin O.E., Proshin A.A., Solovyev V.I., Sistema raboty s raspredelennymi arkhivami rezul'tatov obrabotki sputnikovykh dannykh tsentrov priema NITs "Planeta" (Distributed satellite data processing products archives operation system in SRC Planeta centres), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2012, Vol. 9, No. 5, pp. 55-76.
- Bourtsev M.A., Efremov V.Yu., Mazurov A.A., Matveev A.M., Proshin A.A., Uspensky S.A., Flitman E.V., Raspredelennaya sistema priema i obrabotki dannykh polyarno-orbital'nykh sputnikov v tsentrakh Rosgidrometa (Distributed system for acquisition and processing of polar-orbiting satellites data in RosHydroMet centers), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2008, Issue 5, No. 2, pp. 443-446.
- Egorov V.A., Ilyin V.O., Loupian E.A., Mazurov A.A., Proshin A.A., Flitman E.V., Vozmozhnosti postroeniya avtomatizirovannykh sistem obrabotki sputnikovykh dannykh na osnove programmnogo kompleksa XV_SAT (Possibilities of developing automated satellite data processing systems on the basis of XV_SAT software package), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2004, Issue 1, pp. 431-436.
- Efremov V.Yu., Girina O.A., Kramareva L.S., Loupian E.A., Manevich A.G., Melnikov D.V., Matveev A.M., Proshin A.A., Sorokin A.A., Flitman E.V., Sozdanie informatsionnogo servisa "Distantsionnyi monitoring aktivnosti vulkanov Kamchatki i Kuril" (Creating an information service “Remote Monitoring of Active Volcanoes of Kamchatka and the Kuril Islands”), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2012, Vol. 9, No. 5, pp. 155-170.
- Efremov V.Yu., Krasheninnikova Yu.S., Loupian E.A., Mazurov A.A., Proshin A.A., Flitman E.V., Optimizirovannaya sistema khraneniya i predstavleniya geograficheski privyazannykh sputnikovykh dannykh (Optimized storage and presentation system for georeferenced satellite data), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2007, Issue 4, No. 1, pp. 125-134.
- Efremov V.Yu., Loupian E.A., Mazurov A.A., Proshin A.A., Flitman E.V., Upravlenie i kontrol' rabotosposobnosti sistem avtomatizirovannoi obrabotki sputnikovykh dannykh (Operation control and management of distributed satellite data processing systems), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2004, Issue 1, pp. 467-475.
- Loupian E.A., Mazurov A.A., Nazirov R.R., Proshin A.A., Flitman E.V., Tekhnologiya postroeniya avtomatizirovannykh informatsionnykh sistem sbora, obrabotki, khraneniya i rasprostraneniya sputnikovykh dannykh dlya resheniya nauchnykh i prikladnykh zadach (Development of automated information systems for scientific and application tasks), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2004, pp. 81-89.
- Loupian E.A., Mazurov A.A., Nazirov R.R., Proshin A.A., Flitman E.V., Krasheninnikova Yu.S., Tekhnologii postroeniya informatsionnykh sistem distantsionnogo monitoringa (Technologies for building remote monitoring information systems), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2011, Vol. 8, No. 1, pp. 26-43.
- Loupian E.A., Mazurov A.A., Nazirov R.R., Proshin A.A., Flitman E.V., Krasheninnikova Yu.S., Tekhnologii postroeniya informatsionnykh sistem distantsionnogo monitoringa (Technologies for building remote monitoring information systems), ESIMO'2012 (Conf. The use of funds and resources of the Unified state system of information on the world's oceans to inform maritime activities in the Russian Federation) Book of Abstracts, Obninsk, 25-27 September, 2012, pp. 197.
- Loupian E.A., Savin I.Yu., Bartalev S.A., Tolpin V.A., Balashov I.V., Plotnikov D.E., Sputnikovyi servis monitoringa sostoyaniya rastitel'nosti ("Vega") (Satellite service for vegetation monitoring VEGA), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2011, Vol. 8, No. 1, pp. 190-198.
- Matveev A.M., Mamaev A.S., Proshin A.A., Flitman E.V., Organizatsiya kontrolya nad funktsionirovaniem raspredelennoi sistemy ISDM Rosleskhoz ("ISDM Rosleskhoz" Distributed System Operation Monitoring Organization), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2009, Issue 6, No. 2, pp. 535-541.
- Tolpin V.A., Balashov I.V., Efremov V.Yu., Loupian E.A., Proshin A.A., Uvarov I.A., Flitman E.V., Sozdanie interfeisov dlya raboty s dannymi sovremennykh sistem distantsionnogo monitoringa (sistema GEOSMIS) (The GEOSMIS System: Developing interfaces to operate data in modern remote monitoring systems), Sovremennye problemy distantsionnogo zondirovaniya Zemli iz kosmosa, 2011, Vol. 8, No. 3, pp. 93–108.
- Tolpin V.A., Loupian E.A., Bartalev S.A., Plotnikov D.E., Matveev A.M., Vozmozhnosti analiza sostoyaniya sel'skokhozyaistvennoi rastitel'nosti s ispol'zovaniem sputnikovogo servisa «VEGA» (Possibilities of agricultural vegetation condition analysis with VEGA satellite service), Atmospheric and Oceanic Optics, 2014, Vol. 27, No. 7 (306), pp. 581-586.