key: cord-0058627-m5epf1ks authors: Barrile, Vincenzo; Bilotta, Giuliana; Fotia, Antonino; Bernardo, Ernesto title: Road Extraction for Emergencies from Satellite Imagery date: 2020-08-19 journal: Computational Science and Its Applications - ICCSA 2020 DOI: 10.1007/978-3-030-58811-3_55 sha: c2e1ea24aaf50f4d6f44593ff6024c30520136ee doc_id: 58627 cord_uid: m5epf1ks After earthquakes, international and national organizations must overcome many challenges in rescue operations. Among these, the knowledge of the territory and of the roads is fundamental for international aid. The maps that volunteers make are a valuable asset, showing the roads in the area affected by the seismic events, a knowledge which is necessary to bring rescue. This was very helpful during many earthquakes as in Haiti (on 2010-01-12) and in Nepal (on 2015-04-25) to support the humanitarian organizations. Many volunteers can contribute remotely to mapping little known or inaccessible regions with crowdsourcing actions, by tracing maps from satellite imagery or aerial photographs even if staying far from the affected site. This research, still in progress, aims at experiencing quickly obtaining roads through the so-called Object Based Image Analysis (OBIA), by extracting it from satellite data, semi-automatically or automatically, with a segmentation that starts from concepts of Mathematical Morphology. We compared it with a classification in ENVI and, using an algorithm in GIS, we verified the goodness of the method. The good results obtained encourage further research on fast techniques for map integration for humanitarian emergencies moreover the results were implemented on open street map. Traditional surveying methods are often time-consuming and laborious; instead, development of remote sensing technology in recent years has opened the way to an automatic road detection application. Many are the advantages of satellite remote sensing technology: among others, it can provide nearly real-time updated maps. Taken into consideration optical images, different types of roads (highways, rural roads or streets) are identifiable and classifiable in different areas (rural or suburbs), depending on the spatial resolution. Until a few years ago, some topographic features as shape and structure were not describable or verifiable because of the insufficient spatial resolution of the satellite imagery used for Earth Observation [1] . Thus, the application of these data was greatly limited in fields such as analysis and monitoring of the urban environment [2] . Again, the increasing resolution in the images recently available brings to an increasing ambiguity in the statistical definition of the land use classes or ground cover and, in case of application of the standard multispectral classification, pixel-based, to a decrease of the accuracy of automatic detection [3] . We have chosen to apply OBIA as an analysis capable of extracting high semantic level information from the simple radiometric data acquired by satellite sensors, thanks to the structural approach that we will illustrate later. The strength of this type of analysis in particular is in the ability to extract objects by segmenting the image according to rules that are chosen ad hoc, favoring for example the content of the pixel over the shape or taking into account the shape of the objects to be extracted. In Fig. 1a we report multiresolution segmentation example, that does not take into account the shape factor. Our study area is in Calabria; we considered an area in the Province of Reggio Calabria (Fig. 2a) : Melito di Porto Salvo (Fig. 2b) , that has a population of about 11000 inhabitants. Calabria, the southernmost region of the Italian peninsula, has a mainly hilly surface, which covers 49.2% of its territory, has large mountainous areas that cover 41.8% of its territory, while plains occupy 9% of the region's territory. The province of Reggio Calabria is located in the exact center of the Mediterranean Sea and is composed almost entirely of municipalities overlooking the coasts. Streams and rivers, often seasonal, flow from the internal mountains and the Aspromonte massif. The Strait of Messina separates Reggio Calabria, its capital, from the island of Sicily. The territory of Melito di Porto Salvo is the southernmost of mainland Italy and is characterized by the presence of torrential waterways, with a pebbly bed and dry for most of the year. As satellite imagery, we processed a GeoTIFF from IKONOS-2 acquired on 2001-07-01, 09:51 GMT, and 4.00 meters for the pixel size. It is a multispectral IKONOS image with zero as percent Cloud Cover. Spatial resolution of these images seems suitable for the purpose, considering the usual dimensions of the road network. Image was orthorectified; no further processing was therefore required. The methodology we used is not the classic analysis pixel based of satellite data. This analysis has a limit: the recognition of a semantic information, of low-level, derived only from the energy emitted by the pixel, by measuring its amount, which does not take into account the context [4] . A structural approach like that of OBIA, instead, rises the semantic level by adding topological information and statistics, spatial relationship rules, defining the context. Very similar to this is the manual photo interpretation, but this approach overcomes the limits of a subjective classification: in fact, it is homogenous and can be reproduced. However, these analysis tools that can add the information (structural and morphological) contained in the images to the simple spectral content of the pixels, are from a long time used in other fields such as processing of medical images [5] . Concepts of Mathematical Morphology [6] lead this image analysis to a recognition of objects [7] and a classification based on Fuzzy Logic principles [8] . Furthermore, to every rule we can assign an appropriate weight [9] . We get a structural analysis with high semantic level [10] allowing a wealth of information that we cannot achieve with the pixel-based spectral analysis. Thanks to the direct extraction of vector maps from satellite imagery, it is possible an immediate and full integration in GIS. Moreover, the possibility of introducing rules for the relationships between the obtained objects [11] and the recognition of the context significantly makes the photo interpretation process reproducible, increases recognition and improves the extraction (automatic or semi-automatic) of objects that are on the Earth's surface. OBIA operates a segmentation of the whole image by adequately setting the parameters of shape and color. In this case the appropriate choice of rules for segmentation, as a certain interval similar to perimeter/area ratio (indeed the ratio between the image object border length and, attributable to the area of the image object, the square root of the pixels number). This rule allows recognizing and extracting objects that have an elongated shape, and this will discriminate from the others the shapes of rivers and roads. The rules on spectral content, instead, easily can distinguish roads from rivers. Therefore, it is possible to export quickly the data obtained in GIS or also in Open Street Map, collaborative open source tools. This automated migration process is much faster and more efficient than manual integration which, in addition to not being homogeneous, is also subject to many variables. By choosing an appropriate scale factor, we can size largeness of the achieved polygons, because the scale setting is related to the scale that we want to achieve in cartography. The segmentation process, as will, is multiresolution: in fact, starting from the same image, we can obtain some levels in a hierarchy of polygons at different scale factors. When we reduce the scale factor, the generated polygons are smaller and smaller; therefore, there is less color variability within the polygons, while we have the opposite by increasing the scale factor. There is a relationship among the polygons in these hierarchical levels. All the polygons of lower hierarchical level have a geometrical relationship with polygons of higher level in the hierarchy. Every lower level polygon belongs only to one higherlevel polygon. All the polygons of all segmentation levels constitute a single database in which are all the relationships between the polygons of the same and of different hierarchical levels. The adjacent polygons on a hierarchical level, as the polygons of a lower hierarchical level, as the polygon in which the examined polygon in the upper hierarchical level is contained, are therefore known for each polygon (Fig. 3, Fig. 4) . Furthermore, there is a characteristic circular interaction between processing and classification of image objects for the object-based approach. Specific information segmentation-based, image objects' shape, and scale are all useful for classification. Moreover, the classification, including semantic information and details related to the context, uses the attributes of the image objects as well as the relationship of image objects connected in network, resulting so in an operational classification model. The required geoinformation and required objects are gradually extracted, through classificationprocessing iterative cycles (Fig. 5) , useful for many applications. Therefore, processing units (image objects) are constantly changing shape, mutual relationships as, so, classification. Analogous to the processes of human understanding images, this type of circular processing translates into intermediate states that follow one another, with a growing variation of the class and information more and more abstract about the starting image. In the example explained, we performed a segmentation of the scene on one level, but we can carry out a multiresolution and multilevel segmentation, and therefore a classification. We operate a multiresolution segmentation obtaining the automatic creation of vector polygons, extracted directly from the image and coinciding in the overlapping on raster; thus the final classification prepares a suitable class hierarchy taking into account relationships between the segmentation levels achieved. The segmentation technique starts with one-pixel objects: it is a technique of merging the region from the bottom up. In the following steps, segmentation produces smaller image objects increasingly merged into larger objects [12] . During the clustering process, and through the process, the optimization procedure decreases nh, that is the weighted heterogeneity of the image objects obtained (n is the segment size and h the definition of heterogeneity, arbitrary). For each step, adjacent image objects merge to represent the smallest growth in the chosen heterogeneity. The described process stops when this growth overcomes the threshold we defined with the scale parameter. Multiresolution segmentation is thus a procedure for optimization that acts locally. We obtain the color (or spectral) heterogeneity by summing, in each layer, the standard deviations of the spectral values weighted with the weights we assigned to each layer: where: h s is spectral or color heterogeneity; r c is st. dev. of digital number in c spectral band; q is bands number; w c instead is the weight we assign to c spectral band. If we only perform a minimization of the spectral heterogeneity, however, we obtain image objects or branched segments with a fractally shaped edge [13] , with a heavy effect in structured data, as in radar data. Therefore, we consider it helpful that the spectral heterogeneity criterion is mixed with the spatial heterogeneity criterion, for reducing the deviation from a shape smooth or compact. Here, we describe heterogeneity as deviation from a compact shape as the ratio between the length l of the border and the square root of the pixels (to be precise, the number of pixels) that make up the image object. where: h g_smooth = fractal factor for spatial heterogeneity; l = length of the border; n = in the image object, the number of pixels. The compactness factor (h g_compact ) is the second; it depends from the ratio of polygon axis dimensions: with h g_compact is the factor of compactness; l is the length of the border; b, instead, is the shortest length of the border given by the bounding box on imageobject parallel to the raster. As described, the algorithm of segmentation merges the adjoining polygons starting from each pixel of the image up to the variation of heterogeneity between the two previous polygons and therefore until the new polygon does not overcome the userdefined scale factor, that is the threshold. Until the change in heterogeneity does not overcome the defined threshold, the merge is actually carried out; when the change overcomes the threshold, the two polygons stay separate. The variation in heterogeneity (overall fusion value) among the resulting object and the two starting polygons is: with f = overall fusion value; w f = user defined weight of color, it is against shape. For w must be chosen a value between 0 and 1, where 0 and 1 are also possible: for w f = 1 is just valued the heterogeneity of shape, while for w f = 0 is valued solely the heterogeneity of color. Dh s is the variation in spectral or color heterogeneity between the achieved polygon and the two starting polygons: where: n merge = achieved polygon pixel number; r mergec = st. dev. of digital number in c -spectral band (resultant polygon); n obj1 = pixel number of the first of the two starting polygons; r obj1c = st. dev. of digital number in c -spectral band of the first of the polygons before that they merged; n obj2 = pixel number of the second starting polygon, before the fusion; r obj2c = st. dev. of digital number in c -spectral band of the second polygon before the fusion. Moreover, the variation Dh g in shape heterogeneity, caused by the merge, is assessed by calculating the variation between after and before the fusion. This involves the calculation methods for compactness and smoothness: w g is the user defined weight for smoothness (remember that it is against compactness). For w we must chose a value between 0 and 1, with 0 and 1 possible: if w f = 1 is valued only smoothness, if w f = 0 is valued only compactness. Dh n is the object size, l the perimeter of the object, b the bounding box perimeter. In this case study, multiresolution segmentation created only a single level with the parameters we indicated in the following table, which takes into account the features of the IKONOS dataset (Table 1 ): We applied the chosen level of segmentation on the four bands of the IKONOS dataset. Therefore, we can identify the long shapes by assigning a very high value to the form factor (0.9) and, consequently, a 0.1 value to the color content. We also assign a minimum factor of 0.1 to compactness to identify objects which have a strong perimeter development. The scale factor is 40 because we do not want an overly fragmented image. However, the fragmentation in some areas is quite strong. Figure 6 shows segmentation with identification of objects. We can see some long objects matching waterways, roads and streets. Figure 7 shows variability of the pixels intra-polygon, which would be even greater choosing a largest scale factor. We applied the method illustrated in this article with a version of the eCognition software, created by Definiens and released by Trimble in recent years. Other commercial software for satellite image processing such as ENVI of L3Harris Geospatial can perform image segmentation similarly to what obtained with eCognition in this application. In that case, however, it is necessary to proceed first with the classification of the image and only afterwards focus on the segmentation and extraction of the objects, while in eCognition the classification is based precisely on the previous segmentation. In ENVI, the segmentation process as Feature Extraction is based on a watershed by immersion algorithm developed by Vincent and Soille [14] that equates pixel DN values in an image with elevation points on a topographic surface. Figure 8 shows classification in ENVI, using the maximum likelihood algorithm, in Fig. 9 is the related confusion matrix. Moreover, Spectral Angle Mapper (SAM) is a physically based spectral classification that uses an n-D angle to match pixels to reference spectra. The algorithm determines the spectral similarity between two spectra by calculating the angle between the spectra and treating them as vectors. With SAM, by varying the classification parameters, firstly the riverbeds only were identified (Fig. 10 ) and then the riverbeds and roads (Fig. 11) . Through a process of subtraction between the two images, we obtained the roads. However, in this classification we identified only a few larger roads. We then superimposed on GIS the open street maps (OSM) layers, the segmentationvector -achieved by eCognition, the classifications of ENVI and the result obtained from the aforementioned subtraction. By comparing the distances between the lines that identify the outline of the vector layers, we can compare the results obtained. For this reason, we implemented a function on QGIS for automating the process of calculation of the distance between the different roads, thanks to the field calculator as the starting point in the program attribute table. Results obtained demonstrate that the object classification, compared to a pixelbased classification, gives better results because the deviation between the different layers exceeds one meter and therefore the latter cannot be used without further processing to refine the result (Fig. 12 ). Next step is the import of these shapefile into Open Street Map (Fig. 13 ). Various attempts were made over time to extract road lattices from satellite images, as the need for an automatic road extraction method is also felt due to the continuous development of transport networks. Some of these have been applied to data from radar satellites, which have the advantage of being free from the influence of the weather: the satellite-mounted synthetic aperture radar (SAR). Some studies propose automatic discrimination methods based on a deep neural network (DNN) adapted for roads from single and double polarization SAR Sentinel-1 images, extending the convolutional neural network (CNN). It is adapted for road extraction from images SAR analyzing the potential of using the fully convolutional neural network or Fully Convolutional Network (FCN) [15] , which works for semantic segmentation. SWT (Stroke Width Transform) is an artificial vision algorithm (an image operator) used in Computer Vision to detect text in images. A non-traditional use of this algorithm has been proposed for the extraction of the road network from an optical satellite image [16] . Results obtained with segmentation are encouraging; the precision is very high, since we extracted the objects directly from the image. The structural methodology (using multiresolution segmentation techniques, and then achieving a classification object-based), unlike the classic spectral or pixel-based analysis, is able to make the best use of the wealth of information detected with remotely sensed data, with rapid integration into GIS. Moreover, it allows the direct and quick production of vector maps [17] . There are many possible applications of OBIA [18, 19] . In this work, we have shown that rapid extraction of roads is possible; these results suggest further research on fast techniques for map integration for the purposes of humanitarian emergencies. Constantly updated satellite imagery can be very useful when speed in an emergency becomes vital. Metodologie "Strutturali" su immagini Satellitari per l'analisi Urbana e Territoriale Multiresolution analysis of urban reflectance Texture analysis for urban pattern recognition using fine-resolution panchromatic satellite imagery Classification and feature extraction for remote sensing images from urban areas based on morphological transformations Texture segmentation by biologically-inspired use of neural networks and mathematical morphology Approccio strutturale all'analisi di immagine per la descrizione del territorio: una esplorazione degli strumenti di morfologia matematica Multi-resolution, object-oriented fuzzy analysis of remote sensing data for GIS-ready information Object-oriented analysis applied to high resolution satellite data Advances in mathematical morphology applied to geoscience and remote sensing A hierarchical fuzzy classification approach for high resolution multispectral data over urban areas Object based image analysis for remote sensing An application of object-oriented analysis to very high resolution satellite data on small cities for change detection Watersheds in digital spaces: an efficient algorithm based on immersion simulations A new road extraction method using Sentinel-1 SAR images based on the deep fully convolutional neural network A road extraction method based on high resolution remote sensing image Metodologie avanzate applicate allo studio dell'uso della terra Sistema integrato per il rilievo e la gestione del catasto delle aree incendiate Individuazione di discariche mediante segmentazione del dato satellitare