Cross-platform lidar and sfm-mvs data fusion for dryland ecosystem monitoring. 

Authors: Tyson L. Swetnam^1*, Temuulen T. Sankey², Mitchell P. McClaran¹, Mary Nichols³, Jason McVay², Phillip Heilman³

1 School of Natural Resources and the Environment, University of Arizona, Tucson AZ

2 Informatics and Computing Program, Northern Arizona University, Flagstaff AZ

3 USDA Southwest Watershed Research Center, Agricultural Research Service, Tucson, AZ 85719

*Corresponding author

1064 East Lowell Street 

Tucson Arizona 85719

email: tswetnam@email.arizona.edu 

phone: 1 (520) 621-1052

Abstract

Dryland ecosystems exhibit long periods of senescance punctuated by rapid growth following seasonal precipitation events. Monitoring vegetation via remote sensing to capture new growth as well as changes to structure from herbivory and disturbance requires high spatial and temporal precision. Difficulties to collecting accurate information persist across sensor platforms and at different scales in space and time. In the present study we identify specific strengths and weaknesses for several different modalities, including terrestrial and aerial platforms including sUAS and manned aircraft. Following the summer monsoon in early October 2015 we collected (1) sUAS structure from motion multi-view stereo (sfm-mvs) and (2) lidar, (3) terrestrial lidar, and (4) manned aircraft lidar over the Walnut Gulch Experimental Watershed (WGEW), a long term United States Agricultural Research Station (USDA-ARS) research site located east of the town of Tombstone, Arizona (30.74° N, -110.05° W). In March, May, August and September 2016 we again collected sUAS sfm-mvs and terrestrial lidar over the Santa Rita Experimental Range (SRER), another long term USDA-ARS site south of the city of Tucson, Arizona (31.80° N, -110.84° W). We found aerial lidar (both sUAS and manned aircraft) to be more precise at identifying bare ground elevation than terrestrial lidar or sUAS sfm-mvs, but less precise in measuring herbaceous vegetation than terrestrial lidar or uUAS sfm-mvs. The cost of collecting manned aircraft lidar and sUAS lidar also precluded them from high frequency data collection, whereas the sUAS was not limiting. Despite the utility of the sUAS for monitoring vegetation phenology and structure, the overhead of computational processing became a limiting factor at progressively larger scale (both spatial and temporal).  

Introduction

Keeping pace with technological innovation and analyzing the breath of remotely sensed information available for ecosystem monitoring now far surpasses the skillset of any single traditional field ecologist. Monitoring of species and ecosystems has always been limited by what Levin termed the 'problem of pattern and scale'. Yet today, technology ever more quickly lowers these barriers. Ecologists are no longer limited to working in small monitoring plots and extrapolating observations to estimate the characteristics. They can measure entire ecosystems (and ecoregions) via manned aircraft equiped with lidar and hyper-spectral sensors. At more local scales they monitor hectare size areas with centimeter and even milimeter precision using small unmanned aerial vehicles (sUAS). Three dimensional structure from motion (sfm) reconstruction of surface features and vegetation from passive visible light cameras also promises to greatly enhance our ability to monitor ecosystem. 

Over the last half century remote sensing from aerial and space-based platforms have immeasurably changed the earth and environmental sciences. Conventional orthophotography and satellite imagery are by their sensor design, two dimensional (2-D) data, measured across a wide range of spatial, spectral, and radiometric resolutions. More recently, these passive sensor technologies have begun to be supplemented and to some extent surpassed at the scientific forefront, by laser-based active sensor technology, specifically light detection and range (lidar), which has become the dominant technology used in the earth sciences for measuring the three dimensional structure of vegetation and earth surface phenomena (Glennie et al. 2013, Harpold et al. 2015). Despite the sensor advances the measurement and re-measurement of ecological and geophysical phenomena are still limited by the resolution of the sensors and their availability across time and space (Woodcock and Strahler 1987, Turner 1989, Turner et al. 1989, Levin 1992). In general, we are very good at measuring things at high spatial, spectral, radiometric, or temporal resolution but typically no more than two of these dimensions at one time. 

Dryland ecosystems, characterized as regions where evaporation and vegetation transpiration exceed precipitation, cover 41% of the terrestrial Earth surface (Millenium ecosystem assessment, 2005). 

From 2D to 3D, and back again.

The dominant modes of Geographic Information Systems (GIS) (Estes and Star 1990, Coppeck and Rhind 1991) are moving away from grid (so-called 'raster') and 2-D vector based maps to also include 3-D point and mesh surfaces of volume spaces. Rasters cannot represent all of the information in a 3-D space at one time, nor do they accurately describe 3-D features such as vegetation or vertical relief like cliffs and river banks (Lague et al. 2013). Raster derivatives of point cloud data which are sometimes described as 2.5-D data include: bare earth Digital Elevation Models (DEM)(Kraus and Pfeifer 2001, Sithole and Vosselman 2004), Digital Surface Models (DSM) which include vegetation and human structures on the surface (Zhang et al. 2003), Canopy Height Models (CHM) of vegetation's height above ground level (St-Onge and Achaichia 2001, Lefsky et al. 2002, Popescu et al. 2002), and statistical metrics of the point cloud vertical profile (Reutebuch et al. 2005, McGaughey 2009) which consist of evenly spaced uniform grids. In any case, the availability of 3-D data are increasing and cost decreasing at a rate faster than most organizations can adapt into their land management planning and monitoring protocols.

Discriminating physical surfaces, e.g. bare ground, bathymetry, or infrastructure, versus biological features, e.g. vegetation, from 3-D point clouds to produce accurate DEM and DSM models involve numerous computational techniques with varying levels of complexity (Kraus and Pfeifer 2001, Zhang et al. 2003, Hutton and Brazier 2012). Classification of physical and biological features from point cloud data (Brodu and Lague 2012) require numerous parameterizations which can be both computationally intensive and time consuming to design. Final surface model precision and accuracy further depend upon the user's intended application and may not be desirable for all users, e.g. a fine textured surface for measuring soil properties may be prohibitively inefficient for modelling surface flooding and run-off from precipitation. SfM mapping from terrestrial platforms (Castillo et al. 2015) and from UAS (Rango et al. 2009, Harwin and Lucieer 2012, Anderson and Gaston 2013) suggest that in non-vegetated terrain SfM, if collected with rigorous planning and attention to detail is as good as ALS or even TLS. 

The critical threshold above which a new technology must reach in order to warrant replacing an older established method or technology should be that its added benefit out weighs the associated costs of collecting it and the effort it takes to process the data and get results. Limitations of the technology should also be considered, i.e. whether a loss in precision or accuracy, or a change in spatial or temporal resolution below an existing technique are acceptable given the benefit added. For example, if SfM collected from UAS is as accurate as ALS but its cost per hectare (ha^-1) is significantly lower than the benefit added in UAS-SfM's temporally deeper resolution outweighs the loss in total spatial coverage obtainable by manned-ALS or manned-SfM.  

In the present study we compare four different 3-D remote sensing techniques: terrestrial laser scanning (TLS), UAS aerial laser scanning (UAS-ALS), manned aircraft aerial laser scanning (M-ALS), and UAS-SfM in two different vegetation cover types (shrubland and semi-arid grassland). We were interested in first determining the cross-platform precision, accuracy, and uncertainty and second, identifying applications for two new and less expensive technologies available for UAS: SfM and ALS, relative to two other well established but much more expensive systems: TLS and manned ALS. 

Lidar suffers from high start up costs for equipment and training. Terrestrial Laser Scanning (TLS) units, differential-corrected Global Positioning Systems (dGPS), and Total Stations, which are required to create a precisely geo-referenced scan cost tens to hundreds of thousands of US dollars to purchase or rent and require specialized training to operate. Similarly, aerial laser scanning (ALS) which covers far larger areas at nominally lower resolutions than TLS has an even higher start up investment cost for the aircraft, pilots and technicians, lidar equipment, dGPS, onboard inertial motion unit (IMU), and computational equipment resulting in manned ALS collections being limited to single site visits, or long intervals between revisits.

Table 1: Spatial and temporal resolution, point density, mapping characteristics, and costs of data collection for various 3-D mapping techniques.

*Fails to find bare earth surfaces whenever there is a significant vegetation component.

More recently developed 3-D remote sensing techniques include Structure from Motion (SfM) with multi-view stereo (MVS) or photogrammetric detection and range (phodar) which derive from traditional stereoscopy techniques (i.e. parallax height measurement) and utilize rapid computational algorithms that produce dense 3-D point cloud data which are nearly equivalent to ALS and TLS in accuracy, precision, and point density (Westoby et al. 2012, Fonstad et al. 2013, Smith et al. 2015). Further, SfM-MVS can be conducted from small unmanned aerial systems (sUAS) (Smith et al. 2015) which increases the frequency of data acquisition, but only across smaller footprint relative to traditional ALS. SfM-MVS also benefits from a much lower lower start-up cost (Dandois and and Ellis 2013). SfM-MVS is limited in several ways in which lidar is not: (1) it is a passive sensor technology and is dependent upon illumination of its targets from another light source typically from the sun; and (2) its capacity to generate accurate bare earth models is increasingly limited in areas with dense vegetation cover (Westoby et al. 2012, Nouwakpo et al. 2015). Advancement in sUAS equipped with various active and passive sensor technology portend yet another scale change in the availability of remotely sensed data for natural resource managers and earth scientists (Rango et al. 2009, Harwin and Lucieer 2012, Anderson and Gaston 2013, Javernick et al. 2014). The potential for multi-temporal (daily to hourly) data at high spatial resolution (<1 cm pixel or voxel size) at intermediate scale (1^-1 to 100^-1 hectares [ha^-1]) makes UAS highly desirable for ecological monitoring as these temporal and spatial scales correspond well to eddy covariance flux tower fetch footprints (Beland et al. 2015) as well as hot spots of activity for geophysical changes following disturbances (Lague et al. 2013). With the veritable fire hose of data streaming in from the field data scientists need to develop best practices for data management and data analysis which will ensure the benefits added by technology are achieved to their fullest.

The temporal repeatability of SfM-MVS collections from sUAS suggest we can now observe structural characteristics and change in ecosystems and landscapes which far exceed those of manned ALS.

Establishing where the limitations of these new remote sensing technology are remains a critical component for managers and researchers interested in developing new monitoring protocols and testing scientific hypotheses. For example, terrestrial laser scanning (TLS) has precision and accuracy (<1cm³), point count (10⁶-10⁹), and point density (10⁵ ppm^-2) which far exceeds other technologies, but it is hampered by the short heights above ground level from which it can be collected. TLS scans often have occlusions from objects such as vegetation which block line of sight. Passive sensor remote sensing which includes SfM-MVS is much less expensive than laser scanning, but has numerous dependencies which must be established prior to collecting high quality data, e.g. establishment of multiple ground control points (GCPs), close attention to light environment, sensor resolution, and angle of the image collection; SfM also suffers from an apparent inability to penetrate dense vegetation, in the way lidar can (Dandois and Ellis 2010, 2013, James and Robson 2014). What has not been clearly investigated is how a cross-sensor platform data fusion can eliminate these limitations in such a way that we can maximize our ability to remotely sense our environment over time and space at higher resolutions than are possible with any one technology. In natural resource management this scaling limitation is of particular concern; when the scale of the area under management are so large it is fiscally or physically impossible to collect enough field sampled information to create a statistically robust characterization of ecosystem or landscape phenomena, remote sensing may be required to answer an important question. 

Table 2: Nomenclature used in the text.

Study Area

The Walnut Gulch Experimental Watershed (WGEW) is a long term United States Agricultural Research Station (USDA-ARS) research site located east of the town of Tombstone, Arizona (30.74° N, -110.05° W). The soils vary from high carbonate soils on the western lower watershed to XXX on the eastern upper watershed (Keefer et al. 2008).

For this study we selected the fetch footprint of two eddy covariance flux towers (both towers are part of the AMERIFLUX network): Lucky Hills Shrubland (US-Whs), and Kendall Grassland (US-Wkg). Lucky Hills is characterized as a Chihuahuan desert scrub (Scott et al. 2006, Scott 2010) the dominant species include: Larrea tridentata (creosote), Vachellia vernicosa (white thorn acacia), Flourensia cernua (tarbush), Parthenium incanum (mariola), Rhus microphylla (Little-leaf desert sumac), Condalia warnockii (Warnock's Snakewood), Ephedra spp. (Mormon tea). The Kendall Grassland site is characterized as a semi-arid desert grassland (Scott et al. 2010) dominant species includes: Eragrostis lehmanniana (Lehmann's lovegrass) an invasive grass, and Prosopis spp. (mesquite), native grasses include Hilaria belangeri (curly mesquite), Bouteloua eriopoda (black grama), Bouteloua hirsuta (hairy grama), and Aristida hamulosa (threeawn) (Skirven et al. 2008), other species include Yucca baccata (Banana yucca), Yucca elata (Soaptree yucca), and Agave palmeri (Palmer's agave).

Methodology

Data are broken up into a cross-comparison of four different techniques: TLS, sUAS-ALS, manned ALS, and SfM. The TLS data are treated as the 'observed' reference value based on their absolute positional accuracy which is over an order of magnitude finer than the next closest technology. Establishing absolute location on the Earth's geoid based on GPS readings from different instruments results in inherent errors related to the positional accuracy of the GPS, here we relied on a real time kinematic (RTK) system. To avoid adding additional levels of uncertainty to our comparisons we align the individual point clouds using control points that are clearly identifiable in each dataset.     

1. Georeferencing and alignment of point clouds

Propogation of Uncertainty

Sources of error in the final models include the propogation of errors from each each technology (the measurement error) as well as the model error when using interpolated or mean estimates.

1. The error propagation for two or more measurements with different levels of uncertainty (δA, δB, and δC) are calculated as the squared values to the 1/2 power whose result, R, has the uncertainty: δR = (dA² + dB² + dC²)^1/2 

 USGS Lidar Base Specification (Heidemann 2014) Quality Level 1 (QL1) must have an absolute vertical error of < 29.4 cm in vegetation and < 19.4 cm for non-vegetated surfaces.

The Woolpert lidar on WGEW had a tested 9.6 cm vertical accuracy at a 95 percent confidence level, derived according to NSSDA, in open terrain using 4.9 cm (RMSEz) x 1.96000. Tested against the TIN using independent checkpoints against all points.

The TLS unit has an estimated uncertainty of 6 mm at 100 m distance from the sensor (Reigl doc). 

Some of the error in the subsequent model generation from sfm may be due to rolling-shutter artifacts during flight (Ref.) 

Terrestrial lidar

We used a Reigl VZ-400 terrestrial laser scanner to scan the locations around the two study sites. Immediately prior to both of the sUAS flights and the TLS scanning on 10/8/2015 we established six two meter tall, 10 cm diameter cylinder, lidar targets atop existing Total Station control points for use as control of the TLS scans. At Kendall Grassland the targets were erected atop two USGS benchmarked points (Mary and Michelle: can you fill out this paragraph with how the ground pins was established - also ensure that these statements are factually correct) and four total station and RTK located pins across the drainage from the USGS benchmarks. RiSCAN-PRO? At Lucky Hills the targets were erected upon (Mary: need information about the target location over pins at Lucky Hills) XXXX

Manned Aerial lidar
Aerial lidar was made by Woolpert Inc. over the entire WGER three weeks prior to the first terrestrial laser scanning at Kendall. The lidar has a nominal point spacing of (proposed*0.35m) with relative vertical accuracy of <=8 cm RMSEz between adjacent swaths and a maximum of +-16 cm. The aerial lidar was surveyed using a Real-Time Kinematic GPS Survey as well as a Rapid-Static GPS Survey

Small Unmanned Aerial 

1. Fixed-wing sUAS and Structure from Motion
    The fixed-wing sUAS (eBee) is a small, electric platform with a single pusher propeller at the rear (SenseFly, Switzerland). It is launched by hand and belly lands with a maximum total take-off weight of 750 gr. It is equipped with a custom flight-planning software and ground control station which provides built-in safety redundancies. The fixed-wing sUAS was designed to collect geometrically-corrected aerial photography, used for creating orthorectified photo-mosaics, as well as photogrammetrically produced synthetic 3-dimensional point clouds via structure from motion. The platform can be equipped with one of several sensors at a time, including a visible, near infrared (NIR), multispectral, and thermal camera. We tested the multispectral sensor with four bands: green (520-580nm), red (630-690nm), red edge (720-750nm), and NIR (760-820nm).

   The fixed-wing sUAS data were processed in eMotion and Postflight Terra 3D software (SenseFly, Switzerland), which orthorectifies and mosaics the image tiles. The Postflight Terra 3D software also generates 3D point cloud data via structure from motion using tiepoints from individual image tiles taken from different angles. The fixed-wing sUAS flights were performed at an average flight altitude of 110 m with 80% and 70% vertical and horizontal overlap between image tiles. Given the flight altitude, the multispectral images had 15 cm resolution. The image tiles were successfully georeferenced and calibrated (100%) to generate the synthetic 3D point cloud. 

2. We did not establish any ground control targets for the eBee in Walnut Gulch. Instead we rely on several local features (fence posts, the eddy covariance flux tower, road features) which are scanned by the TLS unit. 

3. Octocopter sUAS and Aerial lidar

The octocopter sUAS (Service-Drone, Germany) weighs 5.5 kg and was developed to carry an additional heavy payload of up to 6.5 kg. The octocopter is controlled via a hand-held remote control transmitter and a ground control station with navigation data link, which sends waypoint navigation information to the craft live from a laptop computer. The octocopter was custom-designed to carry an inertial navigation system (INS), a lidar scanner, and a hyperspectral sensor with a data storage unit on a 3-axis gimbal. The INS has an integrated survey-grade Global Navigation Satellite System (GNSS) and an inertial motion unit (IMU) that correct for errors associated with pitch, roll, and heading (0.05º, 0.05º and 0.5º RMS, respectively) (SBG Systems North America, Inc., Chicago, IL). The hyperspectral sensor is a pushbroom nano-sensor with 272 spectral bands ranging 400-1000nm (Headwall Photonics Inc., Fitchburg, MA). The Velodyne HDL-32E lidar scanner can operate at up to 80-100 m maximum flight altitude and produces 3-dimensional laser point cloud data with 32 laser beams/scan and +/- 2 cm accuracy at 40º vertical field of view and 360º horizontal field of view (Velodyne Acoustics, Inc., Morgan Hill, CA). The spot diameter of the laser beam is 0.03 m² at each beam at 70 m flight altitude. The lidar point density/m² can vary depending on flight altitude and speed. Average point density observed in this study was 35 points/m².