LiDAR Lab
LiDAR Lab 1
Report
Please provide a written report with the following:
1) One to two sentence description of the LiDAR area you chose, and why. Please include screen shots from your PC of the area in your lab report.
2) Include the code you used for generating the surface models from FUSION and other code from LASTools. If you used the GUI instead, include screenshots of your selected processing options.
3) A two to three sentence description of the analysis you performed on the derivative outputs, i.e. What did you do with the data? Please, include screen shots.
4) In your written report demonstrate you completed at least two of the following lab options:
- DEM/DSM generation using FUSION.
- Change detection using two raster surfaces in a GIS.
- An example of batch processing using at least 3 tiles with FUSION or LASTools.
- 3-point picking or fine registration of two point clouds in CloudCompare.
- Cloud-to-Cloud distance or Multi-scale multi-model cloud to cloud comparison (M3C2) in CloudCompare.
5) A two to three sentence description of the results you accomplished with screen shots.
Part 0: Software Installation
This is already done for you on the Lab Machines. If you wish to install on your personal PC follow these directions:
FUSION
Version Check / Software download
Fusion was one of the earliest open-source softwares to be specifically designed to work with forest LiDAR data. It is still one of the most useful software available for batch processing large quantities of LiDAR data.
Fusion's utility is it's command line batch processing feature. Because batch processes can be complicated we'll stick to the basics of surface model generation for the lab.
LASTools
- Download and install the latest version of LAStools.
- Copy the LASzip.dll file into the FUSION folder (file is saved in directory \LAStools\LASzip\dll).
- this will give you added functionality of working with LAZ format files in FUSION.
LASTools is an advanced software for editing point cloud data - it has numerous 'free' tools, but the higher end production level tools (for making DEM-models) are licenced for purchase.
CloudCompare
Download newest stable release.
CloudCompare has some other dependencies which you may need to identify on your machine and install.
CloudCompare is a handy viewer, with other powerful processing capabilities for working directly with point cloud data.
Optional Software
OSGEO & QGIS
Download the OSGEO4w package for Windows (or Mac or Linux). Download QGIS
The OSGEO ecosystem comes with GDAL, SAGA, GRASS which are powerful GIS analysis tools. SAGA-GIS can work directly with LAS data.
Part 1: Get Data (15 minutes)
- Go to the OpenTopography.org website.
- If you would like, go ahead and register yourself. This will give you access to expanded areas and larger download sizes.
- Select an area with available LiDAR data.
- Download as LAS or LAZ
- Download Ground or All Points.
- Select derivative layers that you wish to also generate from the LiDAR point cloud
- Example Jemez-River Snow-on vs Snow-off LiDAR survey
- Save the files to an easily accessible folder space.
Part 2: FUSION demo workflow (1 hour)
FUSION Manual with detailed explanations of command line options
2.1 Filtering data (10 minutes)
First we want to run the FilterData function to remove outliers (points like birds, or airborne debris which are far above the ground level).
Example below, using data which are in a State Plane coordinate reference system, units in feet, I use the outlier function for points within a 30x30 foot area where points that register as being outside the 5th standard deviation of the z-axis are removed.
FilterData /index outlier 5 30 <Drive>:\<folder>\<output_filename>.las <Drive>:\<folder>\<input_filename>.las
2.2 Digital Surface/Elevation Model generation (10 minutes)
Next, I create a digital surface model using the GridSurfaceCreate function at 3 foot resolution. FUSION generates a Digital Terrain Model (DTM) file.
GridSurfaceCreate <Drive>:\<folder>\<subfolder>_dsm\<output_filename>.dtm 1 f f 2 0 2 2 <Drive>:\<folder>\<input_filename>.las
Before generating a Bare Earth Model several considerations should be made before leaving the point cloud environment:
- Will the data need any interpolation - such as smoothing or filtering?
- In some cases a jagged surface may be preferable, such as in complex terrain.
- Sub-meter models will likely require smoothing to reduce the presence of 0 or NoData values.
- Ordering of script for the median and smooth switches in FUSION will modify the output suface.
- *.LAS point spacing is a primary measure for the need potential for smooth or median filtering.
- Look at the vendor report to determine what the nominal density of the data are
- Run FUSION QA/QC with Catalog first
- *.LAS point spacing is a primary measure for the need potential for smooth or median filtering.
- What is the best neighborhood size of the smoothing or filtering?
- The larger the neighborhood the greater the impact of the spline on the model.
- Do you want to keep the local maxima/minima?
- Keeping minima and maxima in a bare earth surface will create pimples, dimples, or pock marks in the final surface.
- Typically the best aesthetic option is to smooth the data with a 3x3 or 5x5 neighborhood filter.
This function becomes more important when you are working in neighborhoods where roof-tops have likely been classified as non-ground points in the LAS cloud by the vendor, rendering the footprints of houses as empty spaces; when the footprint of a house becomes larger than the size of the spline used to create the bare earth surface a pit will result in the final model.
2.3 Canopy Height Model (10 minutes)
Canopy height models (CHM) are generated when you difference the bare earth elevation from the maximum elevation of the pulses [for our models we used 2 ft canopy height model resolution.
When running CanopyModel you must establish whether you want a canopy surface model [units in elevation above mean sea level], or a canopy height model 35 [units in elevation above bare earth].
Here is an example, of the script we used to generate the canopy height models:
CanopyModel /ground:<Drive>:\<folder>\<subfolder>\<ground_filename>.dtm /outlier:-1,100 <Drive>:\<folder>\<subfolder>\<output_filename>.dtm 2 f f 2 0 2 2 <Drive>:\<folder>\<subfolder>\<input_filename>.las
In this example we used the bare earth model we generated from GridSurfaceCreate specified by the /ground switch.
2.4 Converting DEM, DSM, or CHM to an ASC or TIF file (5 minutes)
Because the DTM format is not consumed by programs like ArcGIS, GRASS, or QGIS you will need to convert the model to a more readily available format. FUSION can convert the DTM to an ASCII file or to a GeoTIFF.
DTM2ASCII <Drive>:\<folder>\<subfolder>\<filename>.dtm
or
DTM2TIF <Drive>:\<folder>\<subfolder>\<filename>.dtm
2.5 Change detection analysis: Creating a DEM of Difference (DoD) or a Canopy Height Model (10 minutes)
If you've chosen an area with more than two LiDAR datasets you can attempt to generate a DEM of Difference or DoD.
Repeat the steps above in the FUSION workflow to generate two DEM, DSM, or CHM rasters.
Load in your favorite GIS platform and use Raster Calculator to difference the two surface models.
2.6 Batch Processing (15 minutes)
Open a command prompt window (type 'cmd' into your windows search bar).
Navigate to the directory where your LiDAR tiles are saved
First I generate a tree list from the working folder directory using MS-DOS code command structure http://ss64.com/nt/dir.html
dir /b /s *.las>filelist.txt
Next, I generated bare earth layers from the classified points (bare earth = class 2):
for /F "eol=; tokens=1* delims=,. " %%i in (D:\plot_trees_CA\Points\filelist.txt) do call create_bare_surface %%i
A second batch file called create_bare_surface:
GridSurfaceCreate /class:2 /median:3 /smooth:3 D:\plot_trees_CA\DTM\%~n1.dtm 1 m m 1 11 2 2 %1.las
Next, I generate the canopy height models and .ASC
for /F "eol=; tokens=1* delims=,. " %%i in (D:\plot_trees_CA\Points\filelist.txt) do call create_chm %%i
CanopyModel /ground:D:\plot_trees_CA\DTM\%~n1.dtm /outlier:-1,80 D:\plot_trees_CA\CHM\%~n1.dtm 0.333 m m 1 11 2 2 %1.las DTM2ASCII D:\plot_trees_CA\CHM\%~n1.dtm
Other FUSION Training Modules
3 CloudCompare (1 hour)
Viewing LAS Data and familiarizing yourself in CloudCompare (15 minutes)
Open CloudCompare
Click and drag a LAS or LAZ file into the CloudCompare space.
Accept all of the properties of the cloud you wish to view (this will vary widely by data set).
Clipping, Shifting Clouds (5 minutes)
After you've loaded a point cloud into CloudCompare you can edit it within the tool.
You can do things like clip or shift the cloud by using the scissors (Segment) tool or the two axis arrow (Translate/Rotate) tool.
3-Point Picking and Fine-registration. (10 minutes)
Two or more clouds can be aligned by selecting reference points for each cloud.
Compute Cloud-to-Cloud distance and M3C2 (10 minutes)
Use the Cloud-to-Cloud distance tool or the Multi-scale model to model cloud comparison tool (M3C2) to directly differentiate distance between two point clouds.
Select two point clouds from the DB Tree using the CTRL command. Once selected several of the tool icons in the header will become active.
Click on the M3C2 icon which is a green cylinder with black points.
Set the comparison you which to do (multi-scale, vertical, horizontal)
LASTools Example (10 minutes)
Here I use LAS tools to read the header of an LAS v 1.4 file.
C:\>cd lastools/bin
C:\LAStools\bin>ls
7z.exe lasboundary.exe lasheight_README.txt lasthin.exe
blast lasboundary_README.txt lasindex.exe lasthin_README.txt
blast2dem.exe lascanopy.exe lasindex_README.txt lastile.exe
blast2dem_README.txt lascanopy_README.txt lasinfo.exe lastile_README.txt
blast2iso.exe lasclassify.exe lasinfo_README.txt lastool.exe
blast2iso_README.txt lasclassify_README.txt laslayers.exe lastool_README.txt
bytecopy.exe lasclip.exe laslayers_README.txt lastrack.exe
e572las.exe lasclip_README.txt lasmerge.exe lastrack_README.txt
e572las_README.txt lascolor.exe lasmerge_README.txt lasvalidate.exe
gzip.exe lascolor_README.txt lasnoise.exe lasvalidate_README.txt
las2dem.exe lascontrol.exe lasnoise_README.txt lasview.exe
las2dem_README.txt lascontrol_README.txt lasoverage.exe lasview_README.txt
las2iso.exe lasdiff.exe lasoverage_README.txt laszip.exe
las2iso_README.txt lasdiff_README.txt lasoverlap.exe laszip_README.txt
las2las.exe lasduplicate.exe lasoverlap_README.txt shp2las.exe
las2las14_README.txt lasduplicate_README.txt lasplanes.exe shp2las_README.txt
las2las_README.txt lasgrid.exe lasplanes_README.txt sonarnoiseblaster.exe
las2shp.exe lasgrid_README.txt lasprecision.exe sonarnoiseblaster_README.txt
las2shp_README.txt lasground.exe lasprecision_README.txt txt2las.exe
las2tin.exe lasground_README.txt lassort.exe txt2las_README.txt
las2tin_README.txt lasground_new.exe lassort_README.txt unzip.exe
las2txt.exe lasground_new_README.txt lassplit.exe
las2txt_README.txt lasheight.exe lassplit_README.txt
C:\LAStools\bin>lasinfo -i D:\Walnut_Gulch\Pilot\LAS\12RWA890120.las
lasinfo report for D:\Walnut_Gulch\Pilot\LAS\12RWA890120.las
reporting all LAS header entries:
file signature: 'LASF'
file source ID: 0
global_encoding: 17
project ID GUID data 1-4: AAAEEA25-068A-4069-BBBF-6FDB0C47767B
version major.minor: 1.4
system identifier: 'Woolpert merged'
generating software: 'GeoCue LAS Updater'
file creation day/year: 328/2015
header size: 375
offset to point data: 827
number var. length records: 1
point data format: 6
point data record length: 30
number of point records: 16018679
number of points by return: 16018210 469 0 0 0
scale factor x y z: 0.01 0.01 0.01
offset x y z: 0 0 0
min x y z: 589000.00 3512000.00 1336.76
max x y z: 589999.99 3512999.99 1691.22
start of waveform data packet record: 0
start of first extended variable length record: 0
number of extended_variable length records: 0
extended number of point records: 16018679
extended number of points by return: 16018210 469 0 0 0 0 0 0 0 0 0 0 0 0 0
variable length header record 1 of 1:
reserved 43707
user ID 'LASF_Projection'
record ID 2112
length after header 394
description 'OGC WKT Coordinate System'
WKT OGC COORDINATE SYSTEM:
PROJCS["WGS_1984_UTM_Zone_12N",GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137,298.257223562997]],PRIMEM["Greenwich",0],UNIT["Degree",0.0174532925199433]], PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000],PARAMETER["False_Northing",0],PARAMETER["Central_Meridian",-111],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0],UNIT["Meter",1]]
the header is followed by 4 user-defined bytes
reporting minimum and maximum for all LAS point record entries ...
X 58900000 58999999
Y 351200000 351299999
Z 133676 169122
intensity 768 65280
return_number 1 2
number_of_returns 1 2
edge_of_flight_line 0 1
scan_direction_flag 0 1
classification 1 18
scan_angle_rank -21 21
user_data 0 1
point_source_ID 28 31
gps_time 126462452.589277 126482542.839642
extended_return_number 1 2
extended_number_of_returns 1 2
extended_classification 1 18
extended_scan_angle -3500 3500
extended_scanner_channel 0 1
number of first returns: 16018210
number of intermediate returns: 0
number of last returns: 16018211
number of single returns: 16017742
WARNING: point type is 6 but (legacy) number of point records in header is 16018679 instead zero.
WARNING: for return 6 point type is 1 but (legacy) number of points by return in header is 16018210 instead zero.
WARNING: for return 6 point type is 2 but (legacy) number of points by return in header is 469 instead zero.
overview over extended number of returns of given pulse: 16017742 937 0 0 0 0 0 0 0 0 0 0 0 0 0
histogram of classification of points:
11854224 unclassified (1)
4164436 ground (2)
1 noise (7)
18 Reserved for ASPRS Definition (18)
+-> flagged as withheld: 19
+-> flagged as extended overlap: 8590239
Compressing LAS to LAZ batch process
I used LASTools to compress a large lidar delivery from LAS to LAZ to achieve greater storage efficiency
First I generated a file list of the directory where the LAS files are stored.
Then I created a call command:
for /F "eol=; tokens=1* delims=,. " %%i in (E:\4FRI_Phase_I\4FRI_LiDAR_as_delivered_2014_01_06\Points\All_Returns\filelist.txt) do call 4fri_laz_create %%i
Next, I used the laszip.exe to compress the files. My files are in UTM Zone 12
laszip -i "%1.las" -odir "E:\4FRI_Phase_I\4FRI_LiDAR_as_delivered_2014_01_06\Points\LAZ" -o "%~n1.laz" -utm 12N