Data Processing

From Ecosynth Wiki
Jump to: navigation, search

The Ecosynth data processing pipeline covers the steps necessary for processing images into a 3D point cloud, then post-processing the point cloud data into useful data products like canopy height models (CHMs), digital terrain models (DTMs), and more. This is the most current version of the pipeline; for previous versions including retired platforms and procedures visit the Data Processing Archive.

Contents

Data Processing

There are 3 steps in creating georeferenced 3D point clouds; generate a 3D point cloud with computer vision, georeference the point cloud, and filter the point cloud. Each of these steps can be accomplished with multiple tools. PhotoScan or Ecosynther computer vision (CV) software can be used to generate the point cloud. Georeferencing can be done by supplying camera position coordinates from GPS in the PhotoScan 'Ground Control' tools or running the Ecosynth Aerial spline script. Filtering is accomplished using the Ecosynth Aerial noise removal script, but can be done as part of the fully automated Ecosynth Aerial Main Pre-processing tool or separated in conjunction with the Telemetry Toolset. Below are instructions for each of the processing options.

Prepare Data for Processing

Follow the instructions at EcosynthAerial Pipeline for installing and launching the EcosynthAerial toolset.

Trim Photo Set

The step must be taken regardless of which CV software you choose. Trimming the photo set to include only photo from first to last waypoint, excluding launch/landing waypoints, can be done visually or aided by CV software.

Visually

  1. Examine the photos carefully to locate the location where the copter reaches it's first waypoint and turns to head to the second.
  2. Examine the photos carefully to locate the location where the copter reaches it's last waypoint and turns to head to landing location.
  3. Copy the photos between first and last turn to a separate folder within project folder.

CV Software

  1. Select a subset of photos that includes the first turn of the copter and process this small set quickly in the computer vision software. When the model is complete, you can then see which photo was located at the first turn.
  2. Select a subset of photos that includes the lasts turn of the copter and process this small set quickly in the computer vision software. When the model is complete, you can then see which photo was located at the last turn.
  3. Copy the photos between first and last turn to a separate folder within project folder.

Convert Telemetry

  • If using PhotoScan to produce the 3D point cloud, a Photoscan ground control file needs to be produced from the Arducopter .log file and the folder containing the trimmed photo set. Use the EcosynthAerial 'generate.utilities.interpolate_gps_positions_for_cams' tool to produce the necessary ground control file for Photoscan, following instructions here: EcosynthAerial Generate Utility.
  • If using Ecosynther to produce the 3D point cloud you need to produce a 'GPS Positions' file based on the Arducopter .log file and the 'acquire.utilities.telemetry_to_gps_positions' in the EcosynthAerial Acquire Utility.
  • The following python script file can also be used for executing the necessary commands to produce both a GPS file and camera file for Photoscan:
# ecosynth_parse_telemetry.py
from ecosynth import generate
import sys

#bring in inputs for first parsing the log file to the GPS_positions file
log_file = open(sys.argv[1], 'r')
msl = float(sys.argv[3])
cam_filepath = sys.argv[2]
pscan_file = open(sys.argv[1].split('.')[0] + '_pscan_cams.txt', 'w')
gps_file = open(sys.argv[1].split('.')[0] + '_GPS_positions.txt', 'w')

generate.utilities.interpolate_gps_positions_for_cams(log_file, msl, cam_filepath, pscan_file, gps_file)

log_file.close()
gps_file.close()
pscan_file.close()

This would be executed from a command line as:

python ecosynth_parse_telemetry.py <path_to_log_file> <path_to_folder_with_photos> <float of launch location elevation>

And produces two files, a GPS positions file of just XYZ values for the flight and a Photoscan file with estimates of the camera location for each image from the log file. Ecosynth uses the relative altitude elevation from the Pixhawk log file.

Back to top

Generate 3D Point Clouds with Computer Vision

PhotoScan

These instructions are for running Agisoft PhotoScan v0.9.0 build 1584 (13 October 2012). Refer to the the PhotoScan Manual for specific questions and settings. We often use a screen capture software like Cam Studio to monitor the status of the job by placing the progress indicator in a small area near the system date/clock and making a screen capture every 30 seconds. This makes it possible to identify when the job actually finished.


PhotoScan can be used to create ungeoreferenced 3D point clouds that can then be georeferenced with the Ecosynth Aerial spline script or it can georeference the 3D point clouds using camera position coordinates from the copter's GPS telemetry data. If using PhotoScan to georeference the 3D point cloud it can can also be used to create a georeferenced orthomosaic.

Using PhotoScan to Create Ungeoreferenced 3D Point Clouds

  1. Open PhotoScan
  2. Go to Workflow > Add Photos
    • Add the trimmed flight photos
  3. Go to Workflow > Align Photos
    • Use Accuracy: High, Pair Selection: Generic
  4. Click OK and let it run. This could take 8+ hours depending on the number of photos
  5. Once PhotoScan has completed, save the job to your project folder
  6. Export the point cloud
    • Go to File > Export Points: and select a PLY file for export.
    • Generate an ASCII PLY file, not a binary file, and export colors, but not normals
  7. Export the cameras
    • Go to Tools > Export Cameras
    • Export the point cloud as a Bundler .OUT file, says Yes or OK when prompted that some camera information will be lost, it is OK.

Using PhotoScan to Create Georeferenced 3D Point Clouds

  1. Open PhotoScan
  2. Go to Workflow > Add Photos
    • Add the trimmed flight photos
  3. Upload the file 'cam_xyz_file' created we you used the EcosynthAerial Generate Utility 'generate.utilities.interpolate_gps_positions_for_cams'.
  4. Select appropriate coordinate system in Ground Control Settings (probably UTM WGS84 if the EcosynthAerial Utility was used.)
  5. Go to Workflow > Align Photos
    • Use Accuracy: High, Pair Selection: Ground Control
  6. Click OK and let it run.
    • PhotoScan takes less time to create a model using Ground Control camera positions, but it may still take a few hours.
  7. Once PhotoScan has completed, save the job to your project folder
  8. Export the point cloud
    • Go to File > Export Points: and select a PLY file for export.
    • Generate an ASCII PLY file, not a binary file, and export colors, but not normals
    • NOTE: Follow these steps if using the EcosynthAerial Post-Processing GUI:
      • Export the point cloud from Photoscan as a bundle.OUT file format Tools > Export Cameras, use the 'Local Coordinate System'.
Create Georeferenced Orthomosaic
  1. Build Geometry
    • Go to Workflow > Build Geometry
    • Refer to PhotoScan Manual for setting corresponding to your particular needs
    • This process can take a few hours to complete
  2. Export the orthomosaic
    • Go to File > Export Orthomosaic
    • Refer to PhotoScan Manual for setting corresponding to your particular needs

Ecosynther

Ecosynther is an open source CV software that can be used as an alternative to PhotoScan to create 3D point clouds. A complete set of instruction can be found at the Ecosynther User Guide

Back to top

Point Cloud Post-Processing

After point clouds are created, they need to be georeferenced filtered before products (CHMS) can be produced. Use the Stage 3: Post-Processing and Analysis Tool to process point clouds and automatically produce several canopy surface maps and a color-raster ortho-like product.

  1. Launch the EcosynthAerial tool.
  2. Launch the Post-Processing tool.
    1. Enter the folder name of the project, output files will be created in this folder, and the folder will be placed in the same folder where 'python -m ecosynth' was originally called from.
    2. Enter the value in meters above mean sea level of the altitude of the launch location (the altitude where the copter started from).
    3. Select the Arducopter Log file
    4. Select the bundle.out file produced by Ecosynther or Photoscan in the above steps.
    5. Hit Start Upload.
  • This tool will automatically georeference the point cloud based on the camera locations and GPS telemetry, filter the point cloud for noise, and produce several outputs ready for GIS.
  • NOTE: This tool may run into MemoryError issues when running with point clouds of over 1 million points on a Windows computer. For best results use EcosynthAerial in a 64-bit Ubuntu or Mac OSX OS.

Back to top

Generate a Digital Terrain Model

A digital terrain model (DTM) is a raster data layer that represents the ground surface elevation, typically when aboveground features (e.g., trees, forest canopy, and buildings) have been removed through filtering of 3D point clouds. This filtering can be performed with many different tools and algorithms, but often requires as much art as science in order to get the tool/algorithm parameters to work effectively for a dataset.

A DTM is a necessary component of a 3D remote sensing pipeline for measuring tree height. The elevation of the terrain is subtracted from the elevation of canopy points in order to determine true height above ground.

Previously, we used the ALDPAT and MCC-LIDAR terrain filtering algorithms on Ecosynth point clouds.

Once a point cloud has been filtered to include only ground points, it is necessary to generate a raster DTM.

  1. Add a header to the output file if there is not one already -- typically X,Y,Z is sufficient
  2. Import the layer into ArcGIS, create XY points, and generate a new shapefile of the ground/terrain points
  3. Use interpolation, e.g., Kriging, to generate a raster model from the ground points layer
  4. Crop the raster model to the project AOI

In addition to generating a DTM using a filtering algorithm, it is also possible to use an existing LIDAR DTM. When provided by a LIDAR contractor, these are typically called bare earth products because the contractor has already filtering off the above-ground points.

Back to top

Generate a Canopy Height Model

A canopy height model (CHM) can be a raster or point cloud (vector) dataset where elevation Z values represent the height of that pixel/point above ground. This is different from the output of the #Point Cloud Post-Processing stage that generates a point cloud representing the surface model where each elevation Z value represents height above mean sea level (or geoid or ellipsoid, depending upon the reference datum used).

The CHM is an important data product of the Ecosynth 3D remote sensing pipeline as it is a common dataset used for estimating biomass density, parametrizing fire models, mapping tree crowns and gaps, and predicting species diversity.

Creating a CHM is very simple once a DTM has been created or if an existing LIDAR DTM/bare earth data product is used.

  1. Add a header to the the local filtered point cloud text file that is in the output folder of the point cloud filtering process
    • The header can be simple: X,Y,Z,R,G,B -- but is necessary for ArcGIS to be able to import the file
  2. Import the file into ArcGIS as a text file, and create XY events using the X, Y, and Z fields and in the correct coordinate system. Refer to the ArcGIS help if you are unsure about importing XY coordinates.
  3. Export the XY events layer as a new shapefile
  4. Add in your DTM raster model for this area
  5. Use the Spatial Analyst tool 'Extract values to points' to assign a DTM elevation value to each point cloud point -- this creates a new shapefile with a field RASTERVALU that represents the DTM pixel value below that point
  6. In the new layer that is created from the last step, add a new field 'HEIGHT' (float) and subtract the RASTERVALU from the point Z value
  7. Ta-da! You have now created a layer that shows canopy height instead of elevation!
  • Alternatively, you can also use one of the surface elevation rasters (DSMs) produced in the #Point Cloud Post-Processing stage to produce a CHM by simply subtracting the DTM from the DSM, for example in ArcGIS Raster Calculator.

Back to top

Archive

An archive of this wiki page is located at Data Processing Archive and includes older data processing instructions used in the project, including procedures for using Mikrokopter flight control (ca. 2010), and older processing scripts that have been replaced by the tools available at code.ecosynth.org/ecosynthaerial.

Back to top