Computer vision software automatically creates 3D point clouds from numerous overlapping digital photos without any initial information about the location and orientation of cameras with respect to where the photos were taken.
These 3D point clouds are the data we use to estimate characteristics of the 3D structure of vegetation. This process also associates each 3D point with a color extracted from the source photographs, which may be useful for investigating vegetation spectral characteristics.
Computer vision software follows the basic workflow shown in the figure at right.
- Potential features are identified within each input image, then features are automatically matched across all images to identify relationships between images.
- Next the photogrammetric bundle adjustment is performed based on the image-feature matching input to solve for the optimal 3D location of images, cameras, and features.
- The output is a 3D point cloud of image features and camera locations as shown at the bottom of the figure. This image shows a side view of a computer vision point cloud of a forest with camera points in purple and canopy points in reds, yellows, and greens.
We are currently working with and quantifying the differences between each software package:
Bundler - Photo Tourism Structure from Motion
Bundler is an open-source software developed by Noah Snavley as a Photo Tourism tool for the vast collection of photographs on the internet. The code makes use of the popular SIFT algorithm for automatically finding features in images. We used this software for our first paper on Ecosynth.
Agisoft Photoscan - Advanced Image-Based 3D Modeling Software
The Agisoft Photoscan software performs similar computer vision based 3D reconstructions of photo sets, outputting camera information and 3D point clouds. The software is available as a free trial download for viewing only and can be upgraded to a standard version so that point clouds can be exported or a pro version that offers additional photogrammetric tasks. It is based off of the general computer vision model as described above but with proprietary algorithms.
VisualSFM(VSFM) - A Visual Structure from Motion System
VSFM is another open-source software similar to Noah Snavley's Bundler. VSFM has been optimized to make use of GPU and multicore parallelism. VSFM processes both a sparse 3D construction as well as a dense reconstruction. VSFM is significantly faster than Bundler, but not as fast as Agisoft Photoscan. However as an open-source software it is free for anyone to use freely.
Microsoft Photosynth - Online Scene 3D Resconstruction
Note: Photosynth is no longer used by Ecosynth.
Microsoft's Photosynth is a web-based application for creating and navigating 3D reconstructions of images using computer vision and is based on Noah's Bundler research. Photosynth incorporates additional processing to create a dense 3D point cloud, whereas Bundler outputs a sparse point cloud. Photosynth is significantly faster than Bundler, although it is unknown how this is implemented. Having been available for several years, Photosynth has been cited in several conference proceedings as a tool for generating digital terrain models and 3D objects.
Ecosynth Available as AWS Image
We now have a publicly available AWS image with Ecosynther preloaded onto it that you can mirror and run your datasets through. Instructions for getting started can be found on the blog here: Ecosynth Blog