The DLR 3D-Modeler (short: 3dMo) is a hand-held close-range scan device which allows the online creation of 3D models of arbirary objects, without artificial markers or other references. The system was developed at the German Aerospace Center (DLR) (see also 3dMo @ DLR).


3dmo_imu.jpg

Up to this project the scanner was globally referenced

  • by a stereo camera system and six spheres mounted on the scan device or
  • by the inverse kinematics of a robot arm, when mounted on its wrist.
These external systems restrict the workspace of the scanner and increase its costs and its complexity.



The idea arose to use the 2 cameras on the 3dMo to calculate the position of the device while scanning. Therefore, a visual localization algorithm was developed, which fullfills following constraints:

  • real-time capable at a standard frame-rate (25 Hz)
  • restricted computational resources - especially no special hardware, like e.g. GPU
  • can deal with high dynamics, as typical in hand-held systems (rotational and translational velocities of up to 75° and 0.5m/s)

3dmo_live.jpg

Following steps were necessary to provide such an image-based motion estimation:

  • accurate KLT-based stereo initialization
  • implementing a sped up, patch-based KLT tracking algorithm
  • using the data of an inertial measurement unit (IMU) for landmark propagation
  • a robustified motion estimation based on VGPS
  • an intelligent feature management



Videos

Following video shows the tracking without KLT extension or IMU integration.
3dMo tracking without any propagation

This video shows tracking with linear propagated KLT tracking.
3dMo tracking with linear propagation

And here we have the combination of linear and IMU-based propagation:
3dMo tracking with linear and IMU-based feature propagation


And here we have a scan sequence
3dMo live modeling



People

German Aerospace Center (DLR)
Dipl.-Ing. Klaus Strobl
Dipl.-Ing. Tim Bodenmueller
Dr.-Ing. Michael Suppa

MVP @ TUM
Elmar Mair, M.Sc.
Prof. Dr.-Ing. Darius Burschka