SfT is a computer vision method meant to solve problems in object detection and single-image 3D reconstruction using a deformable 3D model for objects which can substantially deform.
The template is the deformable 3D model that SfT uses. It comprises a textured 3D shape model and a deformation law but can accomodate more information on the object.
SfT takes an input image and a template. SfT can also be process and take advantage of a monocular video showing the object in deformation or a set of unrelated monocular images showing a variety of deformations of the object.
First, SfT achieves detection: it answers whether the object represented by the template is present or absent of the input image. Second, if the object is found, SfT outputs the object's 3D shape, as observed in the input image. SfT may also compute parameters of the camera which took the input image.
Yes, SfT registrer the template to the input image. In other words, it computes correspondences between point whose coordinates are in object model and points whose coordinates are in the input image.
Owing to its youth, there is currently no commercial application of SfT. However, SfT is the first method which facilitates live Augmented Reality and motion capture for deformable objects imaged by a monocular camera. It forms a basis to achieve Augmented Reality guided laparoscopy for deformable organs, special effects in post-production and 3D measurements of deformable objects.
Website created and maintained by Adrien Bartoli.