THREE-DIMENSIONAL MODELING FROM SINGLE PHOTOGRAPHS

Technology

An interactive technique for modeling 3D man-made objects by extracting them from a single photograph is presented. The technique combines the cognitive ability of humans with the computational accuracy of the machine. To extract an object from a given photo, the user draws its part’s profile and sweeps it using simple gestures, to progressively define a 3D body that snaps to the shape outline in the photo. The generated part adheres to various geo-semantic (geometric and semantic) constraints imposed by the global 3D structure and relations between parts.

The human is involved in perceptual tasks such as recognition, positioning, and partitioning, while the computer performs tasks which are computationally intensive or require accuracy. While the user sweeps the primitive, the computer program dynamically adjusts the progressive profile by sensing the pictorial context on the photograph and automatically snapping to it. With such sweep-snap operations the user models 3D parts that adhere to the object in the photographs, while the computer automatically maintains global constraints with other primitives composing the object. Specifically, we support six types of constraints: parallelism, orthogonality, collinear axis endpoints, overlapping axis endpoints, coplanar axis endpoints and coplanar axes.

The Need

The creation and modeling of 3D objects has always been a difficult task even for professionals.

First, a mental idea of what the model should look like should be formed. This conceptual stage requires creativity and inspiration. Then, this idea needs to be implemented by a series of actions using some geometric modeling tools. These steps take time, demand very high proficiency, and a fair amount of skill.

Potential Application

Once the 3D object is extracted, it can be quickly edited and placed back into photos offering object-driven photo editing tasks which are impossible to achieve in image-space.

3D objects from images also borrow their textures from the image forming at least an initial base 3D model. This model can later be edited, refined and inserted into virtual scenes and environments.

Patents

US patent pending

Project manager

Oren Calfon
VP Business Development, ICT

Project researchers

Daniel Cohen-Or
T.A.U Tel Aviv University, Exact Sciences
School of Computer Science

Ariel Shamir
Interdisciplinary Center Herzliya (IDC),

Tao Chen
,