Image understanding, the ultimate goal of computer vision, needs a representation of the scene that includes objects and their relations. Traditional approaches to three-dimensional (3D) scene recovery try to obtain a nearly exact geometric reconstruction of the 3D scene. The solutions found need either multiple views or user interaction. This paper proposes an innovative approach by considering surface estimation as a recognition problem, from a single view, and without user interaction except for the needed a priori training process.

Intuitively, real-world understanding is a progressive process that advances from simple to more complex knowledge. The proposed algorithm is just such a process: it starts with a low-level segmentation that is based on simple cues, and then it segments again by grouping regions of the previous segmentation and computing more complex cues that need increased spatial support. A key point is that it computes a set of different segmentations to avoid misclassification that can cause a unique bad segmentation. The output is a rough scene reconstruction where we have a labeled image with surface classes such as sky, support, horizontal surface, and vertical surface that are also subclassified according to planarity and orientation. As Hoiem et al. notice, this scene layout lacks relationship information regarding the surfaces in the 3D space.

Although the model is simple, I find the approach very promising. However, as one of the most interesting applications presented is vision-based navigation, I would have appreciated comments about processing time and the possibility of real-time processing.