Representations and techniques for 3D object recognition and scene interpretation Derek Hoiem, Silvio Savarese

eng

Part of: Synthesis digital library of engineering and computer science

1. Background on 3D scene models -- 1.1 Theories of vision -- 1.1.1 Depth and surface perception -- 1.1.2 Awell-organized scene -- 1.2 Early computer vision and AI -- 1.3 Modern computer vision

2. Single-view geometry -- 2.1 Consequences of projection -- 2.2 Perspective projection with pinhole camera: 3D to 2D -- 2.3 3D measurement from a 2D image -- 2.4 Automatic estimation of vanishing points -- 2.5 Summary of key concepts --

3. Modeling the physical scene -- 3.1 Elements of physical scene understanding -- 3.1.1 Elements -- 3.1.2 Physical interactions -- 3.2 Representations of scene space -- 3.2.1 Scene-level geometric description -- 3.2.2 Retinotopic maps -- 3.2.3 Highly structured 3D models -- 3.2.4 Loosely structured models: 3D point clouds and meshes -- 3.3 Summary --

4. Categorizing images and regions -- 4.1 Overview of image labeling -- 4.2 Guiding principles -- 4.2.1 Creating regions -- 4.2.2 Choosing features -- 4.2.3 Classifiers -- 4.2.4 Datasets -- 4.3 Image features -- 4.3.1 Color -- 4.3.2 Texture -- 4.3.3 Gradient-based -- 4.3.4 Interest points and bag of words -- 4.3.5 Image position -- 4.3.6 Region shape -- 4.3.7 Perspective -- 4.4 Summary --

5. Examples of 3D scene interpretation -- 5.1 Surface layout and automatic photo pop-up -- 5.1.1 Intuition -- 5.1.2 Geometric classes -- 5.1.3 Approach to estimate surface layout -- 5.1.4 Examples of predicted surface layout -- 5.1.5 3D reconstruction using the surface layout -- 5.2 Make3D: depth from an image -- 5.2.1 Predicting depth and orientation -- 5.2.2 Local constraints and priors -- 5.2.3 Results -- 5.3 The room as a box -- 5.3.1 Algorithm -- 5.3.2 Results -- 5.4 Summary --

Part II. Recognition of 3D objects from an image --

6. Background on 3D recognition -- 6.1 Human vision theories -- 6.1.1 The Geon theory -- 6.1.2 2D-view specific templates -- 6.1.3 Aspect graphs -- 6.1.4 Computational theories by 3D alignment -- 6.1.5 Conclusions -- 6.2 Early computational models --

7. Modeling 3D objects -- 7.1 Overview -- 7.2 Single instance 3D category models -- 7.2.1 Single instance 2D view-template models -- 7.2.2 Single instance 3D models -- 7.3 Mixture of single-view models -- 7.4 2-1/2D layout models -- 7.4.1 2-1/2D layout by ISM models -- 7.4.2 2-1/2D layout by view-invariant parts -- 7.4.3 2-1/2D hierarchical layout models -- 7.4.4 2-1/2D layout by discriminative aspects -- 7.5 3D layout models -- 7.5.1 3D layout models constructed upon 3D prototypes -- 7.5.2 3D layout models without 3D prototypes --

8. Recognizing and understanding 3D objects -- 8.1 Datasets -- 8.2 Supervision and initialization -- 8.3 Modeling, learning and inference strategies --

9. Examples of 2D 1/2 layout models -- 9.1 Linkage structure of canonical parts -- 9.1.1 The view-morphing formulation -- 9.1.2 Supervision -- 9.2 View-morphing models -- 9.2.1 Learning the model -- 9.2.2 Detection and viewpoint classification -- 9.2.3 Results -- 9.3 Conclusions --

Part III. Integrated 3D scene interpretation --

10. Reasoning about objects and scenes -- 10.1 Objects in perspective -- 10.1.1 Object size -- 10.1.2 Appearance features -- 10.1.3 Interaction between objects and scene via object scale and pose -- 10.2 Scene layout -- 10.3 Occlusion -- 10.4 Summary --

11. Cascades of classifiers -- 11.1 Intrinsic images revisited -- 11.1.1 Intrinsic image representation -- 11.1.2 Contextual interactions -- 11.1.3 Training and inference -- 11.1.4 Experiments -- 11.2 Feedback-enabled cascaded classification models -- 11.2.1 Algorithm -- 11.2.2 Experiments -- 11.3 Summary --

12. Conclusion and future directions -- Bibliography -- Authors' biographies

One of the grand challenges of artificial intelligence is to enable computers to interpret 3D scenes and objects from imagery. This book organizes and introduces major concepts in 3D scene and object representation and inference from still images, with a focus on recent efforts to fuse models of geometry and perspective with statistical machine learning