Expandable Bayesian Networks for 3-D Object Description from Multipl Views and Multiple Mode Inputs

Abstract

Computing 3-D object descriptions from images is an important goal of computer vision. A key problem here is the evaluation of a hypothesis based on evidence that is uncertain. There have been few efforts on applying formal reasoning methods to this problem. In multi-view and multi-mode object description problems, reasoning is required on evidence features extracted from multiple images and non-intensity data. One challenge here is that the number of the evidence features varies at runtime because the number of images being used is not fixed and some modalities may not always be available. We introduce an augmented Bayesian network, the expandable Bayesian network (EBN), which instantiates its structure at runtime according to the structure of input. We introduce the use of hidden variables to handle correlation of evidence variables across images. We show an application of an EBN to a multi-view building description system. Experimental results show that the proposed method gives significant and consistent performance improvement over some earlier methods.

Challenges

  • # of image-based evidence features varies w.r.t. # of images !!
    • Example: 5 features from images, 2 features from other sources.
    • The classifier should handle 7, 12, or 17 features according to the # of images.
    • Some evidence features may be missing.
  • Simple combination of classifiers may not work:
    • Some evidence features may be correlated across images (Y1 and Y2).
    • Some evidence features may be correlated within an image (W and Z).
    • No proper way to weight image-based evidence and evidence from other sources.

    Expandable Bayesian Network (EBN)

  • EBN is a Bayesian network template which is instantiated at realtime. It has repeatable nodes (for example, Y, W, and Z in the figure).
  • Non-repeatable hidden nodes (for example, H) handles correlation across images.
  • Correlation within an image is also handled (W and Z).

    EBN Learning

    Learn parameters (CPT entries) on the network instances and project the parameters into constraint space: all the CPT instances have identical values (commutativity).

    Application on Building Detection

    Compared with combination of naive Bayesian classifiers and plain Bayesian networks (10 stratified 5-fold cross-validation):

    Publications

  • Z. Kim, Multi-View 3-D Object Description with Uncertain Reasoning and Machine Learning, Ph.D. Thesis, CS Dept, USC, 2001. [PDF]
  • Z. Kim and R. Nevatia, "ExpandableBayesian Networks for 3-D Object Descriptions from Multiple Views and Multiple Mode Inputs," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 25, no. 6, 2003. [PDF]
  • Z. Kim and R. Nevatia, "Learning Bayesian Networks for Diverse and Varying Numbers of Evidence Sets," Proc. International Conference on Machine Learning, pp. 479-486, 2000. [PDF]

    • [Home]

    Disclaimer on IEEE publications: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.
    IEEE material: Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.