Proceedings of the Fifth International Conference on Information Fusion. FUSION 2002. (IEEE Cat.No.02EX5997)

In support of the AFOSR program in Information Fusion, the CNS Technology Laboratory at Boston University is developing and applying neural models of image and signal processing, pattern learning and recognition, associative learning dynamics, and 3D visualization, to the domain of Information Fusion for Image Analysis in a geospatial context. Our research is focused by a challenge problem involving the emergence of a crisis in an urban environment, brought on by a terrorist attack or other man-made or natural disaster. We aim to develop methods aiding preparation and monitoring of the battlespace, deriving context from multiple sources of imagery (high-resolution visible and low-resolution hyperspectral) and signals (GMTI from moving vehicles, and ELINT from emitters). This context will serve as a foundation, in conjunction with existing knowledge nets, for exploring neural methods in higher level information fusion supporting situation assessment and creation of a common operating picture (COP).

There is an obvious need to be able to integrate both linguistic-based and stochastic-based input information in data fusion. In particular, this need is critical in addressing problems of track association, including cyber-state intrusions. This paper treats this issue through a new insight into how three apparently distinct mathematical tools can be combined: "boolean relational event algebra" (BREA), "one point random set coverage representations of fuzzy sets" (OPRSC), and "complexity-reducing algorithm for near optimal fusion" (CRANOF).

Bayesian networks are an attractive modeling tool for human sensing, as they combine an intuitive graphical representation with efficient algorithms for inference and learning. Temporal fusion of multiple sensors can be efficiently formulated using dynamic Bayesian networks (DBNs) which allow the power of statistical inference and learning to be combined with contextual knowledge of the problem. Unfortunately, simple learning methods can cause such appealing models to fail when the data exhibits complex behavior We first demonstrate how boosted parameter learning could be used to improve the performance of Bayesian network classifiers for complex multimodal inference problems. As an example we apply the framework to the problem of audiovisual speaker detection in an interactive environment using "off-the-shelf" visual and audio sensors (face, skin, texture, mouth motion, and silence detectors). We then introduce a boosted structure learning algorithm. Given labeled data, our algorithm modifies both the network structure and parameters so as to improve classification accuracy. We compare its performance to both standard structure learning and boosted parameter learning. We present results for speaker detection and for datasets from the UCI repository.

Information fusion based on Kalman filtering often suffers from the lack of knowledge about cross correlations between the noise-corrupted signal sources. Covariance intersection filtering provides a general framework for information fusion with incomplete knowledge about the signal sources since it yields consistent estimates for any degree of cross correlation. However, covariance intersection filtering requires optimization of a nonlinear cost function which is a significant drawback with respect to computational complexity. Therefore, a fast covariance intersection algorithm is developed and investigated based on simulation results.

In this work, various linear predictive feature vectors were used to train three different automated neural networks type classifiers for the task of isolated vowel recognition. The features used included linear prediction filter coefficients, reflection coefficients, log area ratios, and the linear predictive cepstrum. The three neural network classifiers used are the multilayer perceptron, radial basis function and the probabilistic neural network. The linear predictive cepstrum of dimension 12 is the best feature especially when training is done on clean speech and testing is done on noisy speech. Three different classifier fusion strategies (linear fusion, majority voting and weighted majority voting) were found to improve the performance. Linear fusion with varying weights is the best method and is most robust to noise.

Measurement and track fusion in decentralised sensor network architectures is investigated. The investigation employs FLAMES/sup TM/, an advanced military scenario generator. This was specifically customised for distributed data fusion experiments and involves a model of the delays in a realistic communication system. Here the delays were used to modify communication bandwidth and evaluate how this affected the performance of the fusion architectures/algorithms. Under certain scenario conditions, it was found that the decentralised measurement fusion system was severely affected by reduced bandwidth. This is because it does not scale: each node loads its communication buffer with every measurement and consequently some measurements are never transmitted. The decentralised track fusion system is a better performer because it does scale: measurements are fused into tracks prior to transmission and thereby more effective use of bandwidth is made. Moreover, it was found that a partially connected decentralised track fusion system achieved almost optimal fused track performance.

Terrain categorization and target detection algorithms applied to hyperspectral imagery (HSI) typically operate on the measured reflectance (of sun and sky illumination) by an object or scene. Since the reflectance is a non-dimensional ratio, the reflectance by an object is nominally not affected by variations in lighting conditions. Atmospheric correction (referred to as atmospheric compensation, characterization, etc.) algorithms (ACAs) are used in applications of remotely sensed HSI data to correct for the effects of atmospheric propagation on measurements acquired by air and space-borne systems. The fast line-of-sight atmospheric analysis of spectral hypercubes (FLAASH) algorithm is an ACA created for HSI applications in the visible through shortwave infrared (Vis-SWIR) spectral regime. FLAASH derives its 'physics-based' mathematics from MODTRAN4.

A formal framework is proposed for defining data fusion processes. Particularly the notion of qualified gain is proposed: gain related to representation, completeness, accuracy and certainty. These notions are applied to a medical monitoring and diagnosis problem where a dynamic Bayesian network is used to model time series of observations and evolving states. The model aims at giving a daily diagnosis. Experiments are under way using data of an already existing system collected on kidney disease patients. Results are be characterized using our notion of qualified gains.

In this paper, we introduce a probabilistic model based on Bayesian networks (BNs) for inferring human fatigue by integrating information from various visual cues and certain relevant contextual information. First, we briefly review the modern physiological and behavioral studies on human fatigue to identify the major causes for human fatigue and the significant factors affecting fatigue. These factors are then extracted from those studies and form the contextual information variables in our fatigue model. Visual parameters, typically characterizing the cognitive states of a person including parameters related to eyelid movement, gaze, head movement, and facial expression, serve as the sensory observations in the fatigue model. The fatigue model is subsequently parameterized based on the statistics extracted from recent studies on fatigue and on our subjective knowledge. Such a model provides mathematically coherent and sound basis for systematically aggregating visual evidences from different sources, augmented with relevant contextual information. The inference results produced by running the fatigue model using Microsoft BNs engine MSBNX demonstrate the utility of the proposed framework for predicting and modeling fatigue.

An architecture to extend the USAF Joint Battlespace Infosphere (JBI) publish and subscribe paradigm is described. The architecture emphasizes the user side of the JBI architecture where the users' computing devices can vary substantially in bandwidth, processing capability, screen size, and connection type. Today's Internet user can connect with the net at any time using a wide variety of devices. Our emphasis is on designing a infrastructure that will intelligently support the efficient rapid access and dissemination of distributed knowledge base repositories to a client independent of computing device or connection. The architecture is presented via five high level designs and views. Our goal is to design and develop an infrastructure that will efficiently and rapidly provide data, knowledge and information to a dynamic user. A hardware ontology is presented for aiding our architecture to provide information tailored for any device a dynamic user may acquire.