Journal of Cognitive Neuroscience

We have developed a near-real-time computer system that can locate and track a subject's head, and then recognize the person by comparing characteristics of the face to those of known individuals. The computational approach taken in this system is motivated by both physiology and information theory, as well as by the practical requirements of near-real-time performance and accuracy. Our approach treats the face recognition problem as an intrinsically two-dimensional (2-D) recognition problem rather than requiring recovery of three-dimensional geometry, taking advantage of the fact that faces are normally upright and thus may be described by a small set of 2-D characteristic views. The system functions by projecting face images onto a feature space that spans the significant variations among known face images. The significant features are known as "eigenfaces," because they are the eigenvectors (principal components) of the set of faces; they do not necessarily correspond to features such as eyes, ears, and noses. The projection operation characterizes an individual face by a weighted sum of the eigenface features, and so to recognize a particular face it is necessary only to compare these weights to those of known individuals. Some particular advantages of our approach are that it provides for the ability to learn and later recognize new faces in an unsupervised manner, and that it is easy to implement using a neural network architecture.

Positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) have been extensively used to explore the functional neuroanatomy of cognitive functions. Here we review 275 PET and fMRI studies of attention (sustained, selective, Stroop, orientation, divided), perception (object, face, space/motion, smell), imagery (object, space/ motion), language (written/spoken word recognition, spoken/ no spoken response), working memory (verbal/numeric, object, spatial, problem solving), semantic memory retrieval (categorization, generation), episodic memory encoding (verbal, object, spatial), episodic memory retrieval (verbal, nonverbal, success, effort, mode, context), priming (perceptual, conceptual), and procedural memory (conditioning, motor, and nonmotor skill learning). To identify consistent activation patterns associated with these cognitive operations, data from 412 contrasts were summarized at the level of cortical Brodmann's areas, insula, thalamus, medial-temporal lobe (including hippocampus), basal ganglia, and cerebellum. For perception and imagery, activation patterns included primary and secondary regions in the dorsal and ventral pathways. For attention and working memory, activations were usually found in prefrontal and parietal regions. For language and semantic memory retrieval, typical regions included left prefrontal and temporal regions. For episodic memory encoding, consistently activated regions included left prefrontal and medial-temporal regions. For episodic memory retrieval, activation patterns included prefrontal, medial-temporal, and posterior midline regions. For priming, deactivations in prefrontal (conceptual) or extrastriate (perceptual) regions were consistently seen. For procedural memory, activations were found in motor as well as in non-motor brain areas. Analysis of regional activations across cognitive domains suggested that several brain regions, including the cerebellum, are engaged by a variety of cognitive challenges. These observations are discussed in relation to functional specialization as well as functional integration.

Event-related potentials (ERPs) associated with face perception were recorded with scalp electrodes from normal volunteers. Subjects performed a visual target detection task in which they mentally counted the number of occurrences of pictorial stimuli from a designated category such as butterflies. In separate experiments, target stimuli were embedded within a series of other stimuli including unfamiliar human faces and isolated face components, inverted faces, distorted faces, animal faces, and other nonface stimuli. Human faces evoked a negative potential at 172 msec (N170), which was absent from the ERPs elicited by other animate and inanimate nonface stimuli. N170 was largest over the posterior temporal scalp and was larger over the right than the left hemisphere. N170 was delayed when faces were presented upside-down, but its amplitude did not change. When presented in isolation, eyes elicited an N170 that was significantly larger than that elicited by whole faces, while noses and lips elicited small negative ERPs about 50 msec later than N170. Distorted human faces, in which the locations of inner face components were altered, elicited an N170 similar in amplitude to that elicited by normal faces. However, faces of animals, human hands, cars, and items of furniture did not evoke N170. N170 may reflect the operation of a neural mechanism tuned to detect (as opposed to identify) human faces, similar to the “structural encoder” suggested by Bruce and Young (1986). A similar function has been proposed for the face-selective N200 ERP recorded from the middle fusiform and posterior inferior temporal gyri using subdural electrodes in humans (Allison, McCarthy, Nobre, Puce, & Belger, 1994c). However, the differential sensitivity of N170 to eyes in isolation suggests that N170 may reflect the activation of an eye-sensitive region of cortex. The voltage distribution of N170 over the scalp is consistent with a neural generator located in the occipitotemporal sulcus lateral to the fusiform/inferior temporal region that generates N200.

A core brain network has been proposed to underlie a number of different processes, including remembering, prospection, navigation, and theory of mind [Buckner, R. L., & Carroll, D. C. Self-projection and the brain. Trends in Cognitive Sciences, 11, 49–57, 2007]. This purported network—medial prefrontal, medial-temporal, and medial and lateral parietal regions—is similar to that observed during default-mode processing and has been argued to represent self-projection [Buckner, R. L., & Carroll, D. C. Self-projection and the brain. Trends in Cognitive Sciences, 11, 49–57, 2007] or scene-construction [Hassabis, D., & Maguire, E. A. Deconstructing episodic memory with construction. Trends in Cognitive Sciences, 11, 299–306, 2007]. To date, no systematic and quantitative demonstration of evidence for this common network has been presented. Using the activation likelihood estimation (ALE) approach, we conducted four separate quantitative meta-analyses of neuroimaging studies on: (a) autobiographical memory, (b) navigation, (c) theory of mind, and (d) default mode. A conjunction analysis between these domains demonstrated a high degree of correspondence. We compared these findings to a separate ALE analysis of prospection studies and found additional correspondence. Across all domains, and consistent with the proposed network, correspondence was found within the medial-temporal lobe, precuneus, posterior cingulate, retrosplenial cortex, and the temporo-parietal junction. Additionally, this study revealed that the core network extends to lateral prefrontal and occipital cortices. Autobiographical memory, prospection, theory of mind, and default mode demonstrated further reliable involvement of the medial prefrontal cortex and lateral temporal cortices. Autobiographical memory and theory of mind, previously studied as distinct, exhibited extensive functional overlap. These findings represent quantitative evidence for a core network underlying a variety of cognitive domains.

We describe a comprehensive linear approach to the problem of imaging brain activity with high temporal as well as spatial resolution based on combining EEG and MEG data with anatomical constraints derived from MRI images. The "inverse problem" of estimating the distribution of dipole strengths over the cortical surface is highly underdetermined, even given closely spaced EEG and MEG recordings. We have obtained much better solutions to this problem by explicitly incorporating both local cortical orientation as well as spatial covariance of sources and sensors into our formulation. An explicit polygonal model of the cortical manifold is first constructed as follows: (1) slice data in three orthogonal planes of section (needle-shaped voxels) are combined with a linear deblurring technique to make a single high-resolution 3-D image (cubic voxels), (2) the image is recursively flood-filled to determine the topology of the gray-white matter border, and (3) the resulting continuous surface is refined by relaxing it against the original 3-D gray-scale image using a deformable template method, which is also used to computationally flatten the cortex for easier viewing. The explicit solution to an error minimization formulation of an optimal inverse linear operator (for a particular cortical manifold, sensor placement, noise and prior source covariance) gives rise to a compact expression that is practically computable for hundreds of sensors and thousands of sources. The inverse solution can then be weighted for a particular (averaged) event using the sensor covariance for that event. Model studies suggest that we may be able to localize multiple cortical sources with spatial resolution as good as PET with this technique, while retaining a much finer grained picture of activity over time.

Nine previous positron emission tomography (PET) studies of human visual information processing were reanalyzed to determine the consistency across experiments of blood flow decreases during active tasks relative to passive viewing of the same stimulus array. Areas showing consistent decreases during active tasks included posterior cingulate/precuneous (Brodmann area, BA 31/7), left (BAS 40 and 39/19) and right (BA 40) inferior parietal cortex, left dorsolateral frontal cortex (BA S), left lateral inferior frontal cortex (BA 10/47), left inferior temporal gyrus @A 20), a strip of medial frontal regions running along a dorsal-ventral axis (BAs 8, 9, 10, and 32), and the right amygdala.

Experiments involving language-related processes tended to show larger decreases than nonlanguage experiments. This trend mainly reflected blood flow increases at certain areas in the passive conditions of the language experiments (relative to a fixation control in which no task stimulus was present) and slight blood flow decreases in the passive conditions of the nonlanguage experiments. When the active tasks were referenced to the fixation condition, the overall size of blood flow decreases in language and nonlanguage tasks were the same, but differences were found across cortical areas. Decreases were more pronounced in the posterior cingulate/precuneous (BAS 31/7) and right inferior parietal cortex (BA 40) during language-related tasks and more pronounced in left inferior frontal cortex (BA 10/47) during nonlanguage tasks.

Blood flow decreases did not generally show significant differences across the active task states within an experiment, but a verb-generation task produced larger decreases than a read task in right and left inferior parietal lobe (BA 40) and the posterior cingulate/precuneous (BA 31/7), while the read task produced larger decreases in left lateral inferior frontal cortex (BA 10/47). These effects mirrored those found between experiments in the language-nonlanguage comparison.

Consistent active minus passive decreases may reflect decreased activity caused by active task processes that generalize over tasks or increased activity caused by passive task processes that are suspended during the active tasks. Increased activity during the passive condition might reflect ongoing processes, such as unconstrained verbally mediated thoughts and monitoring of the external environment, body, and emotional state.

“Theory of mind,” the ability to make inferences about others' mental states, seems to be a modular cognitive capacity that underlies humans' ability to engage in complex social interaction. It develops in several distinct stages, which can be measured with social reasoning tests of increasing difficulty. Individuals with Asperger's syndrome, a mild form of autism, perform well on simpler theory of mind tests but show deficits on more developmentally advanced theory of mind tests. We tested patients with bilateral damage to orbito-frontal cortex (n = 5) and unilateral damage in left dorsolateral prefrontal cortex (n = 5) on a series of theory of mind tasks varying in difficulty. Bilateral orbito-frontal lesion patients performed similarly to individuals with Asperger's syndrome, performing well on simpler tests and showing deficits on tasks requiring more subtle social reasoning, such as the ability to recognize a faux pas. In contrast, no specific theory of mind deficits were evident in the unilateral dorsolateral frontal lesion patients. The dorsolateral lesion patients had difficulty only on versions of the tasks that placed demands on working memory.

Whether observation of distress in others leads to empathic concern and altruistic motivation, or to personal distress and egoistic motivation, seems to depend upon the capacity for self-other differentiation and cognitive appraisal. In this experiment, behavioral measures and event-related functional magnetic resonance imaging were used to investigate the effects of perspective-taking and cognitive appraisal while participants observed the facial expression of pain resulting from medical treatment. Video clips showing the faces of patients were presented either with the instruction to imagine the feelings of the patient (“imagine other”) or to imagine oneself to be in the patient's situation (“imagine self”). Cognitive appraisal was manipulated by providing information that the medical treatment had or had not been successful. Behavioral measures demonstrated that perspective-taking and treatment effectiveness instructions affected participants' affective responses to the observed pain. Hemodynamic changes were detected in the insular cortices, anterior medial cingulate cortex (aMCC), amygdala, and in visual areas including the fusiform gyrus. Graded responses related to the perspective-taking instructions were observed in middle insula, aMCC, medial and lateral premotor areas, and selectively in left and right parietal cortices. Treatment effectiveness resulted in signal changes in the perigenual anterior cingulate cortex, in the ventromedial orbito-frontal cortex, in the right lateral middle frontal gyrus, and in the cerebellum. These findings support the view that humans' responses to the pain of others can be modulated by cognitive and motivational processes, which influence whether observing a conspecific in need of help will result in empathic concern, an important instigator for helping behavior.

Measures of brain activation (e.g., changes in scalp electrical potentials) have become the most popular method for inferring brain function. However, examining brain disruption (e.g., examining behavior after brain injury) can complement activation studies. Activation techniques identify regions involved with a task, whereas disruption techniques are able to discover which regions are crucial for a task. Voxel-based lesion mapping can be used to determine relationships between behavioral measures and the location of brain injury, revealing the function of brain regions. Lesion mapping can also correlate the effectiveness of neurosurgery with the location of brain resection, identifying optimal surgical targets. Traditionally, voxel-based lesion mapping has employed the chi-square test when the clinical measure is binomial and the Student's t test when measures are continuous. Here we suggest that the Liebermeister approach for binomial data is more sensitive than the chi-square test. We also suggest that a test described by Brunner and Munzel is more appropriate than the t test for nonbinomial data because clinical and neuropsychological data often violate the assumptions of the t test. We test our hypotheses comparing statistical tests using both simulated data and data obtained from a sample of stroke patients with disturbed spatial perception. We also developed software to implement these tests (MRIcron), made freely available to the scientific community.

PET images of blood flow change that were averaged across individuals were used to identify brain areas related to lexical (single-word) processing, A small number of discrete areas were activated during several task conditions including: modality-specific (auditory or visual) areas activated by passive word input, primary motor and premotor areas during speech output, and yet further areas during tasks making semantic or intentional demands.