Functional Magnetic Resonance Imaging (fMRI) provides dynamical access into the complex functioning of the human brain, detailing the hemodynamic activity of thousands of voxels during hundreds of sequential time points. One approach towards illuminating the connection between fMRI and cognitive function is through decoding; how do the time series of voxel activities combine to provide information about internal and external experience? Here we seek models of fMRI decoding which are balanced between the simplicity of their interpretation and the effectiveness of their prediction. We use signals from a subject immersed in virtual reality to compare global and local methods of prediction applying both linear and nonlinear techniques of dimensionality reduction. We find that the prediction of complex stimuli is remarkably low-dimensional, saturating with less than 100 features. In particular, we build effective models based on the decorrelated components of cognitive activity in the classically-defined Brodmann areas. For some of the stimuli, the top predictive areas were surprisingly transparent, including Wernicke's area for verbal instructions, visual cortex for facial and body features, and visual-temporal regions for velocity. Direct sensory experience resulted in the most robust predictions, with the highest correlation ($c \sim 0.8$) between the predicted and experienced time series of verbal instructions. Techniques based on non-linear dimensionality reduction (Laplacian eigenmaps) performed similarly. The interpretability and relative simplicity of our approach provides a conceptual basis upon which to build more sophisticated techniques for fMRI decoding and offers a window into cognitive function during dynamic, natural experience.
Deep Dive into Locality and low-dimensions in the prediction of natural experience from fMRI.
Functional Magnetic Resonance Imaging (fMRI) provides dynamical access into the complex functioning of the human brain, detailing the hemodynamic activity of thousands of voxels during hundreds of sequential time points. One approach towards illuminating the connection between fMRI and cognitive function is through decoding; how do the time series of voxel activities combine to provide information about internal and external experience? Here we seek models of fMRI decoding which are balanced between the simplicity of their interpretation and the effectiveness of their prediction. We use signals from a subject immersed in virtual reality to compare global and local methods of prediction applying both linear and nonlinear techniques of dimensionality reduction. We find that the prediction of complex stimuli is remarkably low-dimensional, saturating with less than 100 features. In particular, we build effective models based on the decorrelated components of cognitive activity in the class
Functional Magnetic Resonance Imaging (fMRI) is a non-invasive imaging technique that can quantify changes in cerebral venous oxygen concentration. Changes in the fMRI signal that occur during brain activation are very small (1-5%) and are often contaminated by noise (created by the imaging system hardware or physiological processes). Statistical techniques that handle the stochastic nature of the data are commonly used for the detection of activated voxels. Traditional methods of analysis -which are designed to test the hypothesis that a simple cognitive or sensory stimulus creates changes in a specific brain area -are unable to analyze fMRI datasets collected in "natural stimuli" where the subjects are bombarded with a multitude of uncontrolled stimuli that cannot always be quantified [1,2]. The Experience Based Cognition competition (EBC) [3] offers an opportunity to study complex responses to natural environments, and to test new ideas and new methods for the analysis of fMRI collected in natural environments. The EBC competition provides fMRI data of three human subjects in three 20-minute segments (704 scanned samples in each segment) in an urban virtual reality environment along with quantitative time series of natural stimuli or features (25 in total) ranging from objective features such as the presence of faces to self-reported, subjective cognitive states such as the experience of fear. During each session, subjects were audibly instructed to complete three search tasks in the environment: looking for weapons (but not tools) taking pictures of people with piercings (but not others), or picking up fruits (but not vegetables). The data was collected with a 3T EPI scanner and typically consists of the activity of 35000 volume elements (voxels) within the head. The feature time series was provided for only the first two segments (1408 time samples) and competitive entries are judged on their ability to predict the feature on the third segment (704 time samples, see Fig. 1). At a microscopic level, a large number of internal variables associated with various physical and physiological phenomena contribute to the dynamic changes in the fMRI signal. Because the fMRI signal is a large scale (as compared to the scale of neurons) measurement of neuronal activity, we expect that many of these variables will be coupled resulting in a low dimensional set for all possible configurations of the activated fMRI signal. In this work we seek a low dimensional representation of the entire fMRI dataset that provides a new set of 'voxel-free" coordinates to study cognitive and sensory features.
We denote a three-dimensional volumes of fMRI composed of a total of N voxels by
We have access to T such volumes. We can stack the spatio-temporal fMRI dataset into a N × T matrix,
where each row n represents a time series x n generated from voxel n and each column j represents a scan acquired at time t j . We call the set of features to be predicted f k , k = 1, , • • • , K. We are interested in studying the variation of the set of features f k (t), k = 1, • • • , K describing the subject experience as a function of the dynamical changes of the brain, as measured by X(t).
We use here the global information provided by the dynamical evolution of X(t) over time, both during the training times and the test times. We would like to effectively replace each fMRI dataset X(t) by a small set of features that facilitates the identification of the brain states, and make the prediction of the features easier. Formally, our goal is to construct a map φ from the voxel space to low dimensional space.
where L N . As t varies over the training and the test sets, we hope that we explore most of the possible brain configurations that are useful for predicting the features. The map φ provides a parametrization of the brain states. Figure 2 provides a pictorial rendition of the map φ. The range D, represented in Fig. 2 as a smooth surface, is the set of parameters y 1 , • • • , y L that characterize the brain dynamics. Different values of the parameters produce different “brain states”, associated with different patterns of activation. Note that time does not play any role on D, and neighboring points on D correspond to similar brain states. Equipped with this re-parametrization of the dataset X , the goal is to learn the evolution of the feature time series as a function of the new coordinates
For a given feature f k , the training data provide us with samples of f k at certain locations in D. The map φ is build by globally computing a new parametrization of the set {X(1), • • • , X(T )}. This parametrization is built into two stages. First, we construct a graph that is a proxy for the entire set of fMRI data {X(1), • • • , X(T )}. Second, we compute some eigenfunctions φ k defined on the graph. Each eigenfunctions provides one specific coordinate for each node of the graph.
We represent the fMRI dataset for the training times and test ti
…(Full text truncated)…
This content is AI-processed based on ArXiv data.
