Publications — BRAIN CoGS

A Vector Reward Prediction Error Model Explains Dopaminergic Heterogeneity

Authors: Rachel S Lee, Ben Engelhard, Ilana B Witten, Nathanial D Daw

PUBLICATION: Biorxiv 2022

The hypothesis that midbrain dopamine (DA) neurons broadcast an error signal for the prediction of reward (reward prediction error, RPE) is among the great successes of computational neuroscience1-3. However, recent results contradict a core aspect of this theory: that the neurons uniformly convey a scalar, global signal. Instead, when animals are placed in a high-dimensional environment, DA neurons in the ventral tegmental area (VTA) display substantial heterogeneity in the features to which they respond, while also having more consistent RPE-like responses at the time of reward. Here we introduce a new Vector RPE model that explains these findings, by positing that DA neurons report individual RPEs for a subset of a population vector code for an animal's state (moment-to-moment situation). To investigate this claim, we train a deep reinforcement learning model on a navigation and decision-making task, and compare the Vector RPE derived from the network to population recordings from DA neurons during the same task. The Vector RPE model recapitulates the key features of the neural data: specifically, heterogeneous coding of task variables during the navigation and decision-making period, but uniform reward responses. The model also makes new predictions about the nature of the responses, which we validate. Our work provides a path to reconcile new observations of DA neuron heterogeneity with classic ideas about RPE coding, while also providing a new perspective on how the brain performs reinforcement learning in high dimensional environments.

PUBLICATION: Neuron 2022

Recent work has highlighted that many types of variables are represented in each neocortical area. How can these many neural representations be organized together without interference, and coherently maintained/updated through time? We recorded from large neural populations in posterior cortices as mice performed a complex, dynamic task involving multiple interrelated variables. The neural encoding implied that correlated task variables were represented by uncorrelated neural-population modes, while pairs of neurons exhibited a variety of signal correlations. This finding relates to principles of efficient coding for task-specific information, with neural-population modes as the encoding unit, and applied across posterior cortical regions and layers 2/3 and 5. Remarkably, this encoding function was multiplexed with sequential neural dynamics as well as reliably followed changes in task-variable correlations through time. We suggest that neural circuits can implement time-dependent encoding in a simple way by using random sequential dynamics as a temporal scaffold.

PUBLICATION: BioRXIV 2021

Among the impairments manifested by autism spectrum disorder (ASD) are sometimes islands of enhanced function. Although neuronal mechanisms for enhanced functions in ASD are unknown, the cerebellum is a major site of developmental alteration, and early-life perturbation to it leads to ASD with higher likelihood than any other brain region. Here we report that a cerebellum-specific transgenic mouse model of ASD shows faster learning on a sensory evidence-accumulation task. In addition, transgenic mice showed enhanced sensitivity to touch and auditory cues, and prolonged electrophysiological responses in Purkinje-cell complex spikes and associative neocortical regions. These findings were replicated by pairing cues with optogenetic stimulation of Purkinje cells. Computational latent-state analysis of behavior revealed that both groups of mice with cerebellar perturbations exhibited enhanced focus on current rather than past information, consistent with a role for the cerebellum in retaining information in memory. We conclude that cerebellar perturbation can activate neocortex via complex spike activity and reduce reliance on prior experience, consistent with a weak-central-coherence account in which ASD traits arise from enhanced detail-oriented processing. This recasts ASD not so much as a disorder but as a variation that, in particular niches, can be adaptive.

Publication: Biorxiv 2021

Accumulating evidence in service of sensory decision making is a core cognitive function. However, previous work has focused either on the dynamics of neural activity during decision-making or on models of evidence accumulation governing behavior. We unify these two perspectives by introducing an evidence-accumulation framework that simultaneously describes multi-neuron population spiking activity and dynamic stimulus-driven behavior during sensory decision-making. We apply our method to behavioral choices and neural activity recorded from three brain regions — the posterior parietal cortex (PPC), the frontal orienting fields (FOF), and the anterior-dorsal striatum (ADS) — while rats performed a pulse-based accumulation task. The model accurately captures the relationship between stimuli and neural activity, the coordinated activity of neural populations, and the distribution of animal choices in response to the stimulus. Model fits show strikingly distinct accumulation models expressed within each brain region, and that all differ strongly from the accumulation strategy expressed at the level of choices. In particular, the FOF exhibited a suboptimal ‘primacy’ strategy, where early sensory evidence was favored. Including neural data in the model led to improved prediction of the moment-by-moment value of accumulated evidence and the intended—and ultimately made—choice of the animal. Our approach offers a window into the neural representation of accumulated evidence and provides a principled framework for incorporating neural responses into accumulation models.

RE AD MORE→

Brigitte StarkOctober 21, 2021

An Oscillatory Mechanism for Multi-level Storage in Short-term Memory

Authors: Kathleen P. Champion, Olivia Gozel, Benjamin S. Lankow, G. Bard Ermentrout, Mark S. Goldman

PUBLICATION: BIORXIV 2021

Oscillatory activity is commonly observed during the maintenance of information in short-term memory, but its role remains unclear. Non-oscillatory models of short-term memory storage are able to encode stimulus identity through their spatial patterns of activity, but are typically limited to either an all-or-none representation of stimulus amplitude or exhibit a biologically implausible exact-tuning condition. Here, we demonstrate a simple phase-locking mechanism by which oscillatory input enables a circuit to generate persistent or sequential activity patterns that encode information not only in their location but also in their discretely graded amplitudes.

PUBLICATION: Cell Reports 2021

Cerebellar outputs take multisynaptic paths to reach higher brain areas, impeding tracing efforts. Here we quantify pathways between cerebellum and contralateral thalamic/corticostriatal structures using the anterograde transsynaptic tracer herpes simplex virus type 1 (H129), the retrograde tracer pseudorabies virus (Bartha), adeno-associated virus, and a whole-brain pipeline for neuron-level analysis using light-sheet microscopy. In ascending pathways, sensorimotor regions contained the most labeled neurons, but higher densities were found in associative areas, including orbital, anterior cingulate, prelimbic, and infralimbic cortex. Ascending paths passed through most thalamic nuclei, especially ventral posteromedial and lateral posterior (sensorimotor), mediodorsal (associative), and reticular (modulatory) nuclei. Retrograde tracing revealed descending paths originating largely from somatomotor cortex. Patterns of ascending influence correlated with anatomical pathway strengths, as measured by brainwide mapping of c-Fos responses to optogenetic inhibition of Purkinje cells. Our results reveal parallel functional networks linking cerebellum to forebrain and suggest that cerebellum uses sensory-motor information to guide both movement and nonmotor functions.

Publication: NATURE NEUROSCIENCE IN PRESS 2021

A classic view of the striatum holds that activity in direct and indirect pathways oppositely modulates motor output. Whether this involves direct control of movement, or reflects a cognitive process underlying movement, has remained unresolved. Here we find that strong, opponent control of behavior by the two pathways of the dorsomedial striatum (DMS) depends on a task’s cognitive demands. Furthermore, a latent state model (a hidden markov model with generalized linear model observations) reveals that—even within a single task—the contribution of the two pathways to behavior is state-dependent. Specifically, the two pathways have large contributions in one of two states associated with a strategy of evidence accumulation, compared to a state associated with a strategy of repeating previous choices. Thus, both the cognitive demands imposed by a task, as well as the strategy that mice pursue within a task, determine whether DMS pathways provide strong and opponent control of behavior.

PUBLIcation: Journal of neuroscience methods 2021

Background

The past decade has seen a multitude of new in vivo functional imaging methodologies. However, the lack of ground-truth comparisons or evaluation metrics makes the large-scale, systematic validation vital to the continued development and use of optical microscopy impossible.

New-method

We provide a new framework for evaluating two-photon microscopy methods via in silico Neural Anatomy and Optical Microscopy (NAOMi) simulation. Our computationally efficient model generates large anatomical volumes of mouse cortex, simulates neural activity, and incorporates optical propagation and scanning to create realistic calcium imaging datasets.

publication: bioRxiv 2021

Electron microscopy of biological tissue has recently seen an unprecedented increase in imaging throughput moving the ultrastructural analysis of large tissue blocks such as whole brains into the realm of the feasible. However, homogeneous, high quality electron microscopy staining of large biological samples is still a major challenge. To date, assessing the staining quality in electron microscopy requires running a sample through the entire staining protocol end-to-end, which can take weeks or even months for large samples, rendering protocol optimization for such samples to be inefficient. Here we present an in situ time-lapsed X-ray assisted staining procedure that opens the "black box" of electron microscopy staining and allows observation of individual staining steps in real time.

PUBLIcation: Nature 2021

Hippocampal neurons encode physical variables such as space or auditory frequency in cognitive maps. In addition, functional magnetic resonance imaging studies in humans have shown that the hippocampus can also encode more abstract, learned variables. However, their integration into existing neural representations of physical variables is unknown. Here, using two-photon calcium imaging, we show that individual neurons in the dorsal hippocampus jointly encode accumulated evidence with spatial position in mice performing a decision-making task in virtual reality. Nonlinear dimensionality reduction showed that population activity was well-described by approximately four to six latent variables, which suggests that neural activity is constrained to a low-dimensional manifold. Within this low-dimensional space, both physical and abstract variables were jointly mapped in an orderly manner, creating a geometric representation that we show is similar across mice. The existence of conjoined cognitive maps suggests that the hippocampus performs a general computation—the creation of task-specific low-dimensional manifolds that contain a geometric representation of learned knowledge.

PUBLICATION: biorxiv 2021

Cortical areas seem to form a hierarchy of intrinsic timescales, but whether this is causal to cognitive behavior remains unknown. In particular, decisions requiring the gradual accrual of sensory evidence over time recruit widespread areas across this hierarchy. Here, we causally tested the hypothesis that this recruitment is related to the intrinsic integration timescales of these widespread areas. We trained mice to accumulate evidence over seconds while navigating in virtual reality, and optogenetically silenced the activity of many cortical areas during different brief trial epochs. We found that the inactivation of different areas primarily affected the evidence-accumulation computation per se, rather than other decision-related processes. Specifically, we observed selective changes in the weighting of evidence over time, such that frontal inactivations led to deficits on longer timescales than posterior cortical ones. Likewise, large-scale cortical Ca2+ activity during task performance displayed different temporal integration windows matching the effects of inactivation. Our findings suggest that distributed cortical areas accumulate evidence by leveraging their hierarchy of intrinsic timescales.

Publication: NEURON 2021

Decision-making strategies evolve during training and can continue to vary even in well-trained animals. However, studies of sensory decision-making tend to characterize behavior in terms of a fixed psychometric function that is fit only after training is complete. Here, we present PsyTrack, a flexible method for inferring the trajectory of sensory decision-making strategies from choice data. We apply PsyTrack to training data from mice, rats, and human subjects learning to perform auditory and visual decision-making tasks. We show that it successfully captures trial-to-trial fluctuations in the weighting of sensory stimuli, bias, and task-irrelevant covariates such as choice and stimulus history. This analysis reveals dramatic differences in learning across mice and rapid adaptation to changes in task statistics. PsyTrack scales easily to large datasets and offers a powerful tool for quantifying time-varying behavior in a wide variety of animals and tasks.

Publication: Curr Opin Neurobiology 2020

Neural computations underlying cognition and behavior rely on the coordination of neural activity across multiple brain areas. Understanding how brain areas interact to process information or generate behavior is thus a central question in neuroscience. Here we provide an overview of statistical approaches for characterizing statistical dependencies in multi-region spike train recordings. We focus on two classes of models in particular: regression-based models and shared latent variable models. Regression-based models describe interactions in terms of a directed transformation of information from one region to another. Shared latent variable models, on the other hand, seek to describe interactions in terms of sources that capture common fluctuations in spiking activity across regions. We discuss the advantages and limitations of each of these approaches and future directions for the field. We intend this review to be an introduction to the statistical methods in multi-region models for computational neuroscientists and experimentalists alike.

PUblication: Nature Neuroscience 2020

Recent work has suggested that the prefrontal cortex (PFC) plays a key role in context-dependent perceptual decision-making. In this study, we addressed that role using a new method for identifying task-relevant dimensions of neural population activity. Specifically, we show that the PFC has a multidimensional code for context, decisions and both relevant and irrelevant sensory information. Moreover, these representations evolve in time, with an early linear accumulation phase followed by a phase with rotational dynamics. We identify the dimensions of neural activity associated with these phases and show that they do not arise from distinct populations but from a single population with broad tuning characteristics. Finally, we use model-based decoding to show that the transition from linear to rotational dynamics coincides with a plateau in decoding accuracy, revealing that rotational dynamics in the PFC preserve sensory choice information for the duration of the stimulus integration period.

PUBLICATION: Neuron, 2019

Neural activity throughout the cortex is correlated with perceptual decisions, but inactivation studies suggest that only a small number of areas are necessary for these behaviors. Here we show that the number of required cortical areas and their dynamics vary across related tasks with different cognitive computations. In a visually guided virtual T-maze task, bilateral inactivation of only a few dorsal cortical regions impaired performance. In contrast, in tasks requiring evidence accumulation and/or post-stimulus memory, performance was impaired by inactivation of widespread cortical areas with diverse patterns of behavioral deficits across areas and tasks. Wide-field imaging revealed widespread ramps of Ca2+ activity during the accumulation and visually guided tasks. Additionally, during accumulation, different regions had more diverse activity profiles, leading to reduced inter-area correlations. Using a modular recurrent neural network model trained to perform analogous tasks, we argue that differences in computational strategies alone could explain these findings.

PUBLICATION: NATURE Communications, 2019

To select actions based on sensory evidence, animals must create and manipulate representations of stimulus information in memory. Here we report that during accumulation of somatosensory evidence, optogenetic manipulation of cerebellar Purkinje cells reduces the accuracy of subsequent memory-guided decisions and causes mice to downweight prior information. Behavioral deficits are consistent with the addition of noise and leak to the evidence accumulation process. We conclude that the cerebellum can influence the accurate maintenance of working memory.

Publication: Nature, 2019

There is increased appreciation that dopamine (DA) neurons in the midbrain respond not only to reward and reward-predicting cues but also to other variables such as distance to reward, movements and behavioral choices. Based on these findings, a major open question is how the responses to these diverse variables are organized across the population of DA neurons. In other words, do individual DA neurons multiplex multiple variables, or are subsets of neurons specialized in encoding specific behavioral variables? The reason that this fundamental question has been difficult to resolve is that recordings from large populations of individual DA neurons have not been performed in a behavioral task with sufficient complexity to examine these diverse variables simultaneously.

Current Opinion in Neurobiology, April 2019

Neural circuits can be reconstructed from brain images acquired by serial section electron microscopy. Image analysis has been performed by manual labor for half a century, and efforts at automation date back almost as far. Convolutional nets were first applied to neuronal boundary detection a dozen years ago, and have now achieved impressive accuracy on clean images. Robust handling of image defects is a major outstanding challenge. Convolutional nets are also being employed for other tasks in neural circuit reconstruction: finding synapses and identifying synaptic partners, extending or pruning neuronal reconstructions, and aligning serial section images to create a 3D image stack. Computational systems are being engineered to handle petavoxel images of cubic millimeter brain volumes.

PUBLICATION: biorxiv, 2019

Studies of perceptual decision-making have often assumed that the main role of sensory cortices is to provide sensory input to downstream processes that accumulate and drive behavioral decisions. We performed a systematic comparison of neural activity in primary visual (V1) to secondary visual and retrosplenial cortices, as mice performed a task where they should accumulate pulsatile visual cues through time to inform a navigational decision. Even in V1, only a small fraction of neurons had sensory-like responses to cues. Instead, in all areas neurons were sequentially active, and contained information ranging from sensory to cognitive, including cue timings, evidence, place/time, decision and reward outcome. Per-cue sensory responses were amplitude-modulated by various cognitive quantities, notably accumulated evidence. This inspired a multiplicative feedback-loop circuit hypothesis that proposes a more intricate role of sensory areas in the accumulation process, and furthermore explains a surprising observation that perceptual discrimination deviates from Weber-Fechner Law.

Featured Publications