Wolfram Schultz - Dopamine: from movement via reward to rational choice - ViDA 2020 Virtual Dopamine 2020-05-22 | Wolfram SchultzUniversity of Cambridge05-20-2020Dopamine: from movement via reward to rational choiceGiven that the phasic dopamine reward prediction error signal is suitable for updating neuronal choice signals, we investigated its properties using economic formalisms. Before starting, I should mention that reward coding is not the only phasic dopamine change; dopamine neurons shows also separate, slower, lower and heterogeneous changes related to what can be broadly described as behavioral activation. In our experimental economics studies on the dopamine reward signal, we estimated formal utility functions from choice under risk (Von Neumann-Morgenstern utility). These choices were rational in following first, second and third order stochastic dominance (reflecting value, variance risk, skewness risk). Utility was coded in dopamine neurones as utility prediction error (which incorporates risk into subjective value). Consistent with this neuronal signal, the dopamine response followed first- and second-order stochastic dominance. These data unite concepts from animal learning theory and economic decision theory at the level of single reward neurons.
Anne Collins - One-shot intrinsic reward valuation in humans Virtual Dopamine 2021-07-01 | ViDA 2021 - Tuesday June 22nd 2021Anne Collins UC Berkeley, Department of Psychology and Helen Wills Neuroscience Institute One-shot intrinsic reward valuation in humans Humans continuously need learn to make good choices in our environment – be it using a new video-conferencing set up, or selecting which location in a house is least likely to be interrupted by toddlers during work calls. However, the goals we seek to attain – such as using zoom successfully – are often vaguely defined and previously unexperienced, and in that sense cannot be known by us as being rewarding. How does the brain enable us to immediately encode such one-shot goals as having value as intrinsic reinforcers? We hypothesized that learning to make good choices in such situations leverages classic, dopaminergic reinforcement learning processes, and that executive functions in general, and working memory in particular, play a crucial role in defining the intrinsic reward function for arbitrary, novel outcomes, in such a way that they become reinforcing. I will show results from a novel behavioral protocol, as well as computational and imaging evidence supporting our hypothesis.
Matt Botvinick-Deep reinforcement learning and its neuroscientific implications Virtual Dopamine 2021-07-01 | ViDA 2021 - Tuesday June 22nd 2021Matt BotvinickDirector of Neuroscience and Team Lead in AGI Research, DeepMind ; Honorary Professor, Gatsby Computational Neuroscience Unit, University College LondonDeep reinforcement learning and its neuroscientific implicationsThe last few years have seen some dramatic developments in artificial intelligence research. What implications might these have for neuroscience? Investigations of this question have, to date, focused largely on deep neural networks trained using supervised learning, in tasks such as image classification. In this talk, I'll discuss another area of recent AI work which has so far received less attention from neuroscientists, but which may have more profound implications: Deep reinforcement learning. Deep RL provides a rich framework for studying the interplay among learning, representation and decision-making, offering to the brain sciences a new set of research tools and a wide range of novel hypotheses. I'll provide a high level introduction to deep RL and survey some of its key implications for research on the brain and behavior, with a particular focus on potential implications for understanding dopaminergic function.
Jochen Roeper - Pacemaker mechanism and plasticity of DA SN neurons Virtual Dopamine 2021-07-01 | ViDA 2021 - Thur June 24th 2021Jochen RoeperNeurophysiology Goethe University FrankfurtPacemaker mechanism and plasticity of DA SN neuronsThe mechanisms of pacemaking in dopamine neurons in the substantia nigra (DA SN) are still not clear and particular the role of L-type channels are both controversial and potentially relevant in the context of neuroprotection in Parkinson Disease. Using a mouse model where only Cav1.3 channels are sensitive to the dihydropyridine isradipine we defined the functional role of this low-threshold L-type channel to autonomous pacing in vitro. Dynamic clamp in vitro experiments revealed how Cav1.3 channels also boost high-frequency discharge in the in vivo burst frequnency range of lateral SN DA neurons. Clinically relevant concentrations of isradipine in the low nanomolar range are tested in this setting and compared to systemic effects of isradipine on in vivo firing properties of identified DA SN neurons. Our results demonstated how lateral SN DA neurons can be targeted selectively in vivo. Finally, we identified the acceleration of pacemaker function in DA SN neurons surviving in a partial 6-OHDA lesion model via downregulation of Kv4.3 channels. We demonstrate that DA SN pacing can be targeted in vivo by dihydropyridines and also adapts in surviving neurons after a lesion.
Anissa Abi-Dargham - Dopamine in schizophrenia: from molecules to behavior Virtual Dopamine 2021-07-01 | ViDA 2021 - Wed June 23rd 2021Anissa Abi-Dargham Department of Psychiatry, Stony Brook University Dopamine in schizophrenia: from molecules to behaviorThis lecture will integrate findings from molecular and functional imaging studies in patients with schizophrenia aimed at understanding the topography of dopamine dysregulation in this disease as well as its functional impact on circuitry and behavior. We have used Positron Emission Tomography (PET) imaging of D2 radiotracers combined with pharmacological manipulations of the dopaminergic system, to examine indices of pre and post synaptic dopamine function and, in the same subjects, we used functional imaging to assess the significance of the molecular alterations. These studies have revealed topographical perturbations in presynaptic dopamine function where the associative striatum shows excess storage and release and extrastriatal regions, including cortex and midbrain, show a deficit in dopamine release. To understand the localized striatal enhancement in dopamine, in presence of low midbrain dopamine availability, we examined local striatal mechanisms that may dysregulate dopamine. One important such mechanism is the cholinergic modulation of dopamine release, via the striatal cholinergic interneurons. For this purpose, we measured the vesicular cholinergic transporter, using PET and a radiotracer specific for this target, which relates to the storage capacity for acetylcholine (ACH) in a small cohort of patients and controls. The molecular alterations described are linked to abnormal connectivity of the rostral caudate to the rest of the brain, particularly to cortical regions, and to measures of auditory perceptual bias and hallucinations. In addition, cortical dopamine deficit has implications for the role of cortical D1 receptors in modulating activity and cognitive processes. Understanding these molecular and functional abnormalities is a critical step in developing novel and targeted therapeutics for the various symptom domains of the disease. We will conclude by highlighting a new multisite trial of a D1 agonist in schizophrenia based in part on these findings and an overall model for the role of dopamine in the development and staging of the disease.
David Lovinger - Dopamine, Dope and Sleep Virtual Dopamine 2021-07-01 | ViDA 2021 - Wed June 23rd 2021David Lovinger National Institute on Alcohol Abuse and AlcoholismDopamine, Dope and SleepMost psychoactive drugs alter sleep and long-term use and abuse of drugs can produce sleep disruption. Acute use of cannabis drugs generally promotes sleep, but long-term, heavy cannabis drug users experience sleep disruption. Indeed, sleep problems are often cited as a reason for relapse to cannabis use. The major psychoactive ingredient of cannabis drugs, delta-9-tetrahydrocannabinol (THC) is likely responsible for these effects. Given the role of dopamine (DA) in the neural effects of THC and other drugs of abuse, along with a growing literature implicating DA in the neural mechanisms of sleep, we have examined THC effects on sleep in mice using polysomnography and on striatal DA release using fast-scan cyclic voltammetry (FSCV). A chronic THC exposure regimen that produces tolerance to the drug alters sleep in a sex-dependent manner. Initial doses of THC enhanced non-rapid eye movement (NREM) sleep in both males and females. Following chronic exposure, both males and females showed tolerance to the sleep-inducing THC effects. Male mice exhibited reduced NREM sleep following this exposure, while female mice showed no such change but exhibited increased REM sleep. Examination of DA release in slices from dorsomedial and dorsolateral striatum (DMS and DLS) and nucleus accumbens (NAc), revealed region, sex and exposure-time-dependent THC effects. The largest chronic THC-induced changes were in DMS where males showed increased DA release after acute exposure on the 6th day after chronic exposure. In contrast, females showed decreased DA release in DMS after acute and on day 6 after chronic THC exposure, with similar effects in female DLS and NAc. We are currently exploring drug-induced DA changes in the different striatal subregions using dLight and in vivo fiber photometry. It will be interesting to examine how THC alters DA levels in vivo and how DA may contribute to the sex-dependent drug-induced sleep alterations.
Bernardo Sabatini - Dopaminergic control of cellular state and action selection Virtual Dopamine 2021-07-01 | ViDA 2021 - Wed June 23rd 2021Bernardo SabatiniNeurobiology, Harvard Medical School, HHMI Dopaminergic control of cellular state and action selectionDopamine is known to reinforce actions which, when performed in the right sensory context, led to its release. This is thought to occur via temporally restricted actions on direct and indirect pathway neurons of the striatum. Here we discuss how dopaminergic neuron firing affects the electrical and biochemical state of direct and indirect pathway striatum projection neurons. Furthermore, we demonstrate how the effects of dopamine on each neuron class depends on the receptors they express and how these vary during behavior. Lastly, we present a model for how these effects contribute to action reinforcement and devaluation.
AI & the brain panel - ViDA 2021 Virtual Dopamine 2021-07-01 | ViDA 2021 - Wed June 23rd 2021Moderator: Paul Middlebrooks (Braininspired Podcast)Panelists:Matt Botvinick (DeepMind)Ida Momennejad (Microsoft Research)Ilana Witten (Princeton University)Armin Lak (University of Oxford)Ashok Litwin-Kumar (Columbia University)What can artificial intelligence teach us about how the brain usesdopamine to learn?Recent advances in artificial intelligence have yielded novel algorithms for reinforcement learning (RL), which leverage the power of deep learning together with reward prediction error signals in order to achieve unprecedented performance in complex tasks. In the brain, rewardprediction error signals are thought to be signaled by midbrain dopamine neurons and support learning. Can these new advances in Deep RL help us understand the role that dopamine plays in learning? In this panel experts in both theoretical and experimental dopamine research will discuss this question.--------------------References cited by Ida Momennejad- Homeostatic reinforcement learning for integrating reward collection and physiological stability elifesciences.org/articles/0481101:47:07 -A Reinforcement Learning Theory for HomeostaticRegulation proceedings.neurips.cc/paper/2011/file/9778d5d219c5080b9a6a17bef029331c-Paper.pdf-interaction grounded learning icml.cc/Conferences/2021/ScheduleMultitrack?event=8937-Navigation Turing Test (NTT): Learning to Evaluate Human-Like Navigation icml.cc/Conferences/2021/ScheduleMultitrack?event=10032
Stephen Zhang - Hypothalamic dopamine neurons control the motivation via cAMP-PKA signaling Virtual Dopamine 2021-07-01 | ViDA 2021 - Thur June 24th 2021Stephen ZhangBeth Israel Deaconess Medical CenterHypothalamic dopamine neurons control the motivation to mate via persistent cAMP-PKA signalingTransient neuromodulation can have long-lasting effects on neural circuits and motivational states. We examined dopaminergic mechanisms underlying mating drive and its persistence in male mice. Brief investigation of females primes a male’s interest to mate for tens of minutes, while a single successful mating triggers satiety that gradually recovers over days. We found that both sexual priming and satiety are controlled by specialized anteroventral and preoptic periventricular (AVPV/PVpo) dopamine neurons in the hypothalamus. During investigations of females, dopamine concentration transiently ramps up in the medial preoptic area (MPOA), an area critical for mating behaviors. Optogenetic stimulation of AVPV/PVpo dopamine axons in MPOA recapitulates the priming effects of female exposure. Using in vivo two-photon fluorescence lifetime imaging microscopy (2p FLIM) as well as novel optogenetic tools to track and manipulate intracellular signaling, we show that these priming effects emerge from accumulation of cyclic adenosine monophosphate (cAMP) levels and protein kinase A (PKA) activity that can be sustained for tens of minutes. Dopamine transients in MPOA are abolished following a successful mating, likely ensuring abstinence. Consistent with this idea, inhibiting AVPV/PVpo dopamine neurons selectively demotivates mating, while stimulating these neurons restores the motivation to mate following sexual satiety. Therefore, accumulation or suppression of signals from specialized dopamine neurons regulates mating behaviors across minutes and days.
Roy Wise -A Dopamine History Virtual Dopamine 2021-07-01 | ViDA 2021 - Thur June 24th 2021Roy WiseNIH/NIDAA Dopamine HistoryThe history of dopamine is over 100 years. It began when Sir Henry Dale first tested a compound in 1910; he eventually named the compound dopamine in 1951. A handful of investigators took steps to develop and use the compound, and the major clinical ending was when dopamine depletion was iidentifed in Parkinsonian patients and when L-DOPA was found to alleviate its symptoms. The lack of movement in Parkinsonian patients—particularly those who had been infected in the 1919 flu epidemic—suggested that dopamine was involved in motor function; dopamine-depleted animals were akinetic and failed to seek food or water or other rewards and failed to avoid predictable punishment. These animals had reflex responses to rewards and punishers but could not learn to search for rewards—or to avoid predictable punishers. In normal animals, excitatory environmental stimuli cause burst-firing of dopaminergic neurons. This burst-firing is required for learning to respond to predictive stimuli; this learning involves two mechanisms. First, burst-responses must develop, through experience, to reward-predictors. This happens by a Hebbian process; rewards enable the development of burst-firing in response to stimuli that immediately precede them. Second, burst-firing in response to predictors enables the cellular basis for long-term potentiation (LTP) or long-term depression (LTD) in nearby sensory—glutamatergic—inputs to medium-spiny—GABAergic—output neurons in the striatum. By enabling LTP and LTD, burst-firing of dopamine neurons enables the stamping-in of learned responses to external stimuli; this is one of dopamine’s two major behavioral effects. A second behavioral function is the modulation of motivation. The level of pacemaker firing of the dopamine system drives or helps drive motivation; low levels of baseline dopamine are associated with low motivation, and moderate levels are associated with activation. High levels of dopamine—apparently caused only by addictive drugs—are associate with drug-satiety. The driving of dopaminergic burst-firing is induced by glutamate input (a correlate of, but separate from, glutamate input to the striatum). The driving of pacemaker firing is under the control of hormones and the modulation of inhibitory GABAergic input to the dopamine neurons. These two functions—reinforcement and motivation—are the major behavioral effects of dopamine in the brain, and they appear to account for its secondary effects.See complete list of references here:docs.google.com/document/d/1I2BxQVyJ5-KQC15O1omJVWMeBu-s6B_Sw-ke4cNgBTc
Miriam Matamales - Local D2- to D1-neuron transmodulation in the striatum Virtual Dopamine 2021-07-01 | ViDA 2021 - Thur June 24th 2021Miriam MatamalesUniversity of New South Wales - SydneyLocal D2- to D1-neuron transmodulation in the striatum and its role in goal-directed learningOne of the most intriguing characteristic of the striatum is the random spatial distribution and high degree of intermingling between its D1-(direct) and D2-(indirect) spiny projection neurons (SPNs). The resulting highly entropic mosaic extends through a homogeneous space and is mostly devoid of histological boundaries. The anatomical organisation of these two principal neuronal populations is actively promoted during development and has been highly conserved throughout evolution, and yet its relationship to function is still not fully understood. In a recent study in my team, by mapping a dopamine-dependent transcriptional activation marker in large ensembles of D1- or D2-SPNs in mice, we demonstrated an extensive and dynamic D2- to D1-SPN transmodulation across the striatum that is necessary for updating previous goal-directed learning. We found evidence that activated D2-SPNs access and modify developing behavioural programs encoded by regionally defined ensembles of transcriptionally active D1-SPNs. This process is slow because it depends on the molecular integration of additive neuromodulatory signals. However, with time, it creates the regional functional boundaries that are necessary to identify and shape specific learning in the striatum. Our work therefore suggests that the striatum takes full advantage of the ‘one-to-one’ structure of the binary mosaic and provides unexpected insights into the peculiar histoanatomical organization of this disordered, borderless environment.
Gabriela Izowit - Brain state dependent responses of midbrain dopaminergic neurons Virtual Dopamine 2021-07-01 | ViDA 2021 - Thur June 24th 2021Gabriela IzowitJagiellonian UniversityBrain state dependent responses of midbrain dopaminergic neurons’ to the aversive stimulusFor a long time, it has been assumed that dopaminergic (DA) neurons of the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) respond to reward and aversive stimuli homogenously across the entire population with either increase or decrease of their activity respectively – coding in this manner information about the value of perceived stimuli. This notion was questioned after the identification of midbrain DA neurons that are excited by both rewarding and aversive stimuli. This resulted in division of midbrain DA neurons into two functionally distinct subpopulations: one encoding the value and the other encoding the salience of the stimuli. Additionally, it has been shown that the general state of the brain modulates the electrical activity of midbrain DA neurons, but it remains unknown whether this factor may also influence signalling of value and salience. Our experiments were aimed at answering this question. For this purpose, we recorded responses of VTA and SNc DA neurons to electrical footshocks across alternating brain states of urethane anaesthetized rats. We identified DA neurons based on electrophysiological criteria combined with the use of either juxtacellular recording-labelling technique or optotagging. Besides the previously described populations of value- and salience-coding neurons, we also observed unidentified so far subpopulation of VTA and SNc DA neurons, that changes its type of response to an aversive stimulus depending on the ongoing brain state. Majority of these neurons were inhibited by footshocks during a REM-like brain state, but with the appearance of nREM-like brain state, they changed their type of response to excitation. Based on our observations, it can be hypothesised that there is a subpopulation of DA neurons that are involved in ‘dual-coding’ of both the value and the salience of the stimulus depending on the general state of the brain.
Vikram Gadagkar - Dopamine Neurons Evaluate Natural Fluctuations in Performance Quality Virtual Dopamine 2021-07-01 | ViDA 2021 - Thur June 24th 2021Vikram GadagkarColumbia UniversityDopamine Neurons Evaluate Natural Fluctuations in Performance QualityMany motor skills are learned by comparing ongoing behavior to internal performance benchmarks. Dopamine neurons have been shown to encode performance error in artificial paradigms where error is externally induced, rather than during natural behavior. Here we recorded dopamine neurons in singing birds and examined how spontaneous dopamine spiking activity correlated with natural fluctuations in ongoing song. Antidromically identified basal ganglia-projecting dopamine neurons correlated with recent, and not future, song variations, consistent with a role in evaluation, not production. Furthermore, dopamine spiking was suppressed following the production of outlying vocal variations, consistent with a role for active song maintenance. These data show for the first time that spontaneous dopamine spiking can evaluate natural behavioral fluctuations unperturbed by experimental events.
Garret Stuber - Molecular and Functional Phenotyping of Habenula Circuitry Virtual Dopamine 2021-06-23 | ViDA 2021 - Tuesday June 22nd 2021Garret Stuber Center for the Neurobiology of Addiction, Pain, and Emotion, Department of Anesthesiology and Pain Medicine, Department of Pharmacology, University of Washington, Seattle, WAMolecular and Functional Phenotyping of Habenula CircuitryThe habenula complex is appreciated as a critical regulator of motivated and pathological behavioral states via its output to midbrain nuclei. Despite this, transcriptional definition of cell populations that comprise both the medial (MHb) and lateral habenular (LHb) subregions in mammals remain undefined. To resolve this, we performed single-cell transcriptional profiling and highly multiplexed in situ hybridization experiments of the mouse habenula complex in naïve mice and those exposed to an acute aversive stimulus. Transcriptionally distinct neuronal cell types identified within the MHb and LHb, were spatially defined, differentially engaged by aversive stimuli and had distinct electrophysiological properties. Cell types identified in mice, also displayed a high degree of transcriptional similarity to those previously described in zebrafish, highlighting the well conserved nature of habenular cell types across the phylum. These data identify key molecular targets within habenular cell types, and provide a critical resource for future studies.
Melissa Warden - Ramping activity in midbrain dopamine neurons signifies the use of a cognitive map Virtual Dopamine 2021-06-23 | ViDA 2021 - Tuesday June 22nd 2021Melissa Warden Neurobiology and Behavior, Cornell University Ramping activity in midbrain dopamine neurons signifies the use of a cognitive map Journeys to novel and familiar destinations employ different navigational strategies. The first drive to a new restaurant relies on map-based planning, but after repeated trips the drive is automatic and guided by local environmental cues. Ventral striatal dopamine rises during navigation toward goals and reflects the spatial proximity and value of goals, but the impact of experience, the neural mechanisms, and the functional significance of dopamine ramps are unknown. Here, we used fiber photometry to record the evolution of activity in midbrain dopamine neurons as mice learned a variety of reward-seeking tasks, starting recordings before training had commenced and continuing daily for weeks. When mice navigated through space toward a goal, robust ramping activity in dopamine neurons appeared immediately – after the first rewarded trial on the first training day in completely naïve animals. In this task spatial cues were available to guide behavior, and although ramps were strong at first, they gradually faded away as training progressed. If instead mice learned to run a fixed distance on a stationary wheel for reward, a task that required an internal model of progress toward the goal, strong dopamine ramps persisted indefinitely. In a passive task in which a visible cue and reward moved together toward the mouse, ramps appeared and then faded over several days, but in an otherwise identical task with a stationary cue and reward ramps never appeared. Our findings provide strong evidence that ramping activity in midbrain dopamine neurons reflects the use of a cognitive map – an internal model of the distance already covered and the remaining distance until the goal is reached. We hypothesize that dopamine ramps may be used to reinforce locations on the way to newly-discovered rewards in order to build a graded ventral striatal value landscape for guiding routine spatial behavior.
Willemieke Kouwenhoven - VGluT2 Expression in Dopamine Neurons Contributes to Striatal Reinnervation Virtual Dopamine 2020-11-23 | Willemieke Kouwenhoven Trudeau Lab, University of Montreal VGluT2 Expression in Dopamine Neurons Contributes to Postlesional Striatal Reinnervation A subset of adult ventral tegmental area dopamine (DA) neurons expresses vesicular glutamate transporter 2 (VGluT2) and releases glutamate as a second neurotransmitter in the striatum, while only a few adult substantia nigra DA neurons have this capacity. Recent work showed that cellular stress created by neurotoxins such as MPTP and 6-hydroxydopamine can upregulate VGluT2 in surviving DA neurons, suggesting the possibility of a role in cell survival, although a high level of overexpression could be toxic to DA neurons. Here we examined the level of VGluT2 upregulation in response to neurotoxins and its impact on postlesional plasticity. We first took advantage of an in vitro neurotoxin model of Parkinson's disease and found that this caused an average 2.5-fold enhancement of Vglut2 mRNA in DA neurons. This could represent a reactivation of a developmental phenotype because using an intersectional genetic lineage-mapping approach, we find that ~98% of DA neurons have a VGluT2+ lineage. Expression of VGluT2 was detectable in most DA neurons at embryonic day 11.5 and was localized in developing axons. Finally, compatible with the possibility that enhanced VGluT2 expression in DA neurons promotes axonal outgrowth and reinnervation in the postlesional brain, we observed that DA neurons in female and male mice in which VGluT2 was conditionally removed established fewer striatal connections 7 weeks after a neurotoxin lesion. Thus, we propose here that the developmental expression of VGluT2 in DA neurons can be reactivated at postnatal stages, contributing to postlesional plasticity of dopaminergic axons.
Corey Baimel - Afferent control of local and long-range circuits in the nucleus accumbens Virtual Dopamine 2020-11-23 | Corey BaimelCarter Lab, New York UniversityAfferent control of local and long-range circuits in the nucleus accumbensThe nucleus accumbens (NAc) is a primary input region of the basal ganglia and serves as a critical interface between limbic and motor regions of the brain. In the NAc, information about feelings, emotions and motivations are transitioned into goal-directed actions. However, we are still in the early stages of understanding the cellular and synaptic makeup of NAc circuits. My work aims at understanding how afferent inputs regulate the local and long-range circuits of the NAc medial shell. Medium spiny neurons (MSNs), the principle cell type of the NAc, can be divided into two broad populations based on the expression of dopamine 1 (D1+) or 2 receptors. Addictive drugs strongly rewire inputs onto D1+ MSNs, but whether drug-induced plasticity occurs in distinct populations of D1+ MSNs was unknown. I will present data in which I have identified two discrete subpopulations of D1+ MSNs that project to either the ventral pallidum or the ventral tegmental area. Moreover, that these neurons are differentially contacted by inputs from the ventral hippocampus and the basolateral amygdala, and that cocaine alters these connections in a circuit-specific manner. Intermingled amongst MSNs are cholinergic interneurons, and little is known about how afferent inputs to the NAc regulate these important modulators of NAc function. I will present a second set of data, which explores the circuit and synaptic mechanisms through which long-range inputs to the NAc alter the firing of cholinergic interneurons. Notably, identifying a novel, NAc specific mechanism for afferent driven suppression of firing of cholinergic interneurons.
Polina Kosillo - Cell-autonomous gatekeeping of dopaminergic neurotransmission by mTOR pathway Virtual Dopamine 2020-11-23 | The Future of Dopamine - ViDA Symposium - Nov 19, 2020Polina KosilloBateup Lab, University of California, BerkeleyCell-autonomous gatekeeping of dopaminergic neurotransmission by mTOR pathwayThe neurodevelopmental disorder Tuberous Sclerosis Complex (TSC) is caused by loss-of-function mutations in the TSC1/TSC2 genes which lead to constitutive mTORC1 activation. TSC patients present with significantly elevated rates of dopamine (DA)-related neuropsychiatric conditions, and we hypothesized DA neuron involvement. To test this, I generated mice with conditional Tsc1 deletion selectively in DA neurons (DA-Tsc1 KO) and found that mTORC1 hyperactivity caused profound structural and functional changes. DA-Tsc1 KO neurons had pronounced somato-dendritic and axonal hypertrophy, reduced intrinsic excitability, and a significant impairment in evoked striatal DA release, despite increased DA production. These structural and functional changes were sufficient to induce a selective deficit in cognitive flexibility: DA-Tsc1 KO mice took considerably longer to update their behavioral strategy in a reversal learning task, showing perseveration, commonly reported across neuropsychiatric conditions. I further established that genetic reduction of Raptor, an obligatory protein component of mTORC1, in DA-Tsc1 KO mice reduced mTORC1 hyperactivation and prevented both DA release and cognitive flexibility deficits. In a follow up study, I address whether two arms of mTOR pathway are equivalent in their governance of the DA system. mTORC1 and mTORC2 have distinct functional roles, and suppression of either complex is possible via genetic reduction of the obligatory binding partners, Raptor and Rictor, respectively. I showed that mTORC1 suppression is severely detrimental to DA neurons causing somato-dendritic hypotrophy and corresponding intrinsic hyperexcitability, together with profound reduction in DA synthesis and release. In contrast, mTORC2 suppression has minimal impact on DA neurons. Across the two studies, my work establishes that DA neurons are exquisitely sensitive to mTORC1 signaling which serves as a cell-autonomous gatekeeper of dopaminergic neurotransmission.
Sarah Starosta - Dopamine and the algorithmic basis of foraging decisions Virtual Dopamine 2020-11-23 | The Future of Dopamine - ViDA Symposium - Nov 19, 2020Sarah StarostaKepecs Lab, Washington University School of Medicine St. LouisDopamine and the algorithmic basis of foraging decisionsWe are continually confronted with decisions about whether to stay engaged with the current option or to switch to a new one. These decisions include when to give up waiting in line or when to settle with a partner for life. These decisions have been extensively studied as foraging decisions, yet little is known about the underlying neural basis.Here, we studied these decisions and its neural correlate in mice facing the decision when to leave depleting reward sources. To explore the choice strategy and its neural correlates, we implemented several reward manipulations and performed optical recordings dopamine neuron activity in the Ventral Tegmental Area (VTA). We observed, unexpectedly, that mice tend to leave a depleting source earlier after a higher than expected reward. Critically, this observation allowed us to distinguish between different theoretical models because only one that implemented a decision rule where animals compare the next expected reward to the average of the previous rewards predicted this result. Additionally, we show that this decision rule may be learned via a reinforcement learning (RL) paradigm called R-learning, but is not consistent with classical V-, or Q-learning paradigms. On a neuronal level, we observed that dopaminergic signaling in the VTA best correlates with the Reward Prediction Error (RPE) of R-learning, pointing to a potential learning mechanism that optimizes stay-or-leave choices.Overall, our work offers an algorithmic decision rule and neuronal implementation for an ethologically relevant behavior based on qualitative different model predictions.
HyungGoo Kim - Spiking activity of VTA dopamine neurons during spatial navigation Virtual Dopamine 2020-11-23 | The Future of Dopamine - ViDA Symposium - Nov 19, 2020HyungGoo KimUchida Lab, Harvard UniversitySpiking activity of VTA dopamine neurons during spatial navigationDopamine activity has been postulated to encode a temporal difference (TD) error, a teaching signal used in machine learning. In the previous meeting, we showed that a pooled calcium activity of dopaminergic axons in the ventral striatum gradually ramps up while mice approached a reward location in virtual reality (VR). Using a series of ‘test’ conditions (e.g., teleport and speed manipulations), we demonstrated that these signals represent TD errors instead of values.However, it is possible that a subset of dopamine neurons encode TD errors while others encode values. Furthermore, dopamine release may be different from the spiking activity at the soma (Mohebi et al, 2019). To address these issues, we recorded from a large number of optogenetically identified VTA dopamine neurons (n = 102) in the same tasks. Neurons on average showed a positive ramp. We also found a medio-lateral spatial gradient of ramping activity within the VTA: medial neurons tended to show a more positive ramp whereas lateral neurons did not. Consistent with TD errors, most of the recorded neurons showed a phasic excitation at the time of teleport and the ramping activity increased with speed. A model fit analysis indicated that the activity of most of the neurons can be approximated by the first-order derivative of value. Finally, the predicted calcium responses from spikes are similar to the calcium responses measured in the VTA dopamine cell bodies.Thus, dopamine neurons compute moment-by-moment TD errors at the single neuron level, supporting the central tenet of TD learning.
Andrew Westbrook - Dopamine Promotes Cognitive Effort Virtual Dopamine 2020-11-23 | The Future of Dopamine - ViDA Symposium - Nov 19, 2020Andrew WestbrookFrank Lab, Brown University & Cools Lab, Donders InstituteStriatal Dopamine Promotes Cognitive Effort by Biasing the Benefits versus Costs of Cognitive WorkStimulants like methylphenidate are increasingly used to treat ADHD-like symptoms, and for cognitive enhancement, but the precise mechanisms of action are unknown. Prior theories have alternately pointed to cortical or subcortical effects and modulation of either dopamine or norepinephrine. In this study, we show that striatal dopamine increases willingness to expend cognitive effort for reward, by amplifying the subjective benefits versus the subjective costs of action. Our study utilized convergent evidence including PET measures of dopamine synthesis capacity, methylphenidate, and sulpiride, a D2 receptor agent, to implicate striatal dopamine, in particular. Gaze patterns reveal that attention to benefit versus cost information also promotes cognitive motivation. Furthermore, both methylphenidate and higher dopamine synthesis capacity increase the effect of benefits on the evidence accumulation process. In addition, we find dynamic effects of gaze on choice, where gaze early in a trial magnifies the influence of attended-versus-unattended information while late gaze instead reflects an emerging preference. These findings help resolve conflicting accounts of how gaze impacts economic decision-making more broadly, beyond decisions about cognitive effort.
Genoveva Uzunova - Inhibition of D2 Dopamine Receptors Using Antisense RNA Virtual Dopamine 2020-11-23 | The Future of Dopamine - ViDA Symposium - Nov 19, 2020Genoveva UzunovaPsychiatry fellow with Prof. Eric Hollander, Albert Einstein College of MedicineDepartment of Pharmacology and Physiology, Drexel University School of Medicine, Philadelphia, PAInhibition of D2 Dopamine Receptors Using Antisense RNAOur goal was to develop an antisense RNA approach to block selectively D2 dopamine receptors in brain. We constructed a non-viral plasmid vector expressing antisense RNA to the mouse D2 dopamine receptor. We tested the specificity of the D2 antisense RNA produced by this vector in vitro in HEK293 cells, transfected with the D2 receptor. After transfection, the vector inhibited D2 RNA and D2 receptors. We determined the localization and spread of the D2 antisense RNA expression vector in mouse striatum after stereotaxic injection. We determined the ability of a single intrastriatal injection on the D2 antisense RNA expression vector in mouse brain, to inhibit D2 dopamine receptor RNA and protein. We found that a single intrastriatal injection of the D2 antisense RNA-expression vector in mouse brain caused long-term inhibition of D2-dopamine behaviors modulated by D2 dopamine agonists. Therefore, such a selective strategy may be used to study D2 dopamine-mediated behaviors in vivo (and in vitro).Next, we found that the D2 dopamine antisense RNA expression vector, unlike haloperidol, produces long-term inhibition of D2 dopamine-mediated behaviors in mice for up to one month without causing upregulation of D2 dopamine receptors. Therefore, the D2 antisense RNA strategy may be used as a highly selective gene therapeutic approach to block D2 dopamine receptors in brain to treat disorders with D2 dopaminergic hyperactivity.
Dan Bang - Sub-second dopamine and serotonin signalling in human striatum Virtual Dopamine 2020-11-23 | The Future of Dopamine - ViDA Symposium - Nov 19, 2020Dan BangWellcome Centre for Human Neuroimaging, University College LondonSub-second dopamine and serotonin signalling in human striatum during perceptual decision-makingRecent animal research indicates that dopamine and serotonin, neuromodulators traditionally linked to appetitive and aversive processes, are also involved in sensory inference and decisions based on such inference. We tested this hypothesis in the human brain, by using fast scan cyclic voltammetry (FSCV) to monitor sub-second changes in dopamine and serotonin delivery to the striatum during a visual perceptual decision task. The opportunity to measure fast neuromodulation in humans occurs during neurosurgery where deep brain stimulating electrodes are implanted for the management of movement disorder symptoms (Parkinson’s disease and essential tremor). With minimal deviations from the standard of care, a research probe can be inserted into the striatum, by following the same path as the DBS electrode. During acquisition of FSCV data, patients performed a modified version of the classic random dot motion task that separates sensory uncertainty from decision difficulty in a factorial design. Caudate nucleus recordings (n = 4) revealed multi-scale encoding: in three participants, serotonin tracked sensory uncertainty, and, in one participant, both dopamine and serotonin tracked deviations from expected trial transitions within our factorial design. Putamen recordings (n = 1) supported a cognition-action separation between caudate nucleus and putamen – a striatal sub-division unique to primates – with both dopamine and serotonin tracking decision times. These first-of-their-kind observations in the human brain support a role for sub-second dopamine and serotonin signalling in non-reward-based aspects of cognition and action.
Caroline Jahn - Complementary roles of dopamine and noradrenaline in motivation Virtual Dopamine 2020-11-23 | The Future of Dopamine - ViDA Symposium - Nov 19, 2020Caroline JahnBuschman Lab, Princeton UniversityComplementary roles of dopamine and noradrenaline in motivationThe two catecholamines, noradrenaline and dopamine, have been shown to play comparable roles in behaviour. Both noradrenergic and dopaminergic neurons respond to cues predicting reward availability and novelty. However, even though both are thought to be involved in motivating actions, their roles in motivation have seldom been directly compared. We, therefore, examined the activity of putative noradrenergic neurons in the locus coeruleus and putative midbrain dopaminergic neurons in monkeys cued to perform effortful actions for rewards. The activity in both regions correlated with engagement with a presented option. By contrast, only noradrenaline neurons were also (i) predictive of engagement in a subsequent trial following a failure to engage and (ii) more strongly activated in non-repeated trials, when cues indicated a new task condition. This suggests that while both catecholaminergic neurons are involved in promoting action, noradrenergic neurons are sensitive to task state changes and their influence on behaviour extends beyond the immediately rewarded action.
Ritwik Niyogi- Fast dopamine release signals reward rate to mediate vigour Virtual Dopamine 2020-11-23 | The Future of Dopamine - ViDA Symposium - Nov 19, 2020Ritwik NiyogiDaw Lab, Princeton UniversityFast dopamine release signals reward rate to mediate vigourMesolimbic dopamine release is believed to express motivation by increasing vigor. But unlike phasic dopaminergic spiking, linked to prediction error and learning, there is no clear link between dopamine release (which can be distinct from spiking), decision variables, and vigor. We investigate this question using computational modeling of vigor and dopamine release dynamics reported from the NAc (Mohebi et al., 2019). Theory (Niv. et al., 2007) implies that animals should act faster when the estimated reward rate is higher. Although this idea was applied to slow, tonic dopamine responses, we suggest that the dynamics of reward rate estimation provide a parsimonious interpretation for recent observations of faster peri-event dynamics in NAc release, previously argued to track value. This can also be understood as a running reward rate estimate (hence linked to behavioral vigor), because reward rate can be estimated by filtering event-related signals related to value expectation (i.e., prediction errors), rather than rewards actually received. We show that DA release around task events superimposes at least two timescales of filtering of such error responses, fast and slow. The slower time constant has the same order of magnitude as that relating behavioral vigor to reward. While this account moves us closer to a unified account of dopamine’s dual functions in learning and motivation, a key obstacle remains: the prediction errors implicit in release and explicit in spiking do not always coincide. We discuss whether this discrepancy might be explained by local control of release or by nonlinearities in the release function.
Pedro Jacob - Spaced training forms complementary long-term memories of opposite valence Virtual Dopamine 2020-11-23 | The Future of Dopamine - ViDA Symposium - Nov 19, 2020Pedro F. JacobWaddell Lab at the Centre for Neural Circuits and Behaviour, Oxford UniversitySpaced training forms complementary long-term memories of opposite valence in DrosophilaThe formation of long-term memory (LTM) in many species requires repetitive experience spread over time (spaced training). However, it is currently unclear what changes in the neural network result from spaced training, and why persistent and robust LTM performance emerges. Prior studies from our lab have shown that flies can simultaneously, or sequentially, form parallel avoidance and approach memories, that compete to guide memory-directed behaviour.We therefore tested whether a parallel memory network model could account for aversive LTM.In Drosophila, aversive olfactory LTM is optimal following spaced training, where each trial pairs one of two odours with an electric shock. Flies acquire parallel memories after spaced training, an aversive memory for the shock-paired odor and a slowly emerging ‘safety-memory’ for the explicitly unpaired odor. ‘Safety-memory’ requires repetition, order and rest intervals and relies on specific subsets of rewarding dopaminergic neurons. Calcium imaging of output neurons of the mushroom body, the centre for learning and memory in Drosophila, revealed the co-existence of parallel aversive and safety memories that can be recorded as depression of odor-specific responses in a distributed collection of unique output neurons. Additionally, our imaging of the activity of reward dopaminergic neurons suggests that they gradually acquire the ability to reinforce the ‘safety-memory’ across the repetitive differential spaced training trials.Aversive LTM performance after spaced training involves the summation of complementary odor-specific avoidance and approach memories. This augments LTM performance after spaced training by making the odor preference more certain.
Loren Frank - One way to give a talk Virtual Dopamine 2020-11-09 | There is an art to science presentation. There is a science to science presentation. Whether you are ready to go on the market, or you're ready to present your science at a conference you and your science can benefit from a well-sculpted presentation. Join us to hear one way to give a talk by Loren Frank, PhD (UCSF). This event is organized by ViDA and is a precursor to the ViDA symposium on Nov 19th: the future of Dopamine. Register via Eventbrite: eventbrite.com/e/vida-symposium-the-future-of-dopamine-tickets-124044455305
Veronica Alvarez - Dopamine modulation of striatal microcircuitry and its behavioral implications Virtual Dopamine 2020-05-23 | Veronica A AlvarezIntramural Research Program, National Institutes of HealthDopamine modulation of striatal microcircuitry and its behavioral implications05-22-2020Dopamine is involved in a wide-range of critical striatal functions including reinforcement learning, motivation and vigor. Dopamine participates in these functions via its modulatory effects on striatal neurons and their synaptic connections. Thus, understanding the cellular and synaptic mechanisms underlying dopamine actions in the striatum is key to advancing our knowledge of these critical striatal functions. The data presented will dissect the roles of D1 and D2 receptors, which are the two main types of dopamine receptors expressed in the striatum. While the focus of most studies thus far has been on the action of dopamine on glutamatergic inputs to the striatum, we have found that dopamine also potently suppresses local inhibitory synapses among striatal projection neurons (aka, medium spiny neurons). These local inhibitory synapses mediate lateral inhibition among striatal projection neurons. Therefore, suppression of the collateral inhibition by dopamine can promote the activation of D1 receptor expressing striatal projection neurons when it coincides with activation of D1 receptors and/or excitatory inputs. A combination of in vitro and in vivo studies show that dopamine actions on lateral inhibition provide a circuit mechanism that explains the poorly understood synergistic effect of D1-type and D2-type dopamine receptors on behavior. I will also describe other studies that explore the synaptic and circuit mechanism of alcohol and other drugs of abuse. Most behavioral effects of these drugs rely on an increase of dopamine in the striatum. Mice with targeted deletion of the D2 receptor gene on either indirect-pathway (D2 receptor) projection neurons or midbrain dopamine neuron terminals in the striatum show unique alterations in the cellular and behavioral responses to substances of abuse including cocaine and alcohol. As such, these experiments shed further light on the unique functions of dopamine and D2Rs expressed across the basal ganglia circuitry and the consequent behavioral implications of these circuit effects.
Michael Frank - Striatal dopamine computations in learning about agency - ViDA 2020 Virtual Dopamine 2020-05-23 | Michael FrankBrown UniversityStriatal dopamine computations in learning about agency05-22-2020The basal ganglia and dopaminergic systems are well studied for their roles in reinforcement learning and reward-based decision making. Much work focuses on "reward prediction error" (RPE) signals conveyed by dopamine and used for learning. Computational considerations suggest that such signals may be enriched beyond the classical global and scalar RPE computation, to support more structured learning in distinct sub-circuits ("vector RPEs"). Such signals allow an agent to assign credit to the level of action selection most likely responsible for the outcomes, and hence to enhance learning depending on the generative task statistics. I will present studies from rodents in tasks involving learning about their agency in controlling reward-predictive sensory events while activity from dopamine terminals across a wide range of dorsal striatum is monitored. Findings indicate that spatiotemporal dynamics of striatal dopamine are influenced by instrumental task demands, and that they are used to enhance credit assignment to differentially reinforce the underlying circuits.
Nathaniel Daw - Population codes and prediction errors - ViDA 2020 Virtual Dopamine 2020-05-23 | Nathaniel DawPrinceton UniversityPopulation codes and prediction errors05-22-2020Standard theoretical accounts envision that the midbrain dopamine system broadcasts a uniform prediction error signal through diffuse ascending connections to forebrain. However, I review mounting evidence that now clearly demonstrates that the dopamine response is instead heterogeneous from target area to target area and even from neuron to neuron. I present a new model that suggests that this heterogeneity is inherited from, and ultimately may help to shed light on, the population codes for ongoing task events in afferent cortical and striatal areas. This may provide an unexpected window on a core problem for decision and learning models of all sorts, known as the problem of state: how the brain represents the relevant (and ignores the irrelevant) features of a task at each step.
Josh Berke - Dopamine firing versus dopamine release during motivated behavior - ViDA 2020 Virtual Dopamine 2020-05-22 | Josh BerkeUCSF05-21-2020Dopamine firing versus dopamine release during motivated behavior.Moderator: Howard Fields (UCSF)Co-Moderator: Lauren Burgeno (University of Oxford)We have reported that the spiking of identified VTA dopamine neurons does not account for dopamine release within the NAc, at least under some behavioral conditions. For example, many labs have observed fast ramps in dopamine concentration as rats approach rewards, but these are not accompanied by increased dopamine cell spiking. This work has received substantial interest but also criticism. I will briefly describe our main findings and some key points of controversy, and then present new results on the mechanisms governing dopamine release during reward anticipation.
Yael Niv - Model-based predictions for dopamine - ViDA 2020 Virtual Dopamine 2020-05-22 | Yael NivPrinceton University05-21-2020Model-based predictions for dopamineModerator: Howard Fields (UCSF)Co-Moderator: Lauren Burgeno (University of Oxford)Phasic dopamine responses are thought to encode a prediction-error signal consistent with model-free reinforcement learning theories. However, a number of recent findings highlight the influence of model-based computations on dopamine responses, and suggest that dopamine prediction errors reflect more dimensions of an expected outcome than scalar reward value. In this talk I will focus on these challenges to the scalar prediction-error theory of dopamine, and to the strict dichotomy between model-based and model-free learning, suggesting that these may better be viewed as a set of intertwined computations rather than two alternative systems. Alas, phasic dopamine signals, until recently a beacon of computationally-interpretable brain activity, may not be as simple as we once hoped they were.
David Sulzer - Dopamine synapses and the regulation of synapses during behavior - ViDA 2020 Virtual Dopamine 2020-05-22 | David SulzerColumbia University05-21-2020Dopamine synapses and the regulation of synapses during behaviorModerator: David Lovinger (NIH/NIAAA)Co-Moderator: Didi Mamaligas (UCSF/Gladstone)The output neurons of the striatum (spiny projection neurons, SPNs) are intrinsically silent and are activated only in response to excitation by inputs from the cortex and thalamus. The SPNs in early life are highly excitable, but as learning during critical periods ends, they become far less excitable. We discuss new results (from Ori Lieberman) that this is due to effects of dopamine and macroautophagy on SPN ion channel expression. When dopamine is released simultaneously with cortical input, it acts at D2 receptors to select specific corticostriatal synapses by decreasing release from the less active corticostriatal synapses, a form of synaptic filtering that may underlie learning as animals interact with their environment (from Nigel Bamford and Minerva Wong). This synaptic selection may occur in a very short defined time frame during operant learning when an animal must coordinate a stimulus with a learned behavior (from Avery McGuirt).
Jennifer Whistler - ViDA 2020 Virtual Dopamine 2020-05-22 | Jennifer WhistlerUC Davis05-20-2020Post-endocytic sorting of D2 dopamine receptors underlies drug induced changes in synaptic and behavioral plasticityLow D2 dopamine receptor availability is a key hallmark of substance use disorder for all drugs of abuse. Low D2 receptor both increases the risk of drug seeking, and occurs as a consequence of drug use. Selective loss of D2 receptors alters the balance of excitatory and inhibitory G protein coupled receptor (GPCR) dopamine signaling, as the D1-like dopamine receptors are Gs-coupled whereas the D2-like receptors are Gi-coupled GPCRs. Loss of balance in GPCR-mediated dopamine signaling, in turn, is likely to underlie many of the alterations in synaptic and behavioral plasticity that occur with prolonged drug use. However, the molecular mechanisms underlying drug-induced loss of D2 receptors remain understudied. We have found that while both the D1 the D2 dopamine receptors are endocytosed after activation by dopamine, the D1 receptors are rapidly recycled whereas the D2 receptors are targeted for degradation in the lysosome. We have identified a sorting protein, GPCR-associated sorting protein 1 (GASP1) that binds to D2 and D3 dopamine receptors and is necessary for their dopamine-induced degradation. Mice with a disruption in GASP1 (GASP1-KO) do not downregulate D2 receptors in response to drug of abuse, unlike wild type mice that show profound loss of D2 receptors. In addition, GASP1-KO mice do not show sensitization to the locomotor activating effects of drug nor do they show changes in behavioral flexibility in response to drug. Furthermore, drug-induced alterations in glutamatergic plasticity in dopamine neurons in the ventral tegmental area (VTA) that occur in wild type mice are absent in GASP1-KO mice. These data suggest that post-endocytic targeting of D2-like receptors to the lysosome underlies changes in both behavioral and synaptic plasticity that occur as a consequence of drug use. They also suggest that drugs that can activate the D2 receptor without promoting its endocytosis and degradation could be used to prevent or restore these drug-induced changes in plasticity. In support of this hypothesis, we have found that administration of aripiprazole, a high affinity G protein biased agonist at the D2 receptor that does not promote receptor endocytosis prevents drug-induced changes in synaptic plasticity in the VTA as well as the drug-induced changes in behavioral flexibility. Hence, we have identified a molecular mechanism that underlies the drug-induced loss of D2 receptor, and we have leveraged this finding to identify potential therapeutic interventions that can restore the balance in dopamine signaling.
Carmen Canavier - What Constrains the Maximum Firing Rate in Midbrain Dopamine Neurons? - ViDA 2020 Virtual Dopamine 2020-05-22 | Carmen CanavierLSU Health Sciences Center New Orleans05-20-2020What Constrains the Maximum Firing Rate in Midbrain Dopamine Neurons?The electrical activity of midbrain dopamine (DA) neurons was thought to homogeneously signal reward prediction errors. However, distinctions between DA subpopulations are now becoming increasingly clear. Subpopulations of dopamine neurons are characterized in various ways, including electrophysiology, projection target, anatomical location and gene expression. Lammel et al. 2008 identified two subpopulations based on electrophysiology: conventional, slow firing dopamine neurons in the substantia nigra plus a subset of VTA neurons projecting to the lateral shell of the nucleus accumbens versus atypical, faster firing dopamine neurons in the remainder of the VTA. A key difference was the dynamic range of the subpopulations; the atypical population was able to sustain much higher firing frequencies (~20-25 Hz versus ~10 Hz). Another key difference was the mode of entry into depolarization block that limits the firing frequency. The slow firing population fails abruptly, whereas the fast firing population fails via decrementing action potential height. A single compartment model with a Markov model of the sodium channel that included both fast and slow activation states was able to capture these differences very well. The fast firing population was modeled with the known differences of a slower recovery from inactivation for Kv4.3, a smaller surface area, a weak or absent H current, and a shallower AHP. These differences did not account for the difference in maximal firing rate. However, increasing the tendency to enter the slow inactivated state did account for both the difference in maximal rate and the different mode of entry into depolarization block.
Naoshige Uchida - Dissociating reward prediction error and value in dopamine signals - ViDA 2020 Virtual Dopamine 2020-05-22 | Naoshige UchidaHarvard University05-20-2020Dissociating reward prediction error and value in dopamine signalsPrevious studies have revealed an exceptional correspondence between the activity of midbrain dopamine neurons and a ‘teaching signal’ in reinforcement learning algorithms. In particular, the reward prediction error (RPE) used in the temporal difference (TD) learning algorithm captures aspects of phasic dopamine responses. However, this idea has been challenged by recent observations that dopamine signals ramp up gradually over the timescale of seconds as animals approach a reward location. It has been argued that these slow fluctuations of dopamine are inconsistent with the RPE model, and instead represent the state value, which gradually increases toward a reward location. Whether these slowly fluctuating dopamine signals represent value or RPE, and under what conditions a dopamine ramp occurs, remain elusive. As originally formulated, the TD RPE approximates the derivative of the value function. Based on this core property, we developed a set of experimental paradigms that dissociate RPE from value. We employed visual virtual reality in mice to manipulate the location of the animal and the speed of scene movement independent of the animal’s locomotion. We found that the manipulation of scene movement – teleport and speed manipulations – caused dopamine responses in the ventral striatum that were consistent with TD RPEs but inconsistent with state values. Furthermore, we found that a more abstract, non-navigational stimulus that indicates temporal proximity to reward is sufficient to cause a dopamine ramp. These results indicate that the RPE account of dopamine responses can be extended to slowly fluctuating dopamine signals in addition to phasic dopamine responses, and support the previously untested central tenet of TD RPEs that dopamine neurons signal RPEs through a derivative-like computation over value on a moment-by-moment basis.
Jesse Goldberg - ViDA 2020 Virtual Dopamine 2020-05-22 | Jesse Goldberg (Cornell University)Male songbirds turn off their self evaluation system when they perform for females05-20-2020Attending to mistakes while practicing alone provides opportunities for learning, but self-evaluation during audience-directed performance could distract from ongoing execution. It remains unknown how animals switch between practice and performance modes, and how evaluation systems process errors across distinct performance contexts. We recorded from striatal-projecting dopamine (DA) neurons as male songbirds transitioned from singing alone to singing female-directed courtship song. In the presence of the female, singing-related performance error signals were reduced or gated off and DA neurons were instead phasically activated by female vocalizations. DA neurons can thus dynamically change their tuning with changes in social context.
