RRC ID 77576
Author Amo R, Matias S, Yamanaka A, Tanaka KF, Uchida N, Watabe-Uchida M.
Title A gradual temporal shift of dopamine responses mirrors the progression of temporal difference error in machine learning.
Journal Nat Neurosci
Abstract A large body of evidence has indicated that the phasic responses of midbrain dopamine neurons show a remarkable similarity to a type of teaching signal (temporal difference (TD) error) used in machine learning. However, previous studies failed to observe a key prediction of this algorithm: that when an agent associates a cue and a reward that are separated in time, the timing of dopamine signals should gradually move backward in time from the time of the reward to the time of the cue over multiple trials. Here we demonstrate that such a gradual shift occurs both at the level of dopaminergic cellular activity and dopamine release in the ventral striatum in mice. Our results establish a long-sought link between dopaminergic activity and the TD learning algorithm, providing fundamental insights into how the brain associates cues and rewards that are separated in time.
Volume 25(8)
Pages 1082-1092
Published 2022-8-1
DOI 10.1038/s41593-022-01109-2
PII 10.1038/s41593-022-01109-2
PMID 35798979
PMC PMC9624460
MeSH Animals Cues Dopamine* / physiology Dopaminergic Neurons / physiology Machine Learning Mesencephalon Mice Reward*
IF 20.071
Mice RBRC09545