Reinforcement learning links spontaneous cortical #dopamine impulses to reward

Extrasynaptic levels of dopamine in mouse cortex exhibit spontaneous impulses
•Impulses are broadly distributed in amplitude and time, with a rate of about 0.01/s
•Feedback was used to train mice to volitionally control their spontaneous impulses
•Mice learned to reliably modulate dopamine impulses in order to receive a reward

In their pioneering study on dopamine release, Romo and Schultz speculated “…that the amount of dopamine released by unmodulated spontaneous impulse activity exerts a tonic, permissive influence on neuronal processes more actively engaged in preparation of self-initiated movements….”1 Motivated by the suggestion of “spontaneous impulses,” as well as by the “ramp up” of dopaminergic neuronal activity that occurs when rodents navigate to a reward,2, 3, 4, 5 we asked two questions.

First, are there spontaneous impulses of dopamine that are released in cortex? Using cell-based optical sensors of extrasynaptic dopamine, [DA]ex,6 we found that spontaneous dopamine impulses in cortex of naive mice occur at a rate of ∼0.01 per second. Next, can mice be trained to change the amplitude and/or timing of dopamine events triggered by internal brain dynamics, much as they can change the amplitude and timing of dopamine impulses based on an external cue?7, 8, 9

Using a reinforcement learning paradigm based solely on rewards that were gated by feedback from real-time measurements of [DA]ex, we found that mice can volitionally modulate their spontaneous [DA]ex. In particular, by only the second session of daily, hour-long training, mice increased the rate of impulses of [DA]ex, increased the amplitude of the impulses, and increased their tonic level of [DA]ex for a reward. Critically, mice learned to reliably elicit [DA]ex impulses prior to receiving a reward.

These effects reversed when the reward was removed. We posit that spontaneous dopamine impulses may serve as a salient cognitive event in behavioral planning.