The Misbehaviour of Value and the Discipline of the Will
Most reinforcement learning models of animal conditioning operate under the convenient, though fictive, assumption that Pavlovian conditioning concerns prediction learning whereas instrumental conditioning concerns action learning. However, it is only through Pavlovian responses that Pavlovian prediction learning is evident, and these responses can act against the instrumental interests of the subjects. This can be seen in both experimental and natural circumstances. I will discuss the consequences of importing this competition into a reinforcement learning context, and demonstrate the resulting effects in an omission schedule. The misbehavior created by Pavlovian values can be quite debilitating; we discuss how it may be disciplined. Finally, I will consider the extension of these notions to one idea about the role of serotonin and serotonin reuptake inhibitors in depression.