==== Submitted for NIPS*98 ==== ==== Denver, CO, USA, Nov 30 -- Dec 5, 1998 ==== Sequence Representation in Animals and Networks: Study of a Recurrent Network Trained with Reinforcement Learning Raju S. Bapi & Kenji Doya Computational Neurobiology Group Kawato Dynamic Brain Project, ERATO, JST 2-2 Hikaridai, Seika, Soraku, Kyoto 619-0288, Japan Abstract Neural encoding for sequence identification, memory and production is studied using an Elman-style recurrent network (Elman, 1990) is studied. Novel feature of this network is that learning is implemented using biologically plausible reinforcement learning paradigm. Findings from Tanji & Shima's (1994) experiments on monkeys indicate that there is sequence-specific activity in the supplementary motor area (SMA) and change of motor-plan related activity in the preSMA. A network with anatomical correspondence to known cortico-basal ganglionic structure is designed to understand how sequence-specific activity arises due to training. Simulations demonstrate that context switching in the preSMA layer facilitates learning of multiple sequences and the hidden unit activity in the resultant network resembles what Tanji & Shima observed in monkeys. It is proposed that the basal ganglia (BG) implement an actor and a critic which learn sequences with the help of internal reinforcement signals calculated as predictions of future reinforcement and context inputs from the preSMA and SMA circuitry. Further, flexible shifting among various sequences requires contextual inputs from preSMA.