our computer modelling centre
OFTNAI is currently supporting a computer modelling centre within the Oxford University Department of Experimental Psychology. Computer modellers within the university research centre are investigating motor function in the brain.
Experimental work indicates that during motor tasks such as reaching, the motor areas of the brain work in tandem with other brain areas that represent spatial information such as the position of the hand. Inspired by these findings, we are developing models which combine motor and spatial networks that work together to carry out motor tasks.
Our models are able to learn to perform arbitrary motor sequences, even in the absence of sensory feedback. Further work has shown how motor sequences can be learned with a delayed reward signal at the end of each training sequence.
the impact of our research
Our models can also generate useful novel motor sequences, which were not performed during training. These are all important capabilities in animals.
We understand that our theoretical work in this area will help to inform medical research aimed at understanding and treating disorders affecting the motor functions of the nervous system.
Such disorders may include motor neurone disease, multiple sclerosis, and Parkinson's disease. Regarding engineering applications, our brain-inspired models allow very flexible and adaptive behaviour, which can surpass current robot control systems that rely on either fixed action sequences or learned stimulus-response reactions. Another area of application may be automated character animation in computer generated films and games.
Case study:
Learning movement sequences with a delayed reward signal
A key question in motor learning is how can an animal learn to perform a sequence of movements when the reward signal only occurs at the end of the sequence. We have developed a hierarchical dynamical model of motor function which may help to explain how the brain solves this problem (Stringer, S.M., Rolls, E.T. and Taylor, P., 2006). At the lowest level, the motor system encodes simple movements or primitives, while at higher levels the system encodes sequences of primitives.
Network architecture of the hierarchical dynamical model of motor function. There are networks of state (S) cells which represent the postural state of the agent, motor (M) cells which represent the motor activity, movement selector (MS) cells which represent the command to perform a motor primitive, and high level movement selector (HMS) cells which represent the high level motor programs composed of sequences of primitives.
During learning of a high level motor program, a set of HMS cells are activated. For each attempt at the motor program, a new (possibly random) set of motor primitives is chosen by activating the corresponding MS cells. Although the MS cells remain active throughout the motor program, each motor primitive is only performed when the agent is in the correct part of its state space because the inputs to these cells are modulated by the state cells.
Whenever the agent completes the motor program successfully, the agent receives a reward signal and the synapses from the active HMS cells to the active MS cells are strengthened. This associates the command to perform the high level motor program with the motor primitives required to perform the sequence. After training, activating the same set of HMS cells causes the network to replay the correct sequence of motor primitives.