Sobell Department of Neurophysiology, Institute of Neurology, University College London, London WC1N 3BG, United Kingdom
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Witney, Alice G., Susan J. Goodbody, and Daniel M. Wolpert. Predictive Motor Learning of Temporal Delays. J. Neurophysiol. 82: 2039-2048, 1999. Anticipatory responses can minimize the disturbances that result from the action of one part of the body on another. Such a predictive response is evident in the anticipatory increase in grip force seen when one hand pulls on an object held in the other hand, thereby preventing the object from slipping. It is postulated that such a response depends on predicting the consequences of the descending motor command, as signaled by efference copy, using an internal model of both one's own body and the object. Here we investigate how the internal model learns the temporal consequences of the motor command. We employed two robots to simulate a virtual object held in one hand and acted on by the other. Delays were introduced between the action of one hand on the object and the effects of this action on the other hand. An initial reactive grip force response to the delayed load decayed with the development of appropriate anticipatory grip force modulation. However, no predictive modulation was seen when the object's movement was not generated by the subject, even when the motion was cued by a tone. These results suggest that, when an internal model learns new temporal relationships between actions and their consequences, this learning involves generating a novel response rather than adapting the original predictive response.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Humans demonstrate different responses to
self-produced and externally produced perturbations (Blakemore
et al. 1998; Dufosse et al. 1985
;
Johansson et al. 1992
; Johansson and Westling
1988
; Massion 1992
). For example, when subjects
are required to remove an object held in one hand with the other hand,
anticipatory deactivation of the forearm muscles occurs before the
unloading so that the position of the loaded hand remains unchanged
(for a review see Massion 1992
). Such anticipatory
behavior has been attributed to the ability to predict the consequences
of our own actions (Johansson and Cole 1994
;
Lacquaniti et al. 1992
; Massion 1992
), a
process that requires an internal model of both one's own body and the
external world. Such models are known as forward models because they
capture the forward or causal relationship between actions, as signaled
by efference copy (Jeannerod et al. 1979
; Sperry
1950
; von Helmholtz 1867
; von Holst
1954
), and outcome. Forward models have been proposed to play a
fundamental role in motor planning, execution, and learning
(Jordan 1995
; Jordan and Rumelhart 1992
;
Kawato et al. 1987
; Miall and Wolpert
1996
; Wolpert 1997
; Wolpert et al.
1995
). As the dynamics of the body change during growth, and
novel objects are encountered, the internal models of both the motor
system and external objects must be acquired and continually refined
throughout life. The ability to learn and use internal models is,
therefore, a fundamental property of the motor system. How such
internal models are learned is the focus of the current study.
Grip-force modulation provides an ideal paradigm to study internal
model learning. When subjects pick up an object using a precision grip,
they exert sufficient grip force to prevent the object from slipping
while avoiding excessive grip forces that may result in object breakage
or fatigue (Johansson and Cole 1992, 1994
; Johansson et al. 1992
;
Johansson and Westling 1984
). When the object is held at
rest, the grip force depends both on the weight of the object, that is
load force, and the coefficient of friction of its surfaces. Grip force
levels can be set without somatosensory feedback, anticipating the
physical properties of the object, that include the object's weight,
shape, and friction at its surface (Jenmalm and Johansson
1997
; Johansson and Cole 1994
; Johansson
and Westling 1984
, 1988
). Such object properties are learned through development, indicated by increasing ability to
correctly parameterize grip force to the object being manipulated (Eliasson et al. 1995
; Forssberg et al.
1991
, 1992
, 1995
). This immediate
scaling of the appropriate grip force level has been described as
anticipatory parameter control (Johansson 1996
;
Johansson and Cole 1992
, 1994
) with the
updating of the appropriate grip force level to alterations in the
object's properties, occurring by a process of discrete-event,
sensory-driven control (for reviews see Johansson and Cole
1992
, 1994
). When the object is moved by the
subject, the load force on the fingers must change to accelerate the
gripped object. Without a corresponding change in grip force, the
object would slip. In this self-generated condition the grip force
tends to parallel load force with negligible delay. Such anticipatory
modulation is seen in both discrete (Johansson and Westling
1984
) and continuous self-generated movements (Flanagan and Wing 1993
, 1995
), made in different
directions and at different speeds, and when pulling on fixed objects
(Johansson et al. 1992
). Despite the relatively rapid
response of cutaneous afferents, such anticipatory control cannot be
explained as a reaction to peripheral feedback (Flanagan and
Wing 1995
; Johansson and Westling 1984
) due to
unavoidable feedback delays (Forssberg et al. 1992
; Johansson and Westling 1984
). A system based solely
on feedback control would be ineffective for manipulative actions with
frequencies above ~1 Hz, which would exclude some complex skills
(Johansson and Cole 1994
; Kunesch et al.
1989
). This suggests that the skilled manipulation of objects
requires the CNS to use the motor command, in conjunction with internal
models of both the arm and the object, to anticipate the resulting load
force and thereby adjust grip force appropriately (Blakemore et
al. 1998
; Flanagan and Wing 1997
; Wing et
al. 1997
). Conversely, when the load force is altered by
external agents, grip force lags 60-100 ms behind load force, suggesting a reactive response to the perturbation (Blakemore et
al. 1998
; Cole and Abbs 1988
; Johansson
et al. 1992
; Johansson and Westling 1988
).
Therefore the timing of grip force modulation can be used to quantify
the accuracy of an internal model. Lags between grip and load suggest a
reactive process in which the CNS is unable to predict the load force,
whereas no lag indicates an accurate internal model.
An integral component of such anticipatory behavior is matching the
grip force response temporally to the load force. Johansson and
Westling (1988) examined grip force responses when a small ball
was dropped into a cup that was gripped by the subject. When the ball
was dropped by the experimenter, a reactive grip response occurred
70-80 ms after impact. Conversely, anticipatory grip force responses
occurred when the load force was self-generated, that is the subjects
dropped the ball, independent of the length of the drop. This suggests
that anticipatory grip force modulation can be appropriately timed for
different delay (length of drop). Such varying temporal delays are not
unusual in motor control. For example, the motor command sent to the
hand and foot will have consequences that are both delayed due to
conduction times, and also separate in time due to the different
conduction distances (Bard et al. 1992
).
In the current study we sought to examine how internal models learn
these new temporal relationships between the predicted and actual
consequences of a descending motor command. Consider a subject holding
an object in a precision grip in one hand and generating a force pulse
on it with the other. Such a condition leads to anticipatory grip force
modulation (Blakemore et al. 1998; Johansson et
al. 1992
) precisely timed to prevent slippage while using a low
grip force level (Fig. 1A).
The timing of the response is critical to ensure that the peak in the
grip force matches the peak in the load force. In the current study we
used two robotic interfaces to examine this task when a delay was
introduced between the action of one hand and its consequences on an
object held in the other hand. In the normal nondelayed condition,
pulling on the object with one hand produced equal and opposite forces on both hands at the same time. This is consistent with the situation in which a single object is held between the hands. In the delayed condition the movement of one hand resulted in the production of the
corresponding force on the other hand after a delay of 250 ms (Fig.
1B). On the introduction of the delay, we expect to see two
peaks in the grip force response. The first is the anticipatory
response to the predicted load force that would normally have occurred
without delay as a result of the movement. The second is a grip force
reactive to the delayed load acting on the object (Fig. 1B).
The internal model that previously allowed predictive modulation, for
the nondelayed load, will no longer be appropriate for the task, and
consequently may be updated.
|
By introducing a delay between the action of one hand and the effect on the other, we were able to test between different hypotheses about the adaptation of the internal model. We examined how the grip force modulation changed on repeated presentations of this delayed consequence of action. We can consider two hypotheses of how the internal model learns to produce a predictive response for the delayed load. The first possibility is that the original predictive peak shifts with learning to the new later time of peak load force while the second, reactive, peak decreases. This would be observed as a migration of the first grip force peak and suggests an adaptation of the existing internal model (Fig. 1C, Migration of peak 1). Alternatively, a new predictive peak to the delayed load force could develop, implying the development of a new internal model, appropriate for the novel situation. This would be reflected by a migration of the second grip force peak caused by the new predictive component becoming increasingly predominant, coupled with a decrease in the reactive component of the response. (Fig. 1C, Migration of peak 2).
To examine the specificity of the anticipatory response to the prediction of the consequences of the motor command, we also examined another sensorimotor context in which the object was subject to an externally produced force pulse a fixed interval after a tone. In this "cued" condition, as in the self-generated condition, the load force is predictable, but is now related to a purely sensory stimulus rather than the descending motor command. Both the self-generated and cued conditions were compared with a "noncued" condition in which the object was pulsed at pseudorandom intervals to assess purely reactive grip force responses. The current study is an investigation of adaptation to delays between self-produced actions and their effects. This is in contrast with the studies of conditioning to external sensory stimuli. The cued condition performs the role of a control in the current study.
To assess the predictive component of the grip force response isolated from any effects of sensory feedback from the finger pads, catch trials were randomly interspersed. During a catch trial, in both the cued and self-generated conditions, no load force was generated. This allowed an analysis of the grip force modulation that occurs solely as a consequence of the internal model.
![]() |
METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Subjects
Six right-handed subjects (3 male, 3 female) aged 21-28 yr participated in the experiment. None of the subjects reported sensory or motor deficits. Subjects gave written informed consent before participating in the experiments and were naïve to the research aims.
Apparatus
A six-axis cylindrical force transducer (Nano, ATI) was embedded in a cylindrical object with two parallel grip surfaces of 20 mm diam, spaced 40 mm apart. The two surfaces were covered with sandpaper (Grade No. 210) with the mass of the transducer (50 g) centred halfway between the surfaces. The force transducer allowed three translational forces to be measured with an accuracy of 0.05 N including cross-talk. This object was attached to the end of lightweight, robotic manipulator (Phantom Haptic Interface, Sensable Devices). A second cylinder, of identical size and mass to the first, was attached to a second robotic manipulator. These robots, which are free to move in three dimensions, can exert forces of up to 20 N, in any direction, at its endpoint (backdrive friction 0.02 N, apparent mass at the tip <150 g). The position of the motors (and through the kinematic equations of the robot the position of the object) were sampled on-line by three optical encoders (10,160 counts per revolution, sampling rate 1,000 Hz) mounted on the three motors.
Procedure
All subjects participated in three conditions in a balanced order. In all conditions, subjects were required to keep their right hand still, preventing the gripped object slipping from their grasp. The object was gripped with the tips of their right thumb and index finger. The subject's right forearm was anchored with velcro straps, and for further stability, they grasped a vertically oriented aluminum rod with their three ulnar fingers. This ensured that the subject's thumb and index finger were used to maintain object stability rather than a more general postural response. In each trial a discrete upward force was applied by the right hand robot to the object. To prevent fatigue, short rest periods were given every 60 trials in all conditions.
Noncued condition
To measure reactive grip force modulation, the right hand robot
generated 50 force pulses on the object held in the right hand at
pseudorandom times, on average one pulse every 2.5 s (Fig. 2B). Subjects held the left
hand object above the right hand object and stationary throughout. The
temporal profile of each force pulse was chosen to be smooth and bell
shaped. Specifically, the force profile had the same shape as the
velocity profile of a minimum jerk movement (Flash and Hogan
1985). The duration of the force profile was 200 ms and its
maximum amplitude was 5 N. This therefore generated a peak load force
on the fingers of the right hand of 5 N. This force profile was chosen
to match that produced by the subjects in the self-generated condition
described below.
|
Self-generated condition
In the self-generated condition the load force generated by the robot on the object held in the right hand was produced as a result of the subjects making a single vertical upward pulse with their left hand (Fig. 2C). Subjects were told that the load on the right hand object was produced by this vertical movement of their left hand. Subjects held the object in the left hand above the object in the right.
The relation between movement of the left hand and the force generated
by the robot on the right hand object was simulated as a stiff spring
between the objects in the two hands. These forces therefore mimicked
the condition of a single object held between the hands. The force on
the right hand was given by Ft = K (Lt Rt
D) where
Lt and
Rt were the vertical positions of the
left and right object at time t, K was a fixed spring
constant of 15 N/cm, and D was the initial vertical distance
between the objects at the start of the trial. Therefore to achieve the
5-N load force the movement of the left hand was ~3 mm in amplitude. The force on the left hand was equal and opposite to the force on the
right hand. To allow the subjects to produce the correct load force
amplitude, their left hand position was displayed as a vertically
moving (but horizontally stationary) dot on an oscilloscope. The
oscilloscope also displayed two horizontal lines that represented the
amplitude of movement required to generate 5 N.
On average once every 2.5 s a tone sounded and the subjects had to move their left hand sharply up and down to cause the desired brief 5-N target force pulse to be applied to the right hand object. Subjects were given ~30 practice trials, which enabled them to accurately produce a movement with their left hand that could generate a 5-N amplitude vertical force pulse on the object held in their right hand. This movement was recorded to ensure that subjects produced the appropriate movement throughout the experiment.
The paradigm comprised of 3 phases of 50 preexposure, 250 exposure, and 50 postexposure trials, producing 350 trials in total. In the preexposure phase there was no delay between the vertical movement of the left hand and the production of a vertical force on the object in the right hand. In the exposure phase a delay of 250 ms was introduced between the upward movement of the left hand and the production of the vertical force on the right hand object. This was chosen to be long enough to resolve the two hypothesized grip force peaks. For the postexposure phase, this delay was removed so that these trials were identical to those of the preexposure phase. After the first 20 trials, one catch trial was interspersed pseudorandomly in every 10 trials. In a catch trial, movement of the left hand did not cause a vertical force on the right hand object.
Cued condition
In the cued condition subjects held the object in the left hand above the object in the right as in the self-generated and noncued condition (Fig. 2D). As in the noncued condition the left hand was stationary throughout. On average once every 2.5 s a tone sounded, and 300 ms later an upward pulse of force was generated by the right hand robot on the object held in the right hand. This 300-ms period between the tone and the upward pulse of force was chosen because it is greater than the reaction time to a tone. The temporal profile of this force pulse was identical to that of the noncued condition.
The paradigm comprised of three phases of 50 preexposure, 250 exposure, and 50 postexposure trials: 350 trials in total. In the preexposure phase the vertical force on the right hand always occurred 300 ms after the tone. In the exposure phase a delay of 250 ms was introduced so that the force now occurred 550 ms after the tone. The additional delay of 250 ms between preexposure and exposure phases was the same for the cued and self-generated conditions. Within the final postexposure phase, this delay was removed so that these trials were identical to those of the preexposure phase. Catch trials, in which the tone sounded but no vertical force on the right object was produced were interspersed as in the self-generated condition.
Analysis
For each trial, the position of the left hand, the load (total force tangential to the surface) and grip force were recorded at 250 Hz and then filtered using a Butterworth 5th-order, zero phase lag, low-pass filter with a 10-Hz cutoff.
As a global measure of anticipatory grip force learning, zero lag cross-correlations (r) were performed between grip and load force for each trial. The square of this cross-correlation (r2, the coefficient of determination) was used to quantify the co-variation of the grip and load force. To further quantify the evolution of the grip force response, the amplitude and time, relative to the peak in load force, of the first and second (if present) grip force peaks were found for each trial. In addition, the magnitudes of the grip at 0 and 250 ms (the time of peak load force in different phases of the experiment) were calculated. These measures identify the most important features of the evolving grip force profile.
These dependent measures were analyzed separately for the catch and noncatch trials using repeated measures ANOVAs (with Bonferroni corrections). First, the subjects' average of these measures for the 50 preexposure trials were analyzed as a function of condition (self-generated, cued, and noncued). To examine the changes over the experiment, a repeated measures MANOVA was performed for the dependent measures as a function of condition (self-generated and cued) and time as factors. Time was split into four levels: pre (1st 50 trials before the delay was introduced), early exposure (1st 20 exposure trials as adaptation was fast), late exposure (last 50 exposure trials), and post (50 postexposure trials). As the first peak disappeared by the end of the exposure phase, we examined its disappearance by performing linear regression over the exposure period. All the statistics are based on an analysis on a trial-by-trial basis. For clarity, the plots shown are averages of these trial-by-trial values, averaged across batches of 10 trials and over subjects.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Subjects found the task easy to perform and were able to generate consistent levels of load force. For the self-generated condition, subjects produced bell-shaped movement as required throughout all phases of the experiment (Fig. 3).
|
Preexposure
When the load force was noncued, a reactive grip-force modulation was seen with an average grip-load r2 of 0.29. The peak grip force lagged the peak load by an average of 73 ms (Fig. 4A). In contrast, when the load force was self-generated, a predictive pattern of grip force was seen with a grip-load r2 of 0.73 (Fig. 4B). In this condition, the peak grip force lagged only 26 ms (Fig. 4B) behind the peak load, a significant decrease compared with the noncued condition (P < 0.05). The cued condition showed a grip-force response that was not significantly different from the noncued condition with a grip-load r2 of 0.30 and a peak grip that lagged the peak load by 69 ms (Fig. 4C). Therefore anticipatory grip force modulation was only seen when the load force was self-generated by movement of the other hand.
|
The predictive nature of the grip force modulation was assessed by
examining the modulation during catch trials in which there was no load
force generated by the robot on the object held in the right hand. Any
grip-force modulation therefore is purely predictive because no sensory
stimulation of the fingers is generated in the absence of the load.
Within the self-generated condition, grip force modulation was seen
during the catch trials (Fig. 4B, - - -). This
predictive component had a peak that was significantly (P < 0.05) earlier at 1 ms compared with the 26-ms
lag of the noncatch trials. The peak of grip force in the catch trial
was also significantly smaller than in the noncatch trials
(P < 0.05). This suggests that sensory feedback of
load acts to delay and amplify the peak grip-force response. During the
catch trials in the cued condition, the grip-force modulation was
erratic and of low amplitude and was not consistent with a predictive strategy.
Self-generated condition
In the preexposure phase of the self-generated condition, peak grip force lagged an average of 26 ms behind the load force (Fig. 5A). On the introduction of a 250-ms delay between the motion of the left hand and the load force generated on the object held in the right, two peaks in grip force modulation were seen (Fig. 5B). As expected the first peak corresponded to the grip force response seen in the preexposure catch trials (because the 1st part of the trial is equivalent to a preexposure catch-trial). The second peak initially lagged an average of 59 ms (average over the subjects' 1st exposure trial) behind the peak load force. This represents a reactive grip-force modulation to the unexpectedly delayed load force. Catch trials in the early exposure trials showed a single peak at around 0 ms.
|
As exposure to the novel temporal arrangement continued, the amplitude
of the first peak slowly decayed, eventually becoming the nondominant
modulation (Fig. 5, C and D). By mid-exposure, modulation began to
appear close to the time of actual load force, with this eventually
becoming the predominant grip force peak (Fig. 5C). Late in
the exposure, the catch trials produced a single grip force peak,
predictive of the delayed load force (Fig. 5D). Removal of
the delay in the postexposure phase resulted in grip force modulation
quickly returning to the preexposure response (Fig. 5, E and
F). These features can also be seen in the individual subject's traces shown in Fig. 6,
A-C () for the preexposure, early exposure, and late
exposure phases.
|
These changes and subsequent learning of the new temporal delay are reflected in the grip-load r2 (Fig. 7A). On the introduction of the delay, r2 fell from 0.73 to 0.33 (average of 1st 10 exposure trials). Over the exposure phase, r2 increased significantly (P < 0.05), rising to 0.58. With the removal of the temporal delay, r2 transiently dropped to 0.51 (1st 10 postexposure trials) before returning to a value of 0.72 (last 40 postexposure trials) similar to the preexposure (0.73). These changes reflect that there is learning taking place over the exposure phase.
|
To characterize how this learning occurs, we examined magnitude and timing of the first and second peaks in the the grip force response. In addition, the magnitudes of the grip at 0 and 250 ms, the time of the peak load force during the preexposure and exposure phases, respectively, were examined. These measures identify the most important features of the evolving grip force profile. During the exposure phase the amplitude of the grip force modulation at the first peak decreased significantly (P < 0.05, Fig. 8A) as did the magnitude at 0 ms. However, the time of the first peak did not change significantly (not shown). The height of the second grip peak decreased by 16% over the exposure phase. The time of this second peak (Fig. 8C) shifted significantly toward the time of maximum load force (P < 0.01) so that over the last 50 trials of the exposure period lagged an average of 28 ms behind the load peak as in the preexposure phase. This represents a 37% decrease in the average lag of the second peak compared with the first 20 trials of the exposure phase. This temporal advance coupled with a decreasing amplitude kept the grip force constant at the peak of the load force (250 ms), while allowing the total force exerted to decrease. Thus the grip force at 250 ms did not change significantly during the exposure period. However, when a second discernible peak appeared in the catch trials, its time did not change significantly (Fig. 8C). Over the exposure phase this peak in the catch trials grip was on average 2 ms in advance of the time of peak load. In summary, although the first grip peak only decreased in amplitude, without a change in timing, the second grip peak systematically decreased in both amplitude and lag, consistent with a change from a reactive to a predictive grip force modulation.
|
After the removal of the delay between the motion of the left hand and the load force generated on the right, grip force modulation quickly reverted to being comparable to that grip force modulation during the preexposure phase. The peak of the grip force lagged the load force by 35 ms, a value not significantly different from the 26-ms lag in peak grip force modulation within the preexposure phase.
Cued condition
In the preexposure phase of the cued condition, peak grip force lagged an average of 70 ms behind the load force (Fig. 9A). On the introduction of an additional 250-ms delay between the tone and the load force generated on the object held in the right hand, the grip force immediately shifted to lag the now delayed load force by 81 ms (average of subjects' 1st exposure trial, Fig. 9B). This lag did not significantly decrease over the exposure batch, (Fig. 9, B-D). Removal of the delay in the postexposure phase resulted in grip force modulation returning to the preexposure response (Fig. 9, E and F). The lack of anticipatory behavior is reflected in the low and unchanging grip-load r2 of around 0.32 (Fig. 7B).
|
To characterize the effects of the delay on grip force modulation, the cued condition was analyzed in the same way as the self-generated condition (Fig. 7B) and (Fig. 8, D-F). In general, these plots, which are shown for completeness, showed no systematic changes. There was no significant change in either the amplitude or timing of the grip force modulation during the exposure phase. In particular, the second peak did not become earlier over this period. Although there were discernible peaks in the catch trials, they occurred at random times with a standard deviation in the exposure phase of 88 ms, compared with a standard deviation of 49 ms for the self-generated condition (compare the exposure catch trials in Fig. 8, C and F) and had very small amplitude compared with the noncatch trials (compare the exposure catch and noncatch trials in Fig. 8, B and E). For all phases of the cued condition, the grip force showed a purely reactive modulation.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In this experiment we compared the grip force responses seen when
an object held in a precision grip is pulled on either by the action of
the other hand (self) or externally by a robot unpredictably (noncued)
or predictably (cued by a tone). Anticipatory grip force modulation was
only seen when the load force was self-generated, grip force lagged
load by only 26 ms, consistent with the previous finding of an
anticipatory grip force response when a perturbation is self-generated
(Johansson and Westling 1988). In contrast, in both the
cued and noncued conditions grip force modulation was solely reactive
in response to the imposed load force, with a grip force lag of 69 and
73 ms, respectively. This lag in grip force modulation is consistent
with those previously measured for an externally generated load
(Flanagan and Wing 1993
; Johansson et al.
1992
; Johansson and Westling 1988
). When a
temporal delay was artificially introduced between the pull exerted by
one hand and the consequential production of the load force on the
object held in the other hand (self), a grip force profile with two
peaks was initially seen. This comprised an early grip force peak,
appropriately timed for a nondelayed load, followed by a grip force
reactive to the now delayed load force. Over the course of 250 trials, the grip force became anticipatory while the early, and now
inappropriate, grip force peak decreased in amplitude. Conversely, when
a temporal delay was introduced between the tone and the load force
produced by the robot pull (cued), the grip force remained purely
reactive in response to the load force.
Performance in the absence of a delay
Here it is argued that the crucial difference between the self-generated and cued conditions in the present study is that in the self-generated condition the CNS uses the motor command, in conjunction with internal models of both the arm and the object, to anticipate the resulting load force and thereby adjust grip force appropriately. In both the cued and noncued conditions the load force is not self-produced, therefore this strategy cannot be employed. Whereas in the noncued condition, anticipation is impossible; anticipatory responses could have been seen in the cued condition as the load followed the tone at a predictable time. The lack of anticipatory responses to the tone suggests that this purely sensory cue or context is insufficient to allow prediction of the load force.
These results are in agreement with previous studies in which the
position of a subject's arm was measured in an unloading task
(Dufosse et al. 1985; Paulignan et al.
1989
). When the load supported by a subject on their hand was
removed, a differential effect occurred dependent on the nature of the
unloading. When the load was removed with the contralateral hand, there
was associated anticipatory control of the position of the load-bearing
forearm that acted to minimize the effects of the perturbation.
However, no corresponding anticipatory action was found when unloading was performed externally, even if it was preceded by a warning tone, or
triggered by the subject themselves. Duffose et al.
(1985)
concluded that temporal cues regarding the precise time
that the unloading occurred is, by itself, inadequate information for anticipation.
Blakemore et al. (1998) have shown that efference copy
and sensory feedback must be consistent with a specific context to see
predictive modulation. In their experiment predictive grip force
modulation was present when the feedback experienced by two hands was
consistent with an internal model of a single object. Here we have
shown that an internal model can be learned in which the consequences
of a motor action are delayed, but cannot be learned for a sensorimotor
context in which a tone determines the consequence, within the time
scale of this experiment. Although the action of one hand often
influences objects held in the other it is less clear that tones, in
everyday life, influence objects held in the hand. Therefore it may be
that internal models for grip force modulation can adapt to new
dynamics relating the motion of one hand to the other, but arbitrary
pairings such as tone with load force (cued condition) cannot be
incorporated into a new internal model.
In the present study, examination of grip force responses during catch trials, in which there was no load force generated on the right hand object in any of the conditions, further elucidates the differences between the cued and self-generated conditions. Because there is no sensory stimulation of the finger tips in these trials, any response must be a purely predictive one. Differences in these trials between the conditions can therefore be associated with corresponding differences in prediction. Within the cued condition, there was no consistent grip force modulation during the catch trials. This finding is consistent with the grip force modulation being solely reactive to the imposed load force. However, in the self-generated condition, grip force modulation was seen during the catch trials (Fig. 4B). The peak of this grip force modulation, on average, led the peak load force by 1 ms. The presence of this accurately timed modulation, in the absence of sensory stimulation, demonstrates that in this self-generated condition at least part of the grip force response is anticipatory in nature.
Differences between the grip force response in the catch and noncatch trials in the same condition are attributable to the consequences of sensory stimulation of the finger tips and hand, which is only present in the noncatch trials. In the cued condition, sensory stimulation in these noncatch trials resulted in a reactive grip force peak. However, in the catch trials in this same condition, no grip force modulation is seen. In the self-generated condition, however, the effect of sensory feedback was twofold. First, the peak of the grip force was now delayed compared with the catch trials and occurred on average 26 ms after the peak load force. Second, the amplitude of the grip force modulation was amplified over that of the catch trials by ~13% (Fig. 4B). The differences between the grip force modulation in the catch and noncatch trials in this self-generated condition can be viewed in two ways. First, the grip force seen in the noncatch trials could be made up of a mixture of predictive and reactive processes. This would indicate that the grip force modulation seen in the catch trials is only the anticipatory component of the grip force response. An alternative view is that the grip force modulation seen in the noncatch trials represents a purely anticipatory response that is suppressed if there is no confirmatory sensory feedback signaled by load on the finger pads. For example, if the expected load force is not experienced, as in the catch trials, then the prediction is judged inaccurate and further anticipatory grip force is curtailed leading to a smaller and earlier peak in grip force modulation as observed. Such a view is consistent with the multiple internal model approach (see Modularity and internal models).
Performance in the presence of a delay
During the exposure phase of both the self-generated and the cued conditions, a temporal delay of 250 ms was added to the expected time of the load on the right hand. When this delay was added to the cued condition, the time between the tone and the production of the load increased, but the peak of the grip force lagged the now delayed load force during exposure by an average of 74 ms (Fig. 8F). As in the preexposure phase of this condition, this is consistent with grip force modulation in the cued condition being purely reactive to load force.
When this delay was added to the self-generated condition, the pull of the left hand did not immediately produce a load on the right hand, which would be consistent with the expected forces for a solid object held between the hands. Instead the load was delayed by 250 ms. Initially, two peaks in grip force were seen in the noncatch trials (Fig. 5B). This conforms to what was expected (Fig. 1B). The first peak appeared at the time of the anticipatory grip force of the preexposure phase, in which there was no delay between the pull of one hand and the load on the other. The second peak is identified as a reactive peak, to the now delayed load force. In these first exposure trials, this peak lagged the load force by 59 ms.
The evolution of the double-peaked grip force profile over the course of the exposure phase allows us to select between the two possible hypotheses described in the INTRODUCTION. These two hypotheses both propose that the second peak will become predictive, but differ as to its origin. The first hypothesis suggests that this peak could be caused by the first grip force peak moving to the time of the peak load force, with the later reactive peak decaying, consistent with adaptation of the existing internal model (Fig. 1C, Migration of peak 1). The second hypothesis is that a new predictive peak to the delayed load force could develop, implying the development of a new internal model, appropriate for the novel situation. This would be reflected by a migration of the second grip force peak caused by the new predictive component becoming increasingly predominant, coupled with a decrease in the reactive component of the response. (Fig. 1C, Migration of peak 2). Analysis of the changing amplitude and time of the peaks over the 250 exposure trials showed that the second peak significantly shifted toward the time of peak load force (Fig. 8C), while the first peak remained at its original time and decreased in amplitude (Fig. 8A), thereby supporting the latter hypothesis. This hypothesis proposes that the second grip force peak is formed from two components. The first is a reactive component that decreases in amplitude with learning. The second component is a new predictive component that can be seen in isolation in the catch trials. When this predictive grip force peak is seen in the catch trials, it has almost no lag with respect to the load force. As the predictive modulation grows and the reactive component declines, the peak of the total grip force migrates to a predictive location.
Modularity and internal models
To produce an appropriate grip force response in one hand, in
response to the motion of the other hand, it has been suggested that a
forward internal model is used. A forward internal model (Jordan
1995; Jordan and Rumelhart 1992
; Kawato
et al. 1987
; Miall and Wolpert 1996
;
Wolpert 1997
; Wolpert et al. 1995
)
captures the forward or causal relationship between actions, as
signaled by efference copy (Jeannerod et al. 1979
;
Sperry 1950
; von Holst 1954
), and
predicts outcomes such as load force. Based on this prediction an
appropriate anticipatory grip force can be generated despite sensory
feedback delays associated with the detection of load force by the
fingertips (Johansson and Westling 1984
).
Forward models depend on the dynamics of the limb as well as the properties of external objects. As the dynamics of the limb change throughout life and many new objects are experienced, forward models must be adaptable. The training signals required to update such a model are readily available, the difference between predicted and actual outcome. In the present study we have shown how the CNS adjusts for a system in which a delay of 250 ms is introduced between the action of one hand and its effect on an object held in the other.
We can consider two alternative ways in which this could be achieved.
The first assumes a single internal forward model that adapts to the
new delay. Such an internal model would have to readapt to every
object, even when the object has been previously experienced, leading
to transient errors in grip force production. However, it is known that
when handling multiple objects anticipatory grip force modulation is
seen even on the first trial (Gordon et al. 1993).
Alternatively, a modular approach can be used. We have proposed such a
modular system for object manipulation in which multiple internal
forward models coexist (Blakemore et al. 1998
;
Wolpert and Kawato 1998
).
A specific modular architecture known as the multiple paired
forward-inverse model (Wolpert and Kawato 1998;
Wolpert et al. 1998
) has been proposed for motor
learning and control. The switching process between modules is
determined by two distinct processes. The first uses sensory contextual
cues to determine which module to use before movement initiation. For
grip force, contextual cues include object weight and shape as judged
visually (Jenmalm and Johansson 1997
; Johansson
and Westling 1988
), experience from previous lifts
(Gordon et al. 1993
), and perceived friction from the
objects' surface (Johansson and Westling 1984
,
1987
). Once the movement is initiated the second process
uses the forward model's predictions. As each forward model captures a
distinct dynamic behavior of the limb or object, their prediction
errors can be used during movement to determine in which context the motor system is acting. The signal derived from the forward models can
be used to adjust the prior selection of the modules that occurs before
the movement is initiated and sensory feedback is available. For
example, for an object that appears heavy, prior selection will predict
that a large grip force is needed to lift the object. However, feedback
processes, based on comparing the predicted (from the forward model)
with the actual consequences of action (such as hand acceleration) can
indicate that the object is in fact light thereby using a module
appropriate for a lower grip force. These feedback processes are used
to adjust the predictive feed-forward control and are separate from the
feedback control processes, which can provide reactive responses.
We propose that multiple forward models are in operation in grip force modulation. The changes during the exposure phase of the self-generated condition were consistent with two internal forward models coexisting. The first is a model suitable for handling a real physical object in which there is no delay between the action on it and its response, and we would expect this model to have been acquired early in life by the subject. However, subjects would have to learn a new model for the condition in which a delay is introduced between action and effect, because this is likely to be the first time subjects have experienced such an object. The decrease in the first grip force peak over the course of learning is consistent with the internal model appropriate for no delay being slowly switched off. Similarly, the decrease in lag of the second peaks is consistent with a new model being learned appropriate for the 250-ms delay. The multiple model predicts the pattern of changes seen in the second hypothesis described in the INTRODUCTION, the formation a new internal model allowing predictive grip force modulation appropriate for the novel situation, with both the first peak and the later reactive peak decreasing in amplitude as the predictive response grows (Fig. 1C, Migration of peak 2). This pattern of change was observed. Furthermore, the differences seen between the catch and the noncatch trials can be attributed to the feedback process, which is a comparison of the predicted (from the forward model) and actual sensory feedback curtailing the feed-forward response due to the absence of the predicted sensory feedback. The rapid change back to the preexposure pattern in the postexposure phase may reflect a fast switching process back to a previously learned module.
In summary, these results suggest that the internal model is able to learn new temporal relationships between actions and their consequences, and such learning may involve generating a novel response rather than altering the original predictive response.
![]() |
FOOTNOTES |
---|
Address for reprint requests: D. M. Wolpert, Sobell Department of Neurophysiology, Institute of Neurology, University College London, Queen Square, London WC1N 3BG, UK.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
Received 29 December 1999; accepted in final form 1 June 1999.
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|