1Department of Biomedical Engineering, Marquette University, Milwaukee, Wisconsin 53201; and 2Sensory Motor Performance Program, Rehabilitation Institute of Chicago, Chicago, Illinois 60611
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Scheidt, Robert A., Jonathan B. Dingwell, and Ferdinando A. Mussa-Ivaldi. Learning to Move Amid Uncertainty. J. Neurophysiol. 86: 971-985, 2001. We studied how subjects learned to make movements against unpredictable perturbations. Twelve healthy human subjects made goal-directed reaching movements in the horizontal plane while holding the handle of a two-joint robotic manipulator. The robot generated viscous force fields that perturbed the limb perpendicular to the desired direction of movement. The amplitude (but not the direction) of the viscous field varied randomly from trial to trial. Systems identification techniques were employed to characterize how subjects adapted to these random perturbations. Subject performance was quantified primarily using the peak deviation from a straight-line hand path. Subjects adapted their arm movements to the sequence of random force-field amplitudes. This adaptive response compensated for the approximate mean from the random sequence of perturbations and did not depend on the statistical distribution of that sequence. Subjects did not adapt by directly counteracting the mean field strength itself on each trial but rather by using information about perturbations and movement errors from a limited number of previous trials to adjust motor commands on subsequent trials. This strategy permitted subjects to achieve near-optimal performance (defined as minimizing movement errors in a least-squares sense) while maintaining computational efficiency. A simple model using information about movement errors and perturbation amplitudes from a single previous trial predicted subject performance in stochastic environments with a high degree of fidelity and further predicted key performance features observed in nonstochastic environments. This suggests that the neural structures modified during motor adaptation require only short-term memory. Explicit representations regarding movements made more than a few trials in the past are not used in generating optimal motor responses on any given trial.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
A remarkable and
well-studied ability of the human brain is that of adapting the
execution of limb movements to physical changes in operating conditions
such as those that naturally occur during growth, aging, and exposure
to altered mechanical environments (Bock 1990;
Conditt et al. 1997a
; Dizio and Lackner
1995
; Goodbody and Wolpert 1998
; Happee
1993
; Lackner and Dizio 1994
; Scheidt and
Rymer 2000
; Shadmehr and Mussa-Ivaldi 1994
;
Thoroughman and Shadmehr 1999
). This process is known as
motor adaptation. Motor adaptation is a form of learning that evolves
over a series of movements whereby some original performance of a given
task is restored in the presence of an external perturbation. This
ability to adapt to environmental changes has played an important role in human survival. A species unable to compensate for prevailing winds
or the refraction of light through water would be ill suited to use the
basic tools (such as spears and nets) necessary for fighting off foes
and obtaining food. In such instances, environmental perturbations
influence the control of upper limb movement in an unpredictable way.
A number of studies have investigated the processes involved in motor
adaptation by exposing subjects to specific perturbations and
quantifying the changes in their responses over time. For example, some
experiments have explored the changes in reaching and pointing
movements of the hand induced by displacements or deformations of the
visual field (Flanagan and Rao 1995; Held and
Freedman 1963
; Helmholtz 1925
; Wolpert et
al. 1995
). Other experiments have perturbed the moving arm with
mechanical disturbances that emulated the effects of inertial loads
and/or viscoelastic media (Bock 1990
; Lackner and
Dizio 1994
; Shadmehr and Brashers-Krug 1997
;
Shadmehr and Mussa-Ivaldi 1994
). Each of these studies
employed perturbations with fixed and repeatable structures. For
example, Shadmehr and Mussa-Ivaldi (1994)
used a robotic
device to apply mechanical forces to the hand. These forces had a fixed
linear dependence on the speed of the subject's hand.
However, the perturbations that people encounter in everyday life do not always have a repeatable and consistent structure. Consider, for example, a worker whose job might be to sort packages of varying size and weight into bins, bags, or slots. Each of these packages will have different inertial properties and will impose different loads on the arm as it moves toward the desired target position. If the worker carries out this task for a prolonged time, is it reasonable to expect some adaptation to take place? In this case, the perturbations are not fixed but vary from object to object and follow a given statistical distribution depending both on the object properties and on the sequence of movements in the task. Can the motor system adapt to a variable environment? And if so, how is this adaptation accomplished? Does the motor system use information it acquires on a trial-by-trial basis, or does it attempt to extract some definable statistical property about the perturbations it encounters, such as the mean or the most likely (i.e., the mode) perturbation? Can subject behavior in a stochastic environment reveal how the neural mechanisms involved in motor adaptation use information from previous trials to modify motor commands on subsequent trials? These questions were addressed in a set of experiments that employed engineering methods of systems identification and a robotic system to generate sequences of perturbing force fields having magnitudes that varied randomly from trial to trial.
In the present experiments, adaptation was examined in the context of
goal-directed reaching movements. Twelve subjects executed reaching
movements between two targets in the horizontal plane while holding the
handle of a two-joint robotic manipulator. The robot applied perturbing
forces to the arm during each movement. The amplitude (but not the
direction) of the perturbing force field varied randomly from trial to
trial. Each subject's motor response to the sequence of perturbing
fields was quantified using the peak deviation from a straight-line
hand path. The trial-to-trial sequences of motor errors were analyzed,
and the results demonstrated that subjects did adapt their motor
behavior in response to the random sequences of force fields presented
at the hand. Furthermore subjects compensated for the approximate mean
field of the stochastic sequence. This behavior did not depend on
specific distribution properties of the sequence. Finally, subjects
accomplished this adaptation by using memories of the most recent
perturbations and the most recent performances only. Adaptation was not
accomplished by directly counteracting the mean field strength on each
individual trial. The present findings are consistent with recent
experiments that suggested a prominent function of prefrontal cortex in
the early stages of motor adaptation to perturbing fields
(Shadmehr and Holcomb 1997).
![]() |
METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Twelve human subjects with no known neuromotor disorders
consented to participate in this study. Subjects executed half-second, 20-cm reaching movements with their dominant arm in the horizontal plane while holding the handle of a two-joint, robotic manipulator (Fig. 1A). The robot was
comprised of a five-bar linkage with torque motors controlled by a
dedicated PC (Scheidt et al. 2000). Subjects were
instructed to "reach from the beginning target to the ending target
in one half second." The computer provided qualitative feedback of
movement duration after each trial (either too fast: <0.45 s, too
slow: >0.55 s, or just right: 0.45-0.55 s). Subjects were instructed
to relax after each movement while the manipulandum moved the hand
slowly back to the beginning target. This protocol was designed for
allowing subjects to experience the limb's mechanical environment
along a limited set of trajectories. Reaching movements were directed
away from the subject's body along a line (the positive y
axis) passing through the center of rotation of the shoulder. The
subjects' arms were supported against gravity by a sling attached to
the 8-ft ceiling. The support was adjusted so that the upper arm was
abducted by 90°. The shoulders were restrained using a Velcro torso
support. "Beginning" and "end" targets corresponding to a 20-cm
reach in the plane of the arm were presented on a computer monitor just
above the manipulandum. The position of the hand was displayed as a
small cursor on the overhead monitor. Subjects could see both their arm
and the cursor representing it at all times.
|
The robotic manipulator applied perturbing force fields to the arm
during each movement. A perpendicular viscous field was designed to deflect the hand perpendicularly from its intended path
with a force proportional to hand velocity along its path (Fig.
1B). The forces applied to the subject's hand during the ith movement were defined
![]() |
(1) |
Two stochastic perturbation sequences were used. In experiment
1, four subjects were presented with a sequence of 200 trials in
which the force-field gain, Bi (Fig.
2A), followed a Gaussian distribution (Fig. 2B). This distribution had a nonzero mean
corresponding to information about the perturbation sequence that
subjects might learn. The mean perturbation amplitude was 15.2 Ns/m
with a variance of 24.7 Ns/m. This sequence was designed to ensure
insignificant correlation between perturbation magnitudes on
consecutive trials separated by more than 40 trials (Fig.
2C). The significance of each correlation term was evaluated
by comparing the correlation magnitude at each integer lag value to an
estimate of the 95% confidence interval bounding zero correlation
(2
2/
N) (Box et al. 1994
). All four
subjects were exposed to the same sequence of perturbations. In
experiment 2, eight subjects were presented with a sequence
of 400 trials with a bimodal probability density function (Fig. 2,
D and E). This bimodal sequence was constructed by shuffling together two unimodal sequences with individual Gaussian distributions having means of 6 Ns/m (175 trials) and 25 Ns/m (225 trials), respectively. While each individual subpopulation contained no
significant correlations between perturbations separated by as many as
40 trials, the shuffling process used to construct the bimodal
population gave rise to spurious correlations at trial lags <40 trials
(Fig. 2F). All eight subjects were exposed to the same
sequence of perturbations. This bimodal stochastic perturbation sequence had greatly differing mean (15.5 Ns/m) and mode (25 Ns/m) values and was constructed to distinguish whether subjects adapt more
closely to the mean or the mode of a given perturbation sequence or
whether adaptation gets "trapped" in the smaller, local maximum designed into the bimodal probability distribution function.
|
Data analysis
Simple measures of kinematic and kinetic behavior were used to
assess subject motor performance on each trial during this goal-directed reaching task. "Movement error" was defined as the peak deviation of the hand from a straight-line trajectory passing between the initial and final targets (Krakauer et al.
1999). Movement error was used to quantify kinematic
performance, assuming that subjects intended to make straight-line
movements of their hands. This measure of motor performance has
previously been found to motivate motor adaptation during reaching
(Scheidt et al. 2000
). The peak hand force that was
generated perpendicular to the direction of movement quantified
dynamic performance.
An exponential function was fitted to the trial series of movement
errors to characterize the rate at which subjects compensated for the
random sequence of perturbation gains. This model had three free
parameters: gain, A, time-constant, , and
offset, C
![]() |
(2) |
A regression analysis of movement error versus perturbation amplitude was performed to determine the field strength (i.e., perturbation gain) that subjects adapted to. The strength of correlation between these two variables and the linearity of this relationship were also evaluated. The amplitude of the field strength to which subjects adapted was estimated from the zero crossing of the resulting regression line since the perturbation gain value at which the regression line passed through zero error indicated the field strength at which subjects would exhibit error-free (straight line) trajectories. This analysis, however, provides no explanation for how subjects adapted to this particular field strength. Subjects could adapt by directly counteracting this "zero error" perturbation magnitude itself on each and every trial; i.e., by executing a control strategy that anticipated the same constant field trial after trial. If so, movement errors would vary linearly with perturbation strength. Alternatively, subjects could employ a continuously evolving strategy of using information about perturbations and movement errors from a limited number of previous trials to adjust performance on subsequent trials. Because such a strategy could also result in a linear relationship between movement error and perturbation strength, the regression analysis described in the preceding text could not distinguish these two possibilities.
The preceding regression analysis was extended to evaluate the dependence of movement errors on previous perturbations and previous errors using autocorrelation and cross-correlation analyses. Specifically, the autocorrelation profile of the movement error trial sequence and the cross-correlation between the error and perturbation gain trial sequences were calculated. If subjects anticipate a constant field strength when exposed to an uncorrelated sequence of perturbations (e.g., the mean field strength), then their performance on each trial must also be uncorrelated with that of previous trials. This hypothesis was directly tested by this analysis.
These correlation analysis results were then used to guide construction
of a model of motor adaptation during reaching. Specifically, movement
error on each trial was modeled as a linear combination of previous
movement errors as well as present and previous perturbation amplitudes. The result was a parametric model of motor adaptation that
was linear in its inputs
![]() |
(3) |
The capacity of this model to predict subjects' adaptation to the
random sequence of perturbations was evaluated. These predictions were
compared with the predictions of two alternate, but viable, learning
algorithms. The first alternative model accumulated an explicit
representation of the mean perturbation strength by "memorizing" the perturbation sequence trial by trial. This explicit representation of the running-average mean perturbation was used to compensate for the
perturbation on the next trial. The second alternative model was an
incremental learning algorithm that utilized local weighting of the
most recent movement errors to predict and compensate for the magnitude
of the next perturbation. This model included the possibility of
nonuniform and nonlinear attention models whereby learning could either
attend closely to or ignore trials where the perturbation amplitude was
"surprising" or "irrelevant" (Atkeson et al.
1997). Each model was first fit from the subjects' data from
the initial 100 movements in the experiments. The performance of each
model was then evaluated according to its ability to predict subject
movement errors on the last 100 movements. Model performance was
quantified using the variance accounted for (VAF) as a measure of
goodness-of-fit
![]() |
(4) |
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Subjects compensate for the approximate mean of the random trial sequence of perturbations
An overhead view of averaged hand paths made during
experiment 1 (Fig.
3A; unimodal perturbation
sequence) shows that subjects exhibited substantial kinematic
deviations to both the left and right even though they experienced
forces that pushed only to the left. To compare across trials,
hand-path data were aligned with respect to the onset of movement (the
point in time when hand speed first exceeded 0.1 m/s; Fig.
3B) and averaged into six "bins" of 5 Ns/m width each
(0-5, 5-10, 10-15, etc.). Movements from trials with field strengths
>20 Ns/m resulted in trajectories that deviated markedly to the left
(i.e., in the direction of the applied force). However, hand-path
deviations were consistently toward the right (i.e., opposite to the
direction of the applied force) for fields with gains <10 Ns/m.
Movements made in weaker fields had hand-path errors that were
approximately mirror symmetric to those made in stronger fields.
Kinematic errors made in the weakest fields were nearly identical to
responses observed when perturbing force fields were unexpectedly
removed after adaptation (Scheidt et al. 2000). This
finding is consistent with traditional measures of aftereffects of
adaptation (Shadmehr and Mussa-Ivaldi 1994
). These
aftereffects are a clear indication that subjects compensated for the
perturbations by adopting some automatic and predictive mechanism.
Force fields roughly corresponding to both the mean (average) and mode
(most likely) disturbance (10-15 Ns/m) resulted in movements with the
least curvature. Note that these movements were only approximately
straight, corresponding to the steady-state bias in movement error
(constant C in Eq. 2).
|
Average hand speed profiles (Fig. 3B) typically demonstrated
two distinct peaks. The secondary peak in the hand speed profile could
be the result of several mechanisms including, but not limited to,
active correction under visual feedback, reflex-mediated adjustments due to the mismatch between the intended and actual final joint posture
(the mismatch being due to the perturbation) or the interaction of limb
and manipulandum dynamics (Shadmehr and Mussa-Ivaldi
1994). The present experiment was not designed to distinguish
between these alternatives. Therefore movements were truncated at the point of time near the end of movement where the hand speed profile reached a transient minimum (vertical line in Fig. 3B, solid
lines in Fig. 3A returning to final target location). This
limited subsequent analysis to the portion of movement that was
predominantly feedforward.
An exponential function (Eq. 2) was fit to the
movement error trial series (dark solid line in Fig. 3C),
confirming the presence of a steady-state bias in movement error.
Figure 3C shows a rapid decrease in movement error within
the first 10-20 trials (time constant = 2.4 trials). Time
constants ( in Eq. 2) for all four subjects averaged
3.2 ± 0.74 trials (mean ± SE mean). The residual steady-state movement error (constant C in Eq. 2)
was observed in all four subjects (average 12.3 ± 2.7 mm),
indicating that subjects compensated only approximately for the mean of
the random trial sequence. These observations were consistent across
all four subjects exposed to the unimodal perturbation sequence.
Subjects did indeed adapt in response to the random sequence of perturbations.
Profiles of the hand forces generated perpendicular to the direction of movement (Fig. 3D) provide further evidence of adaptation to the stochastic sequence of perturbations. Subject-generated forces dominated movements made in the weakest fields (the smallest force profile with biphasic shape), whereas robot-generated forces dominated movements made in the strongest fields (the largest profile with monophasic shape). The initial peak in perpendicular force generated by the subject in the weakest field (~12 N) was directed opposite to the forces imposed by the robot and was not necessary to move the hand toward the target. This excessive force caused the limb to deviate substantially from the target, producing a kinetic aftereffect of adaptation. Consequently, restoring forces (the negatively directed peak in Fig. 3E) were required to move the limb to the final target.
An analysis of movement error versus perturbation amplitude (Fig. 3,
E and F) indicated that these two variables were
well fit by a linear relationship within the range of our experiment (r = 0.82 in Fig. 3E; 0.73 < r < 0.84 for all 4 subjects). The point of zero error
on these regression lines indicates the field strength that was best
compensated for through the adaptive process (13.5 Nm/s in Fig.
3E; 12.9 ± 1.2 Ns/m for all 4 subjects in Fig. 3F). This adapted field strength approximated, but did not
quite attain, the mean value of the distribution (P < 0.01; Student's t-test rejecting the null hypothesis
H0:
Badapt =
Adaptation to the approximate mean, not to the mode
An overhead view of averaged hand paths made by one subject from
experiment 2 (Fig.
4A; bimodal perturbation
sequence) shows that again, movements were either deflected to the left
or to the right. Average hand speed profiles (Fig. 4B)
exhibited the same biphasic pattern found in experiment 1. Consequently, the data from experiment 2 were truncated in
same way as in experiment 1. The truncated and averaged hand
movements (Fig. 4A) exhibited consistent deviations toward
the right when movements were made in fields with strengths of 10
Ns/m and toward the left when movements were made in fields with
strengths of
20 Ns/m. Force fields roughly corresponding to the mean
disturbance (10-15 Ns/m) resulted in trajectories with the least
curvature although they were not ideally straight. These results
demonstrate that adaptation to a sequence of perturbations with
randomly varying magnitudes converges to the approximate mean
perturbation magnitude rather than the most likely magnitude.
|
Fitting an exponential function (Eq. 2) to the movement
error trial series (dark solid line in Fig. 4C) produced
results that varied widely across subjects [time constant = 54.8 ± 18.2 (SE) trials; range = [11.2, 167] trials;
n = 8]. Consequently, this traditional measure of
learning suggests that the bimodal perturbation sequence abolished (or
at least slowed) the initial rapid learning observed in
experiment 1. However, as shown in the following text, subject performance in both experiments can be described using a
single, parsimonious description of motor adaptation.
Figure 4D displays average perpendicular hand force profiles measured at the handle in the bimodal experiment. The overall shape of the profiles was similar to those observed in the unimodal experiment with the initial peak in perpendicular force generated in the weakest fields giving rise to the undesirable deviation from the target indicative of aftereffects of adaptation. Again this is a kinetic aftereffect of adaptation similar to that observed in the first experiment. As in the unimodal experiment, the hand force profiles were smooth and the restoring forces generated at the end of the movement did not appear to be distinct pulses.
As in the unimodal experiment, subjects exposed to the bimodal
perturbation sequence exhibited a linear relationship between movement
error and perturbation strength (r = 0.85 in Fig.
4E; 0.83 < r < 0.94 for all 8 subjects). Again, the point of zero error on these regression lines was
taken as the field strength that was best compensated for through the
adaptive process. These eight subjects adapted to an average field
strength of 11.33 Ns/m with a 95% confidence interval of [8.61,
14.04] Ns/m (Fig. 4F). For all eight subjects, the major
and minor peaks of the bimodal probability density function (6 and 25 Ns/m, respectively) both fell outside this 95% confidence interval.
However, although the adapted field strength in the bimodal sequence
was substantial, subjects did not quite compensate for the mean
perturbing field (
Only recent memories contribute to adaptation
The Gaussian-distributed random trial sequence (Fig. 2,
A and B) was used to perturb subjects while
adapting because this input to the motor adaptation process was both
uncorrelated from trial to trial (up to a lag of 40 trials; Fig.
2C) and "rich" spectrally (Marmarelis and
Marmarelis 1978). Driving each subject's motor system with an
uncorrelated trial sequence ensured that any trial-to-trial
correlations observed in that subject's motor output did not originate
from the perturbation sequence but rather from information processing
within the neuromotor controller. Despite this lack of correlation in
the sequence of perturbing fields, significant trial-to-trial
correlations were observed in subjects' motor output (Fig.
5). Correlations between movement error
and perturbation gain (Fig. 5A) exceeded statistical
significance (>95% CI) not only on concurrent trials (i.e., lag = 0) but also on the preceding trial (lag = +1). The sign of the
correlation at lag 1 was opposite that at lag 0, indicating that
subjects attempted to reduce movement error on each trial by countering the previous perturbation. Significant correlations between movement error and perturbation gain extended back no more than two trials for
all of the four subjects exposed to the perturbation sequence with the
unimodal distribution. Movement errors on a given trial also exhibited
substantial correlations with errors generated in the preceding trial
(Fig. 5B). By definition, the autocorrelation function is
symmetric about 0 lag. Thus correlations at any given lag are reflected
at the corresponding lead with no violation of causality. Significant
autocorrelation terms were found at a lag of one trial for three of the
four subjects (the remaining subject showed no significant correlations
beyond lag 0). Again, the sign of this correlation was negative
indicating that subjects attempted to reduce movement errors on each
trial by countering movement errors generated on the previous trial.
|
Similar correlation analyses were performed on the peak hand forces generated perpendicular to the intended direction of movement (Fig. 5, C and D). Correlations between peak hand force and perturbation gain (Fig. 5C) exceeded statistical significance (>95% CI) only on concurrent trials (i.e., at 0 lag) and at a lag of one trial. Significant correlations between peak hand force and perturbation gain extended back no more than two trials for all four subjects in experiment 1. Correlations between movement error and peak hand force (Fig. 5D) exceeded statistical significance only on concurrent trials and, at a lag of one trial, a result entirely consistent with the findings of Fig. 5A.
The significant correlations at zero lag (Fig. 5, A, C, and D) were due in part to mechanical interaction between the robot and the finite impedance of the subject's arm. Larger forces imposed by the robot on the hand resulted in both larger deviations of the hand from its intended path and in greater forces being recorded at the handle. However, significant correlations at nonzero lags cannot be explained by mechanical interactions. These lag 1 correlations indicate that subjects used explicit information regarding the strength of the perturbation from the previous trial to preprogram the motor response on each subsequent trial. Subjects also utilized information about previous performance to update motor behavior on subsequent trials. The lack of significant correlations beyond two previous trials indicates that explicit memory representation of more remote trials was not used during motor adaptation. Had subjects adapted to the stochastic sequence of perturbations by directly counteracting some constant field strength on each trial (e.g., the mean), their motor output would likewise be uncorrelated with the input. This hypothesis is clearly refuted by the present findings. These findings support the hypothesis that explicit information of only one or two previous trials is sufficient to allow subjects to compensate for the approximate mean field strength in a random sequence of perturbations.
Predicting motor performance
The preceding analyses suggest that subject performance
(quantified by movement error) exhibited on any given trial,
i can be predicted solely from the field strength on that
trial (Bi) and from the field strength
and error exhibited on the previous trial
(Bi-1 and
i-1, respectively)
![]() |
(5) |
|
|
Step response analysis
Although the model performed quite well in response to
the stochastic input sequences from which it was originally derived, it
was also important to determine how well this model could predict behavior exhibited in response to nonstochastic sequences of input perturbations. Equation 5 was used to simulate movement
errors in response to a step increase in perturbation strength that
included a simulated "catch trial" near the end of the input
sequence (Fig. 7A, top). This
input sequence was specifically designed to mimic the constant force
field gains and catch trials used in previous motor adaptation
experiments (e.g., Shadmehr and Mussa-Ivaldi 1994).
Average coefficient values from the unimodal experiment (Table 1) were
used to define the model parameters. When presented with a step
increase in perturbation strength, the simulated movement errors
rapidly approached their asymptotic value (within 3-4 trials) and
exhibited a small steady-state error at large trial numbers as did
subjects in both experiments. The model output also exhibited the
classic behavior of an "after-effect" (Shadmehr and
Mussa-Ivaldi 1994
) when the catch trial was introduced at trial
number 75. Furthermore, this model was also able to account for the
observations of Thoroughman and Shadmehr (2000)
that a
single catch trial can transiently degrade the adapted state generated
in response to a consistent perturbing field. Consequently, this very
simple model of motor adaptation succinctly captures the fundamental behavioral characteristics exhibited in both the present experiment and
in more traditional experiments of motor adaptation.
|
Interpretation of model coefficients
To investigate how each term of the model related to
observed behaviors, the output of the model in response to the same
step input (Fig. 7A) was analyzed for various combinations
of model parameters (Fig. 7B). A model that does not rely on
prior experience, and thus has no memory (i.e.,
a1 = b1 = 0; trace 1), can only respond to
the current perturbation and fails to adapt. This is the response one
would expect if subjects were only co-contracting their limb in
response to the perturbations. Increased co-contraction might decrease
the magnitude of the b0 term, but
unless the limb stiffness became exceedingly large, a substantial
residual offset would remain. The amount of residual steady-state error
in the adaptive response is determined by the relative magnitudes of b0 and
b1. When
b0 = b1 (trace 2), then the residual
steady-state error is eliminated. On the other hand, when the
autoregressive term in the model is removed
(a1 = 0; trace 3), the dynamics
associated with initial exposure to the perturbation are eliminated
while the steady-state error is reduced relative to the full model
(Fig. 7A). If this autoregressive term is instead doubled
(trace 4), the initial transients are extended and the steady-state
error is increased. Changing the sign of
a1 (traces 5 and 6) does not alter the
time course of adaptation but causes the model's response to oscillate
within the envelope defined in Fig. 7, A (bottom) and B (trace 4), respectively. Note, however, that changing
the sign of a1 does in fact reduce (but not
eliminate) the steady-state error. Finally, not all choices of
parameters yield stable learning. Setting
a1 > 1.0 yields an unstable algorithm
that never adapts (trace 7).
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The present experiments investigated the ability of unimpaired humans to adapt to a viscous, perpendicular, force-field environment having force-field gains that were unpredictable (and uncorrelated) from trial to trial. Experiments were designed to determine if subjects adapted to the mean force field gain, the most likely field, or whether adaptation would depend on other features of the perturbation sequence's probability density function. Correlation analyses were performed to determine how much motor performance on any given trial was correlated with performance on previous trials. It was found that 1) subjects adapted their motor behavior in response to the random sequence of force field gains, 2) subjects compensated for the approximate mean field of the stochastic sequence, not the most likely field, and 3) subjects compensated using memories of only the most recent perturbations and the most recent performances.
What did subjects adapt to?
Subjects experienced perturbing forces that were always directed
toward their left. If no adaptation had occurred, then all movement
errors would likewise have been directed toward the left. However, for
field strengths of <10 Ns/m, hand-path deviations were made
consistently toward the right in both experiments (Figs. 3A
and 4A). The presence of these oppositely directed errors
(i.e., to the right) indicates that subjects were directly opposing
forces they anticipated encountering and precludes the possibility that they were merely stiffening the arm around some reference trajectory (Conditt et al. 1997a; Flash 1987
;
Shadmehr and Mussa-Ivaldi 1994
). Furthermore movements
made in stronger-than-average force fields were undercompensated,
whereas movements made in weaker-than-average force fields were
overcompensated, suggesting that subjects were compensating
approximately for the mean perturbing force field in both experiments.
This finding was confirmed by linear regression analysis (Figs.
3E and 4E).
Learning rates in the present study (Figs. 3C and
4C) were slower than rates reported for
compensation of inertial loads (within 1 trial) (Bock
1993) but were substantially faster than learning rates reported for consistent (but geometrically complex) viscous environments when subjects were required to reach in several different directions (more than 100 trials) (Bhushan and Shadmehr
1999
). Remarkably, the learning rates observed in
experiment 1 were almost identical to the rates at which
subjects regained adaptation to a predictable perturbing environment
after a single "catch trial" in which the perturbing environment
was unexpectedly removed (~3 trials) (Thoroughman and Shadmehr
2000
). Clearly, subjects adapted to these stochastic environments.
Adaptation modeled as an autoregressive process with external input (ARX process)
Although the linear regression results demonstrated that subjects compensated for the approximate mean perturbation strength in both force-field environments, they did not suggest how the central nervous system accomplished this adaptation. Mathematically, the mean perturbation magnitude is defined as the sum of the individual magnitudes divided by the number of perturbations. Since subjects had no way of knowing all of the perturbation magnitudes until the experiment was completed, it was not possible for them to directly compute the mean field strength. Subjects could have evaluated a "running average" of all trials they experienced so far. However, this strategy would require subjects to retain either explicit working memory of all previously encountered perturbations or explicit memory of the average of all previous perturbations along with a running total of the number of previous perturbations. In either case, the relative importance of the most recent perturbation would decrease linearly as a function of the number of perturbations.
A less demanding alternative would be for subjects to rely only on explicit information regarding only recent experiences. Motor performance (and consequently motor adaptation) may rely on information about past movement performance and/or past perturbations derived from a variety of sensory sources (e.g., muscle spindles, Golgi tendon organs, slowly adapting hand mechanoreceptors, vision, etc.). A general form of this model, one that depends only on information regarding movement errors and perturbation amplitudes (Eq. 3) was examined in the present experiment. One important aspect of this model is that information about experiences in the distant past is retained implicitly in the autoregressive terms (i.e., if Eq. 3 contains at least one nonzero aj term).
The correlation analyses (Fig. 5) demonstrated that movement error on a
given trial i was well predicted from the field strength on
that trial (Bi) and from the field
strength (Bi-1) and movement error
(i-1) exhibited on the previous trial (Eq. 5). Why do subjects compensate for previous movement
errors when the step response analysis suggests that learning would be more rapid and effective if those errors were disregarded altogether (i.e., set a1 = 0 in Eq. 5)
and the most recent perturbation was canceled exactly (i.e., set
b1 =
b0 in Eq. 5; Fig.
7B, trace 2)? Are there unavoidable history dependencies in
the proprioceptive and/or visual sensory pathways that constrain the
motor learning mechanisms in their "choice" of compensatory
strategies? It has been suggested that cancellation of prior movement
errors is important to motor adaptation (e.g., Flanagan and Rao
1995
; Scheidt et al. 2000
; Wolpert et al.
1995
). Equation 5 suggests that compensating for the
most recent movement error exactly (i.e.,
a1 =
1 in Eq. 5) would be
a counter-productive strategy since Eq. 5 becomes unstable
when |a1|
1. This can be seen
by examining the stability of Eq. 5 in the complex
z-domain (Oppenheim and Schafer 1989
). The
z transform of Eq. 5 is
![]() |
(6) |
![]() |
(7) |
Explicit representation of the internal model
Since the relationship between movement error and perturbation
gain was reasonably well fit by a straight line (Figs. 3E
and 4E), the ARX model of subject performance (Eq. 5) was rearranged to yield an expression for the internal model of
the perturbing environment. Specifically, movement error generated on
trial i was regarded as a function of the mismatch between
the actual perturbation experienced on that trial and the expected (or
adapted) perturbation magnitude: i = f(Bi
Badapted). Figures 3E and
4E demonstrate that this relationship was reasonably
described by a linear function
![]() |
(8) |
![]() |
(9A) |
![]() |
(9B) |
![]() |
(10) |
How do the motor adaptation mechanisms estimate the most recent movement error and perturbation strength?
Constructing the internal representation of the perturbing field
strength via Eq. 5 requires accurate estimation of
i-1 and
Bi-1. Movement error is likely to be
sensed both visually (e.g., Wolpert et al. 1995
) and
proprioceptively (Dizio and Lackner 2000
;
Shadmehr and Mussa-Ivaldi 1994
). The current experiments
were not designed to evaluate the relative contributions of different
feedback modalities to motor adaptation but rather to explore how the
neural mechanisms involved in motor adaptation use information from
previous movements (however that information is sensed) to modify
motor commands on subsequent movements. Both visual and proprioceptive
feedback appear to be important (Conditt et al. 1997b
),
although it is not yet clear how this feedback information is combined
in driving motor adaptation.
There are at least two ways the CNS could estimate the most recent
perturbation strength in keeping with the spirit of Eq. 10.
The first strategy would be to estimate the field strength directly
using sensory organs sensitive to the kinetic demands of the task
(e.g., Golgi tendon organs, hand mechanoreceptors, and indirectly,
muscle spindle receptors since they are coupled to the perturbation
through limb tissues with finite impedance). In this case,
Bi-1 would be "measured" directly
and Eq. 10 would be implemented as written. The second way
of estimating the most recent perturbation strength would be to do so
indirectly and recursively, using only the most recent movement error
to update the previous estimation of the perturbing field. In this case, Eq. 10 can be reformulated
![]() |
(11) |
Comparison of the simple autoregressive model with alternative learning strategies
If movement error is linearly related to the perturbing field
amplitude, i = k(Bi
Badapted), then the optimal internal
model in terms of least square error (i.e., the
Badapted that minimizes the sum of
![]() |
(12) |
![]() |
(13) |
|
A second alternative learning strategy describes what could be called
"careless learning" and was motivated in part by the observation
that subjects never made ideally straight movements and almost always
had peak hand deviations exceeding ~1 cm (Figs. 3E and
4E). Perhaps subjects considered movements with such small errors "good enough" for the specified task? This form of learning is careless in the sense that the learner does not attend to small movement errors. The internal representation of the perturbation in a
careless learning model would be updated only when the learner is
"surprised" (i.e., when the movement errors experienced on a given
trial exceed a minimum threshold value,
threshold). An attention model that describes
how well movement errors are attended to is
![]() |
(14) |
![]() |
(15) |
The attention model of Eq. 14 with
threshold = 5 mm is shown in Fig.
9A. The careless learning
algorithm (Eq. 15) was fit to subject
U2's initial movement error data (1st 100 movements). The
model's performance was then evaluated in the final 100 movements (Fig. 9B). Even though the algorithm neglected the smallest
movement errors, overall performance was respectable when driven by the unimodal perturbation sequence (VAF = 64%). However, when the same model was driven by the bi-modal sequence, the algorithm's performance suffered (Fig. 9C). Movement errors were
negatively biased, indicating an inability of the model to compensate
for the sequence of perturbations as well as the subject did. With
threshold = 5 mm, the careless learning
algorithm compensated only for the approximate mean of the minor peak
in the bimodal distribution (4.9 Ns/m, Fig. 9D).
Consequently, a learning algorithm that performs well in the unimodal
sequence may perform poorly in the bimodal sequence unless movement
errors are attended to carefully. The residual curvature observed while
reaching in both stochastic perturbation sequences was likely due to
biomechanical constraints and/or information processing within the
motor control systems and not due to inattention to very small movement
errors.
|
Relation to studies requiring adaptation to consistent perturbation sequences
A recent study of reaching movements by Thoroughman and
Shadmehr (2000) examined movement errors generated by subjects
exposed to predictable perturbing environments with periodic "catch
trials" where the predictable perturbation was unexpectedly removed.
Movement errors generated in the constant-gain curl-field just after
exposure to a catch trial were substantially larger than errors
generated just prior to the catch trial. This increase in error was
attributed to an "unlearning" of the internal model of the
environment. This increase in error decayed on subsequent movements to
the same target and was undetectable by about the third trial following a catch trial. This decay rate was comparable to the rate of adaptation observed in experiment 1 from the present study, even though
the perturbation sequence used in experiment 1 was random.
The decay rates obtained experimentally were comparable to the rate
predicted by Eq. 5 (Fig. 7A). Thoroughman and
Shadmehr fit a system of equations to their movement error data that
captured this experimentally observed unlearning behavior. Following a
rearrangement of terms and substitution of indices, it can be shown
that their system of equations can be represented in the form of
Eq. 5. The similarity in experimental
observations and the successes in equivalent modeling techniques
between the present study and that of Thoroughman and Shadmehr
(2000)
suggest that the processes involved in adapting to
consistent perturbing environments are the same as those involved in
adapting to stochastic perturbing environments.
In conclusion, a sequence of perpendicular viscous force fields with stochastically varying gains triggered an adaptive process that compensated for the approximate mean field gain from that sequence. Furthermore the force-field gain that subjects adapted to was not the most frequently experienced gain nor was adaptation dependent on the particular distribution of perturbations. Although adaptation to the mean field gain would be optimal in the sense that squared movement error would be minimized in the steady state, this strategy is computationally costly and does not allow sufficient flexibility to accommodate efficient learning of nonstationary environments. A simple model of motor performance that depended only on movement error and perturbation gain from the previous trial (Eq. 5) achieved substantial reduction in movement error, while allowing a rapid and appropriate response to long-term changes in the distribution of perturbations (Fig. 7). This simple model predicted subject performance with ~84% variance accounted for (VAF). These findings support the hypothesis that the neural structures modified as a result of motor adaptation do not explicitly retain memories of performances or perturbations beyond one or two trials in the past.
![]() |
ACKNOWLEDGMENTS |
---|
We extend special thanks to Dr. Chris Raasch for creating Fig. 1A.
This work was supported by National Institutes of Health Grants NS-35673 and P50MH-48185.
![]() |
FOOTNOTES |
---|
Address for reprint requests: R. A. Scheidt, Dept. of Biomedical Engineering, Olin Engineering Center, 303, PO Box 1881, Marquette University, Milwaukee, WI 53201-1881 (E-mail: scheidt{at}ieee.org).
Received 27 November 2000; accepted in final form 3 May 2001.
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|