1Department of Mechanical and Aerospace Engineering and Center for Biomedical Engineering, University of California, Irvine, California 92697-3975; and 2Department of Biomedical Engineering, Marquette University, Milwaukee, Wisconsin 53201
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Takahashi, C. D., R. A. Scheidt, and D. J. Reinkensmeyer. Impedance Control and Internal Model Formation When Reaching in a Randomly Varying Dynamical Environment. J. Neurophysiol. 86: 1047-1051, 2001. We investigated the effects of trial-to-trial, random variation in environmental forces on the motor adaptation of human subjects during reaching. Novel sequences of dynamic environments were applied to subjects' hands by a robot. Subjects reached first in a "mean field" having a constant gain relating force and velocity, then in a "noise field," having a gain that varied randomly between reaches according to a normal distribution with a mean identical to that of the mean field. The unpredictable nature of the noise field did not degrade adaptation as quantified by final kinematic error and rate of adaptation. To achieve this performance, the nervous system used a dual strategy. It increased the impedance of the arm as evidenced by a significant reduction in aftereffect size following removal of the noise field. Simultaneously, it formed an internal model of the mean of the random environment, as evidenced by a minimization of trajectory error on trials for which the noise field gain was close to the mean field gain. We conclude that the human motor system is capable of predicting and compensating for the dynamics of an environment that varies substantially and randomly from trial to trial, while simultaneously increasing the arm's impedance to minimize the consequence of errors in the prediction.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Recent evidence suggests that
humans use internal models of the arm and its environment to control
reaching. In the experiments described by Shadmehr and
Mussa-Ivaldi (1994), subjects adapted to perturbing forces
applied by a robotic device. After adaptation, the perturbing forces
were unexpectedly removed, resulting in a reaching path that was
displaced in the direction opposite the force. The presence of this
"aftereffect" was taken as evidence that the nervous system uses an
internal model to control the arm since the movement errors generated
on these trials were mirror-symmetric to those observed during initial
exposure to the perturbations. An alternate strategy
stiffening the
arm by co-activating muscles to compensate for the force field
disturbance [a form of impedance control (Hogan
1985
)]
was rejected because it would not be expected to
produce aftereffects. Subsequent studies have confirmed and elaborated
on the use of internal models in reaching (Brashers-Krug et al.
1996
; Conditt et al. 1997
; Gandolfo et
al. 1996
; Goodbody and Wolpert 1998
;
Sainburg et al. 1999
).
A possible limitation of these previous studies is that they have utilized novel but predictable force fields that lack the trial-to-trial variability commonly experienced in many natural environments (e.g., sorting packages in a mailroom). The objective of the present study was to determine the effect of trial-by-trial variability on the formation of an internal model of the limb's environment.
![]() |
METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Twenty-four unimpaired subjects (8 in each of 3 experimental
groups; 17 male and 7 female; 20 right handed and 4 left handed; 22-58
yr old) participated in the study, approved by the U.C. Irvine IRB. The
seated subject's hand was attached to a lightweight robot arm (PHANToM
3.0, SensAble Technologies) through a customized orthopedic splint
(Fig. 1). Subjects reached alternately to
two target light-emitting diodes (LEDs; 0.625-in. diam), left and right, positioned in front of the left and right shoulders just inside
the boundary of the reaching workspace (left target position = [200, 70,
265] mm; right target position = [200, 70,
265] mm; see coordinate system in Fig. 1). Between reaches, subjects relaxed the arm at a home position over the lap (Fig. 1, home position = [0, 0, 0] mm). After each movement the computer
sounded one of three tone patterns to provide feedback on the reach
speed (too fast, too slow, or just right = 1.2 s). Subjects
were able to perform the task consistently after a few practice trials. Two targets were used to make the task more engaging for the subjects.
|
The robot was programmed to generate a velocity-dependent force field
of the form
![]() |
Protocol
To characterize adaptation to a randomly varying environment, a within-subject repeated measures design was used in which the same subject was exposed first to a predictable environment, then to a random environment. The performance in each field was compared for each subject. Specifically, one group of subjects (the "Mean-Noise" or "MN" group) was exposed to five sequential dynamic environments, called "stages" (Fig. 1). In the first stage ("null field 1") the robot did not actively apply forces to the subject (i.e., k = 0) for 40 reaches. In the second stage ("mean field") the force gain was constant (k = 1) for 60 reaches, producing a leftward perturbation to the hand according to Eq. 1. The third stage ("null field 2") was another null field for 20 reaches. The beginning of this stage allowed measurement of the aftereffect of adaptation in the mean field. In the fourth field ("noise field") the force gain was randomly varied for 60 reaches according to a normal distribution with a mean of 1.0 and a SD of 0.5. The force gain variation was truncated to ±1.1 about the mean of 1.0 to protect against large forces due to random outliers. The effect of the noise field was to apply a slightly different magnitude of force for each reach, but the average magnitude over many reaches was the same as the mean field. The fifth and final stage ("null field 3") was another null field for 20 reaches. The subjects in the second group (the "MM" group) were exposed to a similar sequence of environments, except that instead of being exposed to the noise field in the fourth stage, they were exposed to a second block of mean field perturbations (Fig. 1). Subjects in a third group (the "NM" group) were exposed first to the noise field and then to the mean field, to check for possible ordering effects (Fig. 1).
Data analysis
Since the force field pushed the hand to the left, disturbances to the reaching trajectory were mainly in the horizontal plane. Statistical analysis indicated that trajectories were not significantly disturbed in the vertical direction on initial exposure to or removal of the field. Thus reaching errors were quantified as the area between the trial path and a reference path projected onto the horizontal plane (X-Z plane, Fig. 1). Reach paths that were to the right of the reference path were given positive values, while those to the left were given negative values. The reference path was selected to be the average path of the trials in the last half of null field 1 (trials 21-40). The average was computed by aligning the path data to an initial velocity threshold (200 mm/s) and computing the mean across the corresponding sampling points. For these trials, the subjects had presumably acclimated to the robot but still had no perturbing force field applied to them.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The eight subjects from the MN group reached first in a predictable dynamic environment (the mean field) and then in a random environment (the noise field), the gain of which varied unpredictably from trial to trial. When the robot first imposed the force fields (mean or noise), the subjects produced large initial errors followed by a gradual recovery of their original performance (Fig. 2). On removal of the force fields, the subjects displayed aftereffects (i.e., increased reaching error in the direction opposite to that of the force field). The magnitude of these aftereffects (trials 101 and 181, Fig. 2) was significantly different from the baseline reference error from the first null field (ANOVA, P < 0.001) and decreased with repeated movement. For MN subjects, the noise field aftereffect (trial 181) was significantly smaller than the mean field aftereffect (trial 101; paired t-test, P = 0.01), while the first and second aftereffects for the MM subjects were comparable in size and not statistically different (P > 0.05). The aftereffect ratio for MN subjects (noise aftereffect divided by mean aftereffect, mean = 0.57 ± 0.25 SD) was 42% smaller than the aftereffect ratio for MM subjects (mean 2 aftereffect divided by mean 1 aftereffect, mean = 0.98 ± 0.32).
|
Despite the diminished aftereffect following exposure to the noise field, the MN subjects' performance in the mean and noise fields was not significantly different, in terms of average final error and the variance of error (t-test on SDs, P = 0.17). Additionally, MN subjects showed similar rates of performance improvement in the random and mean fields, with no significant difference in exponential time constants fit to each subject's learning curves.
The question arises as to why the MN subjects performed about as well in the random field as they did in the predictable field, yet had a diminished aftereffect. One possibility was that they did not accurately model the average dynamics of the noise field. Alternately, subjects may have increased the impedance of their arms in the noise field. Increased arm impedance would be expected to reduce the trajectory error when the noise field was unexpectedly removed.
To distinguish these possibilities, we analyzed the reaching behavior
of the MN subjects in the noise field (Fig.
3A). A minimal trajectory
error would be expected whenever the gain of the internal model matched
the external field gain (Goodbody and Wolpert 1998). In
the first movement in the noise field, the behavior across subjects was
consistent with a nearly zero reaching error when the force gain was
zero (Fig. 3A, thick dashed line). Following adaptation, the
behavior across MN subjects was consistent with a minimization of
trajectory error when the noise field gain was equal to 1.03 ± 0.18 (averaged zero crossing of individual subjects' regressions over last
40 reaches), indicating accurate modeling of the average noise field
(Fig. 3A, thick solid line). The movement error
corresponding to the gain of the mean field declined toward zero over
the 1st 20 movements, suggesting that an accurate internal model was
constructed with repeated practice (Fig. 3B). The slopes of
the regression lines (Fig. 3A) also provide a gross measure of the endpoint impedance of the subjects' limbs (i.e., the impedance generated at the robot's handle by the subjects). The steeper slope
during initial exposure to the field corresponded to a lower impedance
than observed later in the experimental sessions (P < 0.001, t-test comparing individual subject's regressions
slopes of last 40 reaches to that of initial reach across subjects). Consequently the initial range of movement errors was larger than the
range of errors observed at the end of the block of random perturbations for the same range of perturbation gains. Therefore the
reduced noise field aftereffect was likely due to an increase in arm
impedance during reaching in the noise field, presumably accomplished
by stiffening the arm about the reference trajectory.
|
It is possible that the MN subjects did not actually learn the mean of the noise field, but simply reverted to their most recently stored dynamic model (i.e., of the mean field) when presented with the noise field. To evaluate this possibility, a third group of subjects was exposed first to the noise field and then to the mean field. The NM group formed an internal model of the random field, as shown by a minimization of error over the last 40 reaches when the gain was 0.82 ± 0.26 (Fig. 3), a value significantly different from zero (P < 0.001). However, this value was significantly less than the 1.03 value for MN subjects (1-sided t-test, P = 0.04), indicating that the model was not as accurate as the one for the MN group. Thus MN subjects apparently used the most recently stored dynamic model to estimate more accurately the mean of the random field, although the ability to form a model of the random field was not dependent on previous exposure to the mean field. The regression slope of initial exposure to the noise field for the NM group was significantly larger than that of the last 40 reaches (P < 0.001, Fig. 3), and the noise field aftereffect was significantly less than the mean field aftereffect (paired t-test, P = 0.021, noise/mean aftereffect ratio = 0.77 ± 0.24, Fig. 2). These findings are consistent with the hypothesis that subjects increased limb impedance in response to exposure to the random field. The slightly greater NM aftereffect ratio may have arisen because, by chance, higher-than-average force field strengths were applied for the NM subjects over the last three reaches in the noise field (Fig. 1D).
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The results of this study have three major implications. First,
substantial variability in the presentation of perturbing environments
does not inhibit the formation of internal models of limb dynamics.
Subjects were capable of compensating for the approximate mean of the
random perturbing environment. The process by which the model is formed
likely involves a moving average computation and possibly operates over
only a few previous reaches (Scheidt and Mussa-Ivaldi
1999; Thoroughman and Shadmehr
2000
). In addition, the process apparently incorporates
a retention mechanism, since previous exposure to a predictable field
produces more accurate modeling of a random field with the same mean.
Retention has also been observed with repeated exposures to a
predictable field in that better performance is achieved during the
second exposure to the predictable field (Brashers-Krug et al.
1996
).
Second, while some previous investigations have de-emphasized the role
of impedance control in motor adaptation because of the presence of
aftereffects (Gandolfo et al. 1996; Shadmehr and Mussa-Ivaldi 1994
), the results of this study suggest that
impedance control can coexist with the application of internal models
for control. Consistent with this result, muscular co-activity at the
wrist was previously shown to be reduced as a difficult movement task
was learned (Milner and Cloutier 1993
). The results of
the present study indicate that when learning to move in a random mechanical environment, impedance is increased while internal models
are developed.
Third, and finally, realistic computational models of motor adaptation should incorporate in their structure two adaptive processes: internal model formation as well as impedance regulation. Identifying the mechanisms and dynamics giving rise to both model formation and impedance regulation are important goals for future motor control research.
![]() |
ACKNOWLEDGMENTS |
---|
This work was supported by a Whitaker Foundation Biomedical Engineering Research Grant.
![]() |
FOOTNOTES |
---|
Address for reprint requests: D. J. Reinkensmeyer, Dept. of Mechanical and Aerospace Engineering, 4200 Engineering Gateway, University of California, Irvine, CA 92697-3975 (E-mail: dreinken{at}uci.edu).
Received 4 December 2000; accepted in final form 5 April 2001.
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|