The Motor System Does Not Learn the Dynamics of the Arm by Rote Memorization of Past Experience

Michael A. Conditt, Francesca Gandolfo, and Ferdinando A. Mussa-Ivaldi

Sensory Motor Performance Program, Rehabilitation Institute of Chicago, Northwestern University, Chicago, Illinois 60611; and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139

    ABSTRACT
Abstract
Introduction
Methods
Results
Discussion
References

Conditt, Michael A., Francesca Gandolfo, and Ferdinando A. Mussa-Ivaldi. The motor system does not learn the dynamics of the arm by rote memorization of past experience. J. Neurophysiol. 78: 554-560, 1997. The purpose of this study was to investigate the learning mechanisms underlying motor adaptation of arm movements to externally applied perturbing forces. We considered two alternative hypotheses. According to one, adaptation occurs through the learning of a mapping between the states (positions and velocities) visited by the arm and the forces experienced at those states. The alternative hypothesis is that adaptation occurs through the memorization of the temporal sequence of forces experienced along specific trajectories. The first mechanism corresponds to developing a model of the dynamics of the environment, whereas the second is a form of "rote learning." Both types of learning would lead to the recovery of the unperturbed performance. We have tested these hypotheses by examining how adaptation is transferred across different types of movements. Our results indicate that 1) adaptation to an externally applied force field occurs with different classes of movements including but not limited to reaching movements and 2) adaptation generalizes across different movements that visit the same regions of the external field. These findings are not compatible with the hypothesis of rote learning. Instead, they are consistent with the hypothesis that adaptation to changes in movement dynamics is achieved by a module that learns to reproduce the structure of the environmental field as an association between visited states and experienced forces, independent of the kinematics of the movements made during adaptation.

    INTRODUCTION
Abstract
Introduction
Methods
Results
Discussion
References

Adaptation refers to a process in which a system recovers previously learned skills after a change in the operating environment. Among the most studied forms of adaptive behavior are the changes in visuomotor control following the modification of the visual field by means of prism glasses (Held and Freedman 1963). Recent investigations of Flash and Gurevich (1992), Shadmehr and Mussa-Ivaldi (1994), Sainburg and Ghez (1995), and Gandolfo et al. (1996) have demonstrated the presence of adaptive processes in the control of multijoint arm movements. In these experiments, subjects were required to execute point-to-point reaching movements while holding the handle of an instrumented manipulandum, which could act either as a passive device or as an active programmable force generator. With the passive manipulandum, subjects produced the typical reaching movement kinematics that have been described by several investigators: the hand moved from start to end location along a straight path and with a bell-shaped velocity profile (Atkeson and Hollerbach 1985; Flash and Hogan 1985; Morasso 1981). These kinematics were drastically altered when the manipulandum generated a disturbing perturbation. In the experiments of Shadmehr and Mussa-Ivaldi (1994) the manipulandum was programmed to generate forces whose amplitude and direction depended linearly on the measured velocity of the hand. These forces altered, in a specific way, the dynamics of the system controlled by the subject's motor activities. Nevertheless, after a period of repeated movements with the perturbed dynamics, the subject recovered the original kinematics. As in the case of visuomotor adaptation, adaptation to a disturbing force field occurred in parallel with the development of aftereffects: when the disturbing field was suddenly removed, after prolonged exposure, the resulting kinematics displayed a change opposite to that induced initially by the same disturbing field. Shadmehr and Mussa-Ivaldi considered these aftereffects as evidence that subjects adapted to a disturbing field by adding to their motor command a preprogrammed force component that was equal and opposite to the field. They suggested that adaptation to a novel force field occurs by creating an "internal model" of this field. The forces generated by this internal model are added to the normal, unperturbed motor commands so as to restore the original movements. This theory of adaptation is appealing because it implies that after a change in the environment dynamics, it is not necessary for the control system to relearn the entire repertoire of previously acquired motor skills. These skills can be recovered by superimposing onto the original command, the output of a single independent module that learns to mimic and cancel the forces produced by the environment. However, the results of Shadmehr and Mussa-Ivaldi (1994) and Flash and Gurevich (1992) are also consistent with the hypothesis that, instead of an internal model of the environmental forces, the motor system learns the specific temporal pattern of forces encountered along each trajectory. This type of adaptation, that we call "rote learning," would generate the appropriate compensatory forces only along the specified trajectory. Rote learning would account for the convergence of perturbed movements toward their unperturbed kinematics as well as the existence of aftereffects.

This investigation was aimed at determining whether the learning of a new force field occurs through the formation of an internal model or through rote learning. Our results indicate that 1) adaptation to an externally applied force field occurs with different classes of movements including but not limited to reaching movements and 2) adaptation generalizes across different movements that visit the same regions of the external field. These findings are not compatible with the hypothesis of rote learning. Instead, they are consistent with the hypothesis that adaptation to changes in movement dynamics is achieved by a module that learns to reproduce the structure of the environmental field as an association between visited states and experienced forces, independent of the kinematics of the movements made during adaptation.

    METHODS
Abstract
Introduction
Methods
Results
Discussion
References

Setup

The techniques were similar to those described in greater detail by Shadmehr and Mussa-Ivaldi (1994). All experiments were performed using a two degree-of-freedom manipulandum (Fig. 1A). Subjects were asked to hold and move the handle of the manipulandum while its position was continuously displayed as a small cursor on a monitor placed above the manipulandum. The subject's arm was supported in the horizontal plane by a low-friction, low-impedance mechanism and the shoulder was restrained using a Velcro torso support. The manipulandum is equipped with position encoders and tachometers that were used to record the position and velocity of the handle. Two torque motors were used to generate a preprogrammed force field (Fig. 1B). When the field was activated, the subject experienced a force at the hand, [Fx Fy] that was linearly related to the velocity of the hand, [&xdot; &ydot;]
&cjs0362;<AR><R><C><IT>F<SUB>x</SUB></IT></C></R><R><C><IT>F<SUB>y</SUB></IT></C></R></AR>&cjs0363; = &cjs0362;<AR><R><C>−13</C><C>−13</C></R><R><C>−13</C><C>12</C></R></AR>&cjs0363;&cjs0362;<AR><R><C><IT><A><AC>x</AC><AC>˙</AC></A></IT></C></R><R><C><IT><A><AC>y</AC><AC>˙</AC></A></IT></C></R></AR>&cjs0363; (1)
where the force is in units of N, the viscosity matrix is in Newtons per meter per second, and the velocity is in meter per seconds. The matrix, B, had two eigenvalues with opposite signs so that there was a resistive viscosity along one eigenvector and a destabilizing assistive force along the other.


View larger version (13K):
[in this window]
[in a new window]
 
FIG. 1. A: planar arm movements were made as the subject grasped and moved the handle of a 2 degree-of-freedom robot manipulandum. B: velocity-dependent force field applied by the manipulandum to the subject's hand as described in Eq. 1.

Tasks

Fourteen subjects with no known neuromotor disorders, ranging in age from 21 to 43 yr, participated in this study. We investigated adaptive learning through two different experiments performed by two different sets of subjects. In the first experiment, adaptation of complex movements, six subjects were required to execute four different drawing movements: a circle, the infinity symbol, a triangle, and a curved diamond figure. In the second experiment, transfer of learning, eight different subjects were required to execute two types of movements: reaching movements and one of the drawing movements, circles. With reaching movements, subjects were presented a square target on the monitor and were required to move the cursor representing the handle inside the target square. No explicit instructions were given concerning the hand path from initial to final location (a distance of ~10 cm). However, explicit instructions were provided concerning duration in order to maintain the movement times within a reasonably short (0.3-0.5 s) and repeatable range. Each drawing movement was obtained by presenting the subjects with one of four templates: 1) a circle, 2) the infinity symbol, 3) a triangle, and 4) a curved diamond figure. All movements were limited to a single execution of the template. No cyclic movements were performed. For all four shapes, the starting and stopping locations were the same, providing the possibility of presenting the templates in random order. Tracing was avoided by presenting the template and the position feedback in two different regions of the monitor. Again, no explicit instructions were given concerning the hand path; however, all subjects chose to traverse the four shapes in a counterclockwise direction. Explicit instructions were provided concerning movement times that were different for each shape.

The first experiment tested the ability of subjects to adapt complex movements to a change in dynamics. Subjects first experienced the "null-field" condition, during which the motors were turned off in the majority of the trials and the manipulandum behaved as a passive device. The subjects then completed the adaptive stage, during which the endpoint force field of Eq. 1 (Fig. 1B) was activated in the majority of the trials while subjects executed the required drawing movements. Finally, the subjects completed another null-field stage, during which the manipulandum was always passive. In a number of random trials during the first null-field stage (12 trials over a total of 192 movements) the field was turned on unexpectedly. These trials were used to assess the response to the field before the onset of adaptation. Similarly, during a number of random trials in the adaptive stage (48 trials over a total of 768 movements), the field was turned off unexpectedly to assess the aftereffects of adaptation (Shadmehr and Mussa-Ivaldi 1994).

The second experiment focused on how subjects performed circles after adaptation to the viscous field while making reaching movements. As in the first experiment, subjects participated in a null-field stage and an adaptive stage. During the null-field stage, subjects executed both reaching movements and circles, termed the control circles, with no applied viscous field. During the adaptive stage, subjects adapted to the endpoint force field of Eq. 1 while making only reaching movements. We then tested for two types of transfer: 1) transfer of aftereffects, in which subjects were asked to execute circular movements in the null field, the transferred aftereffect circles, and 2) transfer of adaptation, in which subjects were asked to execute circular movements in the viscous field, the transferred adaptation circles. For both tests, circular movements were made at random trials during the last part of the adaptive stage for reaching movements. Immediately after the presentation of a circular pattern, the subjects were presented with another set of reaching movements to be performed in the viscous field. Following the tests for transfer, the subject completed another adaptation stage; however, this time the subject made only circles, the direct adaptation circles. In a number of random trials in this adaptive stage (16 trials over a total of 192 movements), the field was turned off unexpectedly allowing us to record the direct aftereffect circles.

Data analysis

Performance was quantified using a figural distance between a template and the observed trajectories. The figural distance between two trajectories, A and B, is based on the repeated measure of the Euclidean distance between each point in one trajectory and all the points in the other. If the trajectory A has n points, {A(1), A(2), . . . , A(n)}, and the trajectory B has m points, {B(1), B(2), . . . , B(m)}, then one may derive an n-dimensional vector
dist<SUB><IT>A−B</IT></SUB>(<IT>i</IT>) = <LIM><OP>min</OP><LL><SUB><IT>j</IT></SUB></LL></LIM>[∥<IT>A</IT>(<IT>i</IT>) − <IT>B</IT>(<IT>j</IT>)∥]
(1 ≤ <IT>i</IT> ≤ <IT>n</IT>)
and an m-dimensional vector
dist<SUB><IT>B−A</IT></SUB>(<IT>j</IT>) = <LIM><OP>min</OP><LL><SUB><IT>i</IT></SUB></LL></LIM>[∥<IT>A</IT>(<IT>i</IT>) − <IT>B</IT>(<IT>j</IT>)∥]
(1 ≤ <IT>j</IT> ≤ <IT>m</IT>)
The vector distA-B contains the distances between the trajectory B and each point in A, whereas the vector distB-A contains the distances between the trajectory A and each point in B. We define the figural distance between A and B as
ε(<IT>A</IT>, <IT>B</IT>) = &cjs0362;<LIM><OP>∑</OP><LL><SUB><IT>i</IT>=1</SUB></LL><UL><SUP><IT>n</IT></SUP></UL></LIM>dist<SUB><IT>A−B</IT></SUB>(<IT>i</IT>) + <LIM><OP>∑</OP><LL><SUB><IT>j</IT>=1</SUB></LL><UL><SUP><IT>m</IT></SUP></UL></LIM>dist<SUB><IT>B−A</IT></SUB>(<IT>j</IT>)&cjs0363;&cjs0948;(<IT>m + n</IT>) (2)
In essence, the figural distance between two trajectories captures the difference in the shapes of the respective paths and is insensitive to differences in speed. All the trajectories measured in our experiments were compared with a baseline template trajectory, which was an average of the control movements for that shape made in the null field. We will refer to the figural distance between a measured trajectory and its corresponding movement template as the figural error. A standard t-test applied to the figural errors was used to compare two sets of movements after an F-distribution was used in conjunction with the ratio of the variances of the two sets of errors to determine whether to use a t-test for equal or unequal variances.

    RESULTS
Abstract
Introduction
Methods
Results
Discussion
References

Adaptation of complex movements

The first issue we addressed was whether or not subjects may adapt complex movements to a change in dynamics. To this end, we asked six subjects to reproduce four different movement shapes. Average trajectories executed by one subject in the null-field condition are shown in Fig. 2A. The shaded area around each trajectory represents ±1 SD. Figure 2A provides an estimate of how consistently each trajectory was executed in the null field. At the end of the null-field period, the perturbing force field shown in Fig. 1B was unexpectedly applied in a number of random trials (initial exposure condition). The resulting trajectories are shown in Fig. 2B. In all cases, the unexpected application of the field caused a geometrically consistent distortion of the trajectory: the movement paths became stretched along the direction in which the field was assistive and contracted along the resistive direction. Next, we asked our subjects to perform 768 movements while the force field in Fig. 1B was kept persistently active (adaptive learning condition). The average trajectories at the end of the adaptive learning period (final 192 trials) are shown in Fig. 2C. These trajectories qualitatively indicate that the subject was able to recover the original performance, as shown in Fig. 2A. At the end of the adaptive stage, we unexpectedly suppressed the disturbing field in a number of random trials (aftereffect condition). The average aftereffects for each shape are shown in Fig. 2D. The trajectories of the aftereffects are deformed in a way that is complementary to the distortion of the initial exposure: where there was a stretch, there is now a contraction and vice versa. The qualitative findings emerging from Fig. 2 are described in quantitative terms by the plots in Fig. 3. This figure shows, for one subject and all four different movements, the figural errors (see Eq. 2, METHODS) of the trajectories at the different stages of learning with respect to the averages of the trajectories for their corresponding shapes in the null field. The average figural errors for the control trajectories in the null field are represented by the dash-dot baselines. The shaded area indicates the movements made when the force field was predominantly activated. The solid lines indicate the temporal evolution of the errors during the learning period. The decrease of this figural error indicates that the original movement kinematics were gradually restored by repeated practice in the field. Note that, for all movements, the biggest adaptive change occurs within the 1st 192 movements in the field (corresponding approximately to 6 min of practice). The dashed lines in the same figure show the evolution of the aftereffects. The temporal course of the aftereffects was specular to the temporal course of adaptation. All six subjects exhibited similar results. The average drop in figural error from the initial exposure trajectories to the fully adapted movements in the field was 55% across all subjects and shapes. These results extend the findings of Shadmehr and Mussa-Ivaldi (1994) to a broader class of trajectories. They show that subjects modified the motor commands so as to cancel the forces of the external field.


View larger version (28K):
[in this window]
[in a new window]
 
FIG. 2. A: average (±SD) baseline hand trajectories for the 4 shapes in the null field (n = 45 for each shape). B: average (±SD) perturbed trajectories obtained by the unexpected exposure to the force field (n = 3). C: average (±SD) trajectories in the viscous field after 4 training sets (n = 45). D: average (±SD) aftereffects after 4 training sets (n = 3). These data were obtained from a single subject.


View larger version (61K):
[in this window]
[in a new window]
 
FIG. 3. Progression of the average figural errors for one of the subjects with all 4 shapes: A, circles; B, infinity symbols; C, triangles; and D, curved diamond figures. Shaded area in each plot represents movements made while the field was predominantly active. Solid lines represent movements in the field and thus show the progression of learning to compensate for the field as the movements converge to the control trajectories (dash-dot lines). Dashed lines show the progression of aftereffects with practice and finally the loss of aftereffects after the removal of the field.

Transfer of learning

It has been suggested that adaptation to an external force field may be accomplished by the CNS building an internal representation of the field. Mathematically, this representation is a mapping between the state of the limb and the appropriate force corresponding to that state (Shadmehr and Mussa-Ivaldi 1994). The idea of an internal model, however, is not the only hypothesis compatible with the observation of adaptive behavior. An alternative hypothesis, which we call rote learning, is that the motor system learns to generate the specific temporal sequence of forces that cancels the external field along each trajectory. In this case, the system would not develop an internal representation of the mapping between limb states and forces.

Rote learning would be compatible with the observation of aftereffects when the external field is suddenly removed. However, rote learning would be unable to produce the correct movements if subjects were asked to generate a trajectory that differs in any substantial way from those that had been previously experienced. In particular, rote learning would fail if the subject were required to execute a movement that goes through states and forces that were explored by previous movements but follows a different temporal sequence. In contrast, such a situation would not constitute a problem with an internal model. In this case, the temporal order in which the states are visited is irrelevant provided the CNS has had the opportunity to develop the adequate pairings with the corresponding forces. To test whether rote learning or an internal model is responsible for adaptation, we developed a paradigm with two sets of movements: a set of linear reaching movements and a set of circle-drawing movements. Reaching movements were oriented in different directions, spanning a range of 360°. The amplitude and speed of the circular movements were chosen so that the circles passed through the same positions and velocities as the reaching movements. The goal of this experiment was to test whether, after adapting to a novel field with reaching movements, subjects had also implicitly adapted the execution of circular movements.

We performed two separate tests on each of eight subjects. In one test, we observed the effects of adapting the reaching movements to a perturbing field on the execution of circular movements in the null field (test for transferred aftereffects). In the other test, we observed the effects of adapting the reaching movements to a perturbing field on the execution of circular movements in the same perturbing field (test for transferred adaptation). The results of the test for transferred compensation were compared with the performance of the same subjects when asked to adapt to the same force field using only circular movements. These movements were performed after the tests for transfer so that when testing for both transferred aftereffects and transferred adaptation, the subjects had not yet experienced the field while making circles.

The results of both tests for one subject are summarized in Fig. 4. The top panels (A-C) refer to the test for transferred aftereffects. Figure 4A shows the average circular trajectory in the null field as obtained in the control experiment. Data presented in Fig. 4B, also from the control experiment, show the average aftereffect trajectory following the adaptation of the circular movements to the disturbing field (direct aftereffect). This trajectory should be compared with the trajectory in Fig. 4C, which shows the aftereffect induced on the circular motions, the transferred aftereffect, after adaptation of the reaching movements to the disturbing field. We must stress that in this latter case, the subject had not executed any circular motions in the perturbing field during the period of adaptation. Thus the visible elongation of the transferred aftereffect trajectory in the same direction as the direct aftereffect trajectory indicates that the motor command for the circular movement was consistently modified by the exposure of the reaching movements to the external field.


View larger version (19K):
[in this window]
[in a new window]
 
FIG. 4. Top panels: circles made in the null field. A: average (±SD) of the control circles made in the null field (n = 47). B: average (±SD) direct aftereffects as a result of adapting to the field directly with circles (n = 15). C: average (±SD) transferred aftereffects as a result of adapting to the field with reaching movements (n = 16). Bottom panels: circles made in the viscous field. D: average (±SD) trajectories during the unexpected initial exposure to the viscous field (n = 16). E: average (±SD) direct adaptation circles made after adapting to the field directly with the circles (n = 27). F: average (±SD) transferred adaptation circles after adapting to the field with reaching movements (n = 31).

The results of the test for transferred adaptation are shown in the bottom panels of Fig. 4 (D-F). The initial effect of the force field during the control experiment is demonstrated by the average trajectory in Fig. 4D. This trajectory is stretched along the direction of the destabilizing forces (Fig. 1B). Figure 4E shows the average trajectory at the end of the direct adaptation stage during the control experiment (the subjects adapted to the field while making circles). This trajectory, compared with the null-field movement in Fig. 4A, shows that the subject fully recovered the initial kinematics after practicing the circular movements in the field. Figure 4F shows the trajectory at the end of an adaptive period. However, in this case the subject adapted to the field by practicing reaching movements instead of circular movements. This trajectory represents the transferred adaptation.

Table 1 provides a summary of the generalization results for all subjects tested in this experiment. In the majority of the cases (for exceptions see the legend of Table 1), after adapting with reaching movements there was significant transfer of both aftereffects and adaptation from the reaching movements to the circular movements. These results are not compatible with the hypothesis that adaptation occurs by rote learning and playback of the external forces. In contrast, the same findings indicate that, at the end of adaptation, subjects were able to reproduce the field forces regardless of the particular exploratory sequences that were used in the adaptive stage.

 
View this table:
[in this window] [in a new window]
 
TABLE 1. Summary of transferred learning

    DISCUSSION
Abstract
Introduction
Methods
Results
Discussion
References

We investigated the process underlying the adaptation of multijoint arm movements to changes in the environmental dynamics in which the movements are performed. To date, similar experiments have been limited to point-to-point reaching movements (Bock 1990, 1993; Flash and Gurevich 1992; Gandolfo et al. 1996; Happee 1993; Lackner and Dizio 1994; Sainburg and Ghez 1995; Shadmehr and Mussa-Ivaldi 1994; Weeks et al. 1996). Our first experiment demonstrated that subjects adapt not only reaching movements but a variety of trajectories when the arm is exposed to a pattern of perturbing forces. With practice, the trajectories of complex movements made in the field converged to the unperturbed trajectories. In addition, when the field was suddenly removed after prolonged exposure, the resulting trajectories exhibited aftereffects, indicating that the subject preprogrammed the appropriate forces to compensate for the perturbation. An important difference between our result and previous adaptation results is that, unlike the point-to-point reaching movements that generally were completed in <500 ms, our complex movements lasted as long as 2 s. The presence of aftereffects throughout an entire movement lasting 2 s provides additional evidence that adaptive learning results from a change in the feed-forward component of the motor command.

The presence of aftereffects in adaptive learning has supported the hypothesis that the human motor control system adapts to changes in the environmental dynamics by constructing an internal representation of those dynamics (Shadmehr and Mussa-Ivaldi 1994). However, an alternative explanation for adaptation and its aftereffects is that the motor control system may learn the temporal sequence of forces experienced along specific trajectories. On subsequent trials, this record of forces would then be played back whenever the same trajectories are repeated. We call this hypothetical mechanism "rote learning," a notion that is consistent with the open-loop motor program theories of motor control (Henry and Rogers 1960; Keele 1968; Schmidt 1975). Rote learning could lead to similar effects as an internal model of the environment. In particular, it could reproduce the appropriate compensatory forces along any specific trajectory. However, rote learning would not lead to a representation of the structure of the field as a functional relation between limb states and perturbing forces. In contrast, an internal model reconstructs this relation. Hence, when a different movement is performed for the first time in the explored region of the field, there is no need to relearn the forces along this new trajectory. To distinguish between rote learning and learning of an internal model, we examined the effects of adapting to a particular field with one set of movements on the performance of a different set of movements that visited the same states. We found that learning does indeed transfer to movements never before performed in the field.

We want to stress that this finding must not be confused with others suggesting that learning of dynamics has a limited domain of generalization. Results by Gandolfo et al. (1996) and Sainburg and Ghez (1995) showed that when subjects learn to compensate a viscous or inertial perturbation along a specific direction, the adaptation does not generalize to movements in different directions. These findings refer to the ability of the control system to extrapolate (or generalize) learning beyond the set of explored positions and velocities. In contrast, our experiments addressed the issue of how the motor system may generalize adaptation to different trajectories in the same range of positions and velocities explored during the learning period. Shadmehr and Mussa-Ivaldi (1994) also found that generalization occurs after adapting to a field within an initial region of the workspace when reaching movements are repeated in a new region with the same pattern of disturbing torques as the initial field. This result is compatible with the hypothesis of rote learning because subjects could cancel out the field in the new region by producing the same temporal sequences of torques learned in the initial region. Our experiments rule out this latter option, leaving only one surviving hypothesis according to which adaptive learning is the product of an internal model of the environment. This model may be expressed in the CNS by specialized modules that operate independently of the structures that support the planning and representation of movement kinematics.

    ACKNOWLEDGEMENTS

  This work was supported by the Office of Naval Research (N00014-95-1-0571, N00014-88-K0372), National Institute of Neurological Disorders and Stroke Grant NS-35673-01, and the Ralph and Marion C. Falk Research Trust Fund.

    FOOTNOTES

  Address for reprint requests: F. A. Mussa-Ivaldi, Sensory Motor Performance Program, Rehabilitation Institute of Chicago, 345 East Superior, Suite 1406, Chicago, IL 60611.

  Received 13 November 1996; accepted in final form 8 April 1997.

    REFERENCES
Abstract
Introduction
Methods
Results
Discussion
References

0022-3077/97 $5.00 Copyright ©1997 The American Physiological Society