Predictive Motor Learning of Temporal Delays

Alice G. Witney, Susan J. Goodbody, and Daniel M. Wolpert

Sobell Department of Neurophysiology, Institute of Neurology, University College London, London WC1N 3BG, United Kingdom


    ABSTRACT
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Witney, Alice G., Susan J. Goodbody, and Daniel M. Wolpert. Predictive Motor Learning of Temporal Delays. J. Neurophysiol. 82: 2039-2048, 1999. Anticipatory responses can minimize the disturbances that result from the action of one part of the body on another. Such a predictive response is evident in the anticipatory increase in grip force seen when one hand pulls on an object held in the other hand, thereby preventing the object from slipping. It is postulated that such a response depends on predicting the consequences of the descending motor command, as signaled by efference copy, using an internal model of both one's own body and the object. Here we investigate how the internal model learns the temporal consequences of the motor command. We employed two robots to simulate a virtual object held in one hand and acted on by the other. Delays were introduced between the action of one hand on the object and the effects of this action on the other hand. An initial reactive grip force response to the delayed load decayed with the development of appropriate anticipatory grip force modulation. However, no predictive modulation was seen when the object's movement was not generated by the subject, even when the motion was cued by a tone. These results suggest that, when an internal model learns new temporal relationships between actions and their consequences, this learning involves generating a novel response rather than adapting the original predictive response.


    INTRODUCTION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Humans demonstrate different responses to self-produced and externally produced perturbations (Blakemore et al. 1998; Dufosse et al. 1985; Johansson et al. 1992; Johansson and Westling 1988; Massion 1992). For example, when subjects are required to remove an object held in one hand with the other hand, anticipatory deactivation of the forearm muscles occurs before the unloading so that the position of the loaded hand remains unchanged (for a review see Massion 1992). Such anticipatory behavior has been attributed to the ability to predict the consequences of our own actions (Johansson and Cole 1994; Lacquaniti et al. 1992; Massion 1992), a process that requires an internal model of both one's own body and the external world. Such models are known as forward models because they capture the forward or causal relationship between actions, as signaled by efference copy (Jeannerod et al. 1979; Sperry 1950; von Helmholtz 1867; von Holst 1954), and outcome. Forward models have been proposed to play a fundamental role in motor planning, execution, and learning (Jordan 1995; Jordan and Rumelhart 1992; Kawato et al. 1987; Miall and Wolpert 1996; Wolpert 1997; Wolpert et al. 1995). As the dynamics of the body change during growth, and novel objects are encountered, the internal models of both the motor system and external objects must be acquired and continually refined throughout life. The ability to learn and use internal models is, therefore, a fundamental property of the motor system. How such internal models are learned is the focus of the current study.

Grip-force modulation provides an ideal paradigm to study internal model learning. When subjects pick up an object using a precision grip, they exert sufficient grip force to prevent the object from slipping while avoiding excessive grip forces that may result in object breakage or fatigue (Johansson and Cole 1992, 1994; Johansson et al. 1992; Johansson and Westling 1984). When the object is held at rest, the grip force depends both on the weight of the object, that is load force, and the coefficient of friction of its surfaces. Grip force levels can be set without somatosensory feedback, anticipating the physical properties of the object, that include the object's weight, shape, and friction at its surface (Jenmalm and Johansson 1997; Johansson and Cole 1994; Johansson and Westling 1984, 1988). Such object properties are learned through development, indicated by increasing ability to correctly parameterize grip force to the object being manipulated (Eliasson et al. 1995; Forssberg et al. 1991, 1992, 1995). This immediate scaling of the appropriate grip force level has been described as anticipatory parameter control (Johansson 1996; Johansson and Cole 1992, 1994) with the updating of the appropriate grip force level to alterations in the object's properties, occurring by a process of discrete-event, sensory-driven control (for reviews see Johansson and Cole 1992, 1994). When the object is moved by the subject, the load force on the fingers must change to accelerate the gripped object. Without a corresponding change in grip force, the object would slip. In this self-generated condition the grip force tends to parallel load force with negligible delay. Such anticipatory modulation is seen in both discrete (Johansson and Westling 1984) and continuous self-generated movements (Flanagan and Wing 1993, 1995), made in different directions and at different speeds, and when pulling on fixed objects (Johansson et al. 1992). Despite the relatively rapid response of cutaneous afferents, such anticipatory control cannot be explained as a reaction to peripheral feedback (Flanagan and Wing 1995; Johansson and Westling 1984) due to unavoidable feedback delays (Forssberg et al. 1992; Johansson and Westling 1984). A system based solely on feedback control would be ineffective for manipulative actions with frequencies above ~1 Hz, which would exclude some complex skills (Johansson and Cole 1994; Kunesch et al. 1989). This suggests that the skilled manipulation of objects requires the CNS to use the motor command, in conjunction with internal models of both the arm and the object, to anticipate the resulting load force and thereby adjust grip force appropriately (Blakemore et al. 1998; Flanagan and Wing 1997; Wing et al. 1997). Conversely, when the load force is altered by external agents, grip force lags 60-100 ms behind load force, suggesting a reactive response to the perturbation (Blakemore et al. 1998; Cole and Abbs 1988; Johansson et al. 1992; Johansson and Westling 1988). Therefore the timing of grip force modulation can be used to quantify the accuracy of an internal model. Lags between grip and load suggest a reactive process in which the CNS is unable to predict the load force, whereas no lag indicates an accurate internal model.

An integral component of such anticipatory behavior is matching the grip force response temporally to the load force. Johansson and Westling (1988) examined grip force responses when a small ball was dropped into a cup that was gripped by the subject. When the ball was dropped by the experimenter, a reactive grip response occurred 70-80 ms after impact. Conversely, anticipatory grip force responses occurred when the load force was self-generated, that is the subjects dropped the ball, independent of the length of the drop. This suggests that anticipatory grip force modulation can be appropriately timed for different delay (length of drop). Such varying temporal delays are not unusual in motor control. For example, the motor command sent to the hand and foot will have consequences that are both delayed due to conduction times, and also separate in time due to the different conduction distances (Bard et al. 1992).

In the current study we sought to examine how internal models learn these new temporal relationships between the predicted and actual consequences of a descending motor command. Consider a subject holding an object in a precision grip in one hand and generating a force pulse on it with the other. Such a condition leads to anticipatory grip force modulation (Blakemore et al. 1998; Johansson et al. 1992) precisely timed to prevent slippage while using a low grip force level (Fig. 1A). The timing of the response is critical to ensure that the peak in the grip force matches the peak in the load force. In the current study we used two robotic interfaces to examine this task when a delay was introduced between the action of one hand and its consequences on an object held in the other hand. In the normal nondelayed condition, pulling on the object with one hand produced equal and opposite forces on both hands at the same time. This is consistent with the situation in which a single object is held between the hands. In the delayed condition the movement of one hand resulted in the production of the corresponding force on the other hand after a delay of 250 ms (Fig. 1B). On the introduction of the delay, we expect to see two peaks in the grip force response. The first is the anticipatory response to the predicted load force that would normally have occurred without delay as a result of the movement. The second is a grip force reactive to the delayed load acting on the object (Fig. 1B). The internal model that previously allowed predictive modulation, for the nondelayed load, will no longer be appropriate for the task, and consequently may be updated.



View larger version (24K):
[in this window]
[in a new window]
 
Fig. 1. Schematic illustration of the possible adaptation to the effects of delaying the consequences of an action. A, top trace: load applied by one hand to an object held in the other hand. Bottom traces: load experienced by the hand holding the object and its grip force that shows anticipatory modulation. B: the expected immediate effects of introducing a delay between the load applied by the one hand and the load experienced by the other. Two peaks in grip force are expected: the 1st a predictive response and the 2nd a reactive response. C: 2 possible types of adaptation to the delayed consequences of action. See text for details.

By introducing a delay between the action of one hand and the effect on the other, we were able to test between different hypotheses about the adaptation of the internal model. We examined how the grip force modulation changed on repeated presentations of this delayed consequence of action. We can consider two hypotheses of how the internal model learns to produce a predictive response for the delayed load. The first possibility is that the original predictive peak shifts with learning to the new later time of peak load force while the second, reactive, peak decreases. This would be observed as a migration of the first grip force peak and suggests an adaptation of the existing internal model (Fig. 1C, Migration of peak 1). Alternatively, a new predictive peak to the delayed load force could develop, implying the development of a new internal model, appropriate for the novel situation. This would be reflected by a migration of the second grip force peak caused by the new predictive component becoming increasingly predominant, coupled with a decrease in the reactive component of the response. (Fig. 1C, Migration of peak 2).

To examine the specificity of the anticipatory response to the prediction of the consequences of the motor command, we also examined another sensorimotor context in which the object was subject to an externally produced force pulse a fixed interval after a tone. In this "cued" condition, as in the self-generated condition, the load force is predictable, but is now related to a purely sensory stimulus rather than the descending motor command. Both the self-generated and cued conditions were compared with a "noncued" condition in which the object was pulsed at pseudorandom intervals to assess purely reactive grip force responses. The current study is an investigation of adaptation to delays between self-produced actions and their effects. This is in contrast with the studies of conditioning to external sensory stimuli. The cued condition performs the role of a control in the current study.

To assess the predictive component of the grip force response isolated from any effects of sensory feedback from the finger pads, catch trials were randomly interspersed. During a catch trial, in both the cued and self-generated conditions, no load force was generated. This allowed an analysis of the grip force modulation that occurs solely as a consequence of the internal model.


    METHODS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Subjects

Six right-handed subjects (3 male, 3 female) aged 21-28 yr participated in the experiment. None of the subjects reported sensory or motor deficits. Subjects gave written informed consent before participating in the experiments and were naïve to the research aims.

Apparatus

A six-axis cylindrical force transducer (Nano, ATI) was embedded in a cylindrical object with two parallel grip surfaces of 20 mm diam, spaced 40 mm apart. The two surfaces were covered with sandpaper (Grade No. 210) with the mass of the transducer (50 g) centred halfway between the surfaces. The force transducer allowed three translational forces to be measured with an accuracy of 0.05 N including cross-talk. This object was attached to the end of lightweight, robotic manipulator (Phantom Haptic Interface, Sensable Devices). A second cylinder, of identical size and mass to the first, was attached to a second robotic manipulator. These robots, which are free to move in three dimensions, can exert forces of up to 20 N, in any direction, at its endpoint (backdrive friction 0.02 N, apparent mass at the tip <150 g). The position of the motors (and through the kinematic equations of the robot the position of the object) were sampled on-line by three optical encoders (10,160 counts per revolution, sampling rate 1,000 Hz) mounted on the three motors.

Procedure

All subjects participated in three conditions in a balanced order. In all conditions, subjects were required to keep their right hand still, preventing the gripped object slipping from their grasp. The object was gripped with the tips of their right thumb and index finger. The subject's right forearm was anchored with velcro straps, and for further stability, they grasped a vertically oriented aluminum rod with their three ulnar fingers. This ensured that the subject's thumb and index finger were used to maintain object stability rather than a more general postural response. In each trial a discrete upward force was applied by the right hand robot to the object. To prevent fatigue, short rest periods were given every 60 trials in all conditions.

Noncued condition

To measure reactive grip force modulation, the right hand robot generated 50 force pulses on the object held in the right hand at pseudorandom times, on average one pulse every 2.5 s (Fig. 2B). Subjects held the left hand object above the right hand object and stationary throughout. The temporal profile of each force pulse was chosen to be smooth and bell shaped. Specifically, the force profile had the same shape as the velocity profile of a minimum jerk movement (Flash and Hogan 1985). The duration of the force profile was 200 ms and its maximum amplitude was 5 N. This therefore generated a peak load force on the fingers of the right hand of 5 N. This force profile was chosen to match that produced by the subjects in the self-generated condition described below.



View larger version (62K):
[in this window]
[in a new window]
 
Fig. 2. A: schematic diagram of the apparatus used to create a virtual object using the 2 robots. Schematics of the noncued condition (B) in which the object in the right hand was pulsed upward at pseudorandom intervals. C: self-generated condition in which the left hand generated an upward movement that caused an upward force on the object in the right hand. D: cued condition in which the object in the right hand was pulsed upward a fixed interval after a tone sounded.

Self-generated condition

In the self-generated condition the load force generated by the robot on the object held in the right hand was produced as a result of the subjects making a single vertical upward pulse with their left hand (Fig. 2C). Subjects were told that the load on the right hand object was produced by this vertical movement of their left hand. Subjects held the object in the left hand above the object in the right.

The relation between movement of the left hand and the force generated by the robot on the right hand object was simulated as a stiff spring between the objects in the two hands. These forces therefore mimicked the condition of a single object held between the hands. The force on the right hand was given by Ft K (Lt - Rt - D) where Lt and Rt were the vertical positions of the left and right object at time t, K was a fixed spring constant of 15 N/cm, and D was the initial vertical distance between the objects at the start of the trial. Therefore to achieve the 5-N load force the movement of the left hand was ~3 mm in amplitude. The force on the left hand was equal and opposite to the force on the right hand. To allow the subjects to produce the correct load force amplitude, their left hand position was displayed as a vertically moving (but horizontally stationary) dot on an oscilloscope. The oscilloscope also displayed two horizontal lines that represented the amplitude of movement required to generate 5 N.

On average once every 2.5 s a tone sounded and the subjects had to move their left hand sharply up and down to cause the desired brief 5-N target force pulse to be applied to the right hand object. Subjects were given ~30 practice trials, which enabled them to accurately produce a movement with their left hand that could generate a 5-N amplitude vertical force pulse on the object held in their right hand. This movement was recorded to ensure that subjects produced the appropriate movement throughout the experiment.

The paradigm comprised of 3 phases of 50 preexposure, 250 exposure, and 50 postexposure trials, producing 350 trials in total. In the preexposure phase there was no delay between the vertical movement of the left hand and the production of a vertical force on the object in the right hand. In the exposure phase a delay of 250 ms was introduced between the upward movement of the left hand and the production of the vertical force on the right hand object. This was chosen to be long enough to resolve the two hypothesized grip force peaks. For the postexposure phase, this delay was removed so that these trials were identical to those of the preexposure phase. After the first 20 trials, one catch trial was interspersed pseudorandomly in every 10 trials. In a catch trial, movement of the left hand did not cause a vertical force on the right hand object.

Cued condition

In the cued condition subjects held the object in the left hand above the object in the right as in the self-generated and noncued condition (Fig. 2D). As in the noncued condition the left hand was stationary throughout. On average once every 2.5 s a tone sounded, and 300 ms later an upward pulse of force was generated by the right hand robot on the object held in the right hand. This 300-ms period between the tone and the upward pulse of force was chosen because it is greater than the reaction time to a tone. The temporal profile of this force pulse was identical to that of the noncued condition.

The paradigm comprised of three phases of 50 preexposure, 250 exposure, and 50 postexposure trials: 350 trials in total. In the preexposure phase the vertical force on the right hand always occurred 300 ms after the tone. In the exposure phase a delay of 250 ms was introduced so that the force now occurred 550 ms after the tone. The additional delay of 250 ms between preexposure and exposure phases was the same for the cued and self-generated conditions. Within the final postexposure phase, this delay was removed so that these trials were identical to those of the preexposure phase. Catch trials, in which the tone sounded but no vertical force on the right object was produced were interspersed as in the self-generated condition.

Analysis

For each trial, the position of the left hand, the load (total force tangential to the surface) and grip force were recorded at 250 Hz and then filtered using a Butterworth 5th-order, zero phase lag, low-pass filter with a 10-Hz cutoff.

As a global measure of anticipatory grip force learning, zero lag cross-correlations (r) were performed between grip and load force for each trial. The square of this cross-correlation (r2, the coefficient of determination) was used to quantify the co-variation of the grip and load force. To further quantify the evolution of the grip force response, the amplitude and time, relative to the peak in load force, of the first and second (if present) grip force peaks were found for each trial. In addition, the magnitudes of the grip at 0 and 250 ms (the time of peak load force in different phases of the experiment) were calculated. These measures identify the most important features of the evolving grip force profile.

These dependent measures were analyzed separately for the catch and noncatch trials using repeated measures ANOVAs (with Bonferroni corrections). First, the subjects' average of these measures for the 50 preexposure trials were analyzed as a function of condition (self-generated, cued, and noncued). To examine the changes over the experiment, a repeated measures MANOVA was performed for the dependent measures as a function of condition (self-generated and cued) and time as factors. Time was split into four levels: pre (1st 50 trials before the delay was introduced), early exposure (1st 20 exposure trials as adaptation was fast), late exposure (last 50 exposure trials), and post (50 postexposure trials). As the first peak disappeared by the end of the exposure phase, we examined its disappearance by performing linear regression over the exposure period. All the statistics are based on an analysis on a trial-by-trial basis. For clarity, the plots shown are averages of these trial-by-trial values, averaged across batches of 10 trials and over subjects.


    RESULTS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Subjects found the task easy to perform and were able to generate consistent levels of load force. For the self-generated condition, subjects produced bell-shaped movement as required throughout all phases of the experiment (Fig. 3).



View larger version (28K):
[in this window]
[in a new window]
 
Fig. 3. Left hand position averaged over all subjects for the different phases of the self-generated condition. Each plot is the average of 1 catch trial and 9 noncatch trials per subject. The solid vertical line indicates the time of the peak load force. A: preexposure, trials 40-50. B: early exposure, trials 70-80. C: mid-exposure, trials 150-160. D: late exposure, trials 280-290. E: early postexposure, trials 300-310. F: late postexposure, trials 320-330.

Preexposure

When the load force was noncued, a reactive grip-force modulation was seen with an average grip-load r2 of 0.29. The peak grip force lagged the peak load by an average of 73 ms (Fig. 4A). In contrast, when the load force was self-generated, a predictive pattern of grip force was seen with a grip-load r2 of 0.73 (Fig. 4B). In this condition, the peak grip force lagged only 26 ms (Fig. 4B) behind the peak load, a significant decrease compared with the noncued condition (P < 0.05). The cued condition showed a grip-force response that was not significantly different from the noncued condition with a grip-load r2 of 0.30 and a peak grip that lagged the peak load by 69 ms (Fig. 4C). Therefore anticipatory grip force modulation was only seen when the load force was self-generated by movement of the other hand.



View larger version (13K):
[in this window]
[in a new window]
 
Fig. 4. Grip force modulation averaged over all subjects for the preexposure phase (trials 20-50) for normal (---) and catch trials (- - -). Traces are aligned to the peak in load force at 0 ms noncued (A), self-generated (B), and cued (C) conditions. Average tangential load forces are shown by dotted lines. Note that the small peaks in the load force after the main peak in the noncued and cued conditions, which occur at the same time as the peak in grip, are due to the grip force the subjects generated not being perfectly orthogonal to the grip surfaces.

The predictive nature of the grip force modulation was assessed by examining the modulation during catch trials in which there was no load force generated by the robot on the object held in the right hand. Any grip-force modulation therefore is purely predictive because no sensory stimulation of the fingers is generated in the absence of the load. Within the self-generated condition, grip force modulation was seen during the catch trials (Fig. 4B, - - -). This predictive component had a peak that was significantly (P < 0.05) earlier at -1 ms compared with the 26-ms lag of the noncatch trials. The peak of grip force in the catch trial was also significantly smaller than in the noncatch trials (P < 0.05). This suggests that sensory feedback of load acts to delay and amplify the peak grip-force response. During the catch trials in the cued condition, the grip-force modulation was erratic and of low amplitude and was not consistent with a predictive strategy.

Self-generated condition

In the preexposure phase of the self-generated condition, peak grip force lagged an average of 26 ms behind the load force (Fig. 5A). On the introduction of a 250-ms delay between the motion of the left hand and the load force generated on the object held in the right, two peaks in grip force modulation were seen (Fig. 5B). As expected the first peak corresponded to the grip force response seen in the preexposure catch trials (because the 1st part of the trial is equivalent to a preexposure catch-trial). The second peak initially lagged an average of 59 ms (average over the subjects' 1st exposure trial) behind the peak load force. This represents a reactive grip-force modulation to the unexpectedly delayed load force. Catch trials in the early exposure trials showed a single peak at around 0 ms.



View larger version (43K):
[in this window]
[in a new window]
 
Fig. 5. Grip force modulation averaged over all subjects for the different phases of the self-generated condition. Each plot is the average of 1 catch trial (- - -) and 9 noncatch trials (---) per subject (---). Traces are aligned to the peak in load force (solid vertical line). A: preexposure, trials 40-50. B: early exposure, trials 70-80. C: mid-exposure, trials 150-160. D: late exposure, trials 280-290. E: early postexposure, trials 300-310. F: late postexposure, trials 320-330.

As exposure to the novel temporal arrangement continued, the amplitude of the first peak slowly decayed, eventually becoming the nondominant modulation (Fig. 5, C and D). By mid-exposure, modulation began to appear close to the time of actual load force, with this eventually becoming the predominant grip force peak (Fig. 5C). Late in the exposure, the catch trials produced a single grip force peak, predictive of the delayed load force (Fig. 5D). Removal of the delay in the postexposure phase resulted in grip force modulation quickly returning to the preexposure response (Fig. 5, E and F). These features can also be seen in the individual subject's traces shown in Fig. 6, A-C (---) for the preexposure, early exposure, and late exposure phases.



View larger version (38K):
[in this window]
[in a new window]
 
Fig. 6. Grip force modulation for each subject (columns S1-S6) for the different phases of the self-generated (---) and cued condition (- - -). Each plot is the average of 10 noncatch trials. Traces are aligned to the peak in load force (solid vertical line). A: preexposure trials (40-50). B: early exposure trials (70-80). C: late exposure trials (290-300). For clarity the cued-condition traces have been offset by 4 N.

These changes and subsequent learning of the new temporal delay are reflected in the grip-load r2 (Fig. 7A). On the introduction of the delay, r2 fell from 0.73 to 0.33 (average of 1st 10 exposure trials). Over the exposure phase, r2 increased significantly (P < 0.05), rising to 0.58. With the removal of the temporal delay, r2 transiently dropped to 0.51 (1st 10 postexposure trials) before returning to a value of 0.72 (last 40 postexposure trials) similar to the preexposure (0.73). These changes reflect that there is learning taking place over the exposure phase.



View larger version (13K):
[in this window]
[in a new window]
 
Fig. 7. Coefficient of determination (r2) of grip with load force against trial number averaged over batches of 10 trials and subjects for the self-generated condition (A) and cued condition (B). Error bars are 1 SE. The vertical solid line indicates the start and end of the exposure phase.

To characterize how this learning occurs, we examined magnitude and timing of the first and second peaks in the the grip force response. In addition, the magnitudes of the grip at 0 and 250 ms, the time of the peak load force during the preexposure and exposure phases, respectively, were examined. These measures identify the most important features of the evolving grip force profile. During the exposure phase the amplitude of the grip force modulation at the first peak decreased significantly (P < 0.05, Fig. 8A) as did the magnitude at 0 ms. However, the time of the first peak did not change significantly (not shown). The height of the second grip peak decreased by 16% over the exposure phase. The time of this second peak (Fig. 8C) shifted significantly toward the time of maximum load force (P < 0.01) so that over the last 50 trials of the exposure period lagged an average of 28 ms behind the load peak as in the preexposure phase. This represents a 37% decrease in the average lag of the second peak compared with the first 20 trials of the exposure phase. This temporal advance coupled with a decreasing amplitude kept the grip force constant at the peak of the load force (250 ms), while allowing the total force exerted to decrease. Thus the grip force at 250 ms did not change significantly during the exposure period. However, when a second discernible peak appeared in the catch trials, its time did not change significantly (Fig. 8C). Over the exposure phase this peak in the catch trials grip was on average 2 ms in advance of the time of peak load. In summary, although the first grip peak only decreased in amplitude, without a change in timing, the second grip peak systematically decreased in both amplitude and lag, consistent with a change from a reactive to a predictive grip force modulation.



View larger version (44K):
[in this window]
[in a new window]
 
Fig. 8. Grip force peak magnitude and time against trial number averaged over all subjects for the self-generated condition (A-C) and for the cued condition (D-F). A and D: magnitude of 1st grip force peak. B and E: magnitude of 2nd grip force peak. C and F: time of 2nd grip force peak. Each point is the average of values from 9 noncatch trials per subject (black-lozenge ) or 1 catch trial ().

After the removal of the delay between the motion of the left hand and the load force generated on the right, grip force modulation quickly reverted to being comparable to that grip force modulation during the preexposure phase. The peak of the grip force lagged the load force by 35 ms, a value not significantly different from the 26-ms lag in peak grip force modulation within the preexposure phase.

Cued condition

In the preexposure phase of the cued condition, peak grip force lagged an average of 70 ms behind the load force (Fig. 9A). On the introduction of an additional 250-ms delay between the tone and the load force generated on the object held in the right hand, the grip force immediately shifted to lag the now delayed load force by 81 ms (average of subjects' 1st exposure trial, Fig. 9B). This lag did not significantly decrease over the exposure batch, (Fig. 9, B-D). Removal of the delay in the postexposure phase resulted in grip force modulation returning to the preexposure response (Fig. 9, E and F). The lack of anticipatory behavior is reflected in the low and unchanging grip-load r2 of around 0.32 (Fig. 7B).



View larger version (44K):
[in this window]
[in a new window]
 
Fig. 9. Grip force modulation averaged over all subjects for the different phases of the cued condition. Each plot is the average of 1 catch trial (- - -) and 9 noncatch trials (---) per subject. Traces are aligned to the peak in load force (solid vertical line). A: preexposure, trials 40-50. B: early exposure, trials 70-80. C: mid-exposure, trials 150-160. D: late exposure, trials 380-290. E: early postexposure, trials 300-310. F: late postexposure, trials 320-330.

To characterize the effects of the delay on grip force modulation, the cued condition was analyzed in the same way as the self-generated condition (Fig. 7B) and (Fig. 8, D-F). In general, these plots, which are shown for completeness, showed no systematic changes. There was no significant change in either the amplitude or timing of the grip force modulation during the exposure phase. In particular, the second peak did not become earlier over this period. Although there were discernible peaks in the catch trials, they occurred at random times with a standard deviation in the exposure phase of 88 ms, compared with a standard deviation of 49 ms for the self-generated condition (compare the exposure catch trials in Fig. 8, C and F) and had very small amplitude compared with the noncatch trials (compare the exposure catch and noncatch trials in Fig. 8, B and E). For all phases of the cued condition, the grip force showed a purely reactive modulation.


    DISCUSSION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

In this experiment we compared the grip force responses seen when an object held in a precision grip is pulled on either by the action of the other hand (self) or externally by a robot unpredictably (noncued) or predictably (cued by a tone). Anticipatory grip force modulation was only seen when the load force was self-generated, grip force lagged load by only 26 ms, consistent with the previous finding of an anticipatory grip force response when a perturbation is self-generated (Johansson and Westling 1988). In contrast, in both the cued and noncued conditions grip force modulation was solely reactive in response to the imposed load force, with a grip force lag of 69 and 73 ms, respectively. This lag in grip force modulation is consistent with those previously measured for an externally generated load (Flanagan and Wing 1993; Johansson et al. 1992; Johansson and Westling 1988). When a temporal delay was artificially introduced between the pull exerted by one hand and the consequential production of the load force on the object held in the other hand (self), a grip force profile with two peaks was initially seen. This comprised an early grip force peak, appropriately timed for a nondelayed load, followed by a grip force reactive to the now delayed load force. Over the course of 250 trials, the grip force became anticipatory while the early, and now inappropriate, grip force peak decreased in amplitude. Conversely, when a temporal delay was introduced between the tone and the load force produced by the robot pull (cued), the grip force remained purely reactive in response to the load force.

Performance in the absence of a delay

Here it is argued that the crucial difference between the self-generated and cued conditions in the present study is that in the self-generated condition the CNS uses the motor command, in conjunction with internal models of both the arm and the object, to anticipate the resulting load force and thereby adjust grip force appropriately. In both the cued and noncued conditions the load force is not self-produced, therefore this strategy cannot be employed. Whereas in the noncued condition, anticipation is impossible; anticipatory responses could have been seen in the cued condition as the load followed the tone at a predictable time. The lack of anticipatory responses to the tone suggests that this purely sensory cue or context is insufficient to allow prediction of the load force.

These results are in agreement with previous studies in which the position of a subject's arm was measured in an unloading task (Dufosse et al. 1985; Paulignan et al. 1989). When the load supported by a subject on their hand was removed, a differential effect occurred dependent on the nature of the unloading. When the load was removed with the contralateral hand, there was associated anticipatory control of the position of the load-bearing forearm that acted to minimize the effects of the perturbation. However, no corresponding anticipatory action was found when unloading was performed externally, even if it was preceded by a warning tone, or triggered by the subject themselves. Duffose et al. (1985) concluded that temporal cues regarding the precise time that the unloading occurred is, by itself, inadequate information for anticipation.

Blakemore et al. (1998) have shown that efference copy and sensory feedback must be consistent with a specific context to see predictive modulation. In their experiment predictive grip force modulation was present when the feedback experienced by two hands was consistent with an internal model of a single object. Here we have shown that an internal model can be learned in which the consequences of a motor action are delayed, but cannot be learned for a sensorimotor context in which a tone determines the consequence, within the time scale of this experiment. Although the action of one hand often influences objects held in the other it is less clear that tones, in everyday life, influence objects held in the hand. Therefore it may be that internal models for grip force modulation can adapt to new dynamics relating the motion of one hand to the other, but arbitrary pairings such as tone with load force (cued condition) cannot be incorporated into a new internal model.

In the present study, examination of grip force responses during catch trials, in which there was no load force generated on the right hand object in any of the conditions, further elucidates the differences between the cued and self-generated conditions. Because there is no sensory stimulation of the finger tips in these trials, any response must be a purely predictive one. Differences in these trials between the conditions can therefore be associated with corresponding differences in prediction. Within the cued condition, there was no consistent grip force modulation during the catch trials. This finding is consistent with the grip force modulation being solely reactive to the imposed load force. However, in the self-generated condition, grip force modulation was seen during the catch trials (Fig. 4B). The peak of this grip force modulation, on average, led the peak load force by 1 ms. The presence of this accurately timed modulation, in the absence of sensory stimulation, demonstrates that in this self-generated condition at least part of the grip force response is anticipatory in nature.

Differences between the grip force response in the catch and noncatch trials in the same condition are attributable to the consequences of sensory stimulation of the finger tips and hand, which is only present in the noncatch trials. In the cued condition, sensory stimulation in these noncatch trials resulted in a reactive grip force peak. However, in the catch trials in this same condition, no grip force modulation is seen. In the self-generated condition, however, the effect of sensory feedback was twofold. First, the peak of the grip force was now delayed compared with the catch trials and occurred on average 26 ms after the peak load force. Second, the amplitude of the grip force modulation was amplified over that of the catch trials by ~13% (Fig. 4B). The differences between the grip force modulation in the catch and noncatch trials in this self-generated condition can be viewed in two ways. First, the grip force seen in the noncatch trials could be made up of a mixture of predictive and reactive processes. This would indicate that the grip force modulation seen in the catch trials is only the anticipatory component of the grip force response. An alternative view is that the grip force modulation seen in the noncatch trials represents a purely anticipatory response that is suppressed if there is no confirmatory sensory feedback signaled by load on the finger pads. For example, if the expected load force is not experienced, as in the catch trials, then the prediction is judged inaccurate and further anticipatory grip force is curtailed leading to a smaller and earlier peak in grip force modulation as observed. Such a view is consistent with the multiple internal model approach (see Modularity and internal models).

Performance in the presence of a delay

During the exposure phase of both the self-generated and the cued conditions, a temporal delay of 250 ms was added to the expected time of the load on the right hand. When this delay was added to the cued condition, the time between the tone and the production of the load increased, but the peak of the grip force lagged the now delayed load force during exposure by an average of 74 ms (Fig. 8F). As in the preexposure phase of this condition, this is consistent with grip force modulation in the cued condition being purely reactive to load force.

When this delay was added to the self-generated condition, the pull of the left hand did not immediately produce a load on the right hand, which would be consistent with the expected forces for a solid object held between the hands. Instead the load was delayed by 250 ms. Initially, two peaks in grip force were seen in the noncatch trials (Fig. 5B). This conforms to what was expected (Fig. 1B). The first peak appeared at the time of the anticipatory grip force of the preexposure phase, in which there was no delay between the pull of one hand and the load on the other. The second peak is identified as a reactive peak, to the now delayed load force. In these first exposure trials, this peak lagged the load force by 59 ms.

The evolution of the double-peaked grip force profile over the course of the exposure phase allows us to select between the two possible hypotheses described in the INTRODUCTION. These two hypotheses both propose that the second peak will become predictive, but differ as to its origin. The first hypothesis suggests that this peak could be caused by the first grip force peak moving to the time of the peak load force, with the later reactive peak decaying, consistent with adaptation of the existing internal model (Fig. 1C, Migration of peak 1). The second hypothesis is that a new predictive peak to the delayed load force could develop, implying the development of a new internal model, appropriate for the novel situation. This would be reflected by a migration of the second grip force peak caused by the new predictive component becoming increasingly predominant, coupled with a decrease in the reactive component of the response. (Fig. 1C, Migration of peak 2). Analysis of the changing amplitude and time of the peaks over the 250 exposure trials showed that the second peak significantly shifted toward the time of peak load force (Fig. 8C), while the first peak remained at its original time and decreased in amplitude (Fig. 8A), thereby supporting the latter hypothesis. This hypothesis proposes that the second grip force peak is formed from two components. The first is a reactive component that decreases in amplitude with learning. The second component is a new predictive component that can be seen in isolation in the catch trials. When this predictive grip force peak is seen in the catch trials, it has almost no lag with respect to the load force. As the predictive modulation grows and the reactive component declines, the peak of the total grip force migrates to a predictive location.

Modularity and internal models

To produce an appropriate grip force response in one hand, in response to the motion of the other hand, it has been suggested that a forward internal model is used. A forward internal model (Jordan 1995; Jordan and Rumelhart 1992; Kawato et al. 1987; Miall and Wolpert 1996; Wolpert 1997; Wolpert et al. 1995) captures the forward or causal relationship between actions, as signaled by efference copy (Jeannerod et al. 1979; Sperry 1950; von Holst 1954), and predicts outcomes such as load force. Based on this prediction an appropriate anticipatory grip force can be generated despite sensory feedback delays associated with the detection of load force by the fingertips (Johansson and Westling 1984).

Forward models depend on the dynamics of the limb as well as the properties of external objects. As the dynamics of the limb change throughout life and many new objects are experienced, forward models must be adaptable. The training signals required to update such a model are readily available, the difference between predicted and actual outcome. In the present study we have shown how the CNS adjusts for a system in which a delay of 250 ms is introduced between the action of one hand and its effect on an object held in the other.

We can consider two alternative ways in which this could be achieved. The first assumes a single internal forward model that adapts to the new delay. Such an internal model would have to readapt to every object, even when the object has been previously experienced, leading to transient errors in grip force production. However, it is known that when handling multiple objects anticipatory grip force modulation is seen even on the first trial (Gordon et al. 1993). Alternatively, a modular approach can be used. We have proposed such a modular system for object manipulation in which multiple internal forward models coexist (Blakemore et al. 1998; Wolpert and Kawato 1998).

A specific modular architecture known as the multiple paired forward-inverse model (Wolpert and Kawato 1998; Wolpert et al. 1998) has been proposed for motor learning and control. The switching process between modules is determined by two distinct processes. The first uses sensory contextual cues to determine which module to use before movement initiation. For grip force, contextual cues include object weight and shape as judged visually (Jenmalm and Johansson 1997; Johansson and Westling 1988), experience from previous lifts (Gordon et al. 1993), and perceived friction from the objects' surface (Johansson and Westling 1984, 1987). Once the movement is initiated the second process uses the forward model's predictions. As each forward model captures a distinct dynamic behavior of the limb or object, their prediction errors can be used during movement to determine in which context the motor system is acting. The signal derived from the forward models can be used to adjust the prior selection of the modules that occurs before the movement is initiated and sensory feedback is available. For example, for an object that appears heavy, prior selection will predict that a large grip force is needed to lift the object. However, feedback processes, based on comparing the predicted (from the forward model) with the actual consequences of action (such as hand acceleration) can indicate that the object is in fact light thereby using a module appropriate for a lower grip force. These feedback processes are used to adjust the predictive feed-forward control and are separate from the feedback control processes, which can provide reactive responses.

We propose that multiple forward models are in operation in grip force modulation. The changes during the exposure phase of the self-generated condition were consistent with two internal forward models coexisting. The first is a model suitable for handling a real physical object in which there is no delay between the action on it and its response, and we would expect this model to have been acquired early in life by the subject. However, subjects would have to learn a new model for the condition in which a delay is introduced between action and effect, because this is likely to be the first time subjects have experienced such an object. The decrease in the first grip force peak over the course of learning is consistent with the internal model appropriate for no delay being slowly switched off. Similarly, the decrease in lag of the second peaks is consistent with a new model being learned appropriate for the 250-ms delay. The multiple model predicts the pattern of changes seen in the second hypothesis described in the INTRODUCTION, the formation a new internal model allowing predictive grip force modulation appropriate for the novel situation, with both the first peak and the later reactive peak decreasing in amplitude as the predictive response grows (Fig. 1C, Migration of peak 2). This pattern of change was observed. Furthermore, the differences seen between the catch and the noncatch trials can be attributed to the feedback process, which is a comparison of the predicted (from the forward model) and actual sensory feedback curtailing the feed-forward response due to the absence of the predicted sensory feedback. The rapid change back to the preexposure pattern in the postexposure phase may reflect a fast switching process back to a previously learned module.

In summary, these results suggest that the internal model is able to learn new temporal relationships between actions and their consequences, and such learning may involve generating a novel response rather than altering the original predictive response.


    FOOTNOTES

Address for reprint requests: D. M. Wolpert, Sobell Department of Neurophysiology, Institute of Neurology, University College London, Queen Square, London WC1N 3BG, UK.

The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Received 29 December 1999; accepted in final form 1 June 1999.


    REFERENCES
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

0022-3077/99 $5.00 Copyright © 1999 The American Physiological Society