Motor Cortical Activity During Drawing Movements: Population Representation During Lemniscate Tracing

Andrew B. Schwartz and Daniel W. Moran

The Neurosciences Institute, San Diego, California 92121


    ABSTRACT
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Schwartz, Andrew B. and Daniel W. Moran. Motor Cortical Activity During Drawing Movements: Population Representation During Lemniscate Tracing. J. Neurophysiol. 82: 2705-2718, 1999. Activity was recorded extracellularly from single cells in motor and premotor cortex as monkeys traced figure-eights on a touch-sensitive computer monitor using the index finger. Each unit was recorded individually, and the responses collected from four hemispheres (3 primary motor and 1 dorsal premotor) were analyzed as a population. Population vectors constructed from this activity accurately and isomorphically represented the shape of the drawn figures showing that they represent the spatial aspect of the task well. These observations were extended by examining the temporal relation between this neural representation and finger displacement. Movements generated during this task were made in four kinematic segments. This segmentation was clearly evident in a time series of population vectors. In addition, the <FR><NU>2</NU><DE>3</DE></FR> power law described for human drawing was also evident in the neural correlate of the monkey hand trajectory. Movement direction and speed changed continuously during the task. Within each segment, speed and direction changed reciprocally. The prediction interval between the population vector and movement direction increased in the middle of the segments where curvature was high, but decreased in straight portions at the beginning and end of each segment. In contrast to direction, prediction intervals between the movement speed and population vector length were near-constant with only a modest modulation in each segment. Population vectors predicted direction (vector angle) and speed (vector length) throughout the drawing task. Joint angular velocity and arm muscle EMG were well correlated to hand direction, suggesting that kinematic and kinetic parameters are correlated in these tasks.


    INTRODUCTION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Changes in kinematic variables characterize the behavior expressed by volitional movement. The structure of all movement is determined by the behavioral goal to be achieved. For example, the time course of velocities taken by the hand when swatting a fly is characteristically different from that when reaching for a glass, even though the path of the hand may be identical. The trajectory of the hand is especially important in drawing movements where the behavioral goal is the path taken by the hand. Subjects tend to select a particular pattern of kinematic parameters from an infinite set that would result in a desired hand path. These consistent patterns are characterized by invariants or rules determined by the neural substrate generating the movement. For instance, there is no reason to expect that within the motor system, the spatial description of the path would be linked to the temporal evolution of the movement. Yet psychophysical results suggest that this is an important aspect of the trajectory planning process. An initial study of handwriting (Viviani and Terzuolo 1982) showed that hand speed and the radius of curvature at each point in the cursive script were inversely related over discrete segments. The slope between these variables changed sharply at segment boundaries located at points in the trajectory where curvature was at a local minimum. A later analysis (Lacquaniti et al. 1983) showed the speed-curvature relation to be exponential. Movement speed was proportional to the radius of curvature raised to the <FR><NU>1</NU><DE>3</DE></FR> power. This relation is equivalent to the ratio of angular velocity to curvature (1/radius of curvature) raised to the <FR><NU>2</NU><DE>3</DE></FR> power. Mathematically, this can be represented as
<IT>V</IT>(<IT>t</IT>)<IT>=</IT><IT>kR</IT>(<IT>t</IT>)<SUP><IT>1/3</IT></SUP> (1)

ω(<IT>t</IT>)<IT>=</IT><IT>kC</IT>(<IT>t</IT>)<SUP><IT>2/3</IT></SUP> (2)
where V(t) is tangential velocity, R(t) is the instantaneous radius of curvature, omega (t) is the angular velocity, C(t) is the instantaneous curvature, and k is a proportionality constant. The "velocity gain factor," k, changes between segments and is related to the length of the segment.

These rules, segmentation and the power law, are not the result of the mechanical process moving the limbs, because the rules are followed for isometric drawing tasks (Massey et al. 1992) and even seem to be an important component in the perception of moving objects (Fagg et al. 1992; Soechting et al. 1986). Furthermore, the value of the exponent (<FR><NU>2</NU><DE>3</DE></FR>) is not determined by an obligatory relation between kinematic variables because it varies in children (however, it is constant in adults) (Sciaky et al. 1987; Viviani and Schneider 1991). The two rules are linked because the transition between segments is delineated by changes in the velocity gain factor k (Soechting and Terzuolo 1987b). With drawings in free space, motion is confined to a plane within a segment, but switches to a different plane between segments (Soechting and Terzuolo 1986, 1987a). Although the plane of movement and the velocity gain factor change instantaneously, there are no abrupt changes in joint torques or electromyographs (EMGs) at the segment boundaries.

The observed figural-kinematic relationships are produced centrally by the neural structures generating the instructions that cause the arm to move. Our previous work has shown that the hand's trajectory is well represented in a population of motor cortical cell activity (Moran and Schwartz 1999b; Schwartz 1993, 1994). Although this activity predicts the hand's trajectory accurately, it is unlikely that the motor cortex is the only structure responsible for the trajectory structure because there are reports of directional tuning with arm movements in a large number of other sensory and motor structures (Bosco and Poppele 1993; Fortier et al. 1989; Graziano et al. 1994; Kutz et al. 1997; Ruiz et al. 1995; Turner and Anderson 1997). Furthermore, although we are dealing with kinematic variables measured in an external coordinate system, the correlation of these variables with kinetic variables during drawing suggests that this type of distinction between classes of variables may not be pertinent in classifying neuronal activity patterns during natural behaviors. In a previous paper (Moran and Schwartz 1999b) we showed that arm joint angles and EMG were highly correlated to hand velocity. This was even more evident in the present results.

This paper, the final in a set of three, examines the interrelation of speed and direction as a figure is drawn and their representations in the activity of motor cortical neurons. The first paper showed that speed and direction could be encoded simultaneously in the activity of a single unit. The prediction interval, defined in the second paper as the time interval between the direction of a population vector and the direction of the movement velocity, varied as a function of curvature in a spiral. Here we show that the prediction interval based on direction varies with curvature within each segment of a figure-eight. Although this prediction interval was highly modulated, a prediction interval based on speed was much less so and could be represented with a constant value. Curvature defines the way a movement is segmented, and this segmentation is obvious in the neuronal firing patterns. The neuronal activity, its timing relative to the movement, the joint angles, EMG, and hand velocity all show an organization related to movement segmentation. This gives further support to theories suggesting that apparently continuous drawing movements are generated by temporal segments of neural activity (Soechting and Terzuolo 1987a; Viviani 1986; Viviani and Cenzato 1985; Viviani and Flash 1995).


    METHODS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Most of the methods used in these studies have been detailed in previous reports (Moran and Schwartz 1999b; Schwartz 1992-1994); only those that are unique to this study will be described here.

Behavioral task

The behavioral apparatus and basic approach are the same as those described in the preceding paper (Moran and Schwartz 1999b). A lemniscate was graphed on a touch-sensitive computer monitor. Superimposed on the static figure was an animated circle (1 cm radius) that moved along the figure each time the finger moved to it. In this way, the animal controlled the speed of the circle, which was always just ahead of the finger. The circle regulated the movement tolerance, which in this case was 1 cm. Each figure-eight was traced as four classes (vertical and horizontal orientation, clockwise, and counterclockwise) presented in five randomized blocks. The projected Lissajous lemniscate was 12 × 12 cm, but with the tolerance of the moving circle, the animals drew figures that were slightly larger than 11 × 11 cm.

Data analysis

Curvature was calculated for both the hand and the neural trajectories using Eq. 3.
<IT>C</IT>(<IT>t</IT>)<IT>=</IT><FR><NU><IT><A><AC>x</AC><AC>˙</AC></A><A><AC>y</AC><AC>¨</AC></A></IT><IT>−</IT><IT><A><AC>x</AC><AC>¨</AC></A><A><AC>y</AC><AC>˙</AC></A></IT></NU><DE>(<IT><A><AC>x</AC><AC>˙</AC></A></IT><SUP><IT>2</IT></SUP><IT>+</IT><IT><A><AC>y</AC><AC>˙</AC></A></IT><SUP><IT>2</IT></SUP>)<SUP><IT>3/2</IT></SUP></DE></FR> (3)
The single and double dot symbols above the variables represent first and second time derivatives, respectively. Derivatives were calculated using spline functions (csakm IMSL, Visual Numerics, Houston, TX) or by differentiating and smoothing with a double-sided, five point exponential routine.

All comparisons between neuronal and behavioral data in this paper are carried out with vector quantities. The neuronal data are represented by population vectors and arm movement data with velocity vectors. Because the time-varying processing associated with continuous drawing movements is of interest here, each task is divided into a time series of vectors, and comparisons are made between corresponding population and velocity vectors. In addition to showing (as we have in the 2 previous papers in this series) that the population vector is an accurate prediction of the velocity vector, we explore here the relation of the time interval between the neuronal and movement vectors and how this prediction interval (PI) is related to vector directions and vector magnitudes. The directional PI was found by applying a spline function to the time series of movement vector directions and interpolating between the 100 values to increase the total number of points to 10,000. A search was then performed for each of the population vector directions to find the nearest match (within 8 bins; ~160 ms) to the interpolated movement direction. The search was halted when the match was within a criterion (0.0005 radians). If this criterion was not met (for instance if the directional range of the movement was slightly larger than those of the population vector directions), then the closest direction within eight bins was used to calculate the time difference. The instantaneous direction was transformed (by adding or subtracting 2pi whenever the direction changed sign) to unwrap the directions. Segment boundaries were defined by calculating the slope of the angular velocity profile (using absolute values) and finding the transition point (a minimum) where the angular velocity went from decreasing to increasing values.

The temporal profiles of the population and velocity vector magnitudes were semi- sinusoidal. Because the period and relative phase of both profiles varied as the figure was drawn, it was difficult to calculate the instantaneous time difference between the two. These data were applied to the Hilbert transform (Bendat and Piersol 1986) to calculate the instantaneous phase of each profile. Based on the Fourier transform, this analysis calculates the phase, frequency, and amplitude of a continuous signal. This would be an ideal analysis to apply to the time series of vector magnitudes, making it possible to compare the phases of the rhythmic signals to get the time lags between them. However, because low-frequency sinusoidal components are emphasized in this transform, it was not possible to get a precise, bin-by-bin time difference between the two profiles with this algorithm. Instead we divided the data into pieces bounded by each extremum. These pieces were monotonic in time so that the speeds within each piece could be interpolated with a cubic spline. The data were splined in such a way that for any speed within the piece, the corresponding instant that the speed occurred could be found. This made it possible to match each calculated population vector magnitude to an interpolated finger speed and to find the corresponding time value of that matching finger speed. The difference between the time value of the matched speed and the time of the population vector was the prediction interval for speed.


    RESULTS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Movement kinematics

Average finger trajectories are displayed in Fig. 1. These data were averaged over 1,680 drawings (5 repetitions × 336 units). Each figure was divided into four segments at angular velocity minima. Individual segments of the trajectory data were normalized to 100 values. The resulting four segments of displacement data were differentiated successively to give values equivalent to instantaneous velocity and acceleration. Segments in each class were averaged together over all the experiments, and the result is shown in Fig. 2. The gray line is zero. Speed (solid line) for each class had a minimum value (4 mm/bin) near the middle of each segment. The acceleration profile was slightly asymmetric with the negative amplitude larger. The profiles across classes were very similar, showing that the segment-averaged data are robust.



View larger version (24K):
[in this window]
[in a new window]
 
Fig. 1. Average finger trajectories. The touch screen coordinates recorded during the trials selected for neuronal analysis were normalized to 100 points and averaged across trials for each class. Points of minimal angular velocity were segment boundaries. Segments are color coded and consistent between classes. Four classes were examined in this study; the lemniscates were oriented vertically and horizontally. Each orientation was drawn from both directions.



View larger version (20K):
[in this window]
[in a new window]
 
Fig. 2. Segment averages of velocity and acceleration magnitudes. The displacements shown in Fig. 1 were differentiated successively to give velocity and acceleration. These data were then averaged over the 4 segments of each class. Velocity (---) is minimal in the middle of each segment and maximal at either end. Acceleration (· · ·) is negative at the beginning of the segment, becoming positive after the velocity minimum. These kinematic profiles of these average segments are very similar across classes.

The same procedure was performed for curvature and angular velocity, and they are displayed in the segment averages of Fig. 3. Both the curvature and angular velocity are maximal in the middle of the segment and minimal (by definition) at the boundaries. The segments are characterized by consistent kinematics: minimum speed and maximum curvature in the middle and maximum speed and minimal curvature at the beginning and end.



View larger version (16K):
[in this window]
[in a new window]
 
Fig. 3. Segment averages of curvature and angular velocity. These averages were calculated with the method described for Fig. 2. Both parameters (---, curvature; · · ·, angular velocity) are maximal in the middle and minimal at the boundaries of the segment.

Population vector direction versus movement direction

Population vectors were composed of responses from 336 cortical cells recorded in 4 different cerebral hemispheres of 2 rhesus monkeys [3 sites were in primary motor cortex; 1, consisting of 71 units, was in dorsal premotor cortex (see Moran and Schwartz 1999a) for recording sites of the same units]. Population vectors constructed only from the motor cortical data were very similar to those constructed from the entire data set. Using only the 71 premotor cortical units resulted in distorted population vectors, a result that is consistent with the analysis performed on these data in the previous paper (Moran and Schwartz 1999b). Data for each task class were divided into 100 bins, with a population and finger velocity vector calculated for each bin. A vectogram comparing these data are shown in Fig. 4. The bottom set of vectors in each pair are the population vectors, the top set are the finger velocities. Movement onset from the start zone was used to align the two sets of vectors. The 10 population vectors before the alignment point were calculated from the spike data immediately preceding movement onset to show how they predict the movement vectors at the beginning of the task. These vectors point in a consistent direction, and their magnitudes are large, suggesting that the hand was moving toward the start location in this interval. The movement vectors occurring in this "prestart" period are not included because position data were not logged until the finger exited the start circle. For the remaining 100 vectors, there is a general correspondence of the population and velocity vectors (direction and magnitude) for each drawing, although there is a variable temporal offset between the neural and movement time series along the horizontal axis.



View larger version (32K):
[in this window]
[in a new window]
 
Fig. 4. Vectograms of population and displacement vectors. The bottom series of each pairing are the population vectors; the top are the displacement vectors. The origins of the 100 vectors in each series are evenly spaced along the abscissa. Although the overall correspondence is very good (r = 0.87, averaged across classes), there are local time shifts along the abscissa.

Population and movement vector directions were related in a characteristic way through each drawing. The directions of these vectors are compared in Fig. 5. Each of the four segments is signified by different colors. Directions of the neural vectors (dotted lines) precede those of the movement. The filled gray profiles show that the temporal offset between the neural and movement vectors tended to be smallest at the beginning and end of each segment. The temporal offset or prediction interval is the shift along the abscissa needed to align the two vector directions. Notice that the temporal pattern of the prediction intervals was very similar across classes. Because there was a linear negative relation between the radius of curvature (inverse of curvature) and the directional PI during spiral drawing (Moran and Schwartz 1999b), we plotted the segment averages of these quantities for each class in Fig. 6. The profiles are inversely related; radius is largest at the beginning and end of the segment. In contrast, the direction PI starts out small, is highest in the middle, and decreases at the end of the segment. These patterns are consistent across the four different task classes. Scatter plots (Fig. 7) of these averaged data show that there is a clear, linear relation (r = -0.95 across classes) between the radius of curvature and direction PI. The slope of these data are -4.2 ms/cm compared with -14.5 ms/cm for the spiral data. This comparison is linear for both the figure 8 and spiral data showing that the direction PI is directly proportional to the radius of curvature. The difference in slopes may be related to the length of the trajectory in a way that is analogous to the velocity gain factor (k of Eqs. 1 and 2) used as the proportionality constant in the formulation of the <FR><NU>2</NU><DE>3</DE></FR> power law (Viviani and Flash 1995). These plots also show that there is a tendency for the radius of curvature to be near constant in the middle of the segment that is represented as the nonlinearities at the peak of the scatter plots for classes one and four.



View larger version (24K):
[in this window]
[in a new window]
 
Fig. 5. Comparison of population and movement vector direction. The directions of the vectors in Fig. 4 were transformed to a range of ±pi . The abscissa data were derived from the average binwidth across the collected trials for that class. - - -, direction of the population; ---, direction of the movement vectors. Colors correspond to the segments described in Fig. 1. Shaded gray profiles are the prediction intervals between the population and movement vector direction (the distance between the dashed and solid lines along the abscissa). The prediction interval (PI) tends to be greatest in the middle of each segment, where the direction slope is greatest, corresponding to maximal curvature.



View larger version (16K):
[in this window]
[in a new window]
 
Fig. 6. Segment average of population curvature and direction prediction interval. Data are averaged within each segment by interpolating the radius of curvature (inverse of curvature, gray line) and the directional prediction interval to 100 points and averaging across corresponding points in each segment for every class.



View larger version (19K):
[in this window]
[in a new window]
 
Fig. 7. Directional prediction interval vs. radius of curvature. Data from Fig. 6 are plotted here in a scatter plot. The 2 parameters are highly correlated (r = -0.95 averaged across classes) in an inverse manner.

Because the radius of curvature is directly related to the cube of the finger speed (power law, described below) it would be expected that the directional prediction intervals would also be directly related to finger speed. There was a good linear relation between these parameters with an average (across classes) correlation coefficient of 0.92 compared with 0.95 for the comparison between radius of curvature and prediction interval.

Population vector magnitude versus finger speed

Magnitudes of the population vectors are plotted against finger speed for each of the drawing tasks in Fig. 8. For display purposes, the vector lengths were normalized by the peak speed for each class. The neuronal data in this figure began at movement onset. When all 110 population vectors (including the 10 vectors before movement onset) are used in a cross-correlation between the population and movement vector magnitudes, the overall correlation (across classes) was r = 0.83 [0.855, 0.641, 0.920, 0.908] with the population vector signal leading the movement by an average of 73 ms (77.1, 59.1, 96.5, and 59.2 ms). The poorest fit between the population vector lengths and movement speeds was for the last segment of class 2. To compare the time differences between these data and the vector direction the bin-by-bin lag was calculated by piecewise splining (METHODS). This can only be calculated from the first to the last extremum in the time profile. Prediction intervals for the vector magnitudes are represented by the filled-in trace. The modulation of prediction intervals is weaker and less consistent than those for directions. Much of the time differences can be accounted for with a constant temporal shift (especially in the vertically oriented figures classes 1 and 3). There appears to be a tendency for an increased PI in the last few bins of the analysis. This is most likely an edge effect. At this point in the task the population vectors are predicting past the end of the movement (notice that they end at a minimum). In contrast, the movement data ended at a speed maximum (touch screen data collection ceased when the finger passed through the last position on the trace). At the end of the task, the finger speed profile also flattened slightly. The combination of these factors make it difficult to estimate the final peak in the last segment of the movement speed trace, leading to apparently prolonged prediction intervals. Because of this unreliability, only the first three segments of each class were used to make an average of the prediction intervals. This and the corresponding modulation of finger speed are plotted in Fig. 9. The mean PI for speed is 75 ms. Although there is a small tendency for the prediction interval profiles to peak in the middle of the segment like those for direction, the profiles are mostly flat (except for class 2). This is reflected in the very small correlation between the profiles (r = -0.028, -0.010, -0.003, -0.020) and shows that most of the temporal shift between vector magnitudes is constant.



View larger version (27K):
[in this window]
[in a new window]
 
Fig. 8. Speed prediction intervals. Prediction intervals were calculated by splining the movement speeds between extrema and finding the exact match for each population vector speed. The incremental number of bins between these 2 values is the speed PI and is plotted on the ordinate of the plots in this figure. In general, the PI modulation is small with a tendency to peak in the middle of the segment.



View larger version (14K):
[in this window]
[in a new window]
 
Fig. 9. Segment averages of speed prediction intervals. Data from Fig. 8 were separated at segment boundaries. These values in each segment were interpolated to give 100 values and then averaged for each class. Except for class 2, the speed PIs (filled profile) tend to be flat with a small tendency to peak in the middle of the segment. Finger speed is shown with the gray line.

Neural trajectories

To compare the population to the finger trajectories, the components of the population vectors were shifted in time to place the two data sets in the same time frame. The magnitudes of the population vectors were shifted by 75 ms (the mean speed PI) and the population vector directions were shifted by the following equation for direction PI
&Dgr;<IT>t</IT><IT>=</IT><IT>a</IT><SUB><IT>0</IT></SUB><IT>+</IT><IT>a</IT><SUB><IT>1</IT></SUB>(<IT>radius<SUB>pv</SUB></IT>) (4)
a0 and a1 were determined with regression by plotting the segment-averaged radius of curvature of the population vectors against the directional PI in the same way that the finger trajectory radius of curvature was plotted in Fig. 7. These values were averaged over classes, giving a0 = 85.6 ms and a1 = -1.33 ms/cm.

The time-shifted population vectors and the movement vectors were integrated in time by adding them tip-to-tail. The resultant plots show the neural and finger trajectories (Fig. 10). The shape and orientation of each figure-eight is clearly recognizable in the corresponding neural trajectory. A correlation technique (Shadmehr and Mussa-Ivaldi 1994) for comparing time series of vectors was applied to the trajectories shown in Fig. 10. The correlation coefficients resulting from the comparison of the population and movement vectors for different classes are shown in Table 1. Movement and population vectors for the same classes were well correlated (r > 0.96). A comparison of movement and population vectors across classes in which the figures were of the same orientation but drawn in the opposite directions (1-3, 2-4) showed a moderate negative correlation. There was very little correlation between movement and population vectors of figures with different orientations (1-2, 3-4, 2-3, 1-4).



View larger version (21K):
[in this window]
[in a new window]
 
Fig. 10. Neural trajectories. Population vectors were adjusted temporally with the average prediction intervals and scaled using the maximum x and y values of the movement trajectory. The vectors were then added tip-to-tail to create the neural trajectories. In the left column, 100 population vectors were used beginning at movement onset. Finger trajectories shown in the right column are the same data shown in Fig. 1.


                              
View this table:
[in this window]
[in a new window]
 
Table 1. Correlation matrix of neural and movement trajectories

Neural representation of behavioral invariants

The validity of the <FR><NU>2</NU><DE>3</DE></FR> power law in this paradigm was tested. Curvature2/3 and angular velocity were calculated for the both the neural and finger trajectories. Plots of these parameters derived from the movement and neural trajectories for each class shown in Fig. 11. The relation between angular velocity and curvature2/3 is linear as demonstrated by regression (Table 2) for both the neural and finger trajectories. Within a movement, the slopes change abruptly between segments so that each segment is distinct from the others. These findings are in agreement with studies using human subjects (Polit and Bizzi 1979; Viviani and Cenzato 1985; Viviani and Terzuolo 1982) and show that the <FR><NU>2</NU><DE>3</DE></FR> power law is applicable to drawing movements performed by monkeys. That this law appears in the population activity of motor cortical cells suggests that it is a relevant feature of central processing and is in agreement with our earlier observations of spiral drawing (Schwartz 1994).



View larger version (17K):
[in this window]
[in a new window]
 
Fig. 11. Power law applied to the finger and neural trajectories. Angular velocity is the angle between successive vectors (radians/s) and curvature is that angle divided by the sum of the adjacent vector magnitudes (radians/cm). The plot of angular velocity and curvature2/3 resulted in a series of straight lines (r = 0.96, on average across classes for neural and movement data), each of which is a different color representing a different segment. Both the neural and movement data showed these characteristics. Monkeys use the same invariant as humans when drawing, and this law is represented in the population activity of the motor cortex.


                              
View this table:
[in this window]
[in a new window]
 
Table 2. Linearity of the <FR><NU>2</NU><DE>3</DE></FR> power law

There is a clear representation of speed and direction in the population vectors of neuronal activity. This dual representation is also evident in the discharge patterns of individual cells. Directional tuning parameters from the centerright-arrowout task were used to generate a profile of simulated discharge rates based on the profile of finger directions as the lemniscate was traced. These are shown in Fig. 12A for the same cell whose responses were described in the previous paper (Moran and Schwartz 1999b). The activity of this cell recorded in the centerright-arrowout task was used to determine the tuning function of this cell. This, in turn, was used to give a predicted discharge rate based on the instantaneous direction of the finger as it traced the lemniscate in the present task. For three of the four classes, this predicted discharge, based only on finger direction, captured the major features of the discharge pattern (correlation coefficient across the 4 classes was 0.68 [r = 0.80, 0.43, 0.88, and 0.60, with lags of 76, 58, 19, and 19 ms for each class]). The inclusion of speed in the model improves its accuracy. The following equation from Moran and Schwartz (1999a) was used for this
<IT>D</IT>(<IT>t</IT>)<IT>−</IT><IT>b</IT><SUB><IT>0</IT></SUB><IT>=∥</IT><IT><A><AC>V</AC><AC>&cjs1164;</AC></A></IT>(<IT>t</IT>)<IT>∥</IT>(<IT>b<SUB>n</SUB></IT><IT>+</IT><IT>b<SUB>y</SUB></IT><IT> sin </IT>[<IT>&thgr;</IT>(<IT>t</IT>)]<IT>+</IT><IT>b<SUB>x</SUB></IT><IT> cos </IT>[<IT>&thgr;</IT>(<IT>t</IT>)]) (5)
where D is the instantaneous cortical activity, b0, bn, bx,and by are constants determined from the centerright-arrowout task, theta  is the movement direction, and <A><AC>V</AC><AC>&cjs1164;</AC></A> is the velocity of the finger. This model (- - -) reflected the shape of the discharge rate profiles better, with a slightly better fit to the data (r = 0.77 [0.75, 0.50, 0.96, and 0.88 at lags of 76, 19, 19, and 0 ms]). Overall, cells in this study fit both models with about the same accuracy (r = 0.6 at a lag of ~80 ms). Directional tuning functions for these cells are broad, so simulated discharge patterns based on the cosine function have broad plateaus in time as the arm approaches and leaves the preferred direction (Schwartz 1993). Inclusion of speed in the modeled discharge transforms the plateaus into peaks, better matching the actual discharge patterns. For example, the activity in this figure was recorded from a cell with a preferred direction of 21° (0° is to the right, 90° is up). In the top trace (class 4), movement in this direction occurs in the middle of the third (green) and fourth (blue) segments. This corresponds to the plateaus in simulated discharge rate of the direction-only model (- - -). However, movement in the preferred direction occurs only in the most highly curved portion of the figure, where the speed is lowest. The more complete direction-speed model attenuates the discharge rate in the slow portions of the figure, resulting in a peak in the straight part of the trajectory (boundary between segments 2 and 3) when the finger speed is high and the direction is still near (within 25°) the preferred direction. It is important to note that the speed sensitivity of these cells will result in discharge rate peaks at segment boundaries where speed is the highest even though the direction is fairly constant.



View larger version (25K):
[in this window]
[in a new window]
 
Fig. 12. A: simulated and actual discharge rate for an example cell. Two simulated discharge rates were generated from finger kinematics. The short dashed line is a simulated discharge based solely on finger direction (Eq. 3, Moran and Schwartz 1999a), whereas the long dashed line was generated using both finger direction and speed (Eq. 1, Moran and Schwartz 1999a). The simulated firing rate based on direction and speed was better correlated with the actual (solid line) firing rate. B: population histogram. The maximum firing rate of each cell was used to normalize its firing rate during the task. The resulting firing rate profiles were averaged across cells. The average maximum rate was used to give units of mean firing rate used on the ordinate. The peaks in rate at each segment boundary are due to the speed sensitivity of individual units.

This argument should be valid for all cells regardless of preferred direction. If a cell's activity is sensitive to speed as described in the model, it should tend to peak in those regions of a drawing movement where speed is highest (i.e., at the segment boundary). To test this, we normalized the firing rate profile of each cell by its maximum rate and summed the profile across cells. With a uniform distribution of preferred directions across the recorded population, modulation of the histogram due to direction should be removed. The resulting modulation will be offset by the average mean rate of activity. Each peak of the population histogram shown in Fig. 12B corresponds to a segment boundary. These peaks are due to the speed sensitivity of the recorded cells.

Because many of the EMG patterns in the centerright-arrowout task were directionally tuned (Moran and Schwartz 1999a), we could use these data to generate simulated EMG patterns for the lemniscate task with the same method used for the cortical units. These also yielded good matches to the actual EMG pattern for most of the muscles, as would be expected from the results of the spiral task when the same technique was used (Moran and Schwartz 1999b). The simulated and actual discharge rate for the pectoralis is shown in Fig. 13 for each of the four classes (n = 5 trials). The simulated pattern using the centerright-arrowout directional tuning data matches well the actual pattern of the pectoralis (r = 0.88). This was true for all the muscles analyzed in this task (pectoralis, triceps, infraspinatus, middle deltoids, and posterior deltoids; r = 0.77, n = 350).



View larger version (19K):
[in this window]
[in a new window]
 
Fig. 13. Simulated (- - -) and actual (---) electromyographic (EMG) activity for pectoralis muscle. Using only finger direction information, a good correlation between simulated and actual EMG was obtained, illustrating that EMG activity and finger kinematics are well coupled.

Even the angular velocities of the joints were highly correlated to the coordinate system of the hand. Joint angles for each of four degrees of freedom about the shoulder and elbow were measured during the task. To assess phase changes between the joint angles and changes in hand direction, the preferred direction of the hand, assigned from the spiral task for each joint, was designated as the preferred direction for that degree of freedom. The cosine function was then used to generate a simulated angular velocity for that joint using the profile of finger movement directions. The finger-based coordinate system yielded very good predictions of joint movement. The results of this simulation for shoulder adduction are shown in Fig. 14 for the four classes and show that the simulated and actual angles are well correlated (r = 0.82). The correlation for all joint angles was 0.77. This shows that individual joint angular velocities were highly correlated to the instantaneous direction of the finger.



View larger version (17K):
[in this window]
[in a new window]
 
Fig. 14. Simulated and actual joint angular velocities. With the use of the same "tuning parameters" found in Moran and Schwartz (1999b) for shoulder adduction, the simulated adduction angular velocity (- - -) is well correlated to actual velocity.


    DISCUSSION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

An object's trajectory can be described completely by its speed and direction. In point-to-point reaching and drawing tasks (Ashe and Georgopoulos 1994; Moran and Schwartz 1999a; Schwartz 1992, 1993), direction and speed have been shown to be well represented in motor cortical single-cell activity. These parameters interact; the amplitude of the directional tuning function is modulated by speed. During drawing, motor cortical activity is modulated continuously such that a population of cell responses accurately predicts the velocity of the finger. In addition to a directional match, population and velocity vector lengths (speeds) were also highly correlated. When the population vectors were integrated in time, the resulting "neural trajectory" closely matched the drawn shape. The fidelity of the cortical representation can be appreciated by the high correlation of the neural and movement trajectories.

With the use of the movement trajectory as a reference, it is possible to calculate the temporal interval between the instantaneous representation of a movement parameter in the cortical population and its execution in the task. As spirals are drawn, the predictive directional signal in motor cortex precedes the movement more as the radius of curvature decreases. This suggests that the intervening processing between motor cortex and movement takes longer when the spatial derivative of direction is larger (Moran and Schwartz 1999b; Schwartz 1994). In the present study, we show again that speed and direction are represented simultaneously in the population. As expected from the spiral drawing results, the direction of the population vectors predicted the movement direction with a longer lead time (prediction interval) in those portions of the movement that were more highly curved. This can help explain some of the behavioral observations characteristic of drawing. Studies of human drawing have revealed two invariants: segmentation and the <FR><NU>2</NU><DE>3</DE></FR> power law. These laws were clearly evident in the finger trajectories of our monkeys as they drew lemniscates. The linkage between the drawing rate and the figural components described by these laws is also captured in the neural data.

The timing of parameter representation in the cortical activity suggests that constraints in neural processing may underlie the power law. The variation in prediction interval is directly related to the spatial derivative of direction. Conversely, the time interval between cortical representation and movement is small in straight movements. These findings are consistent with less downstream parameter processing when direction is near constant in the task. A similar argument has been made for the decision to make a saccade based on a random dot display (Shadlen and Newsome 1996). Direction-sensitive cells in the lateral intraparietal cortex respond with longer latencies to a random dot display that is more difficult to interpret, suggesting that less coherent moving dot patterns require more neural processing before deciding where to saccade.

Psychophysical studies (Soechting and Terzuolo 1987a,b; Sternad and Schaal 1999) have shown hand kinematics in the figure-eight task to be cyclical between movement segments and characteristically interrelated within each segment. Our aim was to determine how these segment-dependent kinematics were related to cortical neuronal activity. The present results show that the cortical population activity closely corresponds to these segment-dependent kinematics. Because the kinematic variables are interrelated, cortical activity is correlated simultaneously to multiple movement parameters. For instance, our data show a strong cubic relation between the directional prediction interval and speed in addition to that between the prediction interval and radius of curvature.

Individual cells have activity patterns that are cosine tuned to EMG activity and joint angular velocities. This relation, derived from the centerright-arrowout task, is robust across tasks where EMG and joint velocity can be used to predict discharge rate. Because EMG, joint angular velocity, and hand velocity are so highly linked in these tasks, it is difficult to categorize the neuronal activity as specifically related to an individual movement parameter. In fact, it is possible that these widespread correlations represent a system strategy to control movement efficiently. On the other hand, population vectors do not yield accurate EMG or joint angular velocity time profiles (Moran et al. 1999; Moran and Schwartz 1999a). To use cortical activity to predict these intrinsic variables, it is possible that a more complex extraction algorithm will be required.

Because the neural trajectory accurately reflects the hand trajectory, it allows us to address directly some of the issues raised by behavioral studies of drawing. Psychophysical results show that these movements are processed in pieces or segments (Soechting and Terzuolo 1987b; Viviani 1986; Viviani and Cenzato 1985), although see Sternad and Schaal (1999). Our results would tend to support this viewpoint. The kinematics (velocity, acceleration, curvature, and angular velocity) during drawing were highly consistent between tasks when the data were collapsed into averaged segments. Direct evidence that movement segmentation is a factor in the central process of movement planning is found in the neural trajectory. When angular velocity minima were used to delineate segments in the neural trajectory, the segments were found to correspond to those of the hand's trajectory. Single-cell activity increases at segment boundaries due to the speed sensitivity of these neurons. This is clear in the population activity as a whole. Consistent with this intensity measure, speed coding can be found in gross measurement of cortical activity using magnetoencephalography (Kelso et al. 1998), and segmentation during drawing should also be found with this technique. Prediction intervals were directly related to curvature when analyzed by segments. Finally, segments were demarcated in the neural trajectory data when plotted as angular velocity against curvature with each segment having a different slope. Kinematics within each segment are consistent across segments and figure orientations. Our data clearly show that the neural activity in motor cortical areas is also consistent with these kinematics. Taken together, these findings suggest that segmentation is an important feature in the planning and execution of drawing movements.

Speech is another type of movement that seems to be planned and produced in elastic units (Monsell 1986). The duration of these "stress groups" gets longer, and the time to begin speaking increases as the length of the utterance increases. This was interpreted as an increase in the processing load associated with retrieving and assembling the units, independent of peripheral activation of the muscles used to speak. Modeling approaches employ algorithms to account for the time-warping associated with the production of these units (Hopfield 1995).

Alternative hypotheses pertaining either to preplanning or an optimal control scheme have been examined relative to the form and kinematics of drawing (Viviani and Flash 1995). In the planning scheme, a blueprint as to the form of the movement (<FR><NU>2</NU><DE>3</DE></FR> power law and isochrony) would exist centrally, whereas in the optimization scheme, the relation between kinematics and figure geometry would be determined by a global constraint; in this case, the minimization of jerk. With the planning scheme, every point is specified along the trajectory. The optimization scheme requires only a few specific via points and was able to account for most of the observed features of the movement (except for those associated with the duration of the overall movement) as well as the <FR><NU>2</NU><DE>3</DE></FR> power law. How then might these two viewpoints converge? During drawing, it is likely that there is some sort of central representation of the figure to draw: a desired trajectory. Our experiments have shown that indeed there is an accurate representation of the upcoming trajectory in the activity of motor cortical cells. It is likely that the criterion of smoothness or minimum jerk could also be recognized centrally, and the inverse relation between speed and curvature would tend to ensure smoothness by minimizing changes in acceleration within a figure. Smoothness may be a necessary condition for merging seamlessly units of movement processing (Viviani and Flash 1995).


    ACKNOWLEDGMENTS

A. Kakavand trained the animals and assisted in the experiments.

This work was supported by Neurosciences Research Foundation, the Barrow Neurological Institute, and National Institute of Neurological Disorders and Stroke Grant NS-26375.


    FOOTNOTES

Address for reprint requests: A. B. Schwartz, The Neurosciences Institute, 10640 John Jay Hopkins Dr., San Diego, CA 92121.

The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Received 22 July 1996; accepted in final form 7 June 1999.


    REFERENCES
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

0022-3077/99 $5.00 Copyright © 1999 The American Physiological Society