 |
INTRODUCTION |
Changes in kinematic variables characterize the
behavior expressed by volitional movement. The structure of all
movement is determined by the behavioral goal to be achieved. For
example, the time course of velocities taken by the hand when swatting a fly is characteristically different from that when reaching for a
glass, even though the path of the hand may be identical. The
trajectory of the hand is especially important in drawing movements
where the behavioral goal is the path taken by the hand. Subjects tend
to select a particular pattern of kinematic parameters from an infinite
set that would result in a desired hand path. These consistent patterns
are characterized by invariants or rules determined by the neural
substrate generating the movement. For instance, there is no reason to
expect that within the motor system, the spatial description of the
path would be linked to the temporal evolution of the movement. Yet
psychophysical results suggest that this is an important aspect of the
trajectory planning process. An initial study of handwriting
(Viviani and Terzuolo 1982
) showed that hand speed and
the radius of curvature at each point in the cursive script were
inversely related over discrete segments. The slope between these
variables changed sharply at segment boundaries located at points in
the trajectory where curvature was at a local minimum. A later analysis
(Lacquaniti et al. 1983
) showed the speed-curvature
relation to be exponential. Movement speed was proportional to the
radius of curvature raised to the
power. This relation is
equivalent to the ratio of angular velocity to curvature (1/radius of
curvature) raised to the
power. Mathematically, this can be
represented as
|
(1)
|
|
(2)
|
where V(t) is tangential velocity,
R(t) is the instantaneous radius of curvature,
(t) is the angular velocity, C(t)
is the instantaneous curvature, and k is a proportionality
constant. The "velocity gain factor," k, changes between
segments and is related to the length of the segment.
These rules, segmentation and the power law, are not the result
of the mechanical process moving the limbs, because the rules are
followed for isometric drawing tasks (Massey et al.
1992
) and even seem to be an important component in the
perception of moving objects (Fagg et al. 1992
;
Soechting et al. 1986
). Furthermore, the value of the
exponent (
) is not determined by an obligatory relation
between kinematic variables because it varies in children (however, it
is constant in adults) (Sciaky et al. 1987
;
Viviani and Schneider 1991
). The two rules are linked because the transition between segments is delineated by changes in the
velocity gain factor k (Soechting and Terzuolo
1987b
). With drawings in free space, motion is confined to a
plane within a segment, but switches to a different plane between
segments (Soechting and Terzuolo 1986
, 1987a
). Although
the plane of movement and the velocity gain factor change
instantaneously, there are no abrupt changes in joint torques or
electromyographs (EMGs) at the segment boundaries.
The observed figural-kinematic relationships are produced centrally by
the neural structures generating the instructions that cause the arm to
move. Our previous work has shown that the hand's trajectory is well
represented in a population of motor cortical cell activity
(Moran and Schwartz 1999b
; Schwartz 1993
,
1994
). Although this activity predicts the hand's
trajectory accurately, it is unlikely that the motor cortex is the only
structure responsible for the trajectory structure because there are
reports of directional tuning with arm movements in a large number of
other sensory and motor structures (Bosco and Poppele
1993
; Fortier et al. 1989
; Graziano et
al. 1994
; Kutz et al. 1997
; Ruiz et al.
1995
; Turner and Anderson 1997
). Furthermore,
although we are dealing with kinematic variables measured in an
external coordinate system, the correlation of these variables with
kinetic variables during drawing suggests that this type of distinction
between classes of variables may not be pertinent in classifying
neuronal activity patterns during natural behaviors. In a previous
paper (Moran and Schwartz 1999b
) we showed that arm
joint angles and EMG were highly correlated to hand velocity. This was
even more evident in the present results.
This paper, the final in a set of three, examines the interrelation of
speed and direction as a figure is drawn and their representations in
the activity of motor cortical neurons. The first paper showed that
speed and direction could be encoded simultaneously in the activity of
a single unit. The prediction interval, defined in the second paper as
the time interval between the direction of a population vector and the
direction of the movement velocity, varied as a function of curvature
in a spiral. Here we show that the prediction interval based on
direction varies with curvature within each segment of a figure-eight.
Although this prediction interval was highly modulated, a prediction
interval based on speed was much less so and could be represented with
a constant value. Curvature defines the way a movement is segmented,
and this segmentation is obvious in the neuronal firing patterns. The
neuronal activity, its timing relative to the movement, the joint
angles, EMG, and hand velocity all show an organization related to
movement segmentation. This gives further support to theories
suggesting that apparently continuous drawing movements are generated
by temporal segments of neural activity (Soechting and Terzuolo
1987a
; Viviani 1986
; Viviani and Cenzato
1985
; Viviani and Flash 1995
).
 |
METHODS |
Most of the methods used in these studies have been detailed in
previous reports (Moran and Schwartz 1999b
;
Schwartz 1992
-1994
); only those that are unique to this
study will be described here.
Behavioral task
The behavioral apparatus and basic approach are the same
as those described in the preceding paper (Moran and Schwartz
1999b
). A lemniscate was graphed on a touch-sensitive computer
monitor. Superimposed on the static figure was an animated circle (1 cm radius) that moved along the figure each time the finger moved to it.
In this way, the animal controlled the speed of the circle, which was
always just ahead of the finger. The circle regulated the movement
tolerance, which in this case was 1 cm. Each figure-eight was traced as
four classes (vertical and horizontal orientation, clockwise, and
counterclockwise) presented in five randomized blocks. The projected
Lissajous lemniscate was 12 × 12 cm, but with the tolerance of
the moving circle, the animals drew figures that were slightly larger
than 11 × 11 cm.
Data analysis
Curvature was calculated for both the hand and the neural
trajectories using Eq. 3.
|
(3)
|
The single and double dot symbols above the variables represent
first and second time derivatives, respectively. Derivatives were
calculated using spline functions (csakm IMSL, Visual Numerics, Houston, TX) or by differentiating and smoothing with a double-sided, five point exponential routine.
All comparisons between neuronal and behavioral data in this paper are
carried out with vector quantities. The neuronal data are represented
by population vectors and arm movement data with velocity vectors.
Because the time-varying processing associated with continuous drawing
movements is of interest here, each task is divided into a time series
of vectors, and comparisons are made between corresponding population
and velocity vectors. In addition to showing (as we have in the 2 previous papers in this series) that the population vector is an
accurate prediction of the velocity vector, we explore here the
relation of the time interval between the neuronal and movement vectors
and how this prediction interval (PI) is related to vector directions
and vector magnitudes. The directional PI was found by applying a
spline function to the time series of movement vector directions and interpolating between the 100 values to increase the total number of
points to 10,000. A search was then performed for each of the population vector directions to find the nearest match (within 8 bins;
~160 ms) to the interpolated movement direction. The search was
halted when the match was within a criterion (0.0005 radians). If this
criterion was not met (for instance if the directional range of the
movement was slightly larger than those of the population vector
directions), then the closest direction within eight bins was used to
calculate the time difference. The instantaneous direction was
transformed (by adding or subtracting 2
whenever the direction changed sign) to unwrap the directions. Segment boundaries were defined
by calculating the slope of the angular velocity profile (using
absolute values) and finding the transition point (a minimum) where the
angular velocity went from decreasing to increasing values.
The temporal profiles of the population and velocity vector magnitudes
were semi- sinusoidal. Because the period and relative phase of both
profiles varied as the figure was drawn, it was difficult to calculate
the instantaneous time difference between the two. These data were
applied to the Hilbert transform (Bendat and Piersol
1986
) to calculate the instantaneous phase of each profile.
Based on the Fourier transform, this analysis calculates the phase,
frequency, and amplitude of a continuous signal. This would be an ideal
analysis to apply to the time series of vector magnitudes, making it
possible to compare the phases of the rhythmic signals to get the time
lags between them. However, because low-frequency sinusoidal components
are emphasized in this transform, it was not possible to get a precise,
bin-by-bin time difference between the two profiles with this
algorithm. Instead we divided the data into pieces bounded by each
extremum. These pieces were monotonic in time so that the speeds within
each piece could be interpolated with a cubic spline. The data were
splined in such a way that for any speed within the piece, the
corresponding instant that the speed occurred could be found. This made
it possible to match each calculated population vector magnitude to an
interpolated finger speed and to find the corresponding time value of
that matching finger speed. The difference between the time value of the matched speed and the time of the population vector was the prediction interval for speed.
 |
RESULTS |
Movement kinematics
Average finger trajectories are displayed in Fig.
1. These data were averaged over 1,680 drawings (5 repetitions × 336 units). Each figure was divided
into four segments at angular velocity minima. Individual segments of
the trajectory data were normalized to 100 values. The resulting four
segments of displacement data were differentiated successively to give
values equivalent to instantaneous velocity and acceleration. Segments
in each class were averaged together over all the experiments, and the
result is shown in Fig. 2. The gray line
is zero. Speed (solid line) for each class had a minimum value (4 mm/bin) near the middle of each segment. The acceleration profile was
slightly asymmetric with the negative amplitude larger. The profiles
across classes were very similar, showing that the segment-averaged
data are robust.

View larger version (24K):
[in this window]
[in a new window]
|
Fig. 1.
Average finger trajectories. The touch screen coordinates recorded
during the trials selected for neuronal analysis were normalized to 100 points and averaged across trials for each class. Points of minimal
angular velocity were segment boundaries. Segments are color coded and
consistent between classes. Four classes were examined in this study;
the lemniscates were oriented vertically and horizontally. Each
orientation was drawn from both directions.
|
|

View larger version (20K):
[in this window]
[in a new window]
|
Fig. 2.
Segment averages of velocity and acceleration magnitudes. The
displacements shown in Fig. 1 were differentiated successively to give
velocity and acceleration. These data were then averaged over the 4 segments of each class. Velocity ( ) is minimal in the middle of each
segment and maximal at either end. Acceleration (· · ·) is
negative at the beginning of the segment, becoming positive after the
velocity minimum. These kinematic profiles of these average segments
are very similar across classes.
|
|
The same procedure was performed for curvature and angular velocity,
and they are displayed in the segment averages of Fig. 3. Both the curvature and angular
velocity are maximal in the middle of the segment and minimal (by
definition) at the boundaries. The segments are characterized by
consistent kinematics: minimum speed and maximum curvature in the
middle and maximum speed and minimal curvature at the beginning and
end.

View larger version (16K):
[in this window]
[in a new window]
|
Fig. 3.
Segment averages of curvature and angular velocity. These averages were
calculated with the method described for Fig. 2. Both parameters ( ,
curvature; · · ·, angular velocity) are maximal in the middle
and minimal at the boundaries of the segment.
|
|
Population vector direction versus movement direction
Population vectors were composed of responses from 336 cortical
cells recorded in 4 different cerebral hemispheres of 2 rhesus monkeys
[3 sites were in primary motor cortex; 1, consisting of 71 units, was
in dorsal premotor cortex (see Moran and Schwartz 1999a
) for recording sites of the same units].
Population vectors constructed only from the motor cortical data were
very similar to those constructed from the entire data set. Using only
the 71 premotor cortical units resulted in distorted population
vectors, a result that is consistent with the analysis performed on
these data in the previous paper (Moran and Schwartz
1999b
). Data for each task class were divided into 100 bins,
with a population and finger velocity vector calculated for each bin. A
vectogram comparing these data are shown in Fig.
4. The bottom set of vectors in each pair
are the population vectors, the top set are the finger velocities.
Movement onset from the start zone was used to align the two sets of
vectors. The 10 population vectors before the alignment point were
calculated from the spike data immediately preceding movement onset to
show how they predict the movement vectors at the beginning of the
task. These vectors point in a consistent direction, and their
magnitudes are large, suggesting that the hand was moving toward the
start location in this interval. The movement vectors occurring in this
"prestart" period are not included because position data were not
logged until the finger exited the start circle. For the remaining 100 vectors, there is a general correspondence of the population and
velocity vectors (direction and magnitude) for each drawing, although
there is a variable temporal offset between the neural and movement
time series along the horizontal axis.

View larger version (32K):
[in this window]
[in a new window]
|
Fig. 4.
Vectograms of population and displacement vectors. The bottom
series of each pairing are the population vectors; the
top are the displacement vectors. The origins of the 100 vectors in each series are evenly spaced along the abscissa. Although
the overall correspondence is very good (r = 0.87, averaged across classes), there are local time shifts along the
abscissa.
|
|
Population and movement vector directions were related in a
characteristic way through each drawing. The directions of these vectors are compared in Fig. 5. Each of
the four segments is signified by different colors. Directions of the
neural vectors (dotted lines) precede those of the movement. The filled
gray profiles show that the temporal offset between the neural and
movement vectors tended to be smallest at the beginning and end of each segment. The temporal offset or prediction interval is the shift along
the abscissa needed to align the two vector directions. Notice that the
temporal pattern of the prediction intervals was very similar across
classes. Because there was a linear negative relation between the
radius of curvature (inverse of curvature) and the directional PI
during spiral drawing (Moran and Schwartz 1999b
), we
plotted the segment averages of these quantities for each class in Fig.
6. The profiles are inversely related;
radius is largest at the beginning and end of the segment. In contrast, the direction PI starts out small, is highest in the middle, and decreases at the end of the segment. These patterns are consistent across the four different task classes. Scatter plots (Fig.
7) of these averaged data show that there
is a clear, linear relation (r =
0.95 across
classes) between the radius of curvature and direction PI. The slope of
these data are
4.2 ms/cm compared with
14.5 ms/cm for the spiral
data. This comparison is linear for both the figure 8 and spiral data
showing that the direction PI is directly proportional to the radius of
curvature. The difference in slopes may be related to the length of the
trajectory in a way that is analogous to the velocity gain factor
(k of Eqs. 1 and 2) used
as the proportionality constant in the formulation of the
power law (Viviani and Flash 1995
). These plots also
show that there is a tendency for the radius of curvature to be near
constant in the middle of the segment that is represented as the
nonlinearities at the peak of the scatter plots for classes one and
four.

View larger version (24K):
[in this window]
[in a new window]
|
Fig. 5.
Comparison of population and movement vector direction. The directions
of the vectors in Fig. 4 were transformed to a range of ± . The
abscissa data were derived from the average binwidth across the
collected trials for that class. - - -, direction of the population;
, direction of the movement vectors. Colors correspond to the
segments described in Fig. 1. Shaded gray profiles are the prediction
intervals between the population and movement vector direction (the
distance between the dashed and solid lines along the abscissa). The
prediction interval (PI) tends to be greatest in the middle of each
segment, where the direction slope is greatest, corresponding to
maximal curvature.
|
|

View larger version (16K):
[in this window]
[in a new window]
|
Fig. 6.
Segment average of population curvature and direction prediction
interval. Data are averaged within each segment by interpolating the
radius of curvature (inverse of curvature, gray line) and the
directional prediction interval to 100 points and averaging across
corresponding points in each segment for every class.
|
|

View larger version (19K):
[in this window]
[in a new window]
|
Fig. 7.
Directional prediction interval vs. radius of curvature. Data from Fig.
6 are plotted here in a scatter plot. The 2 parameters are highly
correlated (r = 0.95 averaged across classes) in
an inverse manner.
|
|
Because the radius of curvature is directly related to the cube of the
finger speed (power law, described below) it would be expected that the
directional prediction intervals would also be directly related to
finger speed. There was a good linear relation between these parameters
with an average (across classes) correlation coefficient of 0.92 compared with 0.95 for the comparison between radius of curvature and
prediction interval.
Population vector magnitude versus finger speed
Magnitudes of the population vectors are plotted against
finger speed for each of the drawing tasks in Fig.
8. For display purposes, the vector
lengths were normalized by the peak speed for each class. The neuronal
data in this figure began at movement onset. When all 110 population
vectors (including the 10 vectors before movement onset) are used in a
cross-correlation between the population and movement vector
magnitudes, the overall correlation (across classes) was
r = 0.83 [0.855, 0.641, 0.920, 0.908] with the
population vector signal leading the movement by an average of 73 ms
(77.1, 59.1, 96.5, and 59.2 ms). The poorest fit between the population
vector lengths and movement speeds was for the last segment of class 2. To compare the time differences between these data and the vector
direction the bin-by-bin lag was calculated by piecewise splining
(METHODS). This can only be calculated from the first to
the last extremum in the time profile. Prediction intervals for the
vector magnitudes are represented by the filled-in trace. The
modulation of prediction intervals is weaker and less consistent than
those for directions. Much of the time differences can be accounted for
with a constant temporal shift (especially in the vertically oriented
figures classes 1 and 3). There appears to be a tendency for an
increased PI in the last few bins of the analysis. This is most likely
an edge effect. At this point in the task the population vectors are
predicting past the end of the movement (notice that they end at a
minimum). In contrast, the movement data ended at a speed maximum
(touch screen data collection ceased when the finger passed through the
last position on the trace). At the end of the task, the finger speed
profile also flattened slightly. The combination of these factors make it difficult to estimate the final peak in the last segment of the
movement speed trace, leading to apparently prolonged prediction intervals. Because of this unreliability, only the first three segments
of each class were used to make an average of the prediction intervals.
This and the corresponding modulation of finger speed are plotted in
Fig. 9. The mean PI for speed is 75 ms.
Although there is a small tendency for the prediction interval profiles to peak in the middle of the segment like those for direction, the
profiles are mostly flat (except for class 2). This is reflected in the
very small correlation between the profiles (r =
0.028,
0.010,
0.003,
0.020) and shows that most of the temporal
shift between vector magnitudes is constant.

View larger version (27K):
[in this window]
[in a new window]
|
Fig. 8.
Speed prediction intervals. Prediction intervals were calculated by
splining the movement speeds between extrema and finding the exact
match for each population vector speed. The incremental number of bins
between these 2 values is the speed PI and is plotted on the ordinate
of the plots in this figure. In general, the PI modulation is small
with a tendency to peak in the middle of the segment.
|
|

View larger version (14K):
[in this window]
[in a new window]
|
Fig. 9.
Segment averages of speed prediction intervals. Data from Fig. 8 were
separated at segment boundaries. These values in each segment were
interpolated to give 100 values and then averaged for each class.
Except for class 2, the speed PIs (filled profile) tend to be flat with
a small tendency to peak in the middle of the segment. Finger speed is
shown with the gray line.
|
|
Neural trajectories
To compare the population to the finger trajectories, the
components of the population vectors were shifted in time to place the
two data sets in the same time frame. The magnitudes of the population
vectors were shifted by 75 ms (the mean speed PI) and the population
vector directions were shifted by the following equation for direction
PI
|
(4)
|
a0 and
a1 were determined with regression by
plotting the segment-averaged radius of curvature of the population
vectors against the directional PI in the same way that the finger
trajectory radius of curvature was plotted in Fig. 7. These values were
averaged over classes, giving a0 = 85.6 ms and a1 =
1.33 ms/cm.
The time-shifted population vectors and the movement vectors were
integrated in time by adding them tip-to-tail. The resultant plots show
the neural and finger trajectories (Fig.
10). The shape and orientation of each
figure-eight is clearly recognizable in the corresponding neural
trajectory. A correlation technique (Shadmehr and Mussa-Ivaldi
1994
) for comparing time series of vectors was applied to the
trajectories shown in Fig. 10. The correlation coefficients resulting
from the comparison of the population and movement vectors for
different classes are shown in Table 1.
Movement and population vectors for the same classes were well
correlated (r > 0.96). A comparison of movement and
population vectors across classes in which the figures were of the same
orientation but drawn in the opposite directions (1-3, 2-4) showed a
moderate negative correlation. There was very little correlation
between movement and population vectors of figures with different
orientations (1-2, 3-4, 2-3, 1-4).

View larger version (21K):
[in this window]
[in a new window]
|
Fig. 10.
Neural trajectories. Population vectors were adjusted temporally with
the average prediction intervals and scaled using the maximum
x and y values of the movement
trajectory. The vectors were then added tip-to-tail to create the
neural trajectories. In the left column, 100 population
vectors were used beginning at movement onset. Finger trajectories
shown in the right column are the same data shown in
Fig. 1.
|
|
Neural representation of behavioral invariants
The validity of the
power law in this paradigm was
tested. Curvature2/3 and angular velocity were calculated
for the both the neural and finger trajectories. Plots of these
parameters derived from the movement and neural trajectories for each
class shown in Fig. 11. The relation
between angular velocity and curvature2/3 is linear as
demonstrated by regression (Table 2) for
both the neural and finger trajectories. Within a movement, the slopes change abruptly between segments so that each segment is distinct from
the others. These findings are in agreement with studies using human
subjects (Polit and Bizzi 1979
; Viviani and
Cenzato 1985
; Viviani and Terzuolo 1982
) and
show that the
power law is applicable to drawing movements
performed by monkeys. That this law appears in the population activity
of motor cortical cells suggests that it is a relevant feature of
central processing and is in agreement with our earlier observations of
spiral drawing (Schwartz 1994
).

View larger version (17K):
[in this window]
[in a new window]
|
Fig. 11.
Power law applied to the finger and neural trajectories. Angular
velocity is the angle between successive vectors (radians/s) and
curvature is that angle divided by the sum of the adjacent vector
magnitudes (radians/cm). The plot of angular velocity and
curvature2/3 resulted in a series of straight lines
(r = 0.96, on average across classes for neural and
movement data), each of which is a different color representing a
different segment. Both the neural and movement data showed these
characteristics. Monkeys use the same invariant as humans when drawing,
and this law is represented in the population activity of the motor
cortex.
|
|
There is a clear representation of speed and direction in the
population vectors of neuronal activity. This dual representation is
also evident in the discharge patterns of individual cells. Directional
tuning parameters from the center
out task were used to generate a
profile of simulated discharge rates based on the profile of finger
directions as the lemniscate was traced. These are shown in Fig.
12A for the same cell whose
responses were described in the previous paper (Moran and
Schwartz 1999b
). The activity of this cell recorded in the
center
out task was used to determine the tuning function of this
cell. This, in turn, was used to give a predicted discharge rate based
on the instantaneous direction of the finger as it traced the
lemniscate in the present task. For three of the four classes, this
predicted discharge, based only on finger direction, captured the major
features of the discharge pattern (correlation coefficient across the 4 classes was 0.68 [r = 0.80, 0.43, 0.88, and 0.60, with
lags of 76, 58, 19, and 19 ms for each class]). The inclusion of speed
in the model improves its accuracy. The following equation from
Moran and Schwartz (1999a)
was used for this
|
(5)
|
where D is the instantaneous cortical activity,
b0, bn,
bx,and by are
constants determined from the center
out task,
is the movement
direction, and
is the velocity of the finger. This
model (- - -) reflected the shape of the discharge rate profiles better, with a slightly better fit to the data (r = 0.77 [0.75, 0.50, 0.96, and 0.88 at lags of 76, 19, 19, and 0 ms]).
Overall, cells in this study fit both models with about the same
accuracy (r = 0.6 at a lag of ~80 ms). Directional
tuning functions for these cells are broad, so simulated discharge
patterns based on the cosine function have broad plateaus in time as
the arm approaches and leaves the preferred direction (Schwartz
1993
). Inclusion of speed in the modeled discharge transforms
the plateaus into peaks, better matching the actual discharge patterns.
For example, the activity in this figure was recorded from a cell with
a preferred direction of 21° (0° is to the right, 90° is up). In
the top trace (class 4), movement in this direction occurs
in the middle of the third (green) and fourth (blue) segments. This
corresponds to the plateaus in simulated discharge rate of the
direction-only model (- - -). However, movement in the preferred
direction occurs only in the most highly curved portion of the figure,
where the speed is lowest. The more complete direction-speed model
attenuates the discharge rate in the slow portions of the figure,
resulting in a peak in the straight part of the trajectory (boundary
between segments 2 and 3) when the finger speed is high and the
direction is still near (within 25°) the preferred direction. It is
important to note that the speed sensitivity of these cells will result in discharge rate peaks at segment boundaries where speed is the highest even though the direction is fairly constant.

View larger version (25K):
[in this window]
[in a new window]
|
Fig. 12.
A: simulated and actual discharge rate for an example
cell. Two simulated discharge rates were generated from finger
kinematics. The short dashed line is a simulated discharge based solely
on finger direction (Eq. 3, Moran and Schwartz
1999a ), whereas the long dashed line was generated using both
finger direction and speed (Eq. 1, Moran and
Schwartz 1999a ). The simulated firing rate based on direction
and speed was better correlated with the actual (solid line) firing
rate. B: population histogram. The maximum firing rate
of each cell was used to normalize its firing rate during the task. The
resulting firing rate profiles were averaged across cells. The average
maximum rate was used to give units of mean firing rate used on the
ordinate. The peaks in rate at each segment boundary are due to the
speed sensitivity of individual units.
|
|
This argument should be valid for all cells regardless of preferred
direction. If a cell's activity is sensitive to speed as described in
the model, it should tend to peak in those regions of a drawing
movement where speed is highest (i.e., at the segment boundary). To
test this, we normalized the firing rate profile of each cell by its
maximum rate and summed the profile across cells. With a uniform
distribution of preferred directions across the recorded population,
modulation of the histogram due to direction should be removed. The
resulting modulation will be offset by the average mean rate of
activity. Each peak of the population histogram shown in Fig.
12B corresponds to a segment boundary. These peaks are due
to the speed sensitivity of the recorded cells.
Because many of the EMG patterns in the center
out task were
directionally tuned (Moran and Schwartz 1999a
), we could
use these data to generate simulated EMG patterns for the lemniscate task with the same method used for the cortical units. These also yielded good matches to the actual EMG pattern for most of the muscles,
as would be expected from the results of the spiral task when the same
technique was used (Moran and Schwartz 1999b
). The simulated and actual discharge rate for the pectoralis is shown in Fig.
13 for each of the four classes
(n = 5 trials). The simulated pattern using the
center
out directional tuning data matches well the actual pattern of
the pectoralis (r = 0.88). This was true for
all the muscles analyzed in this task (pectoralis, triceps, infraspinatus, middle deltoids, and posterior deltoids;
r = 0.77, n = 350).

View larger version (19K):
[in this window]
[in a new window]
|
Fig. 13.
Simulated (- - -) and actual ( ) electromyographic (EMG) activity
for pectoralis muscle. Using only finger direction
information, a good correlation between simulated and actual EMG was
obtained, illustrating that EMG activity and finger kinematics are well
coupled.
|
|
Even the angular velocities of the joints were highly correlated to the
coordinate system of the hand. Joint angles for each of four degrees of
freedom about the shoulder and elbow were measured during the task. To
assess phase changes between the joint angles and changes in hand
direction, the preferred direction of the hand, assigned from the
spiral task for each joint, was designated as the preferred direction
for that degree of freedom. The cosine function was then used to
generate a simulated angular velocity for that joint using the profile
of finger movement directions. The finger-based coordinate system
yielded very good predictions of joint movement. The results of this
simulation for shoulder adduction are shown in Fig.
14 for the four classes and show that the simulated and actual angles are well correlated (r = 0.82). The correlation for all joint angles was 0.77. This shows that individual joint angular velocities were highly correlated to the
instantaneous direction of the finger.

View larger version (17K):
[in this window]
[in a new window]
|
Fig. 14.
Simulated and actual joint angular velocities. With the use of the same
"tuning parameters" found in Moran and Schwartz
(1999b) for shoulder adduction, the simulated adduction angular
velocity (- - -) is well correlated to actual velocity.
|
|
 |
DISCUSSION |
An object's trajectory can be described completely by its speed
and direction. In point-to-point reaching and drawing tasks (Ashe and Georgopoulos 1994
; Moran and Schwartz
1999a
; Schwartz 1992
, 1993
),
direction and speed have been shown to be well represented in motor
cortical single-cell activity. These parameters interact; the amplitude
of the directional tuning function is modulated by speed. During
drawing, motor cortical activity is modulated continuously such that a
population of cell responses accurately predicts the velocity of the
finger. In addition to a directional match, population and velocity
vector lengths (speeds) were also highly correlated. When the
population vectors were integrated in time, the resulting "neural
trajectory" closely matched the drawn shape. The fidelity of the
cortical representation can be appreciated by the high correlation of
the neural and movement trajectories.
With the use of the movement trajectory as a reference, it is possible
to calculate the temporal interval between the instantaneous representation of a movement parameter in the cortical population and
its execution in the task. As spirals are drawn, the predictive directional signal in motor cortex precedes the movement more as the
radius of curvature decreases. This suggests that the intervening processing between motor cortex and movement takes longer when the
spatial derivative of direction is larger (Moran and Schwartz 1999b
; Schwartz 1994
). In the present study, we
show again that speed and direction are represented simultaneously in
the population. As expected from the spiral drawing results, the
direction of the population vectors predicted the movement direction
with a longer lead time (prediction interval) in those portions of the movement that were more highly curved. This can help explain some of
the behavioral observations characteristic of drawing. Studies of human
drawing have revealed two invariants: segmentation and the
power law. These laws were clearly evident in the finger trajectories
of our monkeys as they drew lemniscates. The linkage between the
drawing rate and the figural components described by these laws is also
captured in the neural data.
The timing of parameter representation in the cortical activity
suggests that constraints in neural processing may underlie the power
law. The variation in prediction interval is directly related to the
spatial derivative of direction. Conversely, the time interval between
cortical representation and movement is small in straight movements.
These findings are consistent with less downstream parameter processing
when direction is near constant in the task. A similar argument has
been made for the decision to make a saccade based on a random dot
display (Shadlen and Newsome 1996
). Direction-sensitive
cells in the lateral intraparietal cortex respond with longer latencies
to a random dot display that is more difficult to interpret, suggesting
that less coherent moving dot patterns require more neural processing
before deciding where to saccade.
Psychophysical studies (Soechting and Terzuolo 1987a
,b
;
Sternad and Schaal 1999
) have shown hand kinematics in
the figure-eight task to be cyclical between movement segments and
characteristically interrelated within each segment. Our aim was to
determine how these segment-dependent kinematics were related to
cortical neuronal activity. The present results show that the cortical
population activity closely corresponds to these segment-dependent
kinematics. Because the kinematic variables are interrelated, cortical
activity is correlated simultaneously to multiple movement parameters. For instance, our data show a strong cubic relation between the directional prediction interval and speed in addition to that between
the prediction interval and radius of curvature.
Individual cells have activity patterns that are cosine tuned to EMG
activity and joint angular velocities. This relation, derived from the
center
out task, is robust across tasks where EMG and joint velocity
can be used to predict discharge rate. Because EMG, joint angular
velocity, and hand velocity are so highly linked in these tasks, it is
difficult to categorize the neuronal activity as specifically related
to an individual movement parameter. In fact, it is possible that these
widespread correlations represent a system strategy to control movement
efficiently. On the other hand, population vectors do not yield
accurate EMG or joint angular velocity time profiles (Moran et
al. 1999
; Moran and Schwartz 1999a
). To use
cortical activity to predict these intrinsic variables, it is possible
that a more complex extraction algorithm will be required.
Because the neural trajectory accurately reflects the hand
trajectory, it allows us to address directly some of the issues raised
by behavioral studies of drawing. Psychophysical results show that
these movements are processed in pieces or segments (Soechting and Terzuolo 1987b
; Viviani
1986
; Viviani and Cenzato 1985
), although see
Sternad and Schaal (1999)
. Our results would tend to
support this viewpoint. The kinematics (velocity, acceleration, curvature, and angular velocity) during drawing were highly consistent between tasks when the data were collapsed into averaged segments. Direct evidence that movement segmentation is a factor in the central
process of movement planning is found in the neural trajectory. When
angular velocity minima were used to delineate segments in the neural
trajectory, the segments were found to correspond to those of the
hand's trajectory. Single-cell activity increases at segment
boundaries due to the speed sensitivity of these neurons. This is clear
in the population activity as a whole. Consistent with this intensity
measure, speed coding can be found in gross measurement of cortical
activity using magnetoencephalography (Kelso et al.
1998
), and segmentation during drawing should also be found
with this technique. Prediction intervals were directly related to
curvature when analyzed by segments. Finally, segments were demarcated
in the neural trajectory data when plotted as angular velocity against
curvature with each segment having a different slope. Kinematics within
each segment are consistent across segments and figure orientations.
Our data clearly show that the neural activity in motor cortical areas
is also consistent with these kinematics. Taken together, these
findings suggest that segmentation is an important feature in the
planning and execution of drawing movements.
Speech is another type of movement that seems to be planned and
produced in elastic units (Monsell 1986
). The duration
of these "stress groups" gets longer, and the time to begin
speaking increases as the length of the utterance increases. This was
interpreted as an increase in the processing load associated with
retrieving and assembling the units, independent of peripheral
activation of the muscles used to speak. Modeling approaches employ
algorithms to account for the time-warping associated with the
production of these units (Hopfield 1995
).
Alternative hypotheses pertaining either to preplanning or an optimal
control scheme have been examined relative to the form and kinematics
of drawing (Viviani and Flash 1995
). In the planning scheme, a blueprint as to the form of the movement (
power
law and isochrony) would exist centrally, whereas in the optimization
scheme, the relation between kinematics and figure geometry would be
determined by a global constraint; in this case, the minimization of
jerk. With the planning scheme, every point is specified along the
trajectory. The optimization scheme requires only a few specific via
points and was able to account for most of the observed features of the
movement (except for those associated with the duration of the overall
movement) as well as the
power law. How then might these two
viewpoints converge? During drawing, it is likely that there is some
sort of central representation of the figure to draw: a desired
trajectory. Our experiments have shown that indeed there is an accurate
representation of the upcoming trajectory in the activity of motor
cortical cells. It is likely that the criterion of smoothness or
minimum jerk could also be recognized centrally, and the inverse
relation between speed and curvature would tend to ensure smoothness by
minimizing changes in acceleration within a figure. Smoothness may be a
necessary condition for merging seamlessly units of movement processing (Viviani and Flash 1995
).
A. Kakavand trained the animals and assisted in the experiments.
This work was supported by Neurosciences Research Foundation, the
Barrow Neurological Institute, and National Institute of Neurological
Disorders and Stroke Grant NS-26375.
Address for reprint requests: A. B. Schwartz, The Neurosciences
Institute, 10640 John Jay Hopkins Dr., San Diego, CA 92121.
The costs of publication of this article were defrayed in part by the
payment of page charges. The article must therefore be hereby marked
"advertisement" in accordance with 18 U.S.C. Section
1734 solely to indicate this fact.