Division of Biology, California Institute of Technology, Pasadena, California 91125
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Xing, Jing and Richard A. Andersen. Memory Activity of LIP Neurons for Sequential Eye Movements Simulated With Neural Networks. J. Neurophysiol. 84: 651-665, 2000. Many neurons in macaque lateral intraparietal cortex (LIP) maintain elevated activity induced by visual or auditory targets during tasks in which monkeys are required to withhold one or more planned eye movements. We studied the mechanisms for such memory activity with neural network modeling. Recurrent connections among simulated LIP neurons were used to model memory responses of LIP neurons. The connection weights were computed using an optimization procedure to produce desired outputs in memory-saccade tasks. One constraint for the training process is the "single-purpose" rule, which mimics the fact that once LIP neurons hold the memory activity of a saccade, they are insensitive to further stimuli until the motor action is completed. After training, excitatory connections were developed between units with similar preferred saccade directions, while inhibitory connections were formed between units with dissimilar directions. This "push-pull" mechanism enables the network to encode the next intended eye movement and is essential for programming sequential saccades. In simulating double saccades, the push-pull connections locked the on-going activity in the network for the first saccade until the saccade was made, then a new population of units became active to prepare for the second saccade. The simulated LIP neurons exhibited sensory responses and memory activities similar to those recorded in LIP neurons. We propose that push-pull recurrent connections might be the basic structure mediating the memory activity of area LIP in planning sequential eye movements.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The lateral intraparietal cortex
(LIP) is involved in programming saccadic eye movements
(Andersen and Gnadt 1989; Lynch et al.
1977
). Many LIP neurons exhibit sustained responses to
remembered visual or auditory targets (Mazzoni et al.
1996a
). During delayed-saccade tasks in which the monkey
withheld a saccade to a remembered target for a short period of time,
the response of LIP neurons triggered by the target was sustained until
the saccade was initiated (Andersen et al. 1990a
,b
;
Gnadt and Andersen 1988
). Moreover, neurons could maintain the memory for the saccade even if the monkey was presented with new stimuli during the withholding period. Negatively correlated memory responses have also been observed in LIP, and such responses occurred when the remembered saccade was opposite the neuron's preferred saccadic direction (Barash et al. 1991a
,b
).
Memory activity was further characterized with the
delayed-double-saccade experiments (Mazzoni et al.
1996b
) in which the monkey was trained to memorize two
consecutively flashed targets and to plan two saccades to the targets
in the order that the targets were presented. During the delayed
period, many LIP neurons whose preferred directions were in the
direction of the first saccade fired continuously until the execution
of the saccade. These neurons thus held the correct memory for the
first saccade regardless of the flash of the second target. Neurons
coding for the second saccade started to fire only after the first
saccade was executed. The results indicated that memory activities for
the majority of LIP neurons encode the next planned saccade. On the
other hand, a small percentage of LIP neurons encode the memory of
target locations instead. The sustained responses in all kinds of
delayed-saccade tasks have a common feature: neurons begin to encode a
new saccadic movement only after the current motor plan is disengaged.
We call this the "single-purpose" feature.
Short-term memory activity has been observed in a number of cortical
areas (Funahashi et al. 1989; Gnadt and Andersen
1988
; Goldman-Rakic 1995
; Kalaska and
Grammond 1995
; Quintana and Fuster 1992
).
Several computational studies have proposed that recurrent connections
might be the mechanism for this activity (Cowan 1972
; Dehaene and Changeux 1989
; Fuster 1995
;
Zipser 1991
). The purpose of this report was to study
the mechanisms of saccadic-related memory activities in area LIP.
Especially, we were interested in how the single-purpose feature was
related to programming delayed double saccades. Based on experimental
tasks, we used recurrent neural networks to simulate the memory
features of LIP neurons. We first studied the mechanisms of memory
saccades and then examined an extended model for planning
double-saccades. Preliminary results of this report have been presented
in abstract form (Xing et al. 1995
).
![]() |
METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The model is a three-layered neural network, with a similar
structure to that of Zipser and Andersen model (1988).
The diagram of this model is shown in Fig.
1. The model was not designed to resemble
the complex anatomy of area LIP. It is the typical classic neural
network that can be trained to carry out the required sensorimotor transformations. The input layer, like area LIP, has access to visual
and auditory target locations as well as eye position in the orbit. The
output layer is a topographic map of eye motor errors. The middle
layer, or the hidden layer, is a recurrent network with every unit
receiving activities from all other hidden units. Every unit in the
input layer is connected to each of the hidden units, which are in turn
connected to all the output units. The weights of connections vary
between
1 and +1. They are initially set to small random values
between
0.1 and 0.1. The weights are adjusted to encode the motor
errors of visual or auditory targets at the output map.
|
The input layer consists of a visual map in retinal coordinates, an
auditory map in head-centered coordinates, and eye-position units. The
visual map uses 8 × 8 units to model a 40° to 40° retinal
space. Each of the units has a Gaussian receptive field (RF) with a
1/e width of 15°. The centers of the RFs were equally spaced over the 8 × 8 grid with 10° spacing. These units encode target locations with their activation values between 0 and 1. The
auditory input is modeled using an auditory map of an 8 × 8 array
of units, similar to the visual one. The only difference between the
two input maps is that the auditory units encode target locations in
head-centered coordinates and the visual units encode target locations
in eye-centered coordinates. Eye positions is coded by four sets of
eight units representing horizontal and vertical eye coordinates with
positive and negative slopes. The activation of the units, with various
intercepts and slopes, is thus an increasing function of eye positions.
The middle layer, also called the hidden layer, typically has 30 units
in the simulations presented in this report. Each hidden unit receives
inputs from all three input channels. In addition, each hidden unit
receives recurrent projections from all other hidden units. The
activation of a hidden unit is calculated by first summing all inputs
and then calculating the output as a sigmoidal function of the total
input. At a given simulated time step, the activation of a hidden unit
can be expressed as the following: output activation = 1/[1 + exp(net)] where net = sum of weighted inputs + bias.
The inputs here include the activities of the visual, auditory, and eye-position units at the current time step and the activities of other hidden units at the previous time step. The sigmoid function is chosen as the activation function because it resembles the operation performed by actual neurons that sum inputs, have a threshold, and saturate at high levels of activity. In the middle region of its dynamic range, the sigmoid approximates a linear function.
The output layer is an eye-centered map encoding eye motor errors (ME)
of saccades. An 8 × 8 array of output units is used to represent
MEs topographically. Each of the units covers a 10° space of MEs with
a Gaussian 1/e width of 15°. The activation of the output
units, like the hidden units, is a sigmoidal function of the sum of the
weighted inputs from the hidden units. We use E to represent the
initial eye position, V for the locations of visual targets in
retinal coordinates, and A for the locations of auditory targets
in head-centered coordinates. For simple saccades, ME = V for
visual targets and ME = A E for auditory
targets. For double saccades, we use E0 to represent the initial
eye position, V1 for the location of the first visual target in
retinal coordinates, and A1 for the location of the first auditory
target in head-centered coordinates. E1 represents the eye
position after the first saccade. V2 and A2 indicate the
second visual and auditory targets, respectively. The desired ME output
for the first saccade is ME = V1 for visual targets or
ME = A1
E0 for auditory targets. The ME for the
second saccade is ME = A2
E1 or ME = V2 + E0
E1.
Training process
We use an algorithm "backpropagation-through-time" to train
the network. This algorithm gradually optimizes connection weights to
produce the desired output in a recurrent neural network (Munro et al. 1994; Werbos 1990
; Williams and
Zipser 1995
). We use this algorithm simply to train the network
to perform the required sensorimotor transformations with no intention
to claim that the algorithm is similar to the learning mechanisms in
the brain.
The backpropagation algorithm uses supervised learning. It first
computes an error signal, which is the difference of the desired output
(the teacher signal) and the actual output. This error signal is then
used to update connection weights. The amount of weight change depends
on the error signal, the activities of the two connected units, and an
arbitrary learning rate. In our implementation of the algorithm, the
desired activity Aexp for each output
unit k is determined by the expected ME of a saccadic target. The actual output Ao of an
output unit is computed for a given target location, eye position and
the initial weights. The error signal k for
an output unit k is
![]() |
![]() |
A connection weight Wih from an input
unit to a hidden unit is updated according to
![]() |
A recurrent connection weight Whh from
a hidden unit i to another hidden unit j is
updated according to
![]() |
In a recurrent network, the output of the network accounts for both the current inputs and the activities at earlier times. We run the network in 13 discrete time steps for each training cycle. To compare with experimental recordings, one time step can be viewed as a duration of 100 ms. The time lag of the recurrent connection is one time step. The input of a visual or auditory target location lasts for one time step while an eye-position signal sustains until a saccade is made. The teacher signal, which is the expected ME in the output layer, appears several steps after the onset of a target simulation and lasts for one time step. This signal mimics the command to make a saccade. The weights of the feedforward connections and recurrent connections are updated at the time of this saccade command. Note that we did not simulate the shut-off of the neuronal activity after a saccade is made (i.e., the postsaccadic suppression). Therefore the recurrent activity in the network may sustain indefinitely unless it is turned off by other mechanisms, as detailed later in the extended double-saccade model. Since different training patterns are employed for models of single- and double-memory saccades, details about the training patterns will be described in each section as needed.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Model of memory saccades
MODEL TRAINED WITH SINGLE MEMORY SACCADES. We first trained the model to perform single memory saccades. Twenty-five target locations across the input space and 25 eye positions were chosen as training samples. For each training cycle, a visual or an auditory target at a chosen location was presented at the first time step. A saccadic target was simulated as a dot stimulus with the amplitude of 1. The saccade was made randomly between the fifth to ninth time steps. The model was trained to encode the ME of a saccade at the time step when the saccade was made. The paradigm is illustrated in Fig. 2A. After approximately 3,000 training cycles, the network learned to produce and memorize saccadic MEs correctly to any input pairs of target location and eye position. The performance of the trained network was evaluated by comparing the expected ME for a given target location and eye position with the produced ME at the output layer. We tested 100 random input pairs of eye position and target location for the trained network. The standard deviation of the actual ME outputs from the expected MEs was 2.62°. Figure 2B shows one example of the model output. For simplification, only eight units (which include the one with the maximum response) along one dimension (1-D) of the two-dimensional (2-D) output map are shown. The vertical axis is the 1-D ME and the horizontal axis indicates time steps. The gray level of squares is proportional to the responses of the output units. The horizontal bar indicates the gravity center of the responses. "T" indicates the expected ME of the target. Figure 2B shows that the model produces the correct output and the activity sustains throughout the delay period.
|
MEMORY-SACCADE MODEL TRAINED WITH THE SINGLE-PURPOSE FEATURE. Training and network performance. We retrained the same model in Fig. 1 by applying the single-purpose feature to the training procedure as a constraint. Figure 3A shows a typical training pattern. The target was flashed for one time step as before. In addition to the target, an irrelevant stimulus was presented at a random location during the delay period. The irrelevant stimulus was a dot stimulus lasting for one time step. The network was required to yield the correct ME of the saccade to the target. Thus the activation of the irrelevant stimulus was to be ignored. At the time of the saccade command, the difference between the expected ME and the actual output was computed for each output unit; the weights of connections were adjusted accordingly.
|
|
|
Model of double saccades
LIP neurons participate in planning sequential eye movements. This has been typically studied with double-saccade experiments. In this section, we first summarize the neurophysiological data and then extend the memory-saccade model to make a sequence of two saccades.
PHYSIOLOGICAL RESULTS TO BE MODELED.
The delayed double-saccade tasks by Mazzoni et al.
(1996b) were designed to test whether LIP neurons encoded
sensory locations or motor plans of saccades in sequential eye
movements. The monkeys were required to memorize two targets briefly
flashed in succession during a delay period and to make a sequence of
two saccades to the two targets after the fixation light went off. The
memory activities during the delay period (before the 1st saccade) and during the intersaccadic interval (after the 1st saccade and before the
2nd saccade) were examined. Extracellular recordings showed that during
the delay period, many neurons whose movement fields were in the
direction of the first saccade fired continuously until the first
saccade was made, whereas neurons coding for the direction of the
second saccade started to fire only after the first saccade was
performed. Figure 6 shows the responses
of a typical LIP neuron in different double-saccade tasks. The
left panel shows the two saccades made toward the two
remembered targets. The dashed curve indicates the neuron's RF. This
neuron preferred saccades in the down-left direction. The saccadic
targets are indicated with black dots and labeled as T1 and T2.
Responses of the neuron are shown in the right panel. The
delay period is labeled as M1. The horizontal and vertical eye
positions are plotted under the responses. The first deflection in
these eye traces corresponds to the first saccade and the second
deflection corresponds to the second saccade. In Fig. 6A,
both targets fall in the RF, and only the first saccade is in the
neuron's PD. The neuron fires during the delay period. The sustained
activity goes off after the first saccade is made. In Fig.
6B, the first target is outside the RF and the second target
falls in the RF. The second saccade is in the neuron's PD. The neuron
has a brief response following the flash of the second target, and this
activity does not sustain during the delay period. After the first
saccade is completed, the neuron begins to fire and the activity
sustains until the monkey makes the second saccade. Thus the activity
is related to the second saccade. In Fig. 6C, no targets
fall in RF, but the second saccade is in the neuron's PD. The neuron
has no response to the flash of either target. However, it fires during
the intersaccadic interval and thus codes for the second saccade.
Therefore this neuron encodes a preferred impending movement regardless
of target locations. As shown in Fig. 6C, the activity does
not even depend on sensory stimulations. Seventy-seven percent of LIP
neurons recorded encode the impending saccade. It is concluded that the memory activity of the majority of LIP neurons encodes the next planned
saccade. On the other hand, 16% of neurons encode target locations
instead. These neurons begin to fire after the flash of the second
target, which falls in their RFs, and the activity lasts through the
delay period and the intersaccadic interval. These neurons may
participate in programming subsequent saccades because information
about the second target needs to be held until the first saccade is
performed. The remaining neurons, approximately 7%, were difficult to
classify into one or the other of the two categories.
|
MODEL. Based on the preceding experimental observations, we extended the memory-saccade model to simulate double-saccade tasks. Figure 7 is a diagram of the extended model. Besides the recurrent network in the original memory-saccade model, called recurrent net I (RN-I) here, the extended model has an additional recurrent net in the hidden layer (RN-II). This population of units also receive visual, auditory and eye-position inputs. Its output projects to the primary hidden net (RN-I). Every RN-II unit projects to all RN-I units. Like RN-I units, RN-II units are fully interconnected. The postsaccadic suppression is also built into the model. It artificially resets the activity of RN-I units to the initial state after the first saccade is made. The push-pull structure of the RN-I network is capable of carrying out feature 1, the single-purpose feature; postsaccadic suppression serves feature 2, i.e., turning off the memory activity in RN-I after a saccade is made; and the RN-II network serves the memory buffer for feature 3. This model is expected to produce the following response patterns: RN-I units encode the first saccade, and the activity is sustained while a brief presentation of the second target does not affect the on-going activity in RN-I due to the push-pull mechanism; information about the second target and the initial eye position is maintained in RN-II; the postsaccadic suppression turns off RN-I activity after the first saccade is made; and after the first saccade is performed, RN-I combines the new eye-position information with the input from RN-II and produces a new ME for the second saccade.
|
TRAINING PROCEDURE.
In each training cycle, two targets (T1 and T2), either visual or
auditory, are randomly selected for position and modality and presented
to the network for a duration of one time step. With E0 representing
the initial eye position, V1 for the location of the first visual
target in retinal coordinates, and A1 for the location of the auditory
target in head-centered coordinates, the desired ME output at the end
of the delay period is ME = V1 for visual targets or ME = A1 E0 for auditory targets. After the first saccade is made,
the eye is moved to the new position E1. The desired output at the time
of the second saccade is ME = V2 + E0
E1 or ME = A2
E1. Here V2 and A2 indicate the second visual and auditory targets.
|
RESPONSES OF THE HIDDEN UNITS. After training, most hidden units in RN-I and RN-II developed localized RFs for both visual and auditory inputs. When eye position was centered in the orbit, the visual and auditory RFs of a given unit were usually aligned. The RFs were very large; some of them even occupied up to half of the input space. The responses of the hidden units to visual or auditory targets were gain modulated by initial eye position. Later we will discuss how these gain fields are essential for coordinate transformations.
As in the single-memory-saccade model, most hidden units in the double-saccade model exhibit sustained memory activity to visual and auditory targets. Here we show the typical response patterns of the hidden units to visual targets to make direct comparisons with the experimental data in Fig. 6. Figure 9 illustrates five response patterns of two typical hidden units. Figure 9, A-C, shows the responses of a unit in the RN-I network; Fig. 9, D and E, shows the responses of a unit in the RN-II network. The left panel shows the spatial arrangements of the saccades. The RFs of the units are outlined with the dashed areas. The initial eye positions are indicated with + symbols. The two targets are labeled as T1 and T2, and the two saccades are labeled as S1 and S2. The responses of the units are shown on the right panel with the height of the vertical bars proportional to the responsiveness. The targets are flashed sequentially on the first and fourth steps and the model produces the first saccade (S1) at the 10th time step and the second saccade (S2) at the 13th time step, as indicated (top right).
|
COORDINATE TRANSFORMATIONS. One traditional question about double-saccade tasks is how the motor vector for the second saccade is computed, given that eye position at the time of the second saccade is different from the time when the visual target was flashed. How are the spatial transformations required for double-saccades carried out? To answer this question, we examined how eye-position information is utilized by the hidden units in the model.
We first examined the hidden units in the RN-II network. The RF of a unit was first measured at the central eye position. Next, for 8 × 8 eye positions, the responses to a target presented in the RF were measured. Results showed that the responses of most hidden units were modulated by eye position. The 2-D plot of responses against different eye positions is called gain field (GF) as reported by Andersen et al. (1985)
|
|
|
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The mechanisms for programming memory saccades and sequential
saccades remain unclear to neurophysiologists. A number of
computational models of saccade generation have been proposed.
Dominey and Arbib (1992) proposed a cortical-subcortical
model of the control of saccadic eye movement and suggested that the
parietal cortex may dynamically remap the target locations in saccade
ME maps to program double saccades. The network model developed by
Droulez and Berthoz (1991)
showed that target position
could be memorized in a sensory map and updated with eye-movement
signals. Krommenhoek et al. (1993)
trained a neural
network to compute MEs using information about eye position. These
computational approaches yield valuable insights into memory saccades.
On the other hand, the frameworks in these models did not correspond
well to known neurophysiological data. Given that LIP neurons can
withhold their saccade-related activity and participate in programming
double saccades, the network model in this report studied the memory
activity in area LIP for saccadic eye movements. With the
implementation of the single-purpose rule in the training process, the
network developed lateral excitation-inhibition (the push-pull
structure) that was essential to memory and sequential saccades. The
simulated neurons in our model exhibited properties similar to those
recorded in area LIP. After training to make double saccades, the model
carried out the coordinate transformations required to program double
saccades by the means of gain modulations. In our model, one group of
neurons maintain the sensory memory of saccadic targets, while the
other group of neurons encode the motor plan of an impending saccade.
Thus coding the motor commands of double saccades is achieved by
different neuronal populations rather than by dynamically remapping the
same neuronal population.
One prediction of the model is that neurons corresponding to the memory
buffer RN-II respond to the second target but not the first one.
Mazzoni et al. (1996b) found that 16% of LIP neurons encoded the location of the second target in a memory-saccade task.
These cells were referred as the "sensory memory" cells. It would
be interesting to test experimentally whether these cells encode only
the second target or any visual stimuli within their RFs. Furthermore
the model predicts that the responses of RN-II neurons are gain
modulated by the initial eye position. Due to the push-pull structure,
the activity of RN-II neurons is not affected by the new eye position.
This remains to be tested experimentally.
Push-pull structure
Examining data from various kinds of delayed-saccade experiments,
we found a common feature in the response patterns of LIP neuronsonce
a neuron is engaged in a saccade command, its activity is maintained
irrespective of further stimuli; the neuron starts to respond to
another stimulus only after the saccade being encoded is completed or
the intention of the saccade is dismissed by some high level command.
We called this feature single-purpose. This feature is essential for
the behaviors of a motor system. The eyes can never make saccades
simultaneously to two different spots.
The single-purpose feature is used as a constraint for the training process of our networks. This constraint results in the push-pull structure, i.e., excitatory connections between units with similar preferred saccadic directions and inhibitory connections between units with dissimilar preferred directions. Such an excitation-inhibition structure is the neuronal basis for the single-purpose feature. In the extended model of the double-saccade system, the push-pull structure allows the model to program two saccadic commands sequentially, rather than mixing the two commands into one. Therefore the excitation-inhibition connections ensure that LIP encodes the next planned saccade.
Ideally, a push-pull structure suppresses any irrelevant stimuli that differ from the target location. However, through simulation we found that responses to stimuli close to the target were often sustained rather than suppressed. The minimal distance for an irrelevant stimulus to be suppressed varied from unit to unit but was roughly in an order of about 10°. Within this distance, the output memory activity represented a ME that was a weighted average between the irrelevant stimulus and the target. The function of the connection weights in Fig. 5A reflected this inaccuracy: Excitatory connections could occur to PDs that are 10-20° apart. Several reasons contribute to this inaccuracy: the tuning of the hidden units is broad; the limited number of the hidden units prevents precise excitatory connections; and the training samples of the stimulus locations are often more than 10° apart. We expect that using a large set of hidden units and finer spaced training stimuli, or an attention mechanism, would improve the accuracy of the push-pull structure.
A number of neural network studies have used a push-pull structure as a
memory-storage mechanism (Grossberg and Levine 1975; Seung 1996
; Zhang 1995
).
Typically, adjacent units in these networks excite each other while
distant units inhibit each other. Such an arrangement could prevent
recurrent activities from spreading to the whole network. Thus a
push-pull mechanism also enforces the stability of a recurrent network.
Salinas and Abbott (1996)
recently proposed another
functional role of the push-pull structure in the parietal cortex. They
found that neurons in a recurrently connected network with push-pull
connections could perform a product operation on additive synaptic
inputs. The resulting multiplicative gain modulation is important for
coordinate transformations in the parietal cortex. In our model, the
push-pull structure emerged as the result of the single-purpose
feature. Moreover, the excitation and inhibition were organized
according to the preferred directions of the units, rather than the
geometric positions. Goldman-Rakic (1995)
observed a
similar lateral inhibition structure in the opponent memory field of
neurons in the frontal cortex. Schlag et al. (1998)
found that, in the FEF, cells that encoded similar eye movements
mutually excited each other while silencing those that would produce
conflicting eye movements. Since the single-purpose feature might be
common for cortical areas involved in motor planning, it is likely that
the push-pull structure is a principle applicable to these cortical areas.
An analogy to the single-purpose feature is the winner-take-all
mechanism. The latter has been widely applied to the models of visual
search processes (Braddick 1997; Ferrera and
Lisberger 1995
; Lee et al. 1999
). In a visual
search task, a target is searched among a number of distractors. A
winner-take-all mechanism allows the neurons representing the target
and the distractors to compete against one another. Attention serves to
bias the outcome of this competition toward the direction of the
selected target. As a result, the neuronal response to the target
remains and the response to the distractors is suppressed.
Salzman and Newsome (1994)
also proposed that a
winner-take-all mechanism existed in the motion cortex (area MT and
MST). When more than one motion cue was presented, monkeys chose the
direction encoded by the largest signal in the representation of motion
direction. Braddick (1997)
suggested that local motion
detectors use winner-take-all interactions in global motion analysis.
The single-purpose feature and the winner-take-all mechanism are similar in that both generate only one single output representation. The latter evokes neuronal competition based on the context of stimuli and enhances the response to the target stimulus through attention. Such a mechanism is not suitable for area LIP because LIP neurons are generally insensitive to stimulus context and thus do not support a competition process. The target to be represented in LIP is chosen by motor intention and is not the result of an attention-biased competition. The winner-take-all mechanism handles spatial conflicts in visual selection. The single-purpose feature assures no conflicts in a temporal sequence of motor plans. Neurophysiological data support our assumption that a single-purpose feature exists in area LIP. It would be interesting to test this assumption further by recording the responses of LIP neurons to a target and many distractors presented simultaneously.
Coordinate transformations
A traditional question about planning double saccades is how the
motor command of the second saccade is computed. Two hypotheses have
been proposed. One hypothesis is head-centered coding (Robinson 1975; Sparks and Mays 1983
): the absolute target
location in head-centered coordinates is computed and stored, and then
the new eye position after the first saccade is subtracted. With this
hypothesis, one would expect to find neurons that encode visual targets
explicitly in head-centered coordinates. However, physiological studies
have largely failed to find such neurons. Most LIP neurons have retinal RFs with their responses modulated by eye position. The other hypothesis is retinotopic coding, also called vector subtraction (Bruce and Goldberg 1985
; Scudder 1988
):
the retinal location of the target is stored and then the change of eye
position is subtracted. This hypothesis requires neurons that
explicitly encode the change of eye position.
The simulation results of this report suggest a third possibility:
instead of computing explicit head-centered target locations or the
change of eye position, LIP neurons utilize eye position with the use
of GFs to carry out coordinate transformation through the distributed
activity of many neurons. In the double-saccade model, information
about the second target location is combined with the current
eye-position signal through aligned RF-GF gain modulation to form a
distributed head-centered representation. After the first saccade, the
new eye position comes in and is combined with the head-centered
representation through the opposite RF-GF structure so that the ME of
the second saccade is computed. This model does not require individual
neurons to encode target locations in explicit head-centered
coordinates. The presence of GFs could account for the computation of
double saccades. Moreover, the experimental results by Li et al.
(1995) suggested that a distributed head-centered
representation of targets might be maintained in LIP for programming
sequences of eye movements. Using reversible lesions of LIP, Li et al.
found that the monkeys depended on the new eye position more than the
retinal vectors to make the second saccade. Thus this model fits
current data well.
Theoretically, coordinate transformations suggested by the first two
hypotheses above can be carried out by shift circuits. Quaia et
al. (1998) proposed a shift circuit to simulate RF remapping in
LIP, in which the FEF neurons shifted the RFs of the LIP neurons. However, the large RFs and the distributed coding feature of parietal neurons make it difficult for a precise shift circuit to work. The
modeling results in this report show that the gain modulation is
essential to carry out the coordinate transformations in area LIP.
Using this strategy, neurons may remain in retinotopic coordinates for
visual stimuli. With eye-position modulation, the distributed activity
of these neurons can represent the stimuli in other coordinates. Varying RF-GF structures carries out different kinds of
transformations. Hence the gain modulation along with distributed
coding is an efficient way to achieve sensorimotor transformations
without using complex shift circuits. Other theoretic studies also
revealed the importance of GF properties in coordinate transformations. Goodman and Andersen (1990)
analytically demonstrated
that an aligned GF and RF relationship was required for transformations from oculocentric to craniocentric coordinates. A similar mechanism of
eye-position modulation in the saccadic system was studied by
Krommenhoek et al. (1993
, 1996
). They developed a neural
network in which retinal signals and an efference copy of eye position could be remapped to a ME map in two steps: distributed coding of
head-centered target position at one level and of ME in eye-centered coordinates at another stage.
RF remapping versus ME coding
Experimental data demonstrate that the memory activity of LIP
neurons encodes saccadic eye movements (Snyder et al.
1997). Furthermore it has been shown that LIP neurons
encode motor intention, irrespective of the actual execution of the
planned movements (Bracewell et al. 1996
; Snyder
et al. 1997
). The simulated LIP neurons in our models indeed
encode the impending saccade. On the other hand, Duhamel et al.
(1992)
proposed that LIP neurons encoded sensory stimuli
instead of saccades. In their experiment, as illustrated in Fig.
14A, the monkey was required
to make a saccade to a remembered target, and this saccade would bring
a stimulus onto the RF of the LIP neuron being recorded. It was found
that the neuron responded to the stimulus outside its classic RF when an impending saccade brought the stimuli into the RF. Some neurons became active before the stimulus was brought into the neurons' RFs by
the saccades. Duhamel et al. thus concluded that the RF of the neuron
transiently shifted with the eyes to the retinal location at which the
stimulus could excite the neuron. This hypothesis is diagrammatically
illustrated in Fig. 14B. During fixation, the representation
of the visual scene was stable (left). Immediately before or
during the saccade the cortical representation shifted into the
destination of the intended saccade. The neuron thus began to respond
to the stimulus at a new retinal location (middle). After
the eye movement, the cortical representation shifted back to match the
visual inputs so that the neuron continued to respond to the stimulus
(right).
|
Quaia et al. (1998) proposed a model to explain the
observed shifts of RFs. In their model, a group FEF neurons carry the signal about impeding saccades; LIP phasic-tonic neurons have stable
local RFs and LIP phasic cells have shifting RFs. If a FEF neuron and a
LIP phasic-tonic neuron are active at the same time, a LIP phasic
neuron, whose RF is equal to the difference of the RF of the LIP
phasic-tonic neuron and the motor field of the FEF neuron, is
activated. All pairs of LIP phasic-tonic neurons and FEF neurons, whose
RF/motor field difference is equal, must be connected to the same LIP
phasic neuron. Therefore the RFs of the LIP phasic neurons are shifted
with impending saccades. Such a model requires a specific connectivity:
precise pairings between LIP and FEF neurons. It also requires specific
computations at the dendritic level, i.e., a multiplication between
cells in a pair and a logic OR computation between different
pairs to the same LIP phasic cell. Both the connectivity and the
computations are biologically difficult to implement. Moreover,
although the model explained RF remapping, it did not account for the
coordinate transformations in sequential saccades.
It is interesting to see how our model responds to the paradigm of Fig. 14A. In Fig. 9C, the hidden unit responded to the second target even though the target never appeared in the unit's RF. This response appeared as if the RF of the unit shifted to capture the second target, while in fact there was no RF shift and the response was merely encoding the impending preferred saccade. Figure 14C illustrates the model results in the same experiment. After the target onset, some neurons fire to the first intended saccade, and the neuron being recorded has no response (left) since its RF is not in the preferred direction. Next, after the command to make the saccade is issued, the on-going activity is suppressed by postsaccadic suppression. The network computes the ME of an intended saccade to the stimulus based on the inputs of the new eye position and the information about the stimulus location. As a result, the neuron under recording becomes active since its PD is in the direction of the next intended saccade (middle). Finally, the first saccade is completed, and the neuron continues to fire for the next intended saccade, although this saccade may not actually be executed. Therefore the hidden units in the model can encode stimuli outside the unit's RF using dynamically updated information about eye position. In this model, the cortical representation does not shift toward the stimulus and then shift back. Instead, the activity of one group of cells goes up while the others come down for a new saccade plan. Thus different populations of neurons are engaged and disengaged rather than individual neurons shifting their retinal RFs back and forth.
Duhamel et al. (1992) reported that 44% of LIP neurons
became active before the saccade brought the stimuli into the neurons' RFs. Our model can account for these predictive responses. Before the
first saccade is made, the RN-I network may already begin to compute
the ME of the second saccade using the information about the new eye
position. Therefore the units coding for the second saccade could
become active before the first saccade. Thus the observed predictive
remapping could be the result of the sequential activation of different
populations of LIP neurons rather than jumping RFs. There is
experimental evidence that signals for new eye positions appear in some
LIP neurons before the beginning of a saccade (C. Li, B. Breznen, and R. A. Andersen, unpublished data). In
addition, psychophysical studies by Dassonville et al. (1995)
and Schlag and Schlag-Rey (1995)
indicated that spatial localization during saccades was largely based
on updating of the internal representation of eye position.
Modeling multiple sequential saccades
The present model only simulated single and double saccades. How
would the model handle more than two saccadic targets in a sequence?
Our double-saccade model can be viewed as a schematic version of a
model of multiple-sequential-saccades. In this report, we focused our
model on how the coordinate transformations of two sequential saccades
were carried out. We could extend the model to handle multiple saccades
in the following two ways: the model has more memory buffers each
holding the memory of every additional saccade and the input layer of
the model could correspond either to sensory inputs or to inputs from a
memory system. The first possibility is perhaps too rigid and the
architecture is difficult for brain to implement. In the second
possibility, the target locations and the orders of the presentation
are held in the memory system while RN-I and RN-II networks carry out
the coordinate transformations for the impending saccade. Behavioral and physiological evidence supports this possibility. Training monkeys
to perform more than two sequential saccades is difficult. Barone and Joseph (1989) were able to train monkeys to
make sequential saccades to three fixed target locations. However, they
only observed prefrontal neurons that responded to the first target or
the second target, but no neurons responded to the third target. The
result suggested that the memory for more than two sequential targets was not directly handled by the parietal or the prefrontal cortex.
In summary, the models in this report capture the important characteristics of LIP neurons and provide insights into the mechanisms of LIP in programming eye movements. By optimizing the network to implement various saccadic tasks, two important properties emerge from the model: push-pull recurrent connections and opposite/aligned GF structures. These properties are the basics for programming memory saccades and sequential saccades. The consistency of simulated results and current experimental data suggests that the models are well suited to describe the sensorimotor processing in area LIP and thus can be used as a framework to guide future experiments in understanding the neural functions of LIP.
![]() |
ACKNOWLEDGMENTS |
---|
We thank P. Mazzoni for providing experimental data, C. Li for valuable discussions, and K. Shenoy for valuable comments on the manuscript.
This work was supported by National Eye Institute Grant EY-05522. J. Xing was supported by the Del Webb Foundation fellowship.
![]() |
FOOTNOTES |
---|
Address for reprint requests: R. A. Andersen, Div. of Biology, 216-76, California Institute of Technology, Pasadena, CA 91125 (E-mail: andersen{at}vis.caltech.edu).
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
Received 30 July 1999; accepted in final form 5 April 2000.
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|