1Howard Hughes Medical Institute, Department of Physiology and Biophysics, and Regional Primate Research Center, University of Washington, Seattle, Washington 98195-7290; and 2Howard Hughes Medical Institute and Department of Neurobiology, Stanford University School of Medicine, Stanford, California 94305
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Shadlen, Michael N. and William T. Newsome. Neural Basis of a Perceptual Decision in the Parietal Cortex (Area LIP) of the Rhesus Monkey. J. Neurophysiol. 86: 1916-1936, 2001. We recorded the activity of single neurons in the posterior parietal cortex (area LIP) of two rhesus monkeys while they discriminated the direction of motion in random-dot visual stimuli. The visual task was similar to a motion discrimination task that has been used in previous investigations of motion-sensitive regions of the extrastriate cortex. The monkeys were trained to decide whether the direction of motion was toward one of two choice targets that appeared on either side of the random-dot stimulus. At the end of the trial, the monkeys reported their direction judgment by making an eye movement to the appropriate target. We studied neurons in LIP that exhibited spatially selective persistent activity during delayed saccadic eye movement tasks. These neurons are thought to carry high-level signals appropriate for identifying salient visual targets and for guiding saccadic eye movements. We arranged the motion discrimination task so that one of the choice targets was in the LIP neuron's response field (RF) while the other target was positioned well away from the RF. During motion viewing, neurons in LIP altered their firing rate in a manner that predicted the saccadic eye movement that the monkey would make at the end of the trial. The activity thus predicted the monkey's judgment of motion direction. This predictive activity began early in the motion-viewing period and became increasingly reliable as the monkey viewed the random-dot motion. The neural activity predicted the monkey's direction judgment on both easy and difficult trials (strong and weak motion), whether or not the judgment was correct. In addition, the timing and magnitude of the response was affected by the strength of the motion signal in the stimulus. When the direction of motion was toward the RF, stronger motion led to larger neural responses earlier in the motion-viewing period. When motion was away from the RF, stronger motion led to greater suppression of ongoing activity. Thus the activity of single neurons in area LIP reflects both the direction of an impending gaze shift and the quality of the sensory information that instructs such a response. The time course of the neural response suggests that LIP accumulates sensory signals relevant to the selection of a target for an eye movement.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Primates use vision to guide their interactions with the environment. In wakefulness, the brain generates a steady stream of decisions to shift the gaze, to position the body, and to grasp, avoid, or classify objects, often with the guidance of data from the visual cortex. Unless an action is purely reflexive or purely capricious, a higher level of information processing must link sensation to action. Sensory data must be interpreted to execute, revise, or delay pending action. The goal of this study is to investigate the neural underpinnings of one such interpretive mechanism: a simple decision process in a two-alternative, forced-choice psychophysical paradigm.
We trained monkeys to discriminate opposed directions of motion in a
stochastic random dot display and to report the perceived direction
with a saccadic eye movement to one of two visual targets. At least
three processing stages must be engaged during each trial the monkey
performs (Fig. 1). First, a sensory
process must extract motion information from the visual image and
represent the outcome within the visual cortex. For our task, the
relevant representation of motion resides largely in areas MT and MST
of extrastriate cortex (Britten et al. 1992,
1996
; Celebrini and Newsome 1995
; Croner and Albright 1999
; Newsome and Paré
1988
; Salzman et al. 1992
; Shadlen et al.
1996
). Neurons in MT and MST generate smoothly varying
responses that reflect the amount of motion energy within a specific
band of velocities (direction and speed) to which they are tuned
(Albright 1984
; Maunsell and Van Essen
1983
; Simoncelli and Heeger 1998
; Zeki
1974
). Second, the map of motion direction in MT and MST must
be interpreted, or read out, to form a
categorical decision: is the net motion flow in direction A or
direction B? Third, after a decision is formed, it may need to be
stored in working memory until an operant response is programmed and
executed. In our task, neural signals for guiding the operant response
must ultimately emerge from eye movement-related structures such as the superior colliculus, the frontal eye field, and the lateral intraparietal area (LIP) of the inferior parietal lobe, areas that have
been studied extensively over the past few decades (for reviews, see
Andersen et al. 1992
; Colby and Goldberg
1999
; Schall 1995
). We therefore have a
reasonable base of knowledge concerning the sensory and motor
processing stages that must be engaged during performance of the task,
but we know virtually nothing concerning the key cognitive stage of
decision formation (see also Romo and Salinas 2001
).
|
As an initial step toward analysis of the decision process, we have
studied the activity of neurons in LIP that carry high-level signals
appropriate for identifying salient visual targets and ultimately for
guiding saccadic eye movements. Many neurons in LIP modulate their
level of activity when there is sufficient information to plan a
saccade, even when execution of the saccade may be delayed by several
seconds (Colby and Goldberg 1999; Mazzoni et al.
1996
; Snyder et al. 2000
). Our central question
is whether the activity of these neurons can provide insight into the
process of decision formation during performance of our psychophysical task. Differentiating decision-related activity from strictly sensory
activity is reasonably easy. By requiring the monkey to discriminate
weak, noisy motion signals near psychophysical threshold, we create a
situation in which the decision varies from trial to trial for repeated
presentations of the same motion stimulus (i.e., the monkey decides
correctly on some trials and incorrectly on others). To a first
approximation, sensory activity will reflect the motion in the stimulus
irrespective of what the monkey decides, whereas activity in higher
level circuits that interpret the motion signals should vary
strongly with the monkey's decision.
Differentiating decision-related activity from strictly motor activity,
however, is not so straightforward. In a trivial sense, all motor
signals are decision-related in that they reflect the outcome of the
decision process. The key problem is to differentiate processing stages
in which the decision is actually formed and represented from stages
that simply represent a movement to be executed. We have adopted two
tactics to gain experimental leverage on this issue. First, we have
introduced an instructed delay period between presentation of the
motion stimulus and the "go" signal to execute the saccadic eye
movement. This tactic delays overt motor activity until the end of the
trial, thereby separating the period of motion viewing (hence the
decision) from motor execution. Second, we have taken advantage of a
fact that all psychophysical subjects know well: all decisions are not
created equal. Subjects are certain of decisions made on the basis of
strong sensory information but are quite doubtful of decisions made on
the basis of ambiguous evidence. We assume that neural circuits
intimately linked to the process of decision formation should reflect
this level of certainty either in the amplitude or timing of
decision-related activity (Basso and Wurtz 1998). In
other words, decision-related activity should bear some signature of
the intensity of the sensory stimulus.
We have found that some neurons in LIP are plausible candidates for participating in the decision process. These neurons generate sustained activity that predicts the impending saccade, and thus the monkey's decision. Both the amplitude and timing of this activity reflect the certainty of the decision and cannot be accounted for by any parameter of the eye movement itself that we have investigated.
We have briefly described some of these results elsewhere
(Shadlen and Newsome 1996; Shadlen et al.
1994
).
![]() |
METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Subjects, surgery, and daily routine
We performed experiments on two adult rhesus monkeys
(Macaca mulatta, 1 male and 1 female) weighing 8-9 kg. The
monkeys were surgically implanted with a head-holding device
(Evarts 1968), a scleral search coil for monitoring eye
movements (Judge et al. 1980
), and a recording cylinder
over the intraparietal sulcus. After recovery from surgery, the animals
engaged in daily training or experimental sessions lasting 2-6 h. The
monkeys were trained to perform a two-alternative, forced-choice
direction discrimination task near psychophysical threshold. The
monkeys were also trained on a variety of fixation and saccadic eye
movement tasks as described below.
The monkeys worked for liquid rewards, and their daily water intake was therefore controlled. All surgical and behavioral procedures were in accordance with the U.S. Department of Health and Human Services (National Institutes of Health) Guide for the Care and Use of Laboratory Animals (1996).
Visual stimuli
Visual stimuli were generated on a PC/486 computer using a
Pepper SGT+ graphics board (Number 9 Computer)
attached to a Sony multiscan monitor (60 Hz noninterlaced) placed 57 cm
away from the monkey. The system displayed fixation and saccade targets
as well as the dynamic random-dot motion stimuli used for the direction
discrimination experiments. The motion display was similar to stimuli
used in previous investigations (e.g., see Britten et al.
1992).
Random dots were plotted within a circular aperture of 5-10° diam. Each dot was displayed for one video frame and then replotted 50 ms (3 video frames) later either at an appropriate spatial displacement for apparent motion (typically 3-7°/s velocity) or at a random location. The probability that a particular dot would be displaced in motion is termed the motion coherence, expressed throughout the paper as a percentage. For example, if the coherence is 50%, then a dot that appears in frame 1 has a 0.5 probability of coherent displacement in video frame 4 and an equal chance of being randomly replaced somewhere else in the viewing aperture. Dots that first appear in video frame 2 are not seen in frames 3 and 4 and are subsequently plotted with the appropriate displacement (or randomly) in video frame 5, and so on. The dots were white on a black background and plotted at a density of 16.7 dots per deg2 per s, as in previous studies.
For some experiments, we used the same sequence of random dots for all trials at each coherence-direction combination. The manipulation did not lead to any detectable difference in the LIP response, and we have therefore combined these experiments with those in which a fresh random-number seed was used on every trial.
Electrophysiological recording
We recorded neural activity using tungsten microelectrodes
(impedance 0.8-1.2 M at 1 kHz; FHC) inserted into the cortex
through a 23-gauge stainless steel guide tube that punctured the dura mater. The tip of the guide tube was either in the superficial layers
of area 7a or in the intraparietal sulcus, outside of the cortex. The
guide tube was held in place by a plastic grid fitted inside the
recording chamber (Crist Instruments). The grid enabled us to record
from the same location along the bank of the intraparietal sulcus for
several days.
Signals were amplified and viewed on an oscilloscope screen. Single
units were isolated on the basis of voltage waveform using a
voltage-time window discriminator (Bak Electronics). The time of each
action potential was stored on computer disk to the nearest millisecond, along with the time of trial events that identified the
time of fixation, stimulus onset, stimulus offset, and saccade. Records
of eye position were stored to disk (250 samples/s) on a portion of the
experiments. Data acquisition and experimental control were
accomplished using a PC/486 running a real-time data acquisition system
(Hays et al. 1982). The trial events, spikes, and eye
position data were analyzed off-line using software tools developed in
Matlab (The Mathworks).
Behavioral tasks
The primary goal of the study was to examine the responses of
neurons during performance of a motion discrimination task similar to
one used in previous investigations of areas MT and MST (Britten et al. 1992, 1996
; Celebrini and Newsome
1994
; Newsome and Paré 1988
;
Salzman et al. 1992
). For the present study, neurons
were selected on the basis of their responses during saccadic eye
movement tasks, described below. For all tasks, the monkey was required to fixate a small red spot (the fixation point, FP) until its extinction. If at any time, the gaze fell outside of a 2 × 2° window centered on the FP, the trial was aborted. The window
accommodated the small variation in eye position from trial to trial,
but the monkey's gaze on any one trial was typically stable. A brief
description of each task follows.
DELAYED SACCADES WITHOUT AND WITH MEMORY. Neurons were screened by their responses in a delayed saccade task. On fixating a central spot, a bright red saccade target appeared in the periphery. The monkey was required to maintain fixation until the fixation spot was extinguished and was then required to make a saccade to the target within 500 ms. The delay period between onset of the saccade target and offset of the fixation point ("go" signal) was randomized from 0.5 to 2.0 s. We sampled LIP using this task, searching for those neurons that discharged during the delay period.
On isolating an appropriate cell, we identified the region of the visual field that led to robust responses during the delay period. We will refer to this region as the response field (RF) of the neuron. Some investigators would use the term receptive field or motor field, depending on whether the emphasis is placed on the response to visual targets or the preparation to make an eye movement (Mazzoni et al. 1996MOTION DISCRIMINATION TASK. After delineating the boundaries of the RF, we set up a direction discrimination task after the design illustrated in Fig. 2. One target, henceforth called T1, was placed in the RF of the neuron under study, while a second target, T2, was placed well outside the RF (often in the opposite hemifield). The stimulus aperture was positioned so that the coherent dots moved toward one or the other target on each trial. We positioned the stimulus aperture so as to minimize stimulation of any visual receptive field.
|
![]() |
(1) |
PASSIVE VIEWING OF RANDOM-DOT MOTION. We examined the response to random-dot motion during trials in which the monkey simply fixated. No saccade targets appeared on these trials, and the monkey was rewarded simply for maintaining fixation throughout the motion-viewing period. The dots appeared in the same location as in the discrimination task, outside the neuron's RF. The strength of motion was 51.2% coherence, which matched the strongest motion used in the discrimination experiments. This fixation task was often performed in a separate block of trials but was sometimes randomly interleaved with discrimination and delayed saccade trials. The task is the only one in which saccade targets do not appear shortly after fixation.
DELAYED SACCADES IN THE PRESENCE OF MOTION DISTRACTOR. This task examines the response to visual motion during preparation of a saccadic eye movement that is specified by a single target. The task resembles the discrimination task with the important exception that only one saccade target appears throughout the trial. The motion coherence was 51.2% and the direction was toward or away from the target. Both the direction of motion and the target location were randomized and independent. The monkey was rewarded for making a saccade to the one target. Importantly, the direction of motion had no bearing on the monkey's reward. This task was always performed in a separate block of trials to distinguish it from the discrimination task. Because this task potentially reinforces a dissociation between motion direction and eye movement response, we included it only after obtaining data on the other tasks.
Data analysis
Raw data were stored as spike events timed to the nearest millisecond. These responses were collated into trials along with various time markers to compute standard peristimulus time histograms and rasters, and to count spikes occurring between trial events. Analysis was performed off-line using custom software developed in Matlab (The Mathworks). Each of the intervals comprising our trials (from target onset to motion onset, from motion onset to offset, and from motion offset to the extinction of the fixation point) contained a variable amount of time. We therefore present our data with respect to different event markers (e.g., motion onset). To compute summary statistics, we used the average spike rate between two trial events or in epochs aligned to common trial events (e.g., 1st 500 ms of motion-viewing period).
We performed standard comparisons of means using t- and
F-tests. When examining results across the population of
neurons in our data set, we applied multiple regression models in which
cell identity was incorporated as an independent categorical variable. For example, to analyze the effect of motion strength on neural response (Fig. 9, A-D) we fit the model
![]() |
(2) |
To test whether saccade direction affects the response, we calculated
the probability of obtaining an F-statistic under the null
hypothesis, H0: = 0. The
F-statistic is derived from the extra sum of squares
obtained by fitting a reduced model in which
= 0 (Draper and Smith 1966
). If there are m data
points and n neurons, then for two models that differ by
k = 1 degrees of freedom
![]() |
(3) |
ANALYSIS OF PREDICTIVE ACTIVITY.
We computed a predictive index that describes the association between
neural response and the monkey's decision. The index approximates the
ability of the experimenter to predict the monkey's behavior from the
neural response. It is the probability that a random sample of the
neural response associated with one behavioral choice would exceed the
neural response associated with the other behavioral choice. Denoting
the response associated with the two choices by
y1 and
y2, this is the joint probability over
all possible criteria, , of observing
y1 =
and
y2 <
![]() |
![]() |
(4) |
Saccadic eye movements
For 45 neurons we maintained records of the monkey's eye position during discrimination and saccade trial types. Eye position was sampled at 1 kHz per horizontal and vertical channel and stored on disk at 250 Hz per channel. From these eye position traces we derived the beginning and endpoint of each saccade, its amplitude (AMP), direction (DIR), peak velocity (VMAX), duration (DUR), latency (LAT), and accuracy (ACC). We defined accuracy as the reciprocal of the RMS distance from the mean endpoint. We were interested in whether trial-to-trial variation in the saccadic eye movement affected the neural response.
Histology and identification of recording sites
The animals were killed with an overdose of pentobarbital sodium
(Nembutal) and perfused through the heart with saline followed by a
10% Formalin fixative. Tissue blocks containing the region of interest
were equilibrated in 30% sucrose, then cut in 48-µm sections using a
freezing microtome. Sections at regular intervals through the
intraparietal sulcus were stained for cell bodies with cresyl violet
and for myelinated fibers by the method of Gallyas (Gallyas
1979). We confirmed that our recordings were from neurons in
the lateral bank of the intraparietal sulcus. Figure
3 illustrates a typical histological
section containing several electrode tracks. The guide tube was
directed toward the lateral bank of the IPS (visible in adjacent
sections), and electrode tracks from this guide tube coursed down the
lateral bank for several millimeters before exiting into white matter.
Although we cannot reconstruct individual penetrations made over the
course of many months, it is clear that the bulk of our recordings were from the more posterior and medial region of LIP, corresponding to the
region of LIP that projects to the frontal eye field and area 8Ar
(Andersen et al. 1990
; Cavada and Goldman-Rakic
1989
; Petrides and Pandya 1984
; Schall et
al. 1995
).
|
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Basic response properties on delayed saccade tasks
We recorded from 104 neurons in area LIP of 2 adult rhesus
monkeys. All of the neurons included for analysis were active during a
delayed saccade task and exhibited a clear preference for targets in a
restricted portion of the visual field, termed the response field (RF;
see METHODS). In nearly all cases we ensured that such delay-period activity did not represent a visual response to the saccade target by extinguishing the target after 200 ms and requiring the monkey to make a memory-guided saccade. Figure
4 illustrates such responses for one LIP
neuron. The monkey made memory-guided saccades to eight test locations,
which were arranged concentrically around the fixation point at an
eccentricity of 10°. The response rasters are arranged concentrically
in the figure to denote the saccade direction for each raster. The
response was largest when the remembered target was to the left of
fixation. When the target appeared outside the RF, the response was
attenuated until after the saccade. The mixture of visual,
delay-period, and perisaccadic responses apparent in these rasters has
been described by other investigators (Barash et al.
1991a,b
; Colby et al. 1996
; Gnadt and
Andersen 1988
; Platt and Glimcher 1997
). We used
the delay period activity to guide placement of choice targets and
random dots in the direction discrimination task.
|
Response during motion discrimination
Our primary goal was to ascertain how such neurons respond when the instruction for the saccade is a motion stimulus presented outside the neuron's RF. In this setting, a saccade into or away from the RF indicates the monkey's judgment of direction. We reasoned that the development of neural activity related to the animal's choice might yield insight into the neural underpinnings of decision formation within the cortex. An example from a typical experiment is illustrated in Fig. 5. The responses shown in the left column accompanied trials in which the monkey decided that motion was toward the RF and made a saccade to the corresponding target (T1). For all three motion strengths shown in the figure, the response increased during the motion-viewing period and remained elevated throughout the delay period. Compare this pattern of responses to those accompanying the opposite decision (right column). During the motion-viewing period, the response diminished and remained attenuated through the delay period until the monkey made its saccade to the target outside the RF (T2). For both choices, the largest response modulations occurred during the motion-viewing period, which is the interval in which the monkey must arrive at its judgment of direction. Importantly, the modulation apparent in Fig. 5 does not reflect the sensory stimulus per se: substantial choice-related modulation occurred on the 0% coherence trials, which contained no net motion (bottom row), and on error trials as well (see Fig. 11). Moreover the modulated activity levels persisted throughout the delay period, after the random dots were extinguished. The response seems to reflect the monkey's decision about direction, rather than the actual motion content of the sensory stimulus.
|
In our paradigm, the monkey can plan an appropriate saccadic eye
movement as soon as a decision is made about the direction of motion in
the stimulus, raising the possibility that the activity of neurons like
the one illustrated in Fig. 5 simply reflects preparation for moving
the eyes. This possibility is reinforced by the fact that similar
predictive activity has been seen during performance of this task in
overtly oculomotor structures such as the frontal eye field (Kim
and Shadlen 1999) and superior colliculus (Horwitz and
Newsome 1999a
).
Closer analysis of the data reveals, however, that neural activity in LIP cannot be explained entirely by motor preparation. The histograms in Fig. 5, for example, suggest that the predictive activity varied in intensity as a function of motion strength. The upper set of responses was obtained when the monkey viewed a strong motion stimulus. These trials were easy, and this is reflected in a rapid rise of activity early in the trial. The average spike rate during motion viewing was 39.2 ± 1.3 spikes/s (mean ± SE) for T1 choices and 13.9 ± 0.6 spikes/s for T2 choices. For the more difficult discriminations, shown at the bottom of the figure, the response modulation occurred later in the motion viewing period and never attained the level seen at the strong motion coherences (32.4 ± 1.3 and 17.5 ± 1.3 spikes/s during the motion-viewing period for T1 and T2 choices, respectively).
For the neuron in Fig. 5, decisions for motion away from the RF (T2 choices; right column) were accompanied by a suppression of activity that varied little across motion strengths. However, for many LIP neurons the effect of motion strength was more apparent for T2 choices than for T1 choices, as illustrated in Fig. 6. When the monkey viewed the 0% coherent display and chose the target outside the RF (bottom right raster and PSTH), the average response during the motion-viewing period was 12.4 ± 1.3 spikes/s. When a strong motion stimulus was directed away from the RF, correct T2 choices were associated with an average response of 7.6 ± 0.7 spikes/s (top right; P = 0.0012, t-test).
|
All of the neurons in our data set responded more strongly when the
monkey decided that motion was toward the neuron's RF, and for most,
this difference was evident during the motion-viewing period. For a few
neurons (4 of 104), however, the response did not indicate the
monkey's choice until the delay period; that is, after the random-dot
motion stimulus was turned off. Figure 7
illustrates this unusual pattern of activity. This neuron responded selectively throughout the delay period of the saccadic eye movement tasks (A and B), but did not strongly indicate
the monkey's decision during the motion-viewing period of the
discrimination task. During the delay period, however, the response
modulated in a manner that reflected the impending saccade and thus the
monkey's decision (C and D). The change in
firing rate became evident about 200 ms after the random-dot motion was
turned off. We emphasize that this pattern of response was rare in LIP,
although it occurs with some regularity in prefrontal areas that are
connected to LIP (Kim and Shadlen 1999). The finding is
important, however, because it demonstrates that selecting neurons
based on their presaccadic activity did not guarantee that their
responses would be modulated during the period of motion viewing.
|
The pattern of responses exemplified in Figs. 5 and 6 were representative of the population of LIP neurons encountered in this study. Figure 8 shows the mean response from 104 neurons plotted as a function of time, aligned to 2 events during the trial. On the left, activity recorded during motion viewing is aligned to the onset of random-dot motion; on the right, the activity recorded during the delay period is aligned to the monkey's saccadic eye movement. The solid curves were obtained from the trials in which the monkey judged motion to be toward the RF. Dashed curves reflect the opposite choice. Only correct choices were included in this analysis, except for the weakest motion strength (0% coherence, red), which provides no basis to distinguish correct from incorrect.
|
There are several interesting features in this graph. Like the single units in Figs. 5 and 6, the magnitude of the response reflects the monkey's choice, increasing for T1 choices and decreasing for T2 choices. The rise and fall in spike rate begins in earnest 175 ms after onset of the dots (P < 0.01, t-tests performed on 1st derivative) and continues throughout the motion-viewing period. For trials in which the monkey judges motion to be toward the RF, the increase in activity saturates during the delay period, culminating in a burst of activity just before and during the saccade. For judgments away from the RF, the responses reach an average attenuation of 4-5 spikes/s below baseline during the delay period.
This basic pattern of responses holds qualitatively for all motion strengths, but the traces differ in the exact time course and amplitude of the discharge. Stronger motion stimuli lead to more profound elevation/depression of the responses, and the modulation occurs earlier, on average, for stronger motion, particularly when it is toward the RF. These effects are more apparent during the motion-viewing period than during the delay period. By the time of the saccade, the response is nearly identical for all T1 choices, regardless of the motion strength that led to the decision. The same is true for all saccades to T2. At the time of the saccade, therefore, the average response simply reflects one or the other alternative.
We used a regression analysis to quantify the effect of stimulus
strength on neural response (see METHODS, Eq. 2). Figure 9, A-D,
illustrates the effect of motion strength on the mean spike rate
obtained from four 1/2-s epochs that spanned the motion-viewing period. In each epoch, the response varied with motion strength, increasing when motion was toward the RF and decreasing when motion was
toward T2 (). These effects were quite modest, especially in comparison to the overall differences in activity associated with
T1 and T2 choices (e.g., compare
and
at
any motion strength). The strongest effects were seen in the second
epoch (Fig. 9B), where the response increased by 2.7 spikes/s on average with increasing motion strength toward
T1 (95% CI = 2.0 to 3.6 spikes/s, P < 10
14, nested F) and decreased by 4.2 spikes/s over the range of motion strengths toward T2
(CI = 3.6-4.7 spikes/s, P < 10
15). The smaller effects seen in the other
three epochs were also significant (P values range from 0.02 to 10
12).
|
The result suggests that LIP neurons do not simply encode the endpoint of a planned saccade but reflect through their discharge the quality of the sensory information that instructed the eye movement. However, this interpretation rests on the presumption that all eye movements to a visual target are identical, which is false. We therefore considered the possibility that eye movements varied with the difficulty of the task, and that this variation accounts for the change in neural response heretofore associated with the strength of random-dot motion.
We extended the linear regression analysis to incorporate various
descriptors of the saccadic eye movements. The analysis was performed
on a subset of the data consisting of 45 neurons (30 from monkey
E, 15 from K) for which we had records of eye position.
For each trial, we extracted six descriptors of the saccadic eye
movement: latency, amplitude, direction relative to the target,
accuracy, maximal speed, and duration. Across the 45 experiments, we
found small but significant inverse variations of saccadic latency and
saccade duration with stimulus strength (P < 107 and P < 10
4, respectively; nested F). The
other four saccade descriptors were more variable in their association
with motion strength, but in any given experiment one or more of these
were often significant. We therefore included all of these factors
along with motion strength in a multivariate regression analysis,
fitting the model
![]() |
(5) |
![]() |
The histograms in Fig. 9, E and F, depict the
change in response that accompanied an increase in motion strength from
0 to 51.2% coherence, after controlling for the potential confounding effect of eye movement variation (from the fit to Eq. 5).
The result is comparable to the simple regression obtained for the whole data set (Fig. 9, A-D) in which we ignored variation
in saccade metrics. On average, there was a 3.9-spike/s increase in
response across the range of motion strengths toward the RF (95%
CI = 3.0-4.8 spikes/s; P < 1015, nested F) and a 1.9-spike/s
decrease in response for motion away from the RF (CI = 1.2 to 2.7 spikes/s; P < 10
6). Individual
neurons with significant F ratios (P < 0.01; H0:
1
= 0) are shown by the shaded portion of the
histogram. In all cases, significant regressions revealed the expected
relationship between motion strength and neural response: enhancement
with stronger motion toward the RF and suppression with stronger motion away from the RF. We conclude from this analysis that variation in
saccade metrics does not explain the response modulation accompanying variation in the strength of random-dot motion.
Neural reflection of behavioral bias?
Before motion onset, one might expect neural activity to be
completely uninformative about the monkey's decision, but this is not
so. Examination of Fig. 8 reveals that the response was slightly
stronger before the monkey was shown motion that led eventually to a T1 choice, especially for trials with weaker
motion stimuli (red and green curves). The difference in activity
ranged from 2 spikes/s for the weakest motion (95% CI = 1.53-2.48 spikes/s) to 0.4 spikes/s at the highest motion strength
(CI = 0.21-1.14 spikes/s; P < 0.01 for all but
the 2 largest motion strengths, nested F). We interpret this
early response modulation as a possible correlate of decision bias: a
predisposition to choose T1 or T2 before viewing
the motion stimulus (Basso and Wurtz 1998
). When the
monkey is biased in favor of a T1 choice, activity is
stronger at the outset of the trial; when the bias favors
T2, activity is smaller than average at the outset. Of
course, such variation is likely to precede trials regardless of the
strength of the ensuing motion. However, when the motion is strong, the
direction of moving dots dictates the monkey's decision; trials
beginning with a T1 or T2 bias end up distributed
among both sets of correct choices. Conversely, when the motion
strength is weak, the monkey's initial bias affects the outcome of the
trial, with the result that more trials with an initial T1
bias actually end in T1 choices.
This scenario would produce the small differences in response preceding
the onset of random-dot motion when the monkey makes T1 or
T2 choices, but only when motion is weak. Firm conclusions about the source of these signals would require analysis of neural activity while behavioral bias is systematically manipulated. Such
experiments, carried out recently by Platt and Glimcher
(1999), have shown that signals related to behavioral bias
indeed exist in LIP. We suspect that our data reflect the same
underlying phenomena.
Predicting the decision
The data in Fig. 8 show that the activity of LIP neurons evolves in time, raising the question, when and how well do LIP neurons predict the monkey's choice? To address these issues, we performed an ROC analysis to compute an index of the neuron's predictive activity during the course of the discrimination. The index reflects the degree of separation between the responses associated with choices into and away from the RF and can be interpreted as a probability of correctly classifying a response as belonging to either choice set (see METHODS, Eq. 4).
Figure 10A plots for a single LIP neuron the predictive index as a function of time for five stimulus strengths. As the monkey viewed the random-dot motion, the neuron predicted the monkey's decision with increasing accuracy. This was also our impression during the recording experiments. While listening to the spike discharge over the loudspeaker, we experienced an increasing sense of confidence in predicting the monkey's decision as the trial progressed. By the end of the viewing period, the discharge from the neuron shown in Fig. 10A was nearly flawless in its predictive power, indicating that there was almost no overlap between the distributions of responses associated with T1 and T2 choices. During the delay period, the response remained highly predictive of the monkey's behavior, as evidenced by the curves on the right side of the plot. Although the curves in Fig. 10 bear resemblance to cumulative functions, the calculation is based only on spikes encountered within ±125 ms of the time indicated on the abscissa.
|
The sigmoidal evolution of predictive activity was evident at all motion strengths, but the neuron became predictive sooner at the stronger motion strengths. This observation is better appreciated in the population averages, illustrated in Fig. 10B. For the easier discriminanda, LIP activity was more predictive of the monkey's decision, and the predictive activity emerged earlier in the trial. Consistent with the bias effect discussed in the preceding section, weak predictive activity was evident prior to onset of the motion stimulus for the two weakest motion strengths. The prolonged temporal evolution of activity during motion viewing suggests a process in which LIP neurons accumulate information toward a plateau state that can guide subsequent behavior.
Errors
An advantage of the threshold discrimination task is that it affords an opportunity to examine trials in which the monkey makes errors, thereby providing a natural dissociation between sensory instruction and behavioral response. When the monkey viewed weak motion stimuli, at or below psychophysical threshold, many choices were incorrect. Figure 11 shows an example of the responses obtained from one neuron on trials in which the monkey viewed 12.8% coherent motion, just above psychophysical threshold. The four plots form a contingency table: the top and bottom rows show the responses when the monkey chose the direction toward and away from the RF, respectively. The left and right columns reflect motion direction toward and away from the RF, respectively. Accordingly, the top left and bottom right plots represent trials performed correctly (Fig. 11, A and D), whereas trials in the top right and bottom left represent error trials (Fig. 11, B and C).
|
The data show that both the monkey's choice and the visual stimulus
influenced the activity of this LIP neuron (P < 107 for both effects, 2-way ANOVA with nested
F-statistic, as in Eq. 3). The response was most
profoundly modulated on correct trials, in which the monkey's choice
and the direction of stimulus motion covaried (compare A and
D). The two panels of error trials generated roughly equal
responses that were intermediate between those in A and
D, indicating that, near psychophysical threshold, behavioral choice and motion direction exerted roughly equal effects on
the activity of this neuron during motion viewing. This pattern of
responses lends further support to the notion that LIP encodes both
qualities of the stimulus as well as the monkey's behavioral response.
The pattern of results illustrated for the single neuron in Fig. 11 was evident on a population basis as well. Because few errors occur when the motion cues are strong, we combined data across all experiments to accumulate a sufficient number of error trials for statistical analysis. The graphs in Fig. 12 show average responses aligned to the onset of random-dot motion and the moment of saccade initiation. The black curves illustrate responses when the monkey chose T1; the gray curves correspond to T2 choices. The solid curves represent correct choices; the dashed curves depict the error trials. When the motion strength was weak, responses were similar for correct trials and errors (Fig. 12A, COH = 3.2%). For intermediate motion strengths (B and C), however, the dashed curves fall between the solid curves. Neural activity in LIP remains correlated with the monkey's choice on error trials (i.e., is "predictive"), but the effect was smaller than for correct trials. Notice that the differences between correct and error trials persist until just before the saccadic eye movement.
|
At the two highest motion strengths, the pattern was different. At a coherence of 25.6% (~2 times threshold), the responses on the error trials were nearly indistinguishable, on average, for T1 and T2 choices. The discharge only became predictive of the monkey's impending eye movement during the delay period, about 500 ms before the saccade. At the strongest motion strength (51.2% coherence) the order of the curves reversed: the response was stronger when the monkey erroneously chose the target outside the RF. It is as if the neuron was reporting the proper choice (the direction of stimulus motion), but the monkey changed its mind late in the trial, perhaps as a result of a distraction or lapse of attention. Note that these last curves represent a small number of trials (T1 errors: 122 of 2,567 trials; T2 errors: 64 of 2,433 trials).
One further detail deserves mention. Notice that at the higher motion strengths, average neural activity predicts the monkey's errors in the period before the random dots are shown (Fig. 12, D and E, dashed lines). We noted a similar effect in Figs. 8 and 10 for correct choices at the weakest motion strengths, which we interpreted as a neural correlate of the monkey's behavioral bias state. The fact that the same effect is apparent for error trials, and most strikingly at high coherences, suggests that the monkey's bias might have influenced the monkey's erroneous choices. This is the expected pattern of results if an appreciable fraction of errors at high coherences are explained by lapses (e.g., distraction) and a tendency to default to the current bias state.
Motion sensitivity
The pattern of activity observed on correct and error trials demonstrates that both visual stimulus motion and eye movement direction influence the activity of LIP neurons. To determine whether the visual discriminanda alone activate LIP neurons, we recorded responses to strong motion stimuli from 93 neurons while the monkey performed a passive fixation task. As in the discrimination task, the direction of motion was either toward or away from the RF, and the random dots appeared in an aperture outside the nominal RF. No targets appeared on these trials, however, and there was no delay period. The monkey received a liquid reward simply for maintaining fixation throughout the stimulus presentation period.
We often observed a weak response to the random-dot motion stimulus
that was slightly stronger for motion toward the RF, as illustrated in
Fig. 13, A-C, for a typical
cell. Figure 13A shows that weak, directionally biased
responses occurred in a block of fixation trials obtained before the
monkey performed the discrimination task (mean response = 5.3 ± 0.5 spikes/s vs. 2.5 ± 0.4 spikes/s, P < 0.0005, t-test). We computed a direction index (DI) for each cell using the convention
![]() |
(6) |
|
In 20 neurons, we performed an additional control experiment to dissociate further the directional (sensory) response from oculomotor preparation. Rather than releasing control of oculomotor planning (as on the passive fixation trials), we instructed the monkey to prepare an eye movement that was unrelated to the moving random dots. In this block of trials, a single saccade target was presented at the beginning of each trial, either inside or outside the RF. On one-third of the trials, the monkey simply executed a delayed saccade to this lone target. On the remaining trials, random-dot motion was shown outside the RF for 1-2 s, but its direction was unrelated to the location of the saccade target. The monkey was thus encouraged to ignore the random dots and to make an eye movement to the location of the single target. As before, motion was toward or away from the RF, and the target appeared either inside or outside of the RF. These trials differed from the discrimination trials in two ways: the motion strength was always strong (51.2% COH), and only one saccade target was present.
Figure 14 illustrates results for a typical neuron. In the top row (A and B), a single target appeared in the RF, resulting in a vigorous response in anticipation of the saccade. The response was weaker, however, on trials in which motion was directed away from the RF (means: 38 ± 1.5 vs. 32 ± 1.8 spikes/s, P < 0.04, t-test). When the target appeared outside the RF (C and D), the response was suppressed, but to a greater degree when motion was away from the RF (6.7 ± 1.1 vs. 3.4 ± 0.7 spikes/s, P < 0.02). Clearly, the response of this neuron was dominated by the direction of the pending saccade, but the activity level also depended weakly on motion direction. This pattern was evident over the population of 20 neurons tested in this manner, as shown by the scatter plots (Fig. 14, E and F) and average response functions (Fig. 14G). Before the onset of the dot motion, the response was determined by the location of the target, inside or outside the RF. After motion onset, however, the response was affected by the direction of motion. On trials in which the target was inside the RF, the neurons responded less vigorously for motion away from the RF (Fig. 14G, dashed black curve) than for motion toward the RF (solid black curve). The overall level of responses decreased substantially on trials in which the saccade target was outside the RF, but the responses were nevertheless greater when motion was toward the RF than when it was away. The effect of motion direction was evident until just before the saccadic eye movement (arrows).
|
The data in Fig. 14 strengthen the interpretation that passive directional visual responses are present in some LIP neurons. Had these directional responses been sharply reduced in amplitude or frequency of occurrence in comparison to the visual responses in the passive fixation trials, we would be more inclined to regard them as covert motor planning signals. Lacking supporting evidence for a complex motor planning interpretation, we tentatively conclude that some LIP neurons receive weak directional visual inputs that influence activity during the stimulus presentation interval.
Horwitz and Newsome (1999a) have reported a
class of choice-predictive prelude neurons in the superior colliculus
that exhibit directional visual responses and strong prelude activity
that varies with stimulus coherence. In contrast, neurons that lack directional visual responses exhibit preludes that depend only on
saccade direction, not on stimulus coherence. We repeated the analysis
of Fig. 10 to determine whether choice-predicting neurons in LIP break
down along similar lines. Figure 15
compares the responses of neurons with significant direction bias (DB;
Fig. 15A) to the responses of nondirectional neurons (Fig.
15B). Consistent with the observations of Horwitz and
Newsome, the predictive activity of directional neurons was stronger
overall and was influenced more by stimulus coherence. The dissociation
was not as clean in LIP as in the superior colliculus, however,
since the effect of stimulus coherence on predictive activity is
significant even for the nondirectional neurons. With the use of the
multiple regression strategy in Eq. 2, DB neurons modulated
their discharge by 6.8 spikes/s (95% CI = 6.0-7.6) across the
range of motion strengths, as compared with 4.5 spikes/s (95% CI = 3.3-5.6) for non-DB neurons (P < 10
9 for both effects and for comparison of DB
and non-DB groups, F-tests).
|
Anticipation
A possible interpretation of the predictive activity we have studied is that LIP integrates (in the mathematical sense) motion information that arrives over time in the stochastic visual stimuli. In such a scheme, decisions would be based on the evidence that is accumulated in the networks that code for one or the other target location. Our final set of results suggests that the rate of this accumulation may reflect psychological variables that are not accounted for by motion processing alone.
In our earlier experiments, the monkey always viewed the random-dot motion for 2 s. Later, we randomized this viewing period so that some trials were as short as 500 ms of motion viewing plus the minimal delay period (typically 500 ms). As shown in Fig. 16, this manipulation exerted a substantial impact on the dynamics of the neural response. After exposure to shorter duration trials, the neural responses evolved faster and attained higher firing rates. The dashed curves in Fig. 16 show the average response obtained from 79 neurons recorded in early experiments, before either monkey had experienced motion-viewing periods of <2 s duration. The solid curves were obtained from 44 neurons encountered after introduction of the random duration task. The groups were distinguished only by the date of exposure; therefore much of the data represented by the solid curves includes trials of 2 s duration.
|
Notice that the change in response pattern is evident from before the onset of dot motion and through the delay period, but only on those trials in which the monkey chose the target in its RF (black curves). The result is shown for two stimulus strengths, but the same pattern of results was found at all motion strengths. The result is unlikely to be explained as a sampling artifact because many features of the data were similar before and after the monkey was exposed to the shorter duration trials. For example, the responses associated with T2 choices were similar for both groups of experiments, and the response just preceding saccades was nearly identical. The pattern of responses was apparent for both monkeys, although we recorded fewer neurons before introducing shorter duration trials to monkey K.
If predictive activity in LIP were due entirely to the integration of motion information, then the response should have followed the same time course, irrespective of the average trial duration. In contrast, the data suggest that the monkey's expectations about the time course of the trials can influence the development of predictive activity. When the experiment contains short trials that require a rapid decision, predictive activity in LIP evolves much more rapidly. The data suggest that decision-related activity in LIP is influenced both by the motion information in the random dot stimulus and by the temporal structure of the task.
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
To perform our direction discrimination task, a monkey must use
weak motion information to inform a binary decision whose overt
expression is a saccadic eye movement to one of two possible targets.
This particular decision process requires a linkage between the sensory
representation of motion direction and a behavioral intention to move
the eyes (Newsome 1997). Anatomically, LIP is well
positioned to participate in this linkage. It receives inputs from MT
and MST (Andersen et al. 1990
), and it is connected
reciprocally with the FEF, superior colliculus, and dorsolateral
prefrontal cortex (Cavada and Goldman-Rakic 1989
;
Paré and Wurtz 1997
; Schall et al.
1995
), structures that have been implicated in planning and
executing saccadic eye movements (for reviews see Schall
1997
; Schall and Thompson 1999
). Our primary
finding is that LIP neurons carry complex signals during performance of
our task that are neither sensory nor motor in a classical sense, but
may instead represent an intermediate stage of computation that
mediates the decision process.
LIP neurons exhibit two properties suggestive of a role in the decision process. First, neural activity signals the behavioral alternative chosen by the monkey, and this "predictive" activity emerges early in the trial during the epoch when the decision is presumably formed. Second, the intensity of the predictive activity depends on the strength of the motion stimulus that instructs the behavioral choice. For identical choices resulting in a particular eye movement into an LIP RF, activity during motion viewing is more intense when the decision is based on strong motion signals. As we discuss in the next two sections, these observations are critical for distinguishing decision-related activity from classical sensory and motor activity.
Differentiation from a sensory response
Many LIP neurons can respond passively to visual stimuli appearing
within their RFs (Gnadt and Andersen 1988; see
Colby and Goldberg 1999
for review). Nevertheless, LIP
activity measured in our experiments is easily distinguished from
classical sensory responses. First, passive visual inputs cannot
explain the striking covariation between neural activity and decision
(i.e., predictive activity) evident throughout our experiments. For
repeated presentations of a weak motion stimulus, the monkey chooses
one direction of motion on some trials and the alternative direction on
others. LIP activity varies strongly with the psychophysical decision (and thus with the eye movement) in these trials even though the visual
stimulus remains essentially constant (e.g., Fig. 8). In contrast,
motion-sensitive neurons in MT, an unambiguously sensory area, respond
predominantly to the direction and strength of stimulus motion
(Britten et al. 1993
); MT activity covaries only weakly with what the animal decides (Britten et al. 1996
;
Celebrini and Newsome 1994
).
Second, the time course of the response to random-dot motion in LIP
differs markedly from direction-selective neurons. MT neurons respond
with latencies of 40-80 ms from onset of random-dot motion and
discharge at a constant rate during random dot presentation (Britten et al. 1992, 1993
; Maunsell
1987
; Schmolesky et al. 1998
). The LIP response
to the random dots was first discernable ~175 ms after motion onset
and then built up or attenuated gradually over time (e.g., see Figs. 8
and 12).
Finally, passive visual inputs cannot account for the pronounced
delay-period activity following termination of the random-dot motion
stimulus. Neural activity in MT (in our task at least) is linked
tightly to the presence of the visual stimulus; sustained activity is
not present during the delay period (Seidemann et al.
1998).
Although LIP neurons carry behaviorally significant signals that are not explained by the sensory stimulus, these signals are nevertheless modulated by the strength and direction of motion in the stimulus. Indeed this modulation is the critical feature that distinguishes LIP responses from high-level motor activity.
Differentiation from a motor response
It is considerably more difficult to distinguish decision-related activity from motor activity than from sensory activity. The activity of oculomotor neurons, after all, trivially reveals the outcome of the decision process and may therefore be considered decision related. Even sustained activity during an enforced delay period, which defers the motor act in time, might simply represent an oculomotor plan being held in readiness until receipt of the "go" signal.
What aspects of LIP activity suggest participation in the decision
process and not merely a plan to shift the gaze? First, the critical
response modulations begin early during the period of motion viewing,
presumably as the monkey is forming its decision about direction, but
before the monkey is committed to a particular eye movement response.
Granted, we do not know when, in any given trial, the monkey completes
the decision process, but recent data obtained in speed-accuracy and
reaction-time versions of our task suggest that monkeys integrate
motion information for many hundreds of milliseconds before committing
to a decision, particularly for weaker motion stimuli (Gold and
Shadlen 2000, 2001
; Roitman et al.
1999
). Second, random-dot motion elicits weak, directionally biased responses during passive fixation (Fig. 13), and, in experiments employing single saccade targets (Fig. 14), observations that are difficult to explain in terms of motor planning. Third, activity throughout the bulk of the trial is independent of small variations in
saccade parameters such as speed, accuracy, duration, or latency (Fig.
9, E and F). To the extent that the response is
"oculomotor," therefore it is a high-level signal that indicates in
a general sense the spatial goal of a pending saccade (Andersen
et al. 1992
; Mountcastle et al. 1975
). Fourth,
and most important, the rate and magnitude of the response buildup (or
attenuation) depends on the strength and direction of random-dot motion
(Figs. 8, 9, 11, and 12). These stimulus features systematically affect
the neural response in a manner that cannot be explained by variation
in the monkey's eye movement response. Such a graded representation of
the decision could reflect growing certainty based on the available motion evidence (Basso and Wurtz 1998
; Carpenter
and Williams 1995
; Gold and Shadlen 2001
).
To summarize, LIP activity can be regarded neither as purely motor nor
purely sensory. The activity reflects a combination of sensory- and
motor-like variables. The "sensory" variables that bear on the
decision process are the direction and strength of random-dot motion,
whereas the "motor" variable is the direction and amplitude of a
pending saccadic eye movement. Both properties are evident in the
responses from the majority of single neurons in our sample. It is
natural to suppose that the mixture of sensory and motor properties
could reflect computations that link sensory instruction (motion) with
the behavioral response (eye movement) (see also Romo et al.
2000; Salinas and Romo 1998
).
Computation of a decision variable
In a discrimination experiment, the link between the sensory
representation of motion and the commitment to one or another choice is
thought to involve the computation of a decision variable: a quantity
that is monotonically related to the relative likelihood of one
alternative versus another (Green and Swets 1966).
Neurons in LIP appear to signal a quantity that resembles a decision
variable in our task.
A decision in the motion discrimination task involves the comparison of
sensory responses to visual motion from opposing pools of motion
sensors in the extrastriate visual cortex (Britten et al.
1992; Shadlen et al. 1996
). For example, in a
left-right discrimination, the monkey chooses left if the pooled
"left" response exceeds the pooled "right" response. Therefore
a useful decision variable is the accumulated difference between the
"left" and "right" population responses to the random dots. The
decision, however, is based on the accumulation of sensory data in
time. Thus it evolves during acquisition and remains sustained during
the delay period between random-dot motion and eye movement response.
This notion of accumulation, or temporal integration, provides an attractive computational framework for the response properties observed in LIP. The central idea is that LIP computes the time integral of sensory data weighted for and against the saliency of the neuron's RF as a possible target for the gaze. A rudimentary model is shown in Fig. 17A. The input to the neuron is organized as the sensory evidence for or against a gaze shift up and to the right. The evidence in our task is motion toward or away from the response field. When the random dots are centered on the fixation point, this would correspond to up-right versus down-left direction sensors, and when the stimulus is above the FP, the appropriate comparison is rightward versus leftward (Fig. 17A, inset). Of course, motion is not the only "evidence" that would support an eye movement to the RF. For example, the neuron should also be driven positively by visual neurons that represent visual targets in the RF and suppressed by neurons with receptive fields outside the RF.
|
Sensory evidence, from whatever source, is reflected in the spike rate of neurons in the visual cortex. These signals must then be integrated to achieve a sustained level of discharge. As illustrated in Fig. 17A, we propose that LIP integrates the difference between opposing pools of MT neurons. The magnitude of this difference would be evident in the LIP response, and this response would be sustained even after the MT response returns to baseline, just as the integral of a brief pulse is a sustained step.
To assess the utility of the model for understanding our data, we
performed computer simulations of combined physiological and
psychophysical experiments, including the responses of directional MT
neurons, LIP neurons that integrate the accumulated difference in
firing rate between pools of oppositely directed MT neurons, and
psychophysical decisions based on the firing rates achieved by the LIP
neurons. The responses of MT neurons were simulated using a model of
pooled responses from MT similar to the one described by Shadlen
et al. (1996).
Consider a right-left discrimination task like the one shown in Fig. 17, B-G. According to the model, an LIP neuron whose RF is situated up and to the right of fixation would integrate the difference between rightward and leftward direction signals from MT neurons with receptive fields located above the FP. We simulated responses to 0% coherent motion (B and E) and to a weak motion coherence near psychophysical threshold (C, D, F, and G). We then sorted the "trials" according to the simulated psychophysical choice (right or left), and displayed the simulated neural activity in B-G. The responses of rightward selective MT neurons are shown by the gray histograms; the responses of the leftward selective MT neurons by the solid black curves. The red curves show the integral of the difference between the pools of MT neurons. If the pooled response of the "rightward" MT neurons is larger, this difference is positive, the LIP response increases, and the monkey chooses the target in the RF (top row, B-D). Conversely, larger responses in the "leftward" MT neurons leads to a decreasing LIP response followed by a leftward psychophysical decision (bottom row, E-G).
Figure 17, top row (B-D), represents trials in which the responses of rightward MT neurons exceeds the leftward neurons (resulting in a rightward choice), whether by chance (as in the 0% coherence case, B), or because the stimulus actually contained rightward motion (C), or in error (D) when stochastic response fluctuations lead to a larger response in the rightward sensors despite the fact that the stimulus actually contained weak leftward motion. Notice that the sensory responses are present only during the period of motion viewing and not during the delay period, whereas the integral of this difference persists. Similarly, the bottom row (E-G) represents trials in which the leftward sensory response is larger, and the monkey ultimately moves its eyes to the target outside the movement field. This can occur by chance in the 0% coherence case (E), for erroneous choices when the motion is actually rightward (F), or for correct choices when motion is leftward (G).
The integrated difference accounts qualitatively for several properties of the LIP discharge seen in our data. It explains the dependency of the response on motion strength because the integral rises fastest when motion is toward the RF (Fig. 17C) and attenuates most profoundly when motion is away from the RF (Fig. 17G). For the 0% coherent stimulus, the signals from rightward and leftward sensors are, on average, the same. Nevertheless, on one-half of the trials, the rightward responses exceed the left, and vice versa. This difference is reflected in the integral obtained when we sort the responses by choice. The model predicts the intermediate level of responses that we observed on error trials at weak motion strengths (D and F). For example, in Fig. 17D, an erroneous rightward decision results when the accumulated rightward signal exceeds the accumulated leftward signal, despite the relatively weak response of rightward motion sensors to leftward motion.
While the integrator concept accounts for several important features of
our data, it would require extensive elaboration to account for others.
For example, the proposed integration would need to be scaled to
accommodate the expected trial duration (as demonstrated in Fig. 16).
In addition, the integral of the sensory instruction (the direction of
coherent motion in our experiments) is constant once the instruction
has been processed. According to the integrator idea, therefore, the
delay-period activity ought to remain constant at the level achieved at
the end of the motion-viewing period. Figure 8 contradicts this
prediction: the response during the delay period fails to preserve the
variety of response levels that was evident during the motion-viewing
period. A possible explanation is that the monkey reaches a final
decision about motion direction when the LIP response achieves a fixed
value. This would occur earlier or later on easy or difficult motion trials, respectively, and it would lead to a stereotyped response for
the remainder of the motion-viewing and delay periods. Preliminary observations in monkeys trained to perform a reaction time version of
the direction-discrimination task support this idea (Roitman and
Shadlen 1998).
The model in Fig. 17A requires additional inputs to control
the dynamics of the integration process (Robinson 1989;
Seidemann et al. 1998
). If the neural activity can
reflect sensory information that arrived hundreds of milliseconds ago,
then some other input must tell the neuron when to begin integrating
and when to reset to a preintegration state. These control signals
could also convey information about the prior probability that a
saccade to the RF will occur, and when the saccade will occur
(Basso and Wurtz 1997
; Dorris and Munoz
1998
; Platt and Glimcher 1999
), accounting perhaps for the weak predictive activity that was apparent before the
onset of random-dot motion (Figs. 8, 10, and 12).
Should the integrator concept prove valuable, it would raise questions
about how the computation is achieved. So-called neural integrators are
well established in the control of eye and head position
(McFarland and Fuchs 1992; Robinson
1989
), and their mechanism is currently an active area of
investigation (Aksay et al. 2001
; Seung et al.
2000
). The mechanisms responsible for persistent activity and
temporal integration in these subcortical structures could lend insight
into cortical mechanisms that serve cognitive functions such as
decision making (Gold and Shadlen 2001
).
Relation to other work
A pivotal role for LIP in the association of visual and visuomotor
processing is consistent with recent anatomical studies that place LIP
at a strategic junction between dorsal stream visual areas and the
oculomotor system. Our recordings were concentrated in the posterior
portion of LIP, a region that receives input from the extrastriate
visual cortex and projects to the superior colliculus, frontal eye
field, and the neighboring area 8Ar (Petrides and Pandya
1984; Schall et al. 1995
). Not surprisingly,
neurons with properties similar to area LIP in the context of our
discrimination task have recently been discovered in the intermediate
layers of the superior colliculus (Horwitz and Newsome
1999a
) and dorsolateral prefrontal cortical areas 8A, 8Ar, and
Walker area 46 (Kim and Shadlen 1999
). The existence of
such a network of neurons with similar properties raises questions
about the role of any one area in the decision process.
A useful synthesis, capable of accommodating our findings and those
from other studies, is that LIP represents the sensory instruction to
shift the gaze to the region of the visual field corresponding to the
RF. This instruction is often a visual target within the RF, but as we
have shown with random dots, the instruction can be a stimulus placed
outside the RF as customarily defined. A growing body of evidence
suggests that the parietal cortex encodes the salience of sensory
features that are relevant to particular behavioral acts (for reviews,
see Andersen 1995; Colby and Goldberg 1999
). LIP in particular may be regarded as representing the
saliency of visual space or visual objects for purposes of guiding eye movements (Andersen et al. 1992
; Bracewell et al.
1996
; Gnadt and Breznen 1996
; Gottlieb et
al. 1998
; Mazzoni et al. 1996
; Platt and
Glimcher 1997
; Sereno and Maunsell 1998
),
whereas medially in Brodmann's area 5 the parietal cortex encodes
space for the guidance of reaching movements (Batista et al.
1999
; Seal 1989
; Seal and Commenges
1985
; Snyder et al. 1997
), and further laterally and anteriorly, the anterior intraparietal area signals
saliency pertaining to fine pincer grasp (Lacquaniti et al.
1995
; Murata et al. 1996
; Sakata et al.
1995
). In our task, visual motion serves to instruct a future
gaze shift and thus to render salient one or the other choice target.
Many regions of the association cortex contain neurons that sustain
their response through a time gap. This delay-period activity has been
described as a representation of motor set (di Pellegrino and
Wise 1993; Evarts and Tanji 1976
), movement
preparation (Andersen 1995
; Andersen et al.
1992
; Johnson et al. 1999
; Snyder et al. 1997
), movement suppression (Hikosaka and Wurtz
1989
; Wise et al. 1996
), attention to salient
features (Colby et al. 1996
; Gottlieb et al.
1998
), and working memory (Funahashi et al.
1989
, 1991
; Hasegawa et al. 1998
;
Miller 1999
; Rainer et al. 1999
).
Fuster (1985)
suggested that such delay-period activity
allows the brain to prepare behavior on the basis of past sensory
instruction (see also, Quintana and Fuster 1992
,
1999
). Our results raise the possibility that such
preparatory activity reflects the accumulation (temporal integral) of
sensory information that supports a prepared action. If this is true,
then many neurons identified by the capacity to maintain delay-period
activity might also exhibit modulation related to both the sensory
instruction and to the behavior to be executed. Such neurons, like the
ones described here, could not be regarded as purely motor or as purely
sensory (Alexander and Crutcher 1990
; Hasegawa et
al. 1998
; Leon and Shadlen 1998
; Riehle
et al. 1994
; Romo et al. 1999
; Seal and
Commenges 1985
; Shen and Alexander 1997a
,b
;
Thompson et al. 1996
, 1997
; Zhang et al. 1997
); they constitute the link between these domains.
"Sensory" and "motor" revisited
Our data suggest a reevaluation of the information processing
scheme outlined in Fig. 1, which envisions a separate "decision process" intervening between sensory and motor systems. In its purest
form this scheme suggests that the brain should contain circuits that
represent the outcome of the decision abstractly, with no necessary
co-representation of the sensory stimulus or the subsequent motor
action. What we actually find in LIP (and in the frontal lobe and
superior colliculus) are neurons that reflect the outcome of the
decision, but are also related parametrically to the sensory stimulus
(motion strength and direction) and to specific movements (eye
movements to a particular region of space). Considered together, the
data obtained thus far suggest that the notion of an abstract decision
process may be misguided, at least for monkeys performing this sort of
operantly conditioned discrimination task. Rather, the decision may be
embodied in direct transformations between the relevant sensory and
motor systems (Gold and Shadlen 2000; Horwitz and
Newsome 1999a
; Kim and Shadlen 1999
;
Newsome 1997
; Rizzolatti et al. 1997
).
Decision-related signals carried by LIP neurons appear to exhibit
physiological signatures of their sensory origins as well as of their
motor destination. It is conceivable that versions of the task may be
devised that would force the monkey to hold the decision in a more
abstract form within the cerebral cortex. This possibility should be
tested explicitly in future experiments (Gold et al.
2000
; Horwitz and Newsome 1999b
).
New experiments will also be necessary to determine whether the circuitry we have studied in LIP is a part of the causal pathway linking the sensory representation of motion to the behavioral response. Conceivably, LIP could receive a corollary copy of the decision without actually contributing to formation of the decision. Microstimulation experiments may be able to distinguish these possibilities. If stimulation of LIP changes the animal's psychophysical choices in a predictable manner, we may be more confident that LIP plays a central role in generating discrimination behavior.
![]() |
ACKNOWLEDGMENTS |
---|
We thank C. Colby and M. Goldberg for providing instructions for locating LIP in electrophysiological experiments, J. Groh and E. Seidemann for assisting in several experiments, and J. Gold, G. Horwitz, C. Barberini, J. Muller, and L. Sugrue for valuable feedback on an earlier version of the manuscript. We are grateful to J. Stein for technical assistance and histology. W. T. Newsome and M. N. Shadlen are Investigators of the Howard Hughes Medical Institute.
This research was supported by National Institutes of Health Grants EY-05603, RR-00166, and EY-11378, the McKnight Foundation, and the Human Frontiers Science Program.
![]() |
FOOTNOTES |
---|
Address for reprint requests: M. N. Shadlen, Dept. of Physiology, University of Washington Medical School, Box 357290, Seattle, WA 98195-7290 (E-mail: shadlen{at}u.washington.edu).
1 All graphs and analyses employ a transformation of motion strength to an ordinal scale from most difficult to easiest, which, with the exception of 0% coherence, is identical to a log scale. We selected this transformation because it led to a linear relationship.
Received 9 March 2000; accepted in final form 9 May 2001.
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|