Saccades to Sounds: Effects of Tracking Illusory Visual Stimuli

Jeffrey T. Somers,1,2 Vallabh E. Das,1,2 Louis F. Dell'Osso,1,2 and R. John Leigh1,2,3

 1Department of Neurology,  2Department of Biomedical Engineering, and  3Department of Neuroscience, Department of Veterans Affairs Medical Center and University Hospitals, Case Western Reserve University, Cleveland, Ohio 44106


    ABSTRACT
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Somers, Jeffrey T., Vallabh E. Das, Louis F. Dell'Osso, and R. John Leigh. Saccades to Sounds: Effects of Tracking Illusory Visual Stimuli. J. Neurophysiol. 84: 96-101, 2000. In 10 normal human subjects, we studied the accuracy of memory-guided saccades made to the remembered locations of visual targets and sounds. During the time of stimulus presentation, subjects were smoothly tracking a projected laser spot that was moving horizontally across a tangent screen, sinusoidally ±15° at 0.25 Hz. In one set of experiments, the laser spot moved across a 40° × 28° random dot display that moved synchronously in the vertical plane; this induced a strong illusion that the trajectory of the laser spot was diagonal (variant of Duncker illusion). In control experiments, the laser spot moved across the same display, which was stationary. The visual targets and speakers were at six locations (range ±15°) in the horizontal plane. Saccades made to the remembered locations of targets presented during background motion (illusion) were significantly (P < 0.05) more inaccurate than with the background stationary (control) in 9 of 10 subjects for lights and in 6 of 10 subjects for sounds. As a group, the median change in errors due to the Duncker illusion was ~2.5 times greater for visual compared with acoustic targets (P < 0.001). These findings are consistent with electrophysiological studies which have shown that neurons in the primate lateral intraparietal area (LIP) may respond to both visual and auditory targets and these neurons are also influenced by the Duncker illusion during programming of memory-guided saccades.


    INTRODUCTION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Under natural conditions, subjects make saccades to both visual and auditory stimuli in the environment. Although the frame of reference in which auditory (head-fixed) and visual (retinal) stimuli are encoded differs, subjects make spatially accurate saccades in response to combined visual-acoustic stimuli, even during eye-head gaze shifts (Goossens and Van Opstal 1999).

Normal human subjects mislocalize visual cues if they are presented against a background that is a moving flow field. Thus if a small visual stimulus, such as a laser spot, moves vertically across a stationary background of random dots, the perceived motion of the laser spot is vertical. However, if the background moves horizontally in synchrony with the laser motion, the trajectory of the latter changes from vertical to diagonal (Zivotofsky et al. 1995). This variant of the Duncker illusion (Duncker 1929) causes subjects to mislocalize visual targets that are briefly flashed; saccades made to the remembered locations of targets are much less accurate than if the background remains stationary (Zivotofsky et al. 1996). Similar results can be obtained from monkeys trained for this task (Powell et al. 1999; Zivotofsky et al. 1998). Electrophysiological studies of neurons in the lateral intraparietal area (LIP) in monkeys have indicated that the Duncker illusion causes some cells to shift their receptive fields to perceived, rather than actual, spatial locations during programming of saccades (Powell et al. 1998).

Unlike humans, monkeys require training to make saccades to the locations of sounds (Grunewald et al. 1999). After such training, some neurons in macaque LIP respond during memory-guided saccades to sounds (Grunewald et al. 1999; Linden et al. 1999; Mazzoni et al. 1996). The majority of neurons with such acoustic responses also respond during memory-guided saccades to visual stimuli (Mazzoni et al. 1996). These findings lead us to hypothesize that the Duncker visual illusion would cause human subjects to mislocalize sounds, which would be made manifest by inaccuracy of saccades made to the remembered positions of acoustic stimuli. Thus the goal of this study was to compare the effects of the Duncker illusion on memory-guided saccades with lights and sounds. We found that both responses were affected, but that the Duncker illusion caused greater mislocalization of visual stimuli than of sounds. A brief summary of these findings will appear as an abstract (Somers et al. 2000).

The work reported in this paper constitutes a portion of the research performed by J. T. Somers as part of the requirements for his Masters dissertation.


    METHODS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Subjects

We studied 10 healthy human subjects (8 males and 2 females, aged 23-53 yr); 7 did not know the purpose of the study. None of the subjects had any ocular motor abnormalities. Three subjects were myopes but were able to wear their glasses during testing because the experiments were self-calibrating (see Data collection and analysis) and their heads were stationary. We confirmed that all subjects had normal hearing with a Grason-Stadler GSI 17 audiometer. All subjects gave informed consent in accordance with our Institutional Review Board and the tenets of the Declaration of Helsinki.

Stimuli and recording

The subjects sat in a chair with a headrest in a dark room, draped with heavy black curtains. They viewed a tangent screen at a distance of 85 cm onto which were projected a red laser spot (primary target) and a field of random black dots on a white background (background). The primary target subtended an angle of 0.2° and had a luminance of 2.2 ft-lamberts. The background subtended 40° × 28° with an average luminance of 3.8 ft-lamberts. Both the primary target and background were moved by galvanometers sinusoidally at a frequency of 0.25 Hz; the primary target horizontally and the background vertically. Behind the tangent screen were six speaker-LED pairs mounted on a horizontal arc at a distance of 95 cm. Each speaker served as an acoustic target; it was 3 cm in diameter (1.8° subtended) and had a frequency response of 600-6000 Hz at a rated sound pressure level of 70 dB. A pink noise generator was used to drive the speakers. Background noise, due to a projector fan was 58.3 dB. The LEDs (secondary visual targets) were 3 mm and green with a luminance of 0.85 ft-lamberts when viewed through the tangent screen. All visual and acoustic stimuli were easily detected by all subjects.

Horizontal and vertical eye rotations were measured with six-foot field coils using the magnetic search coil technique (CNC Engineering, Seattle, WA). The coil was precalibrated on a protractor device. The system was 98.5% linear over an operating range of ±20° in both planes, crosstalk between horizontal and vertical channels was <1.5%, and system noise was <0.02°. Subjects wore a scleral search coil (Skalar, Delft, The Netherlands) on their dominant eye. We confirmed that subjects held their head stationary during all trials by recording head position with a second search coil.

Experimental paradigms

There were two experimental paradigms and two control paradigms: 30 responses were collected from each. All trials started with the subject smoothly pursuing the primary target as it moved horizontally ±15° at 0.25 Hz. After 3 s, either a secondary visual target was flashed for 75 ms or an acoustic target sounded for 500 ms. The subjects were instructed to continue to pursue the primary target and not to make a saccade to the location of the secondary visual target or sound until the primary target and background were extinguished 2 s later (the memory period). The subject was allowed 2 s in darkness to settle on the remembered location and a LED would appear at the location of the secondary visual target or sound. The subject would fixate this LED, thereby correcting for any errors in the memory-guided saccade and providing a measure of the target's location. Each trial lasted 10 s and there were six trials for each run of 60 s. Instructions were given for each test paradigm, and some practice was allowed before the eye coil was inserted and data collection begun. Secondary visual targets and sounds were presented in a pseudo-random sequence at six locations: 5°, 10°, or 15° to the right or left of center. The four paradigms were as follows:

1) Visual targets with background moving. The subject pursued the primary visual target as it moved horizontally. The visual background moved vertically ±11° at 0.25 Hz, in synchrony with the primary target motion. We verified that all subjects perceived a strong Duncker illusion of diagonal movement of the target. The secondary visual targets (green LEDs) were presented at the time when the background was at either the upper or lower extent of its cycle. The switch to darkness 2 s later corresponded to the point at which the background had completed one half-cycle. We selected this combination of background motion and timing of secondary target presentation because it had been previously reported to maximize errors of memory-guided saccades (Zivotofsky et al. 1996).

2) Visual targets with background stationary. The subject pursued the primary visual target as it moved horizontally. The visual background was stationary throughout. The secondary visual targets were the green LEDs.

3) Acoustic targets with background moving. The subject pursued the primary visual target as it moved horizontally. The visual background moved vertically ±11° at 0.25 Hz in synchrony with the primary visual target to produce the Duncker illusion. The synchronization of background motion and target presentation was similar to the visual targets with background moving paradigm. The secondary targets were speaker sounds.

4) Acoustic targets with background stationary. The subject pursued the primary visual target as it moved horizontally. The visual background was stationary throughout. The secondary targets were speaker sounds.

In five subjects the order of the paradigms was as listed above and in the other five subjects the order was acoustic targets with background moving, acoustic targets with background stationary, visual targets with background moving, and visual targets with background stationary.

Data collection and analysis

Horizontal and vertical gaze signals were low-pass filtered using Krohn-Hite Butterworth filters with a cutoff at 90 Hz prior to digitization with 16 bit precision at 200 Hz. The beginning and end of saccades were determined using a velocity criteria of 30°/s. Using interactive programs, each trial was analyzed and the following points were identified and measured from the horizontal and vertical records: 1) eye position at the end of the first saccade in darkness; 2) final eye position in darkness (following corrective saccades and drifts); and 3) reference eye position after the secondary target had reappeared and the subject was fixating the target. The reference eye position was used as an internal calibration of the target position and permitted reliable calibration for the subjects who wore glasses during testing. Figure 1 shows a representative response to a visual trial with the background moving and illustrates where these points were measured.



View larger version (20K):
[in this window]
[in a new window]
 
Fig. 1. Representative record of a subject's response to the visual targets with background moving paradigm. Upward displacements indicate rightward or upward motion of the stimuli or eye position. Subject smoothly tracks the primary visual target, which is moving sinusoidally in the horizontal plane. Note that the visual background is moving sinusoidally in the vertical plane, synchronized to primary target motion. After 3 s, the secondary visual target located at left 5° flashes for 75 ms; at this point the primary visual target is at the left and the background is down. Subject continues to track the primary visual target and at 5 s, the room is switched to complete darkness. Subject makes an initial saccade to the remembered location of the secondary target; the horizontal component is quite accurate, but an inappropriate vertical component takes the eye up ~5° (1) corresponding to the direction of background movement during the memory period. The final vertical eye position in darkness (2) remains incorrect and after the secondary target is turned on corrective saccades are made toward the reference position (3).

All trials in which subjects did not follow instructions, such as by making a saccade to the target before the primary target was extinguished, were discarded. This occurred in <3% of trials. From the above measurements we calculated separately for horizontal and vertical responses: 1) initial error, the difference between eye position after the initial saccade and the reference eye position; and 2) final error, the difference between the final eye position in darkness and the reference eye position. For example, in Fig. 1, the initial vertical error was 5.0° and the final vertical error was 3.4°. Note that no vertical gaze displacement was required by any of the secondary targets (visual or acoustic), because they all lay in the horizontal plane (zero vertical position).

First we analyzed responses of individual subjects for one type of secondary stimulus; light or sound. Because these data were not always normal in distribution, we used nonparametric statistical tests and Tukey box plots to display the results. For each type of stimulus, we made paired comparisons of each subject's responses to the illusory or control conditions using the Wilcoxon signed rank test with a significance level of P < 0.05. Second, we analyzed the responses of the 10 subjects as a group, making paired comparisons of the median errors for different experimental and control conditions, for visual or acoustic stimuli. Third, we compared differences between visual and acoustic responses by conducting a paired comparison of the change in vertical errors (experimental condition - control condition) for visual versus acoustic stimuli for corresponding individual trials for each subject. We performed this comparison separately for initial and final error (~300 comparisons for each). Because we used all of the data in two separate comparisons, we applied a Bonferroni correction, giving a critical level of P < 0.025.


    RESULTS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Comparison of errors of responses to visual targets for individual subjects

In the vertical plane, significant increases occurred during visual trials with background moving (illusion) compared with background remaining stationary in 9 of 10 subjects for initial errors and 8 of 10 subjects for final errors. These results are summarized in Fig. 2. In 84% of trials, the direction of the vertical errors was in the direction of background movement during the memory period. In the horizontal plane, no significant differences occurred between experimental or control conditions for initial or final error.



View larger version (23K):
[in this window]
[in a new window]
 
Fig. 2. Tukey box plots (indicating median, 5th, 25th, 75th, and 95th percentiles) showing magnitude (absolute error) of initial and final errors of responses to visual targets for all 10 subjects. Pairs of columns correspond, respectively, to the experimental (background moving) and control (background stationary) conditions. Top: error after the initial saccade. Bottom: final eye position error. Significant differences between experimental and control conditions using the Wilcoxon signed rank test are indicated by asterisks (*P < 0.025).

Comparison of errors of responses to acoustic targets for individual subjects

In the vertical plane, significant increases occurred during acoustic trials with background moving (illusion) compared with background remaining stationary in 6 of 10 subjects for both initial and final errors. These data are summarized in Fig. 3. In 71% of trials, the direction of the vertical errors was in the direction of background movement during the memory period. In the horizontal plane, no significant differences occurred for initial or final error, except for final error in one subject.



View larger version (27K):
[in this window]
[in a new window]
 
Fig. 3. Tukey box plots showing magnitude of initial and final errors of responses to auditory targets for all 10 subjects. Pairs of columns correspond, respectively, to the experimental (background moving) and control (background stationary) conditions. Top: error after the initial saccade. Bottom: final eye position error. Conventions and symbols are as described in Fig. 2.

Group responses to visual and acoustic targets

Comparisons of the median errors of the 10 subjects as a group during experimental (background moving) and control (background stationary) conditions showed a significant increase of both initial and final errors for both visual and acoustic paradigms (P < 0.05). Differences of median initial or final error between illusion and control in the horizontal plane, for both visual and acoustic paradigms, were smaller and not significant (P >=  0.375).

Comparison of errors due to illusory visual stimuli made to visual and acoustic targets

The goal of this analysis was to determine whether there was any systematic difference in the errors of saccades made to lights or sounds. To this end we compared the change of error (error during experimental condition - error during control condition) for the two types of stimuli using the Wilcoxon signed rank test. The results of these paired comparisons (~300 for each case) for vertical errors are summarized in Fig. 4. Three subjects showed a statistically greater change of both initial and final error to visual compared with acoustic targets; one other subject showed a significantly greater change for only final error. For the data pooled from all 10 subjects, the median change in both initial and final error due to the Duncker illusion was ~2.5 times greater for visual compared with acoustic targets (P < 0.001). Similar comparisons for horizontal errors showed no significant differences.



View larger version (30K):
[in this window]
[in a new window]
 
Fig. 4. Paired comparison of change in error (experimental condition result - the same control condition result) for responses to corresponding stimuli to visual or acoustic stimuli. Top: initial errors. Bottom: final errors. Errors that were exactly the same for corresponding stimuli (similar LED or speaker location) lie on the 45° diagonal line. In both plots, most points lie below the diagonal and errors for the whole data set were significantly greater for saccades made to remembered visual targets. Data from each subject (S) are indicated by different symbols.


    DISCUSSION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

The main finding of this study was that tracking a visual target that has an illusory trajectory causes acoustic as well as visual memory-guided saccades to become inaccurate. However, the effect of the illusory stimulus was greater for memory-guided saccades made to remembered locations of visual rather than acoustic targets. To interpret these findings, we first compared these results with prior studies of the effects of the Duncker illusion on saccades to visual targets. We considered the possible neural mechanisms underlying these behaviors, including effects of experimental and clinical lesions.

Comparison of effects of Duncker illusion on saccades to sounds and visual targets

When the Duncker illusion is produced by a small target moving orthogonally to a linear flow field, the percept is one of diagonal target motion; however, the smooth-pursuit tracking of the small target is accurate (Zivotofsky et al. 1995). When memory-guided saccades are made to targets flashed during pursuit of such illusory stimulus motion, they become inaccurate in the plane of the background motion by a factor of ~3 compared with no illusion (i.e., stationary background) (Zivotofsky et al. 1996). Also, the error of the initial saccade tends to be greater than that of the final eye position in darkness. In the plane orthogonal to background movement, no consistent errors of saccades occur. In this study we confirmed each of these findings and also showed a similar, but smaller, effect for acoustic stimuli. The vertical errors occurring during the illusory paradigms (visual or acoustic trials with background moving) were in the same direction as the background motion during the memory period, as previously reported (Zivotofsky et al. 1996). Although the direction of gaze at the time of stimulus presentation may have influenced the responses (Zahn et al. 1979; Zambarbieri et al. 1995), this and other factors were minimized by paired comparison of saccadic errors to similar stimuli with the background either moving or stationary. The effect of the Duncker illusion on memory-guided saccades to sounds was smaller than for visual stimuli, but two factors may account for this difference. First, it is difficult to equate the magnitude of stimuli to different sensory systems. Thus it remains possible that different acoustic stimuli would have produced a larger effect. Second, memory-guided saccades to sounds are generally smaller and less accurate (i.e., more variable) than those made to visual targets (Goossens and Van Opstal 1997). It may be noted from Figs. 2 and 3 that several subjects (both naive and nonnaive) who showed larger errors to visual targets with background moving also showed larger errors to acoustic targets with the background moving. Conversely, certain other subjects showed a trend of similarly smaller errors to both types of stimuli, thus suggesting a common neural mechanism. We next discuss what electrophysiological data are consistent with this result.

Neural basis for mislocalization of acoustic and visual stimuli with the Duncker illusion

Prior studies of the effects of the Duncker illusion have suggested that visual estimates of the direction of gaze are much more important than extraretinal signals in planning eye movements to remembered target locations during such illusory paradigms (Zivotofsky et al. 1996). Monkeys also misperceive the trajectory of the target during the Duncker illusion (Powell et al. 1999), and show similar inaccuracies of memory-guided saccades to visual targets (Zivotofsky et al. 1998). A neural correlate for these behaviors is the demonstration that the receptive fields of some LIP neurons shift to perceived, rather than actual, target locations for saccades made under these experimental conditions (Powell et al. 1998). Could this putative neural substrate for errors of memory-guided saccades to visual stimuli also account for the inaccuracy of saccades to sound locations presented during the Duncker illusion?

It was originally reported that neurons in portions of the parietal cortex corresponding to LIP in macaque did not respond to auditory stimuli (Hyvärinen 1982; Mountcastle et al. 1975). More recent studies have shown than LIP neurons will respond to acoustic targets, but only after monkeys are trained to make saccades to sounds (Grunewald et al. 1999; Mazzoni et al. 1996). After such training, LIP neurons show greater responses when they are engaged in a memory-saccade task than when they are fixating a visual target, and sound stimuli are presented (Linden et al. 1999). Thus overall behavior of such LIP neurons is more "supramodal" than modality-specific and may reflect shifts of attention, target selection, or planning of gaze shifts (Gottlieb and Goldberg 1999; Gottlieb et al. 1998; Linden et al. 1999). Consistent with this view, pharmacological inactivation of LIP with muscimol has little effect on visually-guided saccades, but saccades to remembered locations of visual targets become inaccurate with contralateral hypometria (Li et al. 1999). The effects of pharmacological inactivation of LIP on memory-guided saccades to sounds are unreported.

Although it seems likely that LIP in macaque monkey is involved in planning saccades to remembered sound locations, other areas may also make substantial contributions. For example, movement-related auditory responses have also been reported in the frontal eye fields (Russo and Bruce 1994) and in the superior colliculus (Jay and Sparks 1987). In addition, differences are likely to exist between the properties of LIP of macaque and its homologue in humans. In fact, monkeys are slow to learn how to make saccades to sounds (Grunewald et al. 1999) and so their behavior appears quite different from humans; our subjects were able to respond to both acoustic target paradigms after being shown one or two trials. Nonetheless, there is also abundant evidence that the posterior parietal region in humans is important for directing attention to targets in extrapersonal space. Biparietal lesions are reported to impair perceptual localization of objects in the environment and saccades made toward them (Leigh and Zee 1999; Pierrot-Deseilligny et al. 1986). Thus it seems that measurement of memory-guided saccades to visual and acoustic stimuli and the effects of the Duncker illusion on these responses presents a promising new experimental strategy to investigate the nature of the behavioral disturbance in patients with parietal lobe lesions.


    ACKNOWLEDGMENTS

We thank L.-L. Kim and Dr. Paul K. Jones for help with statistical analysis, Drs. Dmitri Kourennyi and Niels Otani for advice, and C. C. Lambert for technical assistance.

This work was supported by National Eye Institute Grant EY-06717, the Office of Research and Development, Medical Research Service, Department of Veterans Affairs, and the Evenor Armington Fund (to R. J. Leigh).

Present address of V. E. Das: Division of Visual Sciences, Yerkes Regional Primate Research Center, Emory University, Atlanta, GA 30322.


    FOOTNOTES

Address for reprint requests: R. J. Leigh, Dept. of Neurology, University Hospitals, 11100 Euclid Ave., Cleveland, OH 44106-5040 (E-mail: rjl4{at}po.cwru.edu).

The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Received 8 February 2000; accepted in final form 23 March 2000.


    REFERENCES
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

0022-3077/00 $5.00 Copyright © 2000 The American Physiological Society




This Article
Abstract
Full Text (PDF)
Alert me when this article is cited
Alert me if a correction is posted
Citation Map
Services
Email this article to a friend
Similar articles in this journal
Similar articles in ISI Web of Science
Similar articles in PubMed
Alert me to new issues of the journal
Download to citation manager
Search for citing articles in:
ISI Web of Science (1)
Google Scholar
Articles by Somers, J. T.
Articles by Leigh, R. J.
Articles citing this Article
PubMed
PubMed Citation
Articles by Somers, J. T.
Articles by Leigh, R. J.


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
Visit Other APS Journals Online