Faculty of Design, Engineering and Production, Delft University of Technology, NL-2628 BX Delft, The Netherlands
![]() |
ABSTRACT |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
van Beers, Robert J., Anne C. Sittig, and Jan J. Denier van der Gon. Integration of proprioceptive and visual position-information: an experimentally supported model. To localize one's hand, i.e., to find out its position with respect to the body, humans may use proprioceptive information or visual information or both. It is still not known how the CNS combines simultaneous proprioceptive and visual information. In this study, we investigate in what position in a horizontal plane a hand is localized on the basis of simultaneous proprioceptive and visual information and compare this to the positions in which it is localized on the basis of proprioception only and vision only. Seated at a table, subjects matched target positions on the table top with their unseen left hand under the table. The experiment consisted of three series. In each of these series, the target positions were presented in three conditions: by vision only, by proprioception only, or by both vision and proprioception. In one of the three series, the visual information was veridical. In the other two, it was modified by prisms that displaced the visual field to the left and to the right, respectively. The results show that the mean of the positions indicated in the condition with both vision and proprioception generally lies off the straight line through the means of the other two conditions. In most cases the mean lies on the side predicted by a model describing the integration of multisensory information. According to this model, the visual information and the proprioceptive information are weighted with direction-dependent weights, the weights being related to the direction-dependent precision of the information in such a way that the available information is used very efficiently. Because the proposed model also can explain the unexpectedly small sizes of the variable errors in the localization of a seen hand that were reported earlier, there is strong evidence to support this model. The results imply that the CNS has knowledge about the direction-dependent precision of the proprioceptive and visual information.
![]() |
INTRODUCTION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Humans can use different sources of sensory
information for localizing their hand, i.e., for finding out the
hand's position with respect to the body. When the hand is seen,
proprioception and vision simultaneously provide useful information,
and the CNS uses both sources to plan movements (Rossetti et al.
1995). An analysis of variable errors in tasks where a
nonmoving hand had to be localized showed that the available
proprioceptive and visual information are combined in a very efficient
way (van Beers et al. 1996
). In the present study, we
analyze constant errors in localization tasks to gain more insight into
how the multisensory information is integrated.
Various psychophysical studies have been performed to investigate in
what position a hand is localized on the basis of simultaneous proprioceptive and visual information. In most of these studies (e.g.,
Hay et al. 1965; Pick et al. 1969
;
Warren 1980
; Warren and Pick 1970
), wedge
prisms were placed before the subject's eyes to introduce a conflict
between the hand position encoded by vision and the one encoded by
proprioception. If the conflict is not too large, subjects perceive
their seen hand in one single location, which is somewhere between the
two positions where it is perceived on the basis of vision only and
proprioception only, respectively (Warren and Cleaves
1971
). It usually is localized closer to the visually perceived
position than to the position perceived proprioceptively (e.g.,
Hay et al. 1965
; Pick et al. 1969
;
Warren 1980
; Warren and Pick 1970
). This
is consistent with the results of other conflict studies in which the
proprioceptive information was modified by muscle vibration, whereas
the visual information was veridical (DiZio et al. 1993
;
Lackner and Levine 1978
; van Beekum
1980
). There thus seems to be a weighting of the visual and the
proprioceptive information, the greater weight usually being given to
the visual information. This does not mean to say that we fully
understand how this weighting takes place. There are at least two
different ideas about how the weights given to each modality are
determined. According to one idea, the weights are determined by the
precision of the information in each modality (Pick et al.
1969
; Welch et al. 1979
); according to another
idea, they are related to the attention that is directed to each
modality (Canon 1970
, 1971
; Kelso et al.
1975
; Uhlarik and Canon 1971
; Warren and
Schmitt 1978
).
The conflict studies thus have exposed a problem that still is
unresolved: how are the weights given to vision and to proprioception determined when both sources simultaneously provide information about
the hand's position? In this study, we follow a new approach to test
whether the weights are related to the precision of the information in
each modality. Unlike the above-mentioned conflict studies, which all
involved localization in one dimension, we will study localization in
two dimensions. This enables us to distinguish between different
mechanisms the differences of which vanish in one-dimensional space.
Crucial here is the recent finding that the precision of localization
in a horizontal plane on waist level is spatially nonuniform for both
vision and proprioception (van Beers et al. 1998).
Proprioceptive localization is generally more precise in the radial
direction with respect to the ipsilateral shoulder than in the
azimuthal direction. This can be understood from the geometry of the
arm. Visual localization, on the other hand, is more precise in the
azimuthal direction with regard to the cyclopean eye than in the radial
direction. This is at least partly a result from the subjects looking
down on the horizontal plane. The question then is how the visual
information and the proprioceptive information are weighted in
two-dimensional space. One possibility, in line with the
directed-attention hypothesis, is that weights are given to the two
modalities irrespective of the direction-dependent precision. This
would predict that the seen hand is localized somewhere on the
straight line through the two positions where it is localized on
the basis of vision only and proprioception only, respectively. The
exact position at which it is localized on this line depends on the
weights given to each modality. Although this may seem a reasonable
idea, it is not likely to be correct because the actual variable errors in localization of a seen hand were found to be smaller than the variable errors predicted by such a model (van Beers et al.
1996
). This suggests that the CNS uses the available
information more efficiently. A second possibility, in which the
available information is indeed used more efficiently, is that the CNS
takes the direction-dependent precision of proprioceptive and visual
localization into account and uses different weights in different
directions related to this precision. We will show that such a model
predicts that the seen hand generally will be localized off the
straight line through the two positions where it is localized on
the basis of vision only and proprioception only. We tested this
prediction in the following experiment.
Subjects were asked to match the position of a target in a horizontal plane with their unseen left hand. We used three conditions that differed in the modality in which the target was presented: proprioception only (condition P, eyes closed, right hand is target), vision only (condition V, visual target), or both proprioception and vision (condition PV, seen right hand is target). We will refer to conditions P and V as unimodal conditions because the target position was available in only one modality; condition PV is referred to as the bimodal condition. We take advantage of the fact that different constant errors generally are found in the two unimodal conditions. This enables us to study how vision and proprioception are combined without there being any need to introduce an unnatural conflict between the two modalities. However, the subjects also performed the tasks wearing prism goggles. In this way, the difference between the positions indicated in the two unimodal conditions could be varied, which allowed a more effective test of the "direction-dependent" model. The results of this study provide evidence to support the hypothesis that the CNS takes the direction-dependent precision of visual and proprioceptive localization into account when integrating these two types of information. This implies that the CNS has knowledge about the direction-dependent precision of the visual and proprioceptive information.
![]() |
METHODS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Model
We will propose a model that describes how the localization of
one's hand on the basis of simultaneous visual and proprioceptive information is related to proprioceptive and visual localization in
isolation. The outcome of the localization of a hand can be described
by two quantities: the mean position at which the hand is localized and
the precision of this mean position. We will use variances to describe
this precision because the reciprocal of the variance in a signal is a
useful measure of the information in that signal (Fisher
1966). For localization in the horizontal plane, one single
variance is not sufficient to describe the precision because the
variance depends on direction for both vision and proprioception
(van Beers et al. 1998
). The mean position and the
direction-dependent variance, which together describe the outcome of
the localization, can be described concisely by two-dimensional normal
distributions. In general, an n-dimensional normal
distribution can be written as (e.g., Winer et al. 1991
)
![]() |
Figure 1 shows an example of confidence
ellipses of such distributions for visual localization
[PV(x)] and for proprioceptive localization [PP(x)] of the right
hand in the left hemifield of a horizontal plane (we used target
positions in this area, see Fig. 3). The confidence ellipse of a
two-dimensional spatial probability distribution consists of spatial
locations of equal probability. The narrower an ellipse is in a certain
direction, the more precise the hand is localized in this direction.
The relative locations of the two ellipses in the figure reflect what
generally is found in experiments like the present one: when the unseen
right hand is the target that has to be matched with the unseen left
hand, subjects usually point too far to the right (Crowe et al.
1987; Slinger and Horsley 1906
) and when a
visual target has to be matched with the unseen left hand, an overreach
in the forward direction often is found (de Graaf et al.
1995
; Foley and Held 1972
). The shapes, the
relative sizes, and the orientations of the two ellipses in Fig. 1 are
chosen to comply with what was found by van Beers et al.
(1998)
.
|
The distribution for bimodal localization can be found by multiplying
the two distributions for unimodal localization. It can be shown that
the product of two normal distributions is again a normal distribution
(after normalization). The parameters describing the bimodal
distribution PPV(x) are related to
those of the two unimodal distributions as follows:
![]() |
![]() |
The bimodal ellipse derived from the two unimodal ellipses shown in Fig. 1 is indicated in the same figure. It is striking that the center of the bimodal ellipse lies off the straight line through the centers of the two unimodal ellipses. To understand this, one needs to consider the direction of the minor axis of the proprioceptive ellipse. In this direction, proprioceptive localization is more precise than visual localization. Accordingly, in this direction more weight will be attached to proprioception than to vision. Similarly, a larger weight will be attached to vision in the direction of the minor axis of the visual ellipse. As a result, the predicted center of the bimodal ellipse lies off the straight line through the centers of the unimodal ellipses as sketched in Fig. 1. How far the predicted center of the bimodal ellipse lies off the straight line depends on the characteristics of the unimodal ellipses. In general, the more the major axes are perpendicular to each other and the more eccentric the ellipses, the larger the deviation will be. However, when the center of one unimodal ellipse lies in the vicinity of the extended part of the major axis of the other unimodal ellipse, the deviation is small. The relation between the relative locations of the two unimodal ellipses and the predicted position of the bimodal ellipse is illustrated in Fig. 2. The figure shows that the predicted center of the bimodal ellipse generally lies on the same side of the straight line through the centers of the two unimodal ellipses as the intersection of (the extended parts of) the major axes of these two ellipses. It can be shown that this applies to all possible relative locations of the two unimodal ellipses, except for a very small number of relative locations where the center of one unimodal ellipse is on, or very close to, (the extended part of) the major axis of the other unimodal ellipse.
|
The way in which the information is combined can be described as
weighting of the visual and proprioceptive information, the weights
depending on the spatial direction. This can be seen explicitly by
considering the variances in any arbitrary direction in the horizontal
plane. It can be shown that in each direction, the reciprocal of the
bimodal variance is equal to the sum of the reciprocals of the two
unimodal variances in that direction. For one-dimensional normal
distributions, the available information is used with an optimal
efficiency in such a weighting (Ghahramani et al. 1997).
This shows that the model describes the situation in which two
two-dimensional normal distributions are combined such that the
available information is used optimally. It therefore is conceivable
that the model predicts bimodal ellipses that are generally smaller
than those predicted by the much coarser method with fixed, i.e.,
direction-independent, weights (the model tested in van Beers et
al. 1996
). The present model therefore may explain the small
variances reported in van Beers et al. (1996)
that
remained unexplained by the model tested in that study.
To test the proposed model, we verify one of its striking predictions,
namely the prediction for the mean positions. To derive reliable
predictions for these means, the characteristics of the unimodal
distributions have to be known with a certain degree of accuracy for
each subject. However, these characteristics cannot be determined with
sufficient accuracy for individual subjects as an estimation reveals
that this would require each subject to participate for >100 h. We
therefore will base the analysis on the results of our previous study
(van Beers et al. 1998). For proprioception, we found
that the variance in the radial direction with respect to the shoulder
was smaller than the variance in the azimuthal direction. For visual
localization, the variance was smaller in the azimuthal direction with
respect to the cyclopean eye than in the radial direction. We used
these origins (the shoulder and the cyclopean eye) because they make
sense physiologically. However, we could not demonstrate that the
ellipses are indeed oriented toward these origins. The use of slightly
different origins, such as the projection of the body midline on a
horizontal plane, would describe the results equally well. In the
analysis of the present study, we will assume that the major axes of
visual ellipses point toward the cyclopean eye and that the major axes
of proprioceptive ellipses are perpendicular to the imaginary straight
line to the shoulder. Because we have no strong support for this
assumption, we also will analyze the data using the body midline
projection as the origin for both vision and proprioception. From the
assumed orientations and from the positions at which each subject
indicated in the unimodal conditions, we will derive where the model
predicts the mean of the bimodal condition to lie. This prediction,
then, will be compared with where the subject actually pointed.
Subjects
Ten subjects (1 woman, 9 men, aged 19-50 yr), who previously had given their informed consent, participated in this experiment. All subjects were naive as to the purpose of the experiment. The research was performed in accordance with the regulations of the Commissie Mensproeven i.o. of Delft University of Technology. All subjects had normal or corrected-to-normal vision and no one had any history of neuromuscular disorders.
Setup
Subjects were seated on a chair at a table the top of which was 92 cm above the floor. The chair and the table could not be moved during the experiment. The table top was covered with a bright yellow sheet of cardboard. The positions of the targets were indicated by small pieces of black cardboard on this sheet. Because the pieces of cardboard could be felt and seen by the subjects, they could be used to define target positions both proprioceptively (by placing the right hand on them) and visually. We used three target positions which were all in the subjects' left hemifield and relatively far in front of them (see Fig. 3). Assuming the cyclopean eye and the right shoulder as origins for the ellipse orientations, the major axes of the visual and the proprioceptive ellipses were expected to be approximately perpendicular in this area (with the body midline as origin, this relation is exact). The predicted deviations from the straight line were therefore close to maximal for these target positions.
|
We used an OPTOTRAK 2010 system to record the indicated positions. This system measures the three-dimensional coordinates of an infrared emitting diode (IRED) with a precision of 0.5 mm in all directions. An IRED was placed on the nail of the left index finger.
Procedure
At the beginning of each trial, subjects had to put both hands in starting positions. The left hand was placed just in front of the subject, slightly to the left of the subject's midline, on a support 10 cm below the table (see Fig. 3). The starting position for the right hand was on the table top, slightly to the right of the subject's midline. In all conditions, subjects had to match the target position with their unseen left hand touching the underside of the table. The conditions differed with regard to the information about the target position.
CONDITION P. The task was to match the position of the unseen right hand on the table top (a proprioceptive target) with the unseen left hand on the underside of the table. After the experimenter had presented the target number, subjects closed their eyes and moved the tip of their right index finger to the appropriate piece of cardboard so that this fingertip defined the target position. Thereupon the subjects moved their left hand to match the position of the right hand. Subjects did not open their eyes until the indicated position had been recorded. One could argue that this condition is not a purely proprioceptive matching task because subjects saw the target position at the beginning of each trial and thus also could use visual memory to position their left hand. However, we have evidence that the subjects did not rely on visual memory to position their hands in this condition: they quickly found a target with their right hand after having closed their eyes, but in many cases a wrong target. Only after moving their right hand over the cardboard to find the other targets (which they did spontaneously) did they rest their right hand on the, now correct, target. This indicates that visual memory did not provide useful information about the target position.
CONDITION V. Subjects used their unseen left hand on the underside of the table to match the position of a visual target. They moved their left hand after the experimenter had presented the target number. The right hand was not used. It remained at its starting position.
CONDITION PV. Subjects matched the position of the tip of the index finger of the seen right hand on the table top (target position available by both proprioception and vision) with the unseen left hand on the underside of the table. After the experimenter had presented the target number, subjects moved their right hand to the appropriate piece of cardboard. Thereupon the subjects moved their left hand to match the position of the seen right hand.
In all conditions the task was to match the target position as accurately as possible. Usually, subjects first made a relatively fast movement to bring the left hand close to the target position, whereupon the eventual matching was achieved by some slower movements. The total adjustment usually took a few seconds. When satisfied that the left index finger matched the target position, the subject told the experimenter so. The subject then did not move the finger until its stationary position had been recorded. The subjects were asked not to move their trunk with respect to the chair during the entire experiment. Small movements, however, could not be prevented. Head movements were allowed; subjects always fixated the target when it was presented visually. We did not immobilize any body part for two reasons. First, in the analysis we made use of the results of a previous study (van Beers et al. 1998Analysis
We tested the model by analyzing where subjects pointed in condition PV relative to where they pointed in conditions P and V. We performed the following analysis for each target in each series for each subject. We determined the mean of the five positions that were indicated in each of the three conditions. Next, we assumed that the precision of visual localization can be described by a confidence ellipse the major axis of which is directed toward the cyclopean eye. To visualize this, we drew a straight line with this orientation through the mean of condition V. Likewise, we assumed that the precision of proprioceptive localization of the right hand can be described by a confidence ellipse the major axis of which is perpendicular to the imaginary straight line through the position of the hand and the right shoulder. We drew such a line through the mean of condition P. To test the model, we analyzed whether the mean of PV was on the same side of the straight line through the means of the two unimodal conditions as the intersection of the two drawn lines. The effect of the few relative locations for which this method is not correct (see the model description) is expected to be marginal because the predicted deviation from the straight line is small in comparison with the scatter in indicated positions for these relative locations.
Because we had 10 subjects who matched three target positions in three series, we compared the predicted side with the observed side for 90 cases. We used a one-sided sign test to test the null hypothesis that the number of correct predictions is equal to or smaller than the number of incorrect predictions.
In addition to this "binary" analysis, we also determined the distances of the mean of PV from the straight line through the means of P and V for all 90 cases. We made a histogram of these distances and used a one-sided t-test to check whether the observed distribution of distances could correspond to the mean of PV lying on the straight line through the means of P and V.
To test the proposed direction-dependent model in a more direct way, we
performed similar analyses of the location of the mean of
condition PV with respect to the curved
line predicted by the model. This curved line can only be determined
when the two unimodal distribution are known. As argued in the
MODEL section, we could not derive these distributions for
our individual subjects. Therefore we used the means of the results of
the 10 subjects in van Beers et al. (1998) for the
target that was closest to the targets in the present study. For
proprioceptive localization, we assumed that the variance in the radial
direction was 48 mm2, whereas a variance of 64 mm2 was assumed in the azimuthal direction. For visual
localization, these values were 10 and 23 mm2,
respectively. From these distributions and from the means of conditions P and V, we determined the
curved lines predicted by the model. We determined at which side of
this curved line the mean of condition PV lay. This
could be either "inside" the curved line, i.e., on the side of
the straight line, or "outside" the curved line. A two-sided sign
test was used to test the null-hypothesis that the numbers of means of
PV inside and outside the curved line were equal. We also
analyzed the distances of the mean of PV from the curved
line similar to how we analyzed the distances from the straight line.
To investigate whether the choice of ellipse orientations was crucial for the interpretation of our results, we repeated the complete analysis using slightly different orientations. We now used the body midline as the origin for both modalities.
Note that the localization of the indicating, left hand is not used in the analysis. Because the indicator was the same in all conditions, we assume that all observed differences between conditions result from differences in target localization. The uncertainty of localization of the left hand adds only to the scatter in the indicated positions. Note also that different sensory-to-motor transformations were required in different conditions. It might be possible that this influenced the results, but because we have no arguments suggesting that it would (and if it would it is not clear how), we did not take such possible influences into account.
![]() |
RESULTS |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Figure 4A shows an
example of the positions that one subject (SK) indicated in
the series without prism goggles. Like most subjects, this subject
overreached the targets in condition V, which is
consistent with earlier observations (de Graaf et al. 1995; Foley and Held 1972
). In condition
P, this subject pointed slightly too far to the right and
overreached the target. Most subjects pointed too far to the right in
this condition, which is in agreement with the "overlap effect"
reported previously in similar experiments (Crowe et al.
1987
; Slinger and Horsley 1906
).
|
The positions indicated in condition PV relative to those of conditions P and V are judged more easily in Fig. 4B, which shows the means of the indicated positions shown in Fig. 4A. This figure also shows the lines that indicate an assumed orientation of the major axes of the ellipses for visual localization (condition V, the lines point toward the cyclopean eye) and for proprioceptive localization (condition P, the lines are perpendicular to the imaginary straight line to the right shoulder) of the targets. For each target, the intersection of these two lines indicates on which side of the straight line through the means of P and V the model predicts that the mean of PV will lie. In this case, the intersections are to the left of this line for all three targets. The observed mean is indeed on this side for all targets.
When subjects wore prism goggles, they all noticed that the visual field was affected, but none of them understood in what way it was affected. Inquiries after the experiment revealed that no subject had used a cognitive strategy to compensate for the experienced distortion. Figure 4, C and D, show the results of the series in which leftward displacing prism goggles were worn. As expected, in condition V in this series all subjects pointed more to the left than in the first series. To a lesser extent, this also was observed for condition P for all subjects. This shift in condition P reflects the adaptation of proprioceptive position sense that had occurred during the exposure period in which the moving right hand was viewed through the prisms before the series. The shift was to the left because the adaptation diminished the conflict between the two senses. Because there also may have been some adaptation of proprioception about the position of the left hand, the shift actually reflects the difference between the adaptation for the right hand and the left hand. For the subject whose results are shown in Fig. 4, the predicted deviations from the straight line are larger in this series than in the series without prisms. For all target positions, we did indeed find a relatively large deviation on the predicted side of the straight line through the means of P and V.
The results of the series with the rightward displacing prisms are shown in Fig. 4, E and F. In this series, we found a rightward shift of the positions indicated in conditions V and P for all subjects. The shift in condition V was always larger than the shift in condition P. The deviations from the straight line predicted by the model for this subject are small in comparison with those of the series with the leftward displacing prisms. The intersections are to the right of the straight line through the means of P and V for all targets. For one of the targets, the observed mean lies on the opposite side of this line. For the two other targets, the predicted side is correct.
For the subject of Fig. 4, the model prediction was correct in eight of nine cases. For all subjects, the prediction was correct in 61 of the 90 cases. The middle column of Table 1 shows how the numbers are distributed over the various series. For all series, the number of correct predictions is larger than the number of incorrect predictions. We tested the null hypothesis that the number of correct predictions was equal to or smaller than the number of incorrect predictions. A sign test shows that this hypothesis has to be rejected (P < 0.001). The results therefore refute the hypothesis that the mean of condition PV lies on the straight line through the means of conditions P and V or on the side opposite to the one predicted by the model. The null hypothesis has to be rejected also on the basis of only the series with no prisms (P < 0.025). For the series with the leftward displacing prisms, the hypothesis has to be rejected as well (P < 0.01). No significant effect was found for the series with the rightward displacing prisms.
|
We also analyzed the distances of the mean of PV from the straight line through the means of P and V. Figure 5A shows this distance for all 90 cases. In this figure, a distance was chosen to be positive when the mean of PV lay on the side predicted by the model. This figure shows that the distribution of distances is biased in the predicted direction (P < 0.001, two-sided t-test); this confirms the results of the binary analysis.
|
Both types of analysis thus show that a direction-independent (or straight line) model has to be rejected. To examine whether the direction-dependent (or curved line) model provides a better fit for the data, we also determined on which side of the curved lines the means of PV lay. For this purpose, we had to make assumptions about the visual and proprioceptive ellipses (see Analysis). The right-hand column of Table 1 shows that the mean of PV was inside this line in 40 of the 90 cases. Sign tests show that the hypothesis that the mean of PV lies on the curved line cannot be rejected for the total of all 90 cases nor for any of the individual series. Figure 5B shows the distances from the curved line, where a positive distance denotes a mean that lies outside the curved line. This figure shows a smaller bias than Fig. 5A. Statistical analysis proved that the bias from the straight line was indeed larger than the bias from the curved line (P < 0.001, one-sided paired t-test). The bias in the latter case is not significantly different from zero (P > 0.2, two-sided t-test).
Because we have no evidence that the ellipses are oriented as we assumed in the aforementioned analysis, we repeated the analysis using different orientations. We now assumed that the major axis of the visual ellipse points toward the body midline and that the major axis of the proprioceptive ellipse is perpendicular to the imaginary straight line to the body midline. The results of this analysis are very similar to those of the previous analysis. The direction-independent model has to be rejected on the basis of both the binary analysis (P < 0.001, one-sided sign test, see Table 2) and the analysis of the distances (P < 0.001, two-sided t-test, see also Fig. 5C). The binary analysis also shows that the direction-dependent model cannot be rejected for the total of all 90 cases nor for any of the individual series (Table 2). The bias that was present in the distances from the straight line (Fig. 5C) has again been reduced by taking the distances from the curved line (P < 0.001, one-sided paired t-test), and the resulting bias does not differ significantly from zero (P > 0.05, two-sided t-test, see also Fig. 5D).
|
![]() |
DISCUSSION |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
In this study, we have investigated in what location a hand is localized on the basis of simultaneous visual and proprioceptive information relative to the positions at which it is localized on the basis of vision only and proprioception only. The results indicate that a seen hand is generally not localized on the straight line through the two positions where it is localized in the unimodal situations. Instead the location is predicted better by a model that describes this multisensory integration as a direction-dependent weighting of the proprioceptive and the visual information, the weighting being related to the precision of the unimodal information.
Validity of the proposed model
Before discussing the implications of this study, we want to make sure that the results do not rely on an artifact caused by the subject's posture. We are concerned specifically about whether the result that subjects indicated at different locations in different experimental conditions could have been caused by the subjects having had different postures in different conditions (this could have happened for instance because the right hand was at the starting position in condition V, whereas it was at the target position in conditions P and PV). If such differences were present, their effect on the position of the mean of PV relative to the means of P and V is expected to be in the same direction in all three series. A comparison of Fig. 4, B, D, and F, reveals that this is not the case. In the series with the leftward displacing prisms the mean of PV is shifted further to the left from the straight line through the means of P and V than in the series with no prisms; in the series with the rightward displacing prisms, it is shifted to the right. Obviously, this is different from the constant effect expected from condition-dependent postures. We conclude that the significant effects we found do not rely on this artifact.
The hypothesis that the seen hand is localized somewhere on the straight line between the positions at which it is localized on the basis of vision only and proprioception only is refuted by the present data. This is direct support for the proposed model but an alternative explanation could be that the CNS does not represent locations in Cartesian coordinates but in another coordinate system, such as spherical or joint-based coordinates. Interpolating between two locations in such a system and then converting the locations to Cartesian coordinates would produce the interpolated location to lie off the straight line through the outer two locations. Although it may be possible to find a coordinate system that is in agreement with our results, it seems unlikely that such a system also makes sense physiologically. The two above-mentioned systems that do make sense physiologically do not agree with the data as can be concluded from Fig. 4 using the same argument as in the previous paragraph.
The results show that the mean of condition PV was more often on the side predicted by the proposed curved line model than on the opposite side. This suggests that this model gives a better description of the results than a straight line model would do. This result cannot be due to using prism goggles because the effect was also significant for the series with no prisms. The fact that we found no significant effect for the series with the rightward displacing prisms is another indication of the validity of the model: the relative locations of the means of conditions P and V in this series were frequently comparable to situation E in Fig. 2, so that only very small deviations from the straight line were to be expected. In these situations, the predicted deviations were very small compared with the scatter in the indicated positions. As a result, relatively many incorrect predictions (i.e., observed means that lay opposite to the side predicted) were to be expected in this series, which is indeed what we found with rightward displacing prisms. In general, the relatively large number of incorrect predictions (29 of 90) does not disprove the model. Such a number is to be expected from the scatter in the indicated positions and from the uncertainty in the actual ellipse orientations (compare the width of the distribution of distances with its bias in Fig. 5A).
More direct evidence in favor of the curved line model is provided by comparing the distances of the means of condition PV from the straight and curved lines, respectively. Whereas we found a significant bias relative to the straight line, this bias disappears when we analyze the data relative to the curved line. Because no significant bias resulted, we conclude that the curved line model gives a good quantitative prediction of the position in which a seen hand is localized.
Reanalyzing the data using slightly different orientations of the visual and proprioceptive ellipses produced basically the same results. This indicates that the exact ellipse orientations were not crucial and that it was legitimate to use estimated orientations.
The constant errors found in the present study are thus in good
agreement with the predictions of the model. We cannot use the results
of this study to test the variable errors predicted by the model
because the precision of unimodal localization cannot be derived from
these results (that would require an experiment as in van Beers
et al. 1998). However, evidence that the model also gives a
correct description of the variable errors can be found in van
Beers et al. (1996)
. In that study, we found that the positions
indicated in a condition with both proprioception and vision showed
less scatter than was predicted by a direction-independent model. The
difference between predicted and observed variance was found to vary
among the target positions used. It varied qualitatively in a way that
is predicted by the model proposed in the present paper. For instance,
we found the largest difference for the target to the subject's left
(target 1 in that study). The model indeed predicts a large
direction-dependent effect in this area because here the visual and
proprioceptive ellipses are expected to be relatively elongated and
their major axes will be approximately perpendicular. In conclusion,
both the constant errors found in the present study and the variable
errors found in van Beers et al. (1996)
provide evidence
in favor of a model like the one proposed here.
Implications of the model
What conclusions can be drawn from this study about how the CNS
processes simultaneous proprioceptive and visual information about hand
positions? We found that a seen hand is generally not localized on the
straight line through the two positions where it is localized on the
basis of vision or proprioception only. This implies that the CNS uses
direction-dependent weights when combining information from the two
modalities. We could predict on which side of the straight line the
mean of the bimodal condition would lie on the basis of the
direction-dependent precision of visual and proprioceptive
localization. This is strong evidence that the weights are related to
the direction-dependent precision of the information in each modality.
The weights are chosen in such a way that the available information is
used very efficiently, i.e., more efficiently than if the weighting
were direction-independent. Our results do not support the hypothesis
that the weights are determined by the attention directed to each
modality: this hypothesis cannot explain the observed deviations from
the straight lines. However, this does not rule out the possibility
that attention can influence the weights used (for an example, see
Warren and Schmitt 1978).
The finding that the weights used are related to the precision of
the information implies that the CNS has knowledge about this
direction-dependent precision. It would be interesting to know how the
CNS obtains this knowledge. There seem to be at least two possible
ways. First, the CNS may have learned the precision from experience,
for instance, from the errors occurring in reaching movements, which
also may be direction dependent. This would suggest that the knowledge
is stored somehow. Second, the precision may be derived instantaneously
from sensory signals. For example, the hand's position in space might
be derived directly from proprioceptive signals such as muscle spindle
output. To do this, the CNS must be able to transform the muscle
spindle output into spatial coordinates. When this transformation is
made, noise in the muscle spindle output is translated directly into
direction-dependent uncertainty in the derived hand position. The
results of this study do not distinguish between these two
possibilities. Note that both mechanisms will have approximately the
same result because the precision of localization is determined
primarily by the geometry of the sensory system and the density of
sensory organs (as argued in van Beers et al. 1998),
irrespective of the way in which the knowledge about the
direction-dependent precision is obtained.
Another question is how the integration of multisensory information as described by the model actually is accomplished by the CNS. Although neural nets may implicitly carry out such computations, the CNS not necessarily performs an actual multiplication of two probability distributions. Another possibility is that the CNS processes the available information in a way that minimizes some error, for instance, the errors made in reaching movements. If this minimization acts near optimally, which may be achieved after sufficient experience, the system's performance will be reasonably well described by our model. The model thus may describe the overall performance of the CNS rather than the computations it actually performs.
It is possible that the proposed model, or a generalized version that
also minimizes variance, is also applicable in other situations, such
as involving other modalities and in three-dimensional space. This also
is suggested by the results of Ghahramani et al. (1997),
which provide evidence that a similar model would describe correctly
the integration of visual and auditory position information in the
azimuthal direction with respect to the head. A more generalized
version of the model may even be applicable in many other situations,
e.g., situations involving multisensory integration of information
about orientation, velocity, shape, etcetera. It is interesting to note
that the principle of minimizing of variance also may be on the basis
of the planning of goal-directed eye and arm movements (Harris
and Wolpert 1998
).
Finally, it also would be interesting to know how the spatial
information is processed and represented at the neural level. A
possibility is that spatial probability distributions of spatial locations as we used to describe the outcome of localization are actually represented in the CNS. Neurophysiological studies (e.g., Georgopoulos et al. 1984, 1986
; Helms Tillery et
al. 1996
; Kalaska and Crammond 1992
) have shown
that quantities such as hand positions and movement directions are
represented by the activity of large populations of cells. Because
large numbers of cells are involved, not only a single value of the
represented quantity can be derived but possibly also the reliability
of this value. It therefore might be possible to interpret the activity
of a population of cells as a spatial probability distribution.
Theoretical studies (Anderson 1994
; Sanger
1996
; Zemel et al. 1998
) have indicated that
this may indeed be possible and that it may be a very efficient and
useful way to represent spatial information. It therefore would be
interesting to see whether spatial probability distributions indeed can
be derived from neurophysiological data.
![]() |
APPENDIX |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
A two-dimensional normal distribution can be written as (e.g.,
Winer et al. 1991)
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
ACKNOWLEDGMENTS |
---|
We thank J. Smeets and E. Brenner for helpful discussions about the experiment and for putting the prism goggles at our disposal. We thank D. Wolpert and Z. Ghahramani for suggesting an elegant mathematical representation, S. Gielen and an anonymous reviewer for useful suggestions on earlier versions of this paper, and S. McNab for improving the language and clarity of the paper.
![]() |
FOOTNOTES |
---|
Present address and address for reprint requests: R. J. van Beers, Institute of Cognitive Neuroscience, University College London, 17 Queen Square, London WC1N 3AR, United Kingdom.
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
Received 27 July 1998; accepted in final form 16 November 1998.
![]() |
REFERENCES |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|