The Neurosciences Institute, 10640 John Jay Hopkins Drive, San Diego, CA 92121, USA
![]() |
Abstract |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
![]() |
Introduction |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
As a result of efforts to compile databases of corticocortical and corticothalamic pathways in various species (Scannell et al., 1999), a wealth of neuroanatomical data has become available in recent years. The growing need for an in-depth analysis of this information has prompted a variety of approaches. These approaches have focused mainly on structural aspects such as organization into streams and hierarchies (Young, 1992
; Scannell et al., 1995
; Hilgetag et al., 2000
), wiring length (Mitchison, 1991
), total volume occupied by connectivity (Ringo, 1991
) and spatial placement of components (Cherniak, 1995
). Only rarely, however, have structural analyses been placed in the context of functional connectivity and dynamics. In this paper, we aim at a theoretical analysis of neuroanatomical structures, in particular those of the cerebral cortex, in relation to the specific modes of functional connectivity that they can support.
To characterize the anatomical connectivity of a network, we utilize concepts and measures provided by graph theory (Harary et al., 1975; Bollobás, 1985
; Palmer, 1985
). Representing networks as graphs has the important advantage of a parsimonious structural description that allows comparisons of different connection patterns within a common theoretical framework. It is important to note that nervous systems exhibit distinct neuroanatomical patterns at several levels of scale, ranging from the fine structure of single neurons to local circuitry, and finally to pathways linking distinct cortical areas. In this paper, we focus our analysis on the level of pathways that link different areas of the cerebral cortex. This permits direct application of our theoretical analysis to published cortical connection matrices based on actual neuroanatomy (Felleman and Van Essen, 1991
; Scannell et al., 1995
).
To characterize the functional connectivity (Gerstein et al., 1978; Boven and Aertsen, 1990
; Friston, 1994
) of a network it must be implemented as a dynamical system. The neural activity that emerges can be described as a multidimensional stochastic process whose joint probability density function can be characterized by measures of entropy and mutual information (Papoulis, 1991
). We concentrate here on a set of global functional measures that describe the functional connectivity of a given network, such as its entropy, integration and complexity (Tononi et al., 1998b
). Entropy and integration capture, respectively, the overall degree of statistical independence and dependence. Of particular interest for the present study is the measure of complexity, which characterizes the interplay of functional segregation and functional integration, a central organizing principle of the cerebral cortex. More recently, we have also defined information-theoretical measures of matching complexity and degeneracy (Tononi et al., 1996
, 1999
). Matching captures how well a system's functional connectivity matches the statistical structure of input patterns. Degeneracy captures the ability of structurally different subsets of the system to produce similar effects on output patterns. Both measures characterize the interactions of a neural system with an environment on the basis of the network's functional connectivity.
In our analysis of the relationship between structural properties of neuronal networks and their functional dynamics, we proceed in three steps. First, we use artificially produced graphs in a search for distinct anatomical motifs that emerge after selection for different measures of functional connectivity such as entropy, integration and complexity. Second, we examine actual cortical connection matrices and analyze them both structurally using graph theory and functionally by implementing them as dynamical systems. Third, we investigate how interactions with an environment might drive the emergence of actual patterns of anatomical and functional connectivity that are found in the cerebral cortex.
To carry out this program, we have developed an iterative quasi-evolutionary procedure (referred to as graph selection) to search for networks that are of possible biological interest. This procedure, which is based on structural variation and selection, provides a fast and robust way of generating graphs that give rise to specific patterns of functional connectivity, e.g. graphs characterized by high entropy, high integration or high complexity. Selection for each of these different functional measures was found to be correlated with distinct classes of anatomical patterns. Networks obtained by selecting for high complexity were characterized by the formation of densely linked groups (or clusters) that are sparsely and reciprocally interconnected one to another. Graph-theoretical analysis revealed that graphs giving rise to high values of complexity contain a large number of short reciprocal paths or cycles, and that their connections can be arranged with very short overall wiring lengths. A graph-theoretical analysis of connection matrices of the macaque visual cortex and of the cat cortex yielded very similar structural characteristics. Furthermore, we found that when such cortical connection matrices were run as dynamical systems they gave rise to functional connectivity with high complexity values. Using functional cluster analysis we identified dynamic groupings of cortical areas corresponding to the dorsal and ventral streams of the primate visual system. Finally, in selecting for graphs that match specific input patterns or produce specific output patterns, we observed an increase in complexity that was sustained by the distinctive neuroanatomical motifs exhibited by the cerebral cortex.
![]() |
Theory and Computer Implementation |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Throughout this paper, we distinguish between structural and dynamic aspects of neuronal networks that are based, respectively, on their anatomical connectivity and their functional connectivity.
Structural aspects are captured using concepts from graph theory. For the purpose of structural analysis, we describe neuronal systems X comprising n units and k connections as order-n digraphs Gn,k with n vertices and k edges (excluding self-connections, k n2 n). The adjacency matrix Aij of a digraph corresponds to the system's connection matrix Cij, which describes its anatomical connectivity. Dynamic aspects are captured by implementing networks as dynamical systems. Assuming a multi-dimensional Gaussian stochastic process, global statistical measures of activity patterns can be derived from the system's covariance matrix COV, which describes the system's functional connectivity.
Graph Selection
In searching for relationships between patterns of functional connectivity and the underlying anatomy of networks, we will follow two different strategies. First, we investigate a large set of possible patterns of anatomical and functional connectivity, an approach we have adopted initially because it provides a relatively unbiased view of structurefunction relationships. Second, we consider specific examples of actual neuroanatomical structures and investigate the functional dynamics to which they give rise. In pursuing both strategies, we need to bridge the gap between structure and function, by implementing anatomical networks as dynamical systems. This is accomplished by endowing units and connections with dynamical properties (such as synaptic weights, or the ability to sum up inputs and produce outputs, either in a linear or nonlinear fashion), and by injecting inputs or uncorrelated noise. Neural activity patterns can then be analyzed in terms of the deviations from statistical independence produced by their underlying connection patterns.
First, we turn to the problem of evaluating large numbers of networks and their dynamics. We consider global measures of dynamics as functions over a space of networks (graphs) of n units and k connections whose arrangement is described by the connection matrix Cij. The vastness of this space of possible graphs (graph space), even for modest n and k (~10700 individual graphs, for n = 32, k = 256), precludes the use of exhaustive search techniques. To confront this problem, we employ a quasi-evolutionary strategy based on concurrent variation and selection of graphs. Variation is structural and is implemented as the continuous generation of varying anatomical patterns. Selection occurs according to a global measure of dynamics, which acts as a fitness function. To obtain dynamics, graphs are implemented as dynamical systems, i.e. as networks with defined properties of units and connections.
Figure 1 shows a schematic diagram of the computational procedure used. Graph selection takes place in h discrete generations, in which a global functional measure F (e.g. complexity; see below) is computed for each of u individual variant graphs. After the generation of an initial random set, all subsequent generations are created by copying the single graph with the highest selection score Fmax (parent) and deriving u 1 copies (offspring) for which r edges are rewired (mutated). This process is repeated for a total of hmax generations. The generation size u and the rewiring rate r define, respectively, the grain and the volume of the region of graph space around the parent that is explored in each generation. For example, choosing a small value for u and a large value for r will produce offspring that vary widely in comparison to their parent, sparsely occupying a large volume of graph space. Conversely, large values for u and small values for r result in offspring that closely resemble the parent and densely occupy a relatively small volume of graph space.
|
In graph selection, two structural constraints are applied. We placed a constraint on the in-degree (i.e. the number of incoming connections) of each vertex of the graph population. The in-degree was initialized to k/n for each vertex and this value was maintained throughout graph selection (the constant input constraint). In addition, in each generation a strong bias was placed on maintaining offspring graphs as strongly connected (the connectedness constraint). A graph is strongly connected if at least one path exists between all pairs {i, j} of vertices, such that i can be reached from j and j can be reached from i. For all values of n and k used in this study, random graphs were strongly connected with very high probability (k/n > ln(n)) (Bollobás, 1985). Some mutations, however, may result in the formation of additional graph components (i.e. they disconnect the original graph into individual components), and such graphs were deleted from the procedure. While the connectedness constraint is a necessary prerequisite for a valid analysis of structure and function of individual graphs, we found that the constant input constraint could be relaxed without altering the major conclusions of this study.
Graph-theoretical Measures
The ijth entry of a graph's adjacency matrix Aij is equal to 1 if a connection is present between vertex i and vertex j, and 0 otherwise. A pair of vertices can be connected by zero, one or two (reciprocal) connections. We designate the fraction of edges for which a directly reciprocal connection exists as frecip. To describe the possible ways to traverse a graph Gn,k we assemble its path matrix Pqij. Paths are any sequence of adjacent edges from a source vertex vj to a target vertex vi in which all vertices (and therefore all edges) are distinct. The ijth entry of Pqij corresponds to the total number of unique paths containing q edges that exist between vertices vj and vi. A graph with n vertices may contain noncyclic paths composed of up to n 1 edges. If vj = vi, paths are called cycles, with a maximal length of n edges. A strongly connected digraph has bidirectional paths between all pairs {i, j} of its vertices. The matrix of shortest paths Sij has as its elements the lengths of the shortest path linking vertices vj and vi. The global maximum of Sij is also called the diameter diamG of the graph; it is equivalent to the maximal number of steps needed to reach any vi from any vj. We define the global mean of Sij, also called the characteristic path length lpath (Watts and Strogatz, 1998). The cluster index fclust (Watts and Strogatz, 1998
) evaluates how many connections out of all possible ones exist between the immediate neighbors of a vertex, averaged over all vertices.
We propose another graph-theoretical measure that expresses the probability that paths of length q 1 can be completed as cycles of length q, terminating on the source vertex vj. This measure, which we call cycle probability pcyc(q), captures the relative abundance of cycles of length q within a given graph. Note that 0 pcyc(q)
1 and that pcyc(2) = frecip.
To visualize connection matrices as actual graphs plotted in two dimensions, we use a simple search algorithm that rearranges columns and rows of Cij (preserving graph-isomorphism) such that a cost function is minimized. The cost function is given by
![]() |
Dynamics and Global Functional Measures
In order to derive global statistical measures of functional connectivity, graphs are implemented as dynamical systems, i.e. neuronal networks with defined properties of units and connections. We characterize the dynamics of a system through measurements of deviations from statistical independence. We do not consider time delays or activity-dependent effects on connections, such as those that result from voltage dependency or synaptic modification.
Linear System Dynamics
Linear systems were employed in all graph selection computations conducted in the present study, as they allow analytic derivation of their covariance matrices (see below) and are thus computationally inexpensive. (For a discussion of nonlinear dynamics, see Discussion.) As in previous work (Tononi et al., 1994, 1996
), all connections between units i and j (i
j) are excitatory, with uniform and constant strengths wij (0 <wij << 1). In keeping with the constant input constraint (see above), for each unit the sum of the afferent synaptic weights is set to a constant value w (w = wij k/n = 0.8 in all runs shown in this paper). All systems were run by injecting uncorrelated noise into each of the units; the amount of noise was set such that most of the unit's variance was due to their mutual interactions. To assure that observed changes in global functional measures are due to increases in covariance and not due to changes in variance, a small self-inhibitory weight (0 < wii < wij) is added for each unit of the network before global functional measures are computed. A separate constraint optimization is applied such that these self-inhibitory weights limit the variance (as given by the diagonal terms of COV) of each unit to a fixed value
(
= 0.015 throughout this study).
Covariance Matrices
All global measures of dynamics (functional interactions) used in this study are derived from the covariance matrix COV of each system, under the assumption that the activity of its n units can be described as a Gaussian multidimensional stationary stochastic process (see Discussion). Under Gaussian assumptions, all deviations from statistical independence among the units can be described by COV. For linear systems, the matrix COV, reflecting the system's dynamics, can be obtained analytically from the connection matrix Cij. Considering the vector A of random variables that represents the activity of the units of a given system X subject to uncorrelated noise R, we obtain A = Cij * A + R, when the elements settle under stationary conditions. Substituting Q = [1 Cij]1 and averaging over the states produced by successive values of R, we obtain the covariance matrix
![]() |
Global Functional Measures
In this study, we use five global dynamic measures to characterize and also to select for a given system X: entropy H(X), integration I(X), complexity C(X), matching CM(X;S) between system X and input S, and degeneracy D(X;O) of system X producing output O. We note that all of these global measures can be applied to linear as well as nonlinear dynamical systems. Detailed derivations and discussions of these measures are given elsewhere (Tononi et al., 1994, 1996
, 1999
).
The entropy H(X) of a system measures its overall degree of statistical independence; it is the multivariate generalization of variance. Assuming stationarity, the entropy of a system X composed of n units is computed as (Papoulis, 1991)
![]() | (1) |
The integration I(X) measures the overall degree to which a system deviates from statistical independence. This measure is derived as the difference between the entropies of the individual components of X, considered independently, and the entropy H(X) of the entire system:
![]() | (2) |
Complexity expresses the extent to which a system X is both functionally segregated (small subsets of the system tend to behave independently) and functionally integrated (large subsets tend to behave coherently). Originally, neural complexity was defined as the sum of the average mutual information across all bipartitions of the system (Tononi et al., 1994). The mutual information MI between a subset Xk of size k of a system X and its complement X Xk, defined as MI(Xk;X Xk) = H(Xk) + H(X Xk) H(X), measures the portion of entropy shared by Xk and X Xk. We have recently introduced a related mathematical definition of complexity which does not require the full distribution of the average mutual information over all subset sizes (Tononi et al., 1998b
). It expresses the portion of the entropy of a system that is accounted for by the interactions among its elements. This measure is computed as
![]() | (3) |
An input pattern on a sensory sheet S connected to a neural system X produces changes in the system's functional connectivity. Some covariances between units of the system may be increased, while others may be decreased. If these changes match the intrinsic pattern of covariances produced by the system in the absence of input, the mutual information between many subsets of the system X will increase, resulting in an increase in its overall complexity (Tononi et al., 1996). We have expressed the matching, or fit, between a stimulus present on the sensory sheet S and a system X as the total complexity C(X)T when the input is present minus the intrinsic complexity C(X)I and the complexity C(X)S due to direct input from the sensory sheet.
![]() | (4) |
When an output pattern is produced by a system X connected to an output sheet O, many different subsets of the system may have effects on this output sheet. We refer to this ability to affect a set of output units in many different ways (i.e. through different subsets of the system) as the system's degeneracy (Tononi et al., 1999). Degeneracy can be expressed in terms of mutual information between the system's subsets and the output. In the same way as for complexity (equation 3), we have introduced a measure of degeneracy that does not require averaging among subsets of all sizes. This measure expresses the portion of the entropy of the output that is jointly accounted for by different elements of the system. It is computed as
![]() | (5) |
Functional Cluster Analysis
When analyzing a neural system X, an important first step in the analysis is to attempt to identify strongly interactive subsets of elements. Such subsets are characterized by a high degree of statistical dependence within the subset as well as a simultaneous low degree of statistical dependence with elements outside of the subset. Such functional clusters can be identified by measuring for many different subsets the cluster index CI(Xik) defined as (Tononi et al., 1998a)
![]() | (6) |
![]() |
![]() |
Results |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Selection of graphs for high entropy, integration and complexity yields fast convergence for moderate population sizes u and low rewiring rates r (u = 10, r = 1). Figure 2A shows graphs (n = 32, k = 256) and their corresponding COV matrices obtained at several stages (generations h = 1, 100, 500, 2000) during graph selection for high complexity. Plots on the extreme left show a graph drawn from an initial population of u = 10 random graphs; the complexity of its functional dynamics is low (C(X) = 1.92). Subsequent stages show graphs at generations 100 and 500, with gradually increasing complexity. Plots on the extreme right of Figure 2A
show a graph obtained after h = 2000 generations of graph selection; its complexity is significantly increased (C(X) = 2.76). Note that graph selection does not alter n or k, but merely redistributes connectivity while satisfying the additional constraints of constant input' and connectedness' (see above). All observed changes in complexity are due to changes in covariances, as the variances of individual units are fixed (
= 0.015 throughout). Random graphs tend to produce a COV with uniformly low values, while networks with high C(X) have many strong and many weak covariances. Figure 2B
plots the temporal evolution of C(X) over 10 individual graph selection runs for which C(X) was the selection criterion. On average, good asymptotic convergence of C(X) is achieved within 5001000 generations. Figure 2C
shows plots comparing the relative changes of H(X), I(X) and C(X) in the course of graph selection for high H(X), I(X) and C(X), respectively, each for a total of N = 10 individual simulations. In the plots, random graphs occupy a roughly central region from which selected graphs rapidly diverge, settling into distinct subregions. Graphs selected for C(X) and I(X) tend to dissociate. Graphs selected for high C(X) tend to decrease their corresponding I(X), while graphs selected for I(X) decrease their C(X). Graph selection for H(X) initially yields a concomitant increase in C(X). This is to be expected given the requirement to achieve maximal statistical independence under the connectedness' constraint. Note, however, that the complexity of graphs selected for H(X) always remains below the complexity of graphs directly selected for C(X).
|
Structural Analysis and Graph Theory
Each simulation of graph selection yielded graphs whose fine structure differed in detail, i.e. the graphs were structurally non-isomorphic (as revealed by comparing their out-degree distribution). Nonetheless, for any given global selection criterion (e.g. H(X), equation 1; I(X), equation 2; or C(X), equation 3) all solutions obtained from individual simulations shared common structural motifs. This consistency is in marked contrast to the fact that the different global measures yield graphs with radically different structural motifs. Figure 3 shows examples of a random graph and of graphs obtained after 2000 generations of graph selection for H(X), I(X) and C(X), respectively. While all the graphs maintain constant n, k and
, and are strongly connected, even casual visual inspection shows that their connection patterns differ considerably, as do their patterns of covariances. Random graphs (Fig. 3A
) exhibit no topological ordering, with relatively few reciprocal connections present. Their COV exhibits rather uniform values of covariance. Graphs with high H(X) (Fig. 3B
) contain mostly reciprocal connections linking pairs of units that appear spread out without any apparent local clustering. This pattern is consistent with dynamics characterized by a very high degree of statistical independence. Graphs with high I(X) (Fig. 3C
) consist of a central group of densely and reciprocally interconnected units, surrounded by a loose mesh of units that receive input from the central group and interact only weakly with each other. This pattern is consistent with maximal deviation from statistical independence of the entire system. Increasing k generally increases the size of the central group (not shown), without changing the overall pattern. Graphs with high C(X) (Fig. 3D
) contain a very high proportion of reciprocal connections, and appropriate reordering of their Cij indicates the presence of dense groups of vertices. These groups appear to be linked by a relatively small number of reciprocal bridges. This pattern is consistent with the emergence of a functionally segregated and functionally integrated system.
|
|
While the index fclust (Watts and Strogatz, 1998) gives an indication of shared local neighborhoods or cliques' in graphs, it fails to discriminate between graphs that consist of a single large group and those that are partitioned into multiple interconnected groups of vertices (see Table 1
). In contrast, the distribution of cycle probabilities pcyc(q) allows one to distinguish between these cases and provides an indication of the group size. Figure 4A
shows the distribution of the cycle probability for cycles of length q, pcyc(q), for each of the cases considered. Clearly, complex graphs are characterized not only by a high proportion of cycles of length q = 2 (equivalent to directly reciprocal connections, frecip) but also by significantly higher proportions of cycles of length q = 3 and q = 4. The eventual drop in cycle probabilities for longer cycles provides an indication of the average size of the individual groups. In random graphs, cycles of all lengths occur with approximately equal probability (which is mainly a function only of the overall connection density, i.e. pcyc(q)
k/(n2 n)). In graphs giving rise to high H(X), despite a very high probability for cycles of length q = 2, the probability of cycles of length q = 3 is close to zero. This is consistent with the requirement for the production of functional connectivity with high entropy (disorder) and reflects the apparent (Fig. 3B
) lack of groups of vertices. Graphs giving rise to high I(X) have pcyc(q) distributions very similar to random graphs.
|
A number of cerebral cortical connection matrices have become available for analysis. Of these, we chose the matrix of inter-connections between segregated areas of the macaque monkey visual cortex (Felleman and Van Essen, 1991) (n = 32, k = 315) and of interconnections between cat cortical areas (Scannell and Young, 1993
; Scannell et al., 1995
) (n = 65, k = 1136). We have examined these matrices in two ways. First, we analyzed them structurally (Table 2
), in particular for the abundance of cycles of varying length pcyc(q) (Fig. 4B,C
). Second, we have run them as dynamical systems to determine the degree of complexity of their functional connectivity (Fig. 5
) and the existence of functional clusters of highly interactive areas (Fig. 6
).
|
|
|
To facilitate the structural analysis we converted the original connection matrices of the macaque visual cortex (Felleman and Van Essen, 1991) and the cat cortex (Scannell et al., 1995
) to a binary format. In this format, elements of the resulting matrices signify the absence (0) or presence (1) of a pathway, without regard to its density or strength. In addition, following other authors (e.g. Young, 1992), the original macaque matrix was modified to eliminate areas PIT, CIT and STP; their connections were reassigned to areas PITd/PITv, CITd/CITv and STPp/STPa, respectively.
We compare structural data obtained from cortical matrices with structural data obtained from random graphs and from graphs selected for high C(X), both of equivalent size and degree of connectivity. Table 2 shows that graph-theoretical measures for the macaque visual cortex and the cat cortex produce values similar to those for complex graphs obtained by graph selection. In particular, cortical networks have small diameters and characteristic path lengths, but high frecip and fclust. Actual values for frecip may turn out to be somewhat higher as more anatomical information about reciprocal connections becomes available. Plotting the distribution of pcyc(q) for cortical matrices (Fig. 4B,C
) reveals values for pcyc(q) for cycles of lengths at least up to q = 5 that are significantly above those obtained for random graphs, but that fall within the same range as those obtained for graphs selected for high C(X). The slow decline in pcyc(q) indicates a relatively large group size, probably in excess of five cortical areas. These results confirm the finding (Young, 1992
; Hilgetag et al., 2000
) that cortical areas can be partitioned into distinct streams or clusters based on their anatomical connectivity.
Functional Analysis
We implemented the macaque visual cortex (Felleman and Van Essen, 1991) and the cat cortex (Scannell et al., 1995
) as dynamical systems, to determine whether the resulting functional connectivity has high or low complexity. In attempting to approximate the large-scale dynamics of cortical systems, we chose to implement them as linear systems, noting that not much is known at present about the extent of nonlinear functional interactions between areas of the macaque visual system. Cortical matrices (
= 0.01, wij = 0.04) were run by injecting uncorrelated noise and deriving the system's covariance matrix. The results are shown in Figure 5
. Both cortical matrices gave rise to functional connectivity (Fig. 5A,B
) with high complexity (Fig. 5C,D
). The values for complexity were significantly above those obtained from a number of random graphs of equivalent n and k. We produced structural variants of cortical connection matrices by randomly rewiring a fraction of their pathways. In virtually all cases, the complexity of the functional connectivity was decreased, suggesting that actual cortical connection matrices represent configurations of pathways that are near-optimal with respect to complexity. This observation was robust with respect to changes in dynamical parameters (i.e.
, wij) of cortical matrices.
The high complexity of cortical matrices is due to their anatomical organization into distinct groupings of areas, the presence of which was revealed by structural analysis. Next, we attempted to identify coherent sets of cortical areas by examining patterns of functional connectivity and performing functional cluster analysis (Tononi et al., 1998). Figure 6 shows the results of such an analysis performed on the macaque visual cortex. Figure 6A
lists values for the cluster index CI(Xik) (equation 6) for a given subset size k, ranked by their statistical significance. Figure 6B
displays the corresponding subsets i (corresponding to functional clusters). Reading Figure 6B
from top to bottom provides a cross-section of the hierarchical organization of functional clusters in the macaque visual cortex. The first areas to be excluded from a cluster of size k = 30 were areas MIP and MDP; this may be explained by the lack of anatomical information for these areas. Next, area VOT dissociates away from the functional cluster (k = 29). VOT is an area with uncertain attachment in anatomical clustering schemes (Hilgetag et al., 1999). Next, over a series of steps, a set of six areas (PITd, PITv, CITd, CITv, AITd, AITv), all components of the inferior temporal cortex, dissociated from the main functional cluster (k = 23). These areas are all considered components of the ventral stream of visual cortex (Hilgetag et al., 1998
). Then a second set of areas (STPp, STPa, TF, TH, 46 and 7a), also considered ventral, is found to dissociate (k = 17). These areas appear to form a subcomponent of the ventral stream that maintains stronger functional interactions with the dorsal stream (see also Fig. 7C
). Given the definition of cluster boundaries, elements of each cluster can be ordered according to the amount of their mutual information with the other members of the cluster (here, k = 17; Fig. 7A
). The original cortical connection and covariance matrices can then be rearranged (Fig. 7B,C
; compare Fig. 7C
with Fig. 5A
) according to this ordering scheme. This reveals that areas within the dorsal and the ventral streams, respectively, are highly interactive, while functional interactions between the streams are more limited. A subset of ventral areas (STPp, STPa, TF, TH, 46 and 7a) interact more strongly with dorsal areas, rendering them strong candidates for mediating cross-stream interactions.
|
Graph Selection for Matching and Degeneracy
The analysis and simulations presented so far suggest that graphs giving rise to complex functional dynamics share distinct structural motifs, characterized by local groups of vertices and economical wiring lengths. In the last part of this study we consider graph selection for networks that efficiently match an input stimulus or produce an output stimulus with high degeneracy. This step is motivated by the notion that selectional processes shaping real neuronal networks during development and evolution occur while inputs are received and outputs are produced, i.e. as networks interact with an external environment (Edelman, 1987).
To produce graphs with high matching M(X;S) (equation 4), graph selection is conducted as described previously, but while an external input pattern is presented. This pattern is described by a covariance matrix COVS impinging on a sensory sheet S that is connected to a subset of units of the system X. An example of a network that has been selected to match a particular input pattern is shown in Figure 8A. After reordering (Fig. 8A
, middle and right), it is evident that the internal connectivity of the network has arranged itself to form distinct groups of units. Arrows in Figure 8A
(right) indicate the positions of the units receiving input from S.
|
![]() |
In all cases, selection for matching or degeneracy resulted in graphs that gave rise to highly complex intrinsic functional dynamics (Fig. 9). These graphs show characteristic structural motifs similar to those of graphs selected for high complexity (Table 3
; compare with Table 1
). Thus, our simulations suggest that driving networks towards increased matching or increased degeneracy produces intrinsic dynamics of increasing complexity. This is accompanied by the emergence of neuroanatomical motifs that, as we have shown earlier, are capable of generating complex dynamics.
|
|
![]() |
Discussion |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
The relationship between structural and functional characteristics of a set of networks depends to some degree on the kind of dynamical system that is implemented. We chose a linear systems approach, in order to allow a computationally efficient analytic derivation of the system's covariance matrix. We have, however, tested other nonlinear dynamics, in particular implementations of spiking neurons linked by excitatory AMPA-ergic synaptic connections, following previous work (Lumer et al., 1997). Architectures characterized by interconnected groups of units produced high values for complexity, indicating that relationships similar to the ones sketched out for linear systems in this paper might also hold for at least one class of nonlinear systems (unpublished observations).
Consistent Structural Motifs and Functional Dynamics
A key question at the outset of this study was: are there consistent and coherent regions of graph space with distinctive structural motifs such that graphs within these regions produce dynamics characterized by high entropy, integration or complexity? Our simulations show that this is indeed the case (see Fig. 3). Two points are worth noting. First, for a given selection criterion, irrespective of the initial condition (always a sample of random graphs with given n and k), all simulations converge on specific sets of solutions. Second, for different selection criteria, these specific sets of solutions have very different structural motifs. After appropriate reordering of vertices, even a simple visual inspection of the resulting matrices revealed striking differences between networks that were selected to produce different kinds of dynamics (Fig. 3
).
The anatomical pattern associated with high complexity is particularly striking (Figs 2, 3 etc.). It consists of groups of densely interconnected vertices that are more sparsely interconnected among each other. Clearly, this anatomical pattern mirrors the dual requirements for functional segregation and integration, requirements that are met by the functional organization of the cerebral cortex, as reviewed elsewhere (Zeki, 1993
; Mountcastle, 1998
). It is important to note that over wide ranges of key parameters (n, k,
, r, u, wij) graph selection for high complexity never produced graphs that substantially deviated from this general pattern.
To quantify specific structural motifs, we used several graph-theoretical measures. We employed measures introduced by Watts and Strogatz (Watts and Strogatz, 1998), the characteristic path length lpath and cluster index fclust, as well as associated measures, such as the diameter diamG and the fraction of reciprocal connections, frecip. Networks that give rise to complex dynamics are associated with values for lpath and fclust that are typical of so-called small-world networks. We noted, however, that fclust does not distinguish between graphs containing a single large group of vertices and others containing multiple smaller ones that are interconnected; also, it gives no indication of the group size. To overcome these limitations, we introduced another measure, the cycle probability pcyc(q), which expresses the likelihood that any path of length q 1 can be completed as a cycle of length q. Unlike fclust, which evaluates local connectivity only within a vertex's immediate neighborhood, pcyc(q) expresses the probability of finding cycles over all lengths q = {1, . . ., n}. In particular, pcyc(q) can distinguish between networks that are composed only of one central group (see Fig. 3
, Integration) and others composed of many relatively dense, but globally interconnected groups (see Fig. 3
, Complexity). While fclust is high in both cases (Table 1
), the distribution of pcyc(q) shows very significant differences (Fig. 4A
). Furthermore, the value of q at which pcyc(q) falls below corresponding values for random networks (see Fig. 4A
) corresponds to the average group size within the network.
Cycle probability is a measure of the relative proportion of reentrant loops (Edelman, 1987, 1989
) of length q within the network. Short reentrant loops (e.g. q = 2, 3, 4), which couple sets of units tightly into groups, are clearly prevalent in networks that give rise to complex dynamics (Tononi et al., 1994
). In addition, such networks maintain long reentrant loops that link individual groups and ensure global coherency. This distribution of reentrant loops is consistent with their role in integrating neuronal activity within and between distributed and segregated brain areas (Sporns et al., 1991
; Tononi et al., 1992
). All other classes of networks we examined contain a much smaller proportion of reentrant loops (Fig. 4A
).
Analysis of Cortical Connection Matrices
Our chief goal was to determine how the structural motifs found in the large-scale anatomy of the cerebral cortex relate to global measures of its functional connectivity. For this purpose, we applied our combined structural and functional analysis to two examples of published cortical connection matrices (Felleman and Van Essen, 1991; Scannell et al., 1995
). We found that the structural organization of the cortex into groups of densely linked areas produces functional connectivity of high complexity and gives rise to distinct functional clusters of highly interactive cortical areas.
We employed graph-theoretical measures in a structural analysis of the macaque visual cortex and the cat cortex. Both cortical matrices showed structural characteristics very similar to those of networks selected for high complexity (Table 2). Their distribution of pcyc(q) (Fig. 4B,C
) revealed a strong prevalence of short cycles, indicating the presence of anatomical groups of areas. These results are consistent with the topological analyses carried out previously (Young, 1992
, 1993
; Scannell et al., 1995
; Hilgetag et al., 1998
, 2000
). Note that the lack of information about the existence of many reciprocal pathways produces a tendency to underestimate their abundance (Jouve et al., 1998
); if more such pathways actually existed, the values for structural and functional parameters are likely to be even more closely related to those of complex networks.
In addition to structural analyses and comparisons, we investigated the functional dynamics of cortical connection matrices. In general, evaluating covariance (functional connectivity) patterns is not simply equivalent to examining anatomical connectivity in the absence of dynamics. Although values for covariance within a linear system reflect the underlying anatomical connectivity, it is important to note that covariance values capture the total effect of all connections within a system on deviations from statistical independence between two units, not just the impact of any direct connection linking them (Vanduffel et al., 1997). Furthermore, the differential contributions of a connection to a covariance depend on the underlying connection pattern, not just the number of connections present. For example, connections organized into cycles (re-entrant loops) contribute more towards covariance than connections that are unidirectional.
Dynamics was obtained by implementing cortical matrices as linear systems and deriving their covariance matrix, thus allowing direct comparison with graphs obtained by graph selection. While a linear system may be only a rough approximation of the nonlinear interactions between individual spiking neurons, neuroimaging studies indicate that it may capture more accurately the large-scale dynamics of interacting brain areas (McIntosh and Gonzalez-Lima, 1994; McIntosh et al., 1994
). In addition, there is at present very little information on specific nonlinear interactions between areas of the macaque visual system.
We analyzed the functional connectivity of the macaque visual cortex, in order to identify the cortical areas that participate in the formation of functional clusters (Fig. 6). Figure 6B
provides a summary description of the cluster structure. Note that functional clustering does not produce a singular best configuration; rather, Figure 6B
provides a hierarchical cross-section that captures nested subclusters as well as more clearly defined subdivisions. Our analysis revealed the presence of two dominant clusters of areas, corresponding to the dorsal and ventral streams of the visual system. Several areas were identified as unrelated to either of these streams, in particular areas MIP and MDP (presumably due to lack of connectional information) and area VOT, which in our analysis tended to associate with the ventral stream. Within the ventral stream, several areas could be identified that are initially clustered together with dorsal areas. These were areas STPp, STPa, TF, TH, 46 and 7a, most of which maintain relatively strong covariance with the dorsal stream (Fig. 7C
). This fact renders them likely candidates for cross-stream interactions between the dorsal and ventral streams of the macaque visual cortex.
In addition to identifying functional clusters, we also found that the functional connectivity produced by cortical connection matrices has high complexity (Fig. 5) compared with that of equivalent random networks. In fact, when actual cortical matrices are rewired (by randomizing a small proportion of their existing connections), the vast majority of resulting variants is less complex (Fig. 5B,C
). This provides an indication that cortical connection matrices represent connection patterns that are near optimal with respect to the overall complexity of their functional connectivity.
Functional and dynamic analyses of cortical connection matrices could be extended in a number of ways. The implementation of dynamics should be based on a more realistic large-scale cortical model (Tononi et al., 1992; Tagamets and Horwitz, 1998
), including nonlinear or modulatory interactions where observed (Salin and Bullier, 1995
; Büchel and Friston, 1997
). Connectional information should take into account available anatomical data on the density, strength or efficacy of cortical pathways. Future analyses of functional connectivity could feasibly be carried out in the context of specific tasks, e.g. during the presentation of specific inputs or during production of specific outputs. Finally, functional cluster analysis can be usefully applied to experimentally observed patterns of functional connectivity (McIntosh et al., 1999
), obtained from neurophysiological or neuroimaging studies (Tononi et al., 1998).
The Economy of Wiring in Networks
We find that the graphs that result from selection for highly complex dynamics can be placed in physical space in such a fashion that the cost of wiring is very low. We do not take graph selection as a realistic approximation of neural evolution or development; in fact, as Young and Scannell have pointed out (Young and Scannell, 1996), in real brains the positioning of vertices precedes the formation of edges between them. However, we think it is significant that complex dynamics is associated with potentially low wiring costs. Other authors have argued that, in the course of evolution, connectivity has been subjected to considerable pressure to reduce its overall wiring length (Mitchison, 1991
), to maximize connectivity while minimizing volume (Murre and Sturdy, 1995
), or to place brain components in order to minimize wiring length (Cherniak, 1995
). It is unlikely, however, that evolutionary pressure on wiring alone can be responsible for the specific patterns of connectivity we actually observe today. In our view, anatomical structures have evolved to accommodate particular kinds of functional and dynamic interactions that support adaptive behavior in a complex environment. This functional criterion, in combination with elementary developmental rules (e.g. the tendency of neurons to connect to other neurons in their immediate vicinity) and indisputable volume constraints on wiring, may provide a more satisfactory account to explain the observed patterns of anatomical connectivity. The link between wiring and functional connectivity could profitably be investigated further by refining graph selection, for example by embedding graphs in two- or three-dimensional space, by including temporal features such as conduction delays (Ringo et al., 1994
) and by incorporating explicit developmental rules in the wiring process.
Biological Networks and the Evolution of Complexity
The description of networks as binary adjacency matrices in itself does not make any assumptions about local characteristics of vertices or transmission properties of edges. Vertices and edges may stand for neurons and synaptic connections, molecular species and their chemical/biochemical interactions (e.g. genetic regulatory circuits or metabolic pathways) (Kauffman, 1985, 1993
) or organisms in an ecosystem (e.g. food webs) (Cohen and Newman, 1985
). In fact, the descriptions and tools offered by graph theory are sufficiently general to be applicable to a wide range of biological networks, from neurobiology to ecology, evolution and genetics. Even though the rules and equations describing system dynamics are likely to differ widely, the principles of structural and functional organization may turn out to be similar. Our focus on complexity as a biologically relevant mode of organization is justified by the ubiquitous coexistence of local and global interactions in biological systems.
Much speculation has surrounded the issue of whether organismic evolution may be characterized by an increase in overall complexity, and if so, what the origin of this complexity might be. The ideas range from the affirmative notion that complexity should increase in evolution, e.g. due to selectional advantages of mechanically more efficient organisms (Bonner, 1988) or to the cumulative effect of component additions (Saunders and Ho, 1976
), to the skeptical view that evidence for complexity as a pervasive directional trend in evolution is lacking (McShea, 1996
). The present paper touches upon these issues indirectly through a consideration of functional connectivity of the brain as a prime contributor to the evolutionary and adaptive success of an organism. In previous work, we have defined the complexity of functional neuronal interactions as a reflection of the capacity of a neural system to rapidly integrate distributed information. We have noted that driving selectional networks to match complex input patterns efficiently and to produce complex output patterns with high degeneracy also produces high intrinsic complexity (Tononi et al., 1996
, 1999
). This is due to the formation of neuroanatomical patterns that we associate with complex dynamics. As ecological demands increase, as disparate sources of environmental information need to be integrated, and as rich and varied output patterns are required for adaptive behavior, there is likely to be selection for neural repertoires capable of matching of signals as well as for degenerate pathways that tend to increase robustness against failure. We suggest that the complexity of an organism's neural circuits will increase as a consequence.
![]() |
Notes |
---|
Address correspondence to Dr Olaf Sporns, The Neurosciences Institute, 10640 John Jay Hopkins Drive, San Diego, CA 92121, USA. Email: sporns{at}nsi.edu; URL: www.nsi.edu/users/sporns.
![]() |
References |
---|
![]() ![]() ![]() ![]() ![]() ![]() ![]() |
---|
Bollobás B (1985) Random graphs. London: Academic Press.
Bonner JT (1988) The evolution of complexity. Princeton, NJ: Princeton University Press.
Boven KH, Aertsen A (1990) Dynamics of activity in neuronal networks give rise to fast modulation of functional connectivity. In: Parallel processing in neural systems and computers (Eckmiller R, Hartmann G, Hauske G, eds), pp 5356. Amsterdam: Elsevier.
Büchel C, Friston KJ (1997) Modulation of connectivity in visual pathways by attention: cortical interactions evaluated with structural equation modeling and fMRI. Cereb Cortex 7:768778.[Abstract]
Changeux J-P, Courrege P, Danchin A (1973) A theory of the epigenesis of neuronal networks by selective stabilization of synapses. Proc Natl Acad Sci USA 70:29742978.[Abstract]
Cherniak C (1995) Neural component placement. Trends Neurosci 18:522527.[ISI][Medline]
Cohen JE (1988) Threshold phenomena in random structures. Discrete Appl Math 19:113128.[ISI]
Cohen JE, Newman CM (1985) A stochastic theory of community food webs. I. Models and aggregate data. Proc R Soc Lond B 224:421448.[ISI]
Edelman GM (1987) Neural Darwinism. New York: Basic Books.
Edelman GM (1989) The remembered present. New York: Basic Books.
Erdös P, Rényi A (1960) On the evolution of random graphs. Publ Math Inst Hung Acad Sci 5:1761.
Felleman DJ, Van Essen DC (1991) Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1:147.[Abstract]
Friston KJ (1994) Functional and effective connectivity in neuroimaging: a synthesis. Hum Brain Map 2:5678.
Gerstein GL, Perkel DH, Subramanian KN (1978) Identification of functionally related neural assemblies. Brain Res 140:4362.[ISI][Medline]
Goodhill GJ, Simmen MW, Willshaw DJ (1995) An evaluation of the use of multidimensional scaling for understanding brain connectivity. Phil Trans R Soc Lond B 348:265280.[ISI][Medline]
Harary F, Norman RZ, Cartwright D (1975) Structural models: an introduction to the theory of directed graphs. New York: Wiley.
Hilgetag CC, Burns GAPC, O'Neill MA, Young MP (1998) Cluster structure of cortical systems in mammalian brains. In: Computational neuroscience (Bower JM, ed), pp. 4146. New York: Plenum Press.
Hilgetag CC, Burns GAPC, O'Neill MA, Scannell JW, Young MP (2000) Anatomical connectivity defines the organisation of clusters of cortical areas in macaque monkey and cat. Phil Trans R Soc Lond B (in press).
Holland JH (1992) Adaptation in natural and artificial systems. Cambridge, MA: MIT Press.
Jouve B, Rosenstiehl P, Imbert M (1998) A mathematical approach to the connectivity between the cortical visual areas of the macaque monkey. Cereb Cortex 8:2839.[Abstract]
Kauffman SA (1985) New questions in genetics and evolution. Cladistics 1:247265.
Kauffman SA (1993) The origins of order. Oxford: Oxford University Press.
Lumer ED, Edelman GM, Tononi G (1997) Neural dynamics in a model of the thalamocortical system. I. Layers, loops and the emergence of fast synchronous rhythms. Cereb Cortex 7:207227.[Abstract]
McIntosh AR, Gonzalez-Lima F (1994) Structural equation modeling and its application to network analysis in functional brain imaging. Hum Brain Map 2:222.
McIntosh AR, Grady CL, Ungerleider LG, Haxby JV, Rapoport SI, Horwitz B (1994) Network analysis of cortical visual pathways mapped with PET. J Neurosci 14:655666.[Abstract]
McIntosh AR, Rajah MN, Lobaugh NJ (1999) Interactions of prefrontal cortex in relation to awareness in sensory learning. Science 284: 15311533.
McShea DW (1996) Metazoan complexity and evolution: is there a trend? Evolution 50:477492.[ISI]
Mitchison G (1991) Neuronal branching patterns and the economy of cortical wiring. Proc R Soc Lond B 245:151158.[ISI][Medline]
Mountcastle VB (1998) Perceptual neuroscience: the cerebral cortex. Cambridge, MA: Harvard University Press.
Murre JMJ, Sturdy DPF (1995) The connectivity of the brain: multi-level quantitative analysis. Biol Cybern 73:529545.[ISI][Medline]
Nicolelis MAL, Tinone G, Sameshima K, Timo-Iaria C, Hong YC, Van de Bilt MT (1990) Connection, a microcomputer program for storing and analyzing structural properties of neural circuits. Comp Biomed Res 23:6481.[ISI][Medline]
Palmer EM (1985) Graphical evolution. New York: Wiley.
Papoulis A (1991) Probability, random variables, stochastic processes. New York: McGraw-Hill.
Ringo JL (1991) Neuronal interconnection as a function of brain size. Brain Behav Evol 38:16.[ISI][Medline]
Ringo JL, Doty RW, Demeter S, Simard PY (1994) Time is of the essence: a conjecture that hemispheric specialization arises from interhemispheric conduction delay. Cereb Cortex 4:331343.[Abstract]
Salin PA, Bullier J (1995) Corticocortical connections in the visual system: structure and function. Physiol Rev 75:107154.
Saunders PT, Ho MW (1976) On the increase in complexity in evolution. J Theor Biol 63:375384.[ISI][Medline]
Scannell JW, Young MP (1993) The connectional organization of neural systems in the cat cerebral cortex. Curr Biol 3:191200.[ISI]
Scannell JW, Blakemore C, Young MP (1995) Analysis of connectivity in the cat cerebral cortex. J Neurosci 15:14631483.[Abstract]
Scannell JW, Burns GAPC, Hilgetag CC, O'Neil MA, Young MP (1999) The connectional organization of the corticothalamic system of the cat. Cereb Cortex 9:277299.
Sporns O, Tononi G, Edelman GM (1991) Modeling perceptual grouping and figure-ground segregation by means of active reentrant circuits. Proc Natl Acad Sci USA 88:129133.[Abstract]
Tagamets M-A, Horwitz B (1998) Integrating electrophysiological and anatomical data to create a large-scale model that simulates a delayed match-to-sample human brain imaging study. Cereb Cortex 8: 310320.[Abstract]
Tononi G, Sporns O, Edelman GM (1992) Reentry and the problem of integrating multiple cortical areas: simulation of dynamic integration in the visual system. Cereb Cortex 2:310335.[Abstract]
Tononi G, Sporns O, Edelman GM (1994) A measure for brain complexity: relating functional segregation and integration in the nervous system. Proc Natl Acad Sci USA 91:50335037.[Abstract]
Tononi G, Sporns O, Edelman, GM (1996) A complexity measure for selective matching of signals by the brain. Proc Natl Acad Sci USA 93:34223427.
Tononi G, McIntosh AR, Russell DP, Edelman GM (1998a) Functional clustering: identifying strongly interactive brain regions in neuroimaging data. Neuroimage 7:133149.[ISI][Medline]
Tononi G, Edelman GM, Sporns O (1998b) Complexity and coherency: integrating information in the brain. Trends Cogn Sci 2:474484.[ISI]
Tononi G, Sporns O, Edelman GM (1999) Measures of degeneracy and redundancy in biological networks. Proc Natl Acad Sci USA 96: 32573262.
Vanduffel W, Payne BR, Lomber SG, Orban GA (1997) Functional impact of cerebral connections. Proc Natl Acad Sci USA 94:76177620.
Watts DJ, Strogatz SH (1998) Collective dynamics of small-world networks. Nature 393:440442.[ISI][Medline]
Young MP (1992) Objective analysis of the topological organization of the primate cortical visual system. Nature 358:152155.[ISI][Medline]
Young MP (1993) The organization of neural systems in the primate cerebral cortex. Proc R Soc Lond B 252:1318.[ISI][Medline]
Young MP, Scannell JW, O'Neill MA, Hilgetag CC, Burns G, Blakemore C (1995) Non-metric multidimensional scaling in the analysis of neuroanatomical connection data and the organization of the primate cortical visual system. Phil Trans R Soc Lond B 348:281308.[ISI][Medline]
Young MP, Scannell JW (1996) Component-placement optimization in the brain. Trends Neurosci 19:413414.[ISI][Medline]
Zeki S (1993) A vision of the brain. Oxford: Blackwell.