SPECIAL COMMUNICATION
Voxx: a PC-based, near real-time volume rendering system for biological microscopy

Jeffrey L. Clendenon1, Carrie L. Phillips1, Ruben M. Sandoval1, Shiaofen Fang2, and Kenneth W. Dunn1

1 Department of Medicine, Division of Nephrology and Indiana Center for Biological Microscopy, Indiana University School of Medicine, Indianapolis 46202-5116; and 2 Department of Computer and Information Science, Indiana University-Purdue University at Indianapolis, Indianapolis, Indiana 46202-5132


    ABSTRACT
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Confocal and two-photon fluorescence microscopy have advanced the exploration of complex, three-dimensional biological structures at submicron resolution. We have developed a voxel-based three-dimensional (3-D) imaging program (Voxx) capable of near real-time rendering that runs on inexpensive personal computers. This low-cost interactive 3-D imaging system provides a powerful tool for analyzing complex structures in cells and tissues and encourages a more thorough exploration of complex biological image data.

confocal; three-dimensional; two-photon; voxels


    INTRODUCTION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

ADVANCES IN BIOCHEMISTRY and molecular biology have provided unprecedented insights into molecular interactions, but they also have increased our appreciation of the architectural organization of cells. With the development of specific probes, optical microscopy has allowed biochemical experiments to be conducted within individual cells, essentially using the cell as a test tube. The development of methods to express fluorescent chimeras of endogenous proteins has permitted the distribution and dynamics of specific proteins to be analyzed in living cells and in intact animals. Optical microscopy has become increasingly essential to an integrated approach to modern biomedical research.

Most microscopic studies produce two-dimensional (2-D) images that provide limited information about the three-dimensional (3-D) organization of cells and tissues. Although the development of confocal and multiphoton microscopy has made it possible to collect high-resolution 3-D images, true 3-D microscopy is still in its infancy. This is primarily due to the difficulty of analyzing 3-D images. In the past, 3-D visualization and image analysis systems were expensive, so they have not been widely available to biomedical researchers (6). Consequently, researchers have collected large quantities of images that sometimes have been incompletely explored and analyzed.

Fortunately, computer graphics hardware has developed to the point where high-speed 3-D graphics capabilities are common even on inexpensive personal computers (PCs). We have developed volume visualization software (Voxx) that takes advantage of such low-cost 3-D graphics hardware. Voxx promotes the exploration of complex microscopy data by providing interactive inspection and manipulation of 3-D images on PCs, tasks that until recently required much more expensive workstations or special-purpose voxel processors (2). As explained later in this article, we are making Voxx freely available to interested researchers.


    METHODS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Image acquisition. Image volumes may be generated from vertical series of 2-D images collected with the use of a microscope system that provides optical sectioning, such as a confocal or multiphoton microscope. Examples of images from such microscopy systems are shown in RESULTS. In each case, samples were mounted in aqueous medium, and images were collected with the use of a Bio-Rad MRC1024 confocal/two-photon system (Hercules, CA) fitted to a Nikon Eclipse inverted microscope (Melville, NY) with a ×60 water-immersion, NA 1.2 objective. Illumination for the multiphoton fluorescence excitation was provided by a Spectra-Physics (Mountain View, CA) Tsunami Lite Titanium-Sapphire laser. By matching the refractive index and the immersion medium with that of the mounting medium, this system avoids spherical aberration and has enabled us to collect high-contrast image volumes up to 200 µm in depth.

The 3-D visualization process should be considered during image acquisition. In addition to acquiring images with the required contrast and resolution, one should attempt to collect image volumes with a vertical spacing chosen so that consecutive optical sections overlap, but without oversampling to the degree that image volumes become excessively large. As shown below, 3-D rendering speed decreases significantly as the size of an image volume increases. In the examples given in RESULTS, image volumes were collected with optical sections spaced 0.4-0.5 µm apart.

Image processing. 3-D rendering was performed using Voxx, a voxel-based 3-D rendering program that we have developed. This program renders sets of images in back-to-front order, combining them by using alpha blending (a technique in which images in foreground layers are combined with images in background layers, and the transparency of the volume is manipulated by varying image opacity coefficients) or maximum intensity projection and user-defined transfer functions (1). Voxx supports any combination of color-indexed (8-bit grayscale or pseudocolor) and true-color (32-bit RGBA) image stacks. A graph-based editor can be used to make real-time color and opacity table modifications, permitting one to employ pseudocolor or contrast enhancement and gamma correction by using nonlinear intensity mappings (Fig. 1). Currently, only the color-indexed format permits real-time modification of color and opacity values. User-defined color and opacity tables and images may be saved and loaded using several file formats. Voxx can currently import Bio-Rad PIC, Zeiss LSM, and raw voxel files. In the future we plan to add support for additional file formats and also to provide 3-D filtering and other graphical tools.


View larger version (72K):
[in this window]
[in a new window]
 
Fig. 1.   Screen shot of Voxx showing a rendering of peanut agglutinin-labeled proximal tubules, along with dialog boxes (left and center) used when making movies and the graph-based color and opacity editor window (right).

Voxx is written in C++ and uses OpenGL, GLUT, and GLUI to make the bulk of the code independent of the host window system. The program currently runs on Microsoft Windows (Redmond, WA), and versions of Voxx for the Apple Mac OS (Cupertino, CA) and Linux ports are also being developed. Enhanced rendering capabilities are provided via OpenGL extensions when video boards equipped with suitable graphics processors are used, such as the NVIDIA GeForce family of processors (NVIDIA, Santa Clara, CA). 2-D images were prepared using Adobe Photoshop (Adobe, Mountain View, CA).

Biological samples. Images were collected from cultured Madin-Darby canine kidney (MDCK) or pig kidney epithelial (LLC-PK-1) cells, whole microdissected embryonic (day 17) mouse kidneys, or vibratome sections of mouse heart (postnatal day P1) or rat (adult) and mouse (day P5) kidneys, cut between 60 and 150 µm in thickness. Handling, care, and euthanasia of mice and rats conformed to institutional animal care guidelines. Rodent tissues fixed in 4% paraformaldehyde/1× PBS were washed and incubated with fluorescently labeled lectins (Vector Labs, Burlingame, CA) as previously described (3). Nuclei have been labeled with either 4',6-diamidino-2-phenylindole (DAPI) or TO-PRO-3 dye (Molecular Probes, Eugene, OR).


    RESULTS
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Examples of images rendered using Voxx. Please refer to the Supplementary Material1 for this article (published online at the American Journal of Physiology-Cell Physiology web site) to view the movies (Movies 2-4) generated using Voxx. An optical section of a newborn mouse kidney, labeled with fluorescent peanut agglutinin and then imaged by two-photon microscopy, is shown in Fig. 2A. Although this image clearly contains proximal tubules, it is not until the entire set of 215 optical sections is reconstructed that the interconnected nature of the renal tubular network is apparent. Movie 2A shows the addition of each focal plane to the volume in a slowed down version of the back-to-front rendering process. The entire rendered volume is shown in Fig. 2B, but the 3-D structure of this volume is more obvious in Movie 2B. Movie 2B, generated by using a movie capture function in Voxx, is intended to simulate the experience of interactively using the program, a process in which the user "grabs" the image volume by using the mouse-controlled cursor and then changes the orientation and position of the volume in near real time. The movies presented in the Supplementary Material have been configured to play at approximately the same frame rates used in interactive sessions. While the movies themselves are useful for presentation, we emphasize that the real power of this software lies in the interactive exploration of the volume provided by fast rendering. Voxx allows users to explore image volumes just as they would inspect a 3-D object in their hands.


View larger version (30K):
[in this window]
[in a new window]
 
Fig. 2.   Proximal tubules in the superficial cortex of newborn mouse labeled with peanut agglutinin-rhodamine and imaged by two-photon microscopy. A: a single optical section from the collected image volume. B: entire volume as rendered using Voxx. An animation showing the sequential addition of 215 image planes is shown in Movie 2A (all movies can be viewed online as Supplementary Material to this article). Movie 2B shows the manipulation of the entire image volume. Both movies are intended to simulate the experience of interactive manipulation of the image volumes using Voxx. These volumes were rendered at ~14 frames/s. In each case the field is 100 µm across.

A rendering of a larger image volume (347 optical sections, each 256 × 256 pixels) of peanut agglutinin-rhodamine-labeled embryonic kidney is shown in Fig. 3A. The complex 3-D organization of the branching ureteric bud is more obvious in Movie 3A. Movie 3A also shows that the two-photon microscope provides excellent resolution and contrast even at depths of 140 µm in this kidney section.


View larger version (81K):
[in this window]
[in a new window]
 
Fig. 3.   More examples of microscopy volumes rendered using Voxx. A: a rendered image volume of branching ureteric bud in a developing embryonic kidney, labeled with peanut agglutinin-rhodamine and imaged using two-photon microscopy. B: a rendered volume of polarized Madin-Darby canine kidney cells whose microtubules have been labeled using anti-tubulin immunofluorescence and imaged by confocal microscopy. C: two-photon, triple labeling of rat kidney. Sections were labeled with the nuclear stain DAPI (cyan), Texas red phalloidin (red) to localize filamentous actin, and a fluorescein-conjugated lectin (green) from Dolichos biflorus to label the collecting duct. D: a confocal image volume of a dividing LLC-PK-1 cell, whose microtubules and chromosomes have been labeled using anti-tubulin immunofluorescence and a DNA-binding dye (TO-PRO-3), respectively. Movies 3A through 3D show these volumes being manipulated. The field shown in A is 100 µm across, that in B is 40 µm across, that in C is 170 µm across, and that in D is 45 µm across.

Voxx is also useful for qualitative assessment of complex intracellular structures. Figure 3B shows a monolayer of MDCK cells whose tubulin has been labeled by immunofluorescence. In this case, a confocal image volume was first filtered with the use of unsharp masking (an image-sharpening technique) to highlight the fine structure of microtubules before rendering. Movie 3B shows the orientation of microtubules and prominently displays the primary cilium of each cell. In this volume the optical sections were collected with vertical spacing larger than horizontal pixel spacing. This vertical spacing results in images that appear flattened upon reconstruction. For example, if the magnification is such that each pixel is 0.1 µm wide and the sequential optical sections were collected every 0.3 µm, the vertical dimension must be stretched by a factor of three for the proportions to be represented appropriately in the reconstruction. Voxx provides the capability to scale each axis independently, which can be used to correct for this distortion. In the case of Fig. 3B, the vertical dimension was stretched twofold relative to the horizontal dimensions to compensate for the fact that optical sections were collected with a spacing twice that of the horizontal pixel spacing.

Fluorescence microscopy is somewhat unique in that multiple structures or proteins are frequently labeled with markers that fluoresce in different colors. These different proteins are then visualized in the same volume by each being displayed as a "channel" with a unique color. Voxx supports the simultaneous display of multiple channels and provides for the independent manipulation of the color and opacity of each channel. Figure 3C and Movie 3C show triple labeling of tubules in the rat renal cortex, in which vibratome sections were labeled with the nuclear stain DAPI, Texas red phalloidin to localize filamentous actin, and fluorescein-conjugated Dolichos biflorus lectin to label collecting ducts. The top portion of the volume shows a partial section through a Bowman's capsule with the glomerulus missing. In the collecting duct (center and bottom), a checkerboard staining pattern is produced by the localization of principal cells by the lectin (green). Throughout the tubule, the nuclei from the intercalated cells are visible, whereas those from the principal cells are masked by lectin staining. In the lower left corner of the volume, an arteriole staining intensely with the phallotoxin is visible. The actin-rich apical region of proximal tubules (interspersed throughout the volume) also stains intensely with phalloidin.

An example of multicolor labeling of intracellular structures is shown in Fig. 3D and Movie 3D, which show mitotic LLC-PK-1 cells, imaged by confocal microscopy, whose microtubules have been labeled by immunofluorescence and whose DNA has been labeled with TO-PRO-3.

Different portions of an image volume may be selectively highlighted by manipulating opacity in Voxx. Lightly labeled structures may be rendered nearly transparent by using low opacity values to emphasize brightly labeled structures. Alternatively, increasing the opacity of low-intensity pixels emphasizes dim structures and yields a volume that appears more like a solid object. A single optical section of a surface blood vessel of a newborn heart labeled with fluorescent wheat germ agglutinin and imaged by two-photon microscopy is shown in Fig. 4A. When the dim structures are rendered with low opacity, the image volume is dominated by bright fibrillar labeling of the subepicardial connective tissue (Fig. 4B and Movie 4B). When dim structures are rendered with higher opacity, the volume is dominated by a more diffuse labeling of cell surfaces, resulting in a more solid appearance to the image volume and the emphasis of the epicardium (Fig. 4C and Movie 4C).


View larger version (34K):
[in this window]
[in a new window]
 
Fig. 4.   Newborn mouse heart tissue, labeled with fluorescent wheat germ agglutinin and imaged by two-photon microscopy. A: a single focal plane. B: the image volume rendered with a low opacity for low-intensity voxels. C: the same image volume rendered with high opacity for low-intensity voxels. Movies 4B and 4C show the manipulation of these volumes. Each field is 100 µm across.

Recommended system configurations for Voxx. Voxx is freely available to researchers for noncommercial use and can be downloaded (http://nephrology.iupui.edu/imaging/voxx/download). The program currently runs on Microsoft Windows, with development of Mac OS and Linux ports underway. Video boards equipped with NVIDIA GeForce2 GTS graphics processors seem to offer the best cost-performance ratio at this time, although we also use boards equipped with NVIDIA GeForce2 Ultra, Quadro2 Pro, and GeForce3 processors. Future versions of Voxx will include support for some features that are available only on GeForce3-based boards. To maximize the rendering speed, one should use AGP 4x (not PCI) versions of video boards, and the boards should be equipped with at least 64 MB of DDR memory.

The memory requirements of the program depend on the number and size of the image stacks and the number of color channels per stack. Most microscopists seldom collect more than sixty-four 512 × 512 images. We recommend that PCs running Windows 2000 be equipped with at least 256 MB of memory when rendering such a single-channel volume and 384 MB for a three-channel volume. However, image volumes collected using two-photon microscopy may be significantly larger. On occasion, our two-photon volumes contain 256 image planes with up to three channels, requiring up to 768 MB of memory. Voxx renders volumes this large at around 1 frame/s. As shown below (see Fig. 5), typical microscope image volumes are rendered at between 10 and 30 frames/s.


View larger version (27K):
[in this window]
[in a new window]
 
Fig. 5.   Rendering speed as a function of the size of the image volume and the configuration of the host computer. The rendering speed of the image volume used for Fig. 4 was recorded for the original 512 × 512 section size and for a 256 × 256 subsection. In each case, the rendering speed was recorded after a different number of optical sections was loaded. This test was conducted by running Voxx on two personal computers equipped with NVIDIA GeForce2 GTS and Quadro2 Pro graphics processors, while a third system equipped with a VolumePro 500 voxel processor and 3Dlabs Oxygen GVX420 video board was running Revli.

Voxx performance. We compared the volume rendering performance of three PC-based systems. One was a Dell (Round Rock, TX) Precision 330 equipped with a 1.4 GHz Pentium 4 processor, 1.5 GB of RDRAM memory, and an NVIDIA Quadro2-based video board. The second was a more modest Dell Dimension 4100 equipped with an 800 MHz Pentium III, 512 MB of SDRAM memory, and an NVIDIA GeForce2 GTS board. The third, a Dell Precision 420 with two 933 MHz Pentium III processors, 1 GB of RDRAM memory, and an Oxygen GVX420 video board (3Dlabs, Sunnyvale, CA), was equipped with a special-purpose voxel rendering board (VolumePro 500-2X; TeraRecon, San Mateo, CA). The systems equipped with NVIDIA-based video boards were running Voxx, whereas the VolumePro-based system was running Revli (TeraRecon), a volume-rendering program written for the VolumePro (4).

As shown in Fig. 5, the two NVIDIA-based systems perform nearly identically, which demonstrates that there is no reason to buy the more expensive Quadro2 Pro graphics cards or Pentium 4 and RDRAM-based PCs to run the current version of Voxx. Figure 5 also shows that the rendering speeds of the two NVIDIA-based systems are similar to the rendering speeds of the third system equipped with the more expensive VolumePro voxel processor. The VolumePro-based system outperformed the NVIDIA-based systems for image stacks smaller than 256 × 256 × 256, but the rendering speeds of the systems were very similar for larger stacks containing 512 × 512 images. These results demonstrate that voxel-based rendering using video boards equipped with NVIDIA graphics processors seems to have a major cost-performance advantage over PCs using VolumePro 500 voxel processors. The NVIDIA-based PCs also seem to provide a significant cost-performance advantage over graphics workstations. We find that these PC systems outperform our SGI (Mountain View, CA) Octane SE workstation (data not shown), and others have shown that even the slower GeForce 256 renders small image stacks at speeds similar to the very expensive Reality graphics subsystem used on an SGI Onyx2 workstation (5).


    DISCUSSION
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

Confocal, multiphoton, and digital deconvolution microscopy have provided biologists with the powerful capability to collect high-resolution image volumes, an approach that is critical to characterizing the complex organization of cells and tissues. However, the proliferation of 3-D microscopy has been slowed by the lack of effective and affordable systems for inspecting and evaluating image volumes.

Our goal was to develop software that would provide near real-time rendering of multichannel image stacks by using computer systems affordable enough so that individuals could purchase systems for use in their offices and labs. In the past, most voxel-based programs had to run on very expensive SGI workstations to achieve near real-time rendering speeds. The recently developed VolumePro voxel processors (2) support volume rendering on PCs, but the current VolumePro 500-2X board is limited to single-channel data and adds thousands of dollars to the cost of each imaging system.

Consequently, we developed Voxx, a program that uses the high-performance 3-D graphics processors already present on many low-cost video boards in PCs. These processors can achieve rendering speeds similar to VolumePro voxel processors and the graphics subsystems used on many SGI workstations. The combination of our free Voxx program and such low-cost video boards should make 3-D image analysis systems available to a larger number of biologists, thus promoting a more thorough exploration of complex biological structures imaged using confocal and multiphoton microscopy systems. We encourage other developers of voxel-based imaging software to add support for these low-cost 3-D graphics processors to their programs as well.


    ACKNOWLEDGEMENTS

We thank Dr. Robert Bacallao for providing data for Fig. 3B.


    FOOTNOTES

This work was supported by the Indiana University Strategic Directions Initiative (K. W. Dunn and S. Fang) and a grant (InGen) from the Lilly Foundation to the Indiana University School of Medicine.

Address for reprint requests and other correspondence: K. W. Dunn, Dept. of Medicine, Division of Nephrology, Indiana Univ. School of Medicine, 1120 South Drive, Fesler Hall 115, Indianapolis, Indiana 46202-5116 (E-mail: kwdunn{at}iupui.edu).

1 Supplemental material to this article (Movies 2-4) is available online at http://ajpcell.physiology.org/cgi/content/full/282/1/C213/DC1.

The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Received 6 July 2001; accepted in final form 29 August 2001.


    REFERENCES
TOP
ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
REFERENCES

1.   Lichtenbelt, B, Crane R, and Naqvi S. Introduction to Volume Rendering. Englewood, NJ: Prentice Hall, 1998.

2.   Pfister, H, Hardenbergh J, Knittel J, Lauer H, and Seiler L. The VolumePro real-time ray-casting system. In: Proceedings of SIGGRAPH 99, 1999, p. 251-260.

3.   Phillips, CL, Arend LJ, Filson AJ, Kojetin DJ, Clendenon JL, Fang S, and Dunn KW. Three-dimensional imaging of embryonic mouse kidney by two-photon microscopy. Am J Pathol 158: 49-55, 2001[Abstract/Free Full Text].

4.   Revli Application User's Guide. Mitsubishi Electric Information Technology Center America, Inc., 1999. ftp://ftp.rtviz.com/pub/docs/v2.0/revliusersguide.pdf.

5.  Rezk-Salama C, Engel K, Bauer M, Greiner G, and Ertl T. Interactive volume rendering on standard PC graphics hardware using multi-textures and multi-stage rasterization. Proceedings of SIGGRAPH/Eurographics Graphics Hardware Workshop 2000 (August 2000). ftp://faui90.informatik.uni-erlangen.de/pub/Publications/2000/Publ.2000.5.ps.gz.

6.   White, NS. Visualization systems for multidimensional CLSM images. In: Handbook of Biological Confocal Microscopy, edited by Pawley JB. New York: Plenum, 1995, p. 211-254.


Am J Physiol Cell Physiol 282(1):C213-C218
0363-6143/02 $5.00 Copyright © 2002 the American Physiological Society