Description of problem.
The problem that my project addresses, as I currently envision it, is a volume rendering based on the photographic data of the slices of one or both of the Visible Human datasets. Volume rendering necessitates having a transfer function to calculate the accumulation of light through the rendered volume, and in the case of the human body this is most likely a representation of the densities of the tissues shown. Another approach, however, is to represent transitions between values as being more dense, resulting in the appearance of relatively solid outer boundaries around areas of like color, which are assumed to be body structures. I am leaning towards the second approach, as it is more abstract and requires less anatomical information.
The [hopefully] innovative aspect of this project is that it proposes to volume render based on color opposed to the actual density data that would be gotten from many scientific visualization applications, or indeed from the corresponding tomographic data of the Visible Human itself.
So I would say that from a volume rendering perspective, this is a somewhat unusual visualization; and from a Visible Human perspective, most people who are volume rendering are doing so based on the CT and MRI data, not the color of every pixel, which is what we're doing. Note that the advantage of this is that we should get much higher resolution.
First, the Visible Human Project's own list of research that's going on, (http://www.nlm.nih.gov/research/visible/visible_human.html). Some of the links are NPAC's planar and axial viewers, NLM-Maryland's Visible Human Explorer -- both of which show cross-sections (coronal, axial, sagittarial) rather than attenuation-based 3-D images.
In fact, Loyola, CieMed, Queensland Tech, Michigan, the Colorado CHS, Stanford, Penn, WUStL, are all doing cross-sections, not volumetric views.
Cross-sectioning is not particularly difficult, once registration issues have been worked out. Some of these systems produce output that is also tagged with segmentation data (i.e. you click on a point and a status line reads "left ventricle"). But it isn't real-time. Nor is it what we are really doing here.
Some very well-known work by Bill Lorensen and the GE group (http://www.crd.ge.com/esl/cgsp/projects/vm/) involves extraction of surfaces from the volumetric data. They are getting excellent results, but this is not volume rendering per se.
The Vesalius Project (http://www.cs.stevens-tech.edu/vesalius/) had the best images at the Visible Human Conference if for no other reason than that they eschewed specular reflections. They have an interesting segmentation algorithm which we may or may not want to look into -- we aren't doing segmentation as such but we do want to save color regions. However, we are more likely to work this by considering local gradients. They are also doing surface reconstruction, not volume rendering.
They are using CT data -- the project's scope includes live patients as well as the Visible Human, which is admittedly, for applications like this, just a very interesting "hard-wired" set of data. They're using VolVis.
This is a volume renderer that uses their own implementation of ray-casting, and is optimized for massively parallel systems. They are using MRI/CT data.
Are using CT data as their basis. How are they volume rendering? Using VRen (http://www-unix.mcs.anl.gov/~hudson/htmlbase/vren.html) for Onyx -- something to look into.
is doing volume rendering of the VH with an experimental toolkit called eTDIPS (experimental Two/Three Dimensional Image Processing System). It seems to have the volume rendering well under control ("Interactive dialogs and graphical feedback allow accurate specification of the rendering transfer function, viewpoint parameters, lighting controls, object rotation, and volume of interest.") and that could be worth looking into. They, too, are using CT/MRI data -- "The color and opacity values of the transfer function are specified using direct modification of nodes on a 2D graph."
To reiterate, what differentiates this project is that it does not use the MRI or CT data, but the photographic data.
As for the volume rendering aspect, I found it difficult to find any work that attempts to do photograph- or color-based volume rendering. This isn't too surprising, considering there has probably never been such a volume of points where the color of every point is known over all three dimensions. A search of SIGGRAPH's bibliography database turned up articles and papers from 1988 to 1998 about volume rendering, none of which appeared to be relevant. The ACM Digital Library, similarly, turns up papers about color maps and Victoria Interrante's paper about 3D lines and surface shape and so forth, but nothing that seems to be congruent to what I'm doing.
In conclusion, it seems the literature hasn't got much to say about what we're doing here, so let's see where it takes us.
Update
If you are reading this online and you think I'm missing something, by all means contact me! Recently I was made aware that the words I should have been looking for are "image-based volume rendering." Marc Levoy and Pat Hanrahan are doing this at Stanford. But to quote their page,
The general notion of generating new views from pre-acquired imagery is called image-based rendering. . . . We are investigating a new image-based rendering technique that does not require depth information or image correspondences, yet allows full freedom in the range of possible views.Well, depth information and image correspondence is exactly what we have in the Visible Human photographs, and we intend to make full use of it.
Steve Matuszek
October 7, 1998
This document also available at
http://www.cs.umbc.edu/~wyvern/691/litrev.html.