Clockwise from top, Maryellen L. Giger, PhD, Heber MacMahon, MD, Hiroyuki Yoshida, PhD, and Samuel G. Armato III, PhD, Kurt Rossmann Laboratories for Radiologic Image Research, the University of Chicago.

Can we obtain more information from digital images than our eyes can see? That question has been asked by university and commercial laboratories around the world for more than 20 years, and we now know that the answer is, clearly, yes. One of the most active laboratories in the pursuit of this latent information is the Department of Radiology at the University of Chicago.

What eventually became one of the premier laboratories for imaging research began in 1967, when Kurt Rossmann, PhD, joined the faculty of the University of Chicago as professor and director of the Section of Radiological Sciences. He had previously worked in a vendor-based research laboratory, where he had established the concepts of quantum mottle and modulation transfer function in screen-film systems. Charles E. Metz, PhD, and Kunio Doi, PhD, joined him in 1969.

Rossmann died in 1976 at the age of 50. The following year, the university established the Kurt Rossmann Laboratories for Radiologic Image Research, which have two overarching goals: to improve the diagnostic accuracy of imaging and to minimize radiation exposure of patients. There are more than 10 PhD and MD faculty members, and a research staff of more than 30.

“We were working on image quality as it related to diagnostic work,” recalls Doi, now Ralph W. Gerard Professor of Radiology and director of the Laboratories. “Much of the focus in the early 1980s was how using digital images for PACS [picture archiving and communications system] could help radiology departments and hospitals manage images economically. But we felt that attention needed to be paid to the benefits of digital imaging in the daily diagnostic tasks of radiologists. We believed that processing to help define the location or nature of lesions, what is now called ‘computer-aided diagnosis,’ would be helpful. Because we knew that it would be difficult to develop useful schemes, we selected important targets, namely lung cancer, breast cancer, and cardiac lesions, as our primary goals.”

Work on these projects continues, and projects have been started on other lung diseases, colorectal cancer precursor lesions, and trabecular analysis for determining the degree of osteoporosis. “Computer-aided detection is a broad field, and it is expanding all the time,” notes Heber MacMahon, MD, associate director of Rossmann Laboratories, who as interim chairman and professor of radiology and chief of the chest section, provides a clear link between the laboratory and the clinic. “Traditionally, people think of it as detection of objects on images, but it is helpful to look at the entire gamut of programs that can be used to enhance diagnosis: not just nodule marking but nodule analysis and visual enhancement software.

“One reason these programs are not more widely employed is that most people are still using film,” MacMahon continued. “Until people are reading at a workstation as a routine, they cannot take advantage of this technology. Once they are in a soft-copy environment, it becomes more useful and attractive.”

Kunio Doi, PhD, director, Kurt Rossmann Laboratories for Radiologic Image Research, University of Chicago.

Funding has been provided by the National Institutes of Health, the Department of Energy, the United States Army, the American Cancer Society, The Whitaker Foundation, and industry. Many patents have been licensed for commercial development through UC Tech, the University of Chicago’s technology transfer office, and some products based on them have earned Food and Drug Administration approval. Almost 1,000 publications have described the work at the Rossmann Laboratories.

The Continuing Quest

Despite the recent arguments by a few investigators that screening mammography is not effective, the generally accepted view is that early breast cancer detection saves lives. A few months ago, Decisions in Axis Imaging News described the software recently approved by the Food and Drug Administration for detecting mammographic lesions with features of breast cancer (clustered microcalcifications and masses). Some of that software was originally developed at the Rossmann Laboratories,1 where work continues on methods to obtain even more information from breast images.2,3

One goal is to determine noninvasively whether a lesion is benign or malignant; ie, diagnosis rather than detection. Lesions identified by mammography, ultrasonography, or MRI have been extracted by computer for analysis of features such as margin sharpness, spiculation, and contrast with surrounding normal tissue. These features are evaluated using an artificial neural network or linear discriminant analysis for the modalities individually and in combination. In 2000, Maryellen L. Giger, PhD, professor of radiology and director of the Graduate Programs in Medical Physics, and her colleagues demonstrated a system with which an unknown breast lesion could be assessed for its likelihood of malignancy. A paper that will appear soon in Radiology describes a trial of the program on a new series of lesions identified by screening mammography. A follow-on project involving multimodality analysis was demonstrated at the RSNA meeting last year.

For each lesion, the system displays the likelihood of malignancy in three ways. First, a percentage figure is provided. Second, the system depicts similar lesions from an atlas of more than 300 cases in which the result of the biopsy is known. Benign lesions are outlined in green and malignant lesions in red. Third, the distribution of benign and malignant cases in the atlas is shown as a graph and the position of the unknown lesion is marked according to its features.

This three-way portrayal of the results might seem overly complex. However, as Giger notes, “once you have developed an analytical system, you need to consider how to interface it with the human beings. Some radiologists want just the number, some want to look at similar cases, and some prefer the graphic output. You must remember that no matter how great a system is, if it cannot be used efficiently and effectively, it will not be used at all.”

Another project of the breast imaging group is focused on risk assessment. For this work, 14 features of breast parenchyma have been extracted in order to characterize the density and texture. In one series of experiments, tissue patterns of women at low risk were compared with those of women with mutations of the BRCA1/BRCA2 gene, which predispose them to breast cancer. In the second series, the features were correlated with risk factors identified by standard epidemiologic models of breast cancer risk. Both approaches identified the same features as risk markers, namely, high density, low contrast, and coarse parenchyma.4 Another approach, in which the cancer-containing and noncancerous breasts of breast cancer patients were compared, produced the same results. The laboratories are now working to validate their analysis prospectively.

“Eventually, we want to integrate this analysis with computer-aided detection to see if you can obtain an estimate of the breast cancer risk for an individual patient,” Giger says. “In other words, we are seeking radiographic markers of breast cancer risk.”

Helping Radiologists in the Chest

The complex normal anatomy of the chest makes it a particular challenge to the radiologist. Rossmann Laboratories has been working on several programs to make the task of interpreting these images easier.5 Much of the work has focused on nodules appearing on chest radiographs and, more recently, CT scans, but work also is being done on programs to help in the identification and evaluation of interstitial lung disease, infiltrates, and pneumothorax, and for the measurement of heart size.

The studies of nodules have two goals: to improve detection and to characterize them as benign or malignant. Two computer programs are being tested. In a study published in 1999, 146 participants examined 20 normal and 20 abnormal digitized chest radiographs without and with the use of computer-aided detection. All types of participants-chest radiologists, other radiologists, radiology residents, and nonradiologist physicians-were significantly more accurate in detecting nodules when using the computer program.6 More recently, the Rossmann team tested an artificial neural network for diagnosis.7 For this work, chest radiographs containing 34 primary lung cancers and 22 benign nodules were digitized. Subjective features of the images were noted by radiologists, while a computer program identified objective features that could be correlated with the subjective findings. An artificial neural network was then used to characterize the lesions. The network was much more accurate in distinguishing benign and malignant lesions when it used objective features in its analysis, and it was more accurate than the radiologists in characterizing the lesions.

During their work on detection and evaluation of lung nodules on chest radiographs, researchers at the Rossmann Laboratories have developed or validated techniques for making nodules more conspicuous. One technique, called interval or temporal subtraction, has been commercialized in Japan but is still in clinical trials in the United States. An earlier chest image from a patient is used to create a mask for the more recent image so that only the changes from the previous study are seen. As chest imaging becomes more common, a higher percentage of patients will have an earlier film, making temporal subtraction applicable to more patients. In another technique, energy subtraction, which has been approved by the FDA, two datasets are obtained during the radiographic examination, one at lower energy and one at higher energy. Three images are then produced: a standard radiograph, a soft tissue image, and a bone-only image. The technique requires no more radiation than does a standard chest radiograph and may reduce the number of chest studies that must be acquired to make a diagnosis. One of the large validation trials of the technology was performed at Rossmann Laboratories.

A third technique is in earlier stages of development at the Rossmann Laboratories. Here, the computer uses the image of one side of the chest to create a mask for the other, enhancing the detection of unilateral abnormalities, including nodules, pneumonia, emphysema, and pneumothorax. In a retrospective study, 50 normal and 50 abnormal chest radiographs were employed.8 The percentage of images that were considered at least adequate for diagnosis was increased from 71% to 91% by contralateral subtraction and image enhancement. This method would be useful in patients in whom no earlier chest radiographs were available.

Despite the continued use of chest radiography, whether by traditional film-screen or by new digital methods, the popularity of chest CT is growing. This practice imposes a great burden on radiologists.

“The ability of CT to obtain so many slices so quickly is overwhelming the ability of radiologists to interpret them visually,” notes MacMahon. “I have a scan on my computer right now that consists of 271 slices, each 1.25 mm thick. Of course, we can move quickly up and down through the slices, but it is a lot of information!”

Samuel G. Armato III, PhD, assistant professor of radiology, and his colleagues are working on computer methods to help radiologists detect nodules, both on diagnostic CT scans and on low-dose scans that are being tested for lung cancer screening. One program creates segmented lung volumes to which various gray-level thresholds and a connectivity scheme are applied to identify areas of a specified minimum size that are distinguishable from the background. The computer then examines each volume so identified and selects those that are most likely to represent nodules, which can be presented to the radiologist for review. In a set of 43 diagnostic chest CT scans, the program produced an overall sensitivity of 70% with an average of 1.5 false-positive findings per section. Better results were found with the images having only one or two nodules: 89% sensitivity and 1.3 false-positive findings per section.9

“We have applied this program to several databases now, and the results are quite consistent,” Armato says. “We are trying to reduce the number of false-positive findings further to make the program more acceptable in a clinical environment. We are also working on a computer method of classifying nodules as suspicious for malignancy or probably benign. Our eventual goal is a program that will mark everything out of the ordinary and then note which of the abnormalities needs special attention.”

The potential use of low-dose CT scans for early detection of lung cancer is a hotly debated topic at present, which might suggest that screening is the principal role of computer-aided detection. This is not true. “Whatever the outcome of the debate over screening, we still are going to have to detect and evaluate nodules on chest CT scans,” MacMahon points out. “Even in the unlikely event that people totally give up on screening, in view of the amount of chest CT that is being done for other reasons, we will be able to benefit from computer-aided detection.”

Assisting Virtual Colonoscopy

The high toll exacted by colorectal cancer every year could be significantly reduced by early identification and excision of premalignant polyps discovered by regular colonoscopy. However, there is considerable resistance by the public to this unpleasant examination. With the commercial availability of three-dimensional reconstruction of CT images, imaging laboratories became interested in using CT colonography (also called virtual colonoscopy) to identify clinically significant polyps: those measuring 5 mm or more.

Hiroyuki Yoshida, PhD, assistant professor of radiology, described the goal of the virtual colonoscopy project as “developing a computer program that will review all the images, find these polyps, and point them out to the radiologist.” He notes that a CT scan of the colon produces 600 to 700 slices per patient, which require at least 10 minutes in the scanner and can take as long as an hour to review.

“This is not practical for a screening test. What we want is a method that takes less than 5 minutes. Also, although experts may have a sensitivity of 100% in finding important polyps, the average accuracy for radiologists reading CT colonography is about 70% to 90%.”

An early report of the efficacy of the current version of the program appeared in February of this year.10 Nine patients with polyps and 32 without polyps were examined by CT after standard bowel cleansing and air insufflation, with helical scans being obtained with the patients supine and prone. The patients then underwent colonoscopy. With a computer analysis time of about 10 minutes, all of the significant polyps were detected (100% sensitivity), with an average of 2.5 false-positive findings per patient. Most of the structures falsely labeled as polyps by the computer could be readily identified as nonpolyps by the radiologists, “but we are working to reduce the false-positive rate,” Yoshida says. Several more cases have been analyzed retrospectively since the paper was published, and the sensitivity continues to be 100%. The research team hopes to begin a prospective study by the end of this year.

Some vendors have expressed interest in incorporating the program into their software.

“Some vendors would like to add our program to their software so that suspicious lesions are highlighted during their three-dimensional flythroughs of the colon,” Yoshida reports.


Those working in computer-aided diagnosis describe their goal as diagnosis made by a physician using computer output as a second opinion. The intent is to improve the diagnostic accuracy and consistency of the radiologist. It is already clear that computers can perform the tasks of detecting and even characterizing many abnormalities well, thereby increasing the productivity and accuracy of their human users.

Judith Gunn Bronson, MS, is a contributing writer for Decisions in Axis Imaging News.


  1. Giger ML, Huo Z, Kupinski MA, Vyborny CJ. Computer-aided diagnosis in mammography. In: Sonka M, Fitzpatrick MJ, eds. Handbook of Medical Imaging. Vol 2: Medical Imaging Processing and Analysis. Bellingham, Wash: SPIE; 2000:915??”1004.
  2. Jiang Y, Nishikawa RM, Schmidt RA, Metz CE, Giger ML, Doi K. Improving breast cancer diagnosis with computer-aided diagnosis. Acad Radiol. 1999;6:22??”33.
  3. Huo Z, Giger ML, Vyborny CJ. Computerized analysis of multiple mammographic views: potential usefulness of special view mammograms in computer-aided diagnosis. IEEE Trans Med Imag. 2001;20:1285??”1292.
  4. Huo Z, Giger ML, Wolverton DE, Zhong W, Cumming S, Olopade OI. Computerized analysis of mammographic parenchymal patterns for breast cancer risk assessment: feature selection. Med Phys. 2000;27:4??”12.
  5. Swett H, Giger ML, Doi K. Computer vision and decision support. In: Hendee W, Wells P, eds. Computer Vision and Decision Support. New York: Springer-Verlag; 1997:297??”342.
  6. MacMahon H, Engelmann R, Behlen FM, et al. Computer-aided diagnosis of pulmonary nodules: results of a large-scale observer test. Radiology. 1999;213:723??”726.
  7. Nakamura K, Yoshida H, Engelmann R, et al. Computerized analysis of the likelihood of malignancy in solitary pulmonary nodules with use of artificial neural networks. Radiology. 2000;214:823??”830.
  8. Li Q, Katsuragawa S, Ishida T, et al. Contralateral subtraction: a novel technique for detection of asymmetric abnormalities on digital chest radiographs. Med Phys. 2000;27:47??”55.
  9. Armato SG 3rd, Giger ML, MacMahon H. Automated detection of lung nodules in CT scans: preliminary results. Med Phys. 2001;28:1552??”1561.
  10. Yoshida H, Masutani Y, MacEneaney P, Rubin DT, Dachman AH. Computerized detection of colonic polyps at CT colonography on the basis of volumetric features: pilot study. Radiology. 2002;222:327??”336.