Computer-aided detection (CAD) has made its breakthrough in mammography. That is the conclusion from several recent publications and presentations on trials covering thousands of women. Whether the economics permit its wide use is, as always, another question.
Do Current CAD Systems work?
The utility of CAD has been tested in several ways. At the Elizabeth Wende Breast Clinic in Rochester, NY, all mammograms are examined by two readers, and whenever a cancer is found and confirmed by biopsy, the previous year’s films are looked at by a panel of radiologists to determine whether the cancer is visible in retrospect. The lesions are then allocated to one of three categories: no evidence of cancer, lesion visible but three of the five radiologists judge it below the threshold for action, or visible and three of the five would have taken action (actionable lesions).
In a study reported at the 2001 Radiological Society of North America? meeting, all of the actionable cancers from the year 2000 (15% of the total cancers found by screening or diagnostic mammography) were examined by CAD, which had to mark the correct site and type of lesion to be judged accurate. Among these lesions were 59 masses or architectural distortions, 11 microcalcifications, and 8 masses with calcium. The CAD system identified 87% of the cancers overlooked on the earlier mammograms: 83% of the masses and 100% of the calcifications with or without microcalcifications.1
“Even when double reading is routinely used with screening mammography, the use of CAD has the potential to decrease the number of false-negative interpretations by 87%,” the investigators concluded.
“We were excited by these findings,” said Stamatia V. Destounis, MD,? radiologist at the clinic, who presented the paper. “So we started a prospective study. After both readers examine a screening mammogram, we have CAD look at it. So far, in more than 7,000 screening mammograms, we have found 15 cancers, of which the machine identified 12. Of particular interest, the machine prompted us to work up three lesions that we would not have worked up otherwise; that is, it found three extra cancers.”
The Wende clinic also tested CAD on those cancers that were found by only one of the two readers. From 1996 to 2000, 71 of the 938 cancers found by screening mammography were in this category. The CAD system found 53 of the lesions (75%), making it superior to the average diagnostic accuracy of the two radiologists.2
Ermelinda Bonaccio, MD, radiologist at the clinic, who presented the paper, cautioned that this was a retrospective study.
“We have no way to determine whether the radiologist would have found the cancer after seeing the marks made by the CAD system, and using CAD would significantly have increased our recall rate. But we think the study shows that there is a real potential for CAD as a second reader. Not a perfect second reader, by any means, but if you had no choice, having CAD would be better than working alone.”
Reasoning that a large number of readers would be useful in a clinical trial of new equipment,? Robert A. Schmidt, MD, associate professor, New York University, now at the University of Chicago, and his colleagues presented 100 screening mammograms to 100 general radiologists, three expert readers, and a CAD system. The films contained 50 biopsy-proven cancers in 45 patients.3 For a radiologist to make the correct diagnosis, he or she had to mark the cancer in the correct breast within a third of the breast dimension of the actual location and to specify the type of lesion (calcification or mass). Stricter standards were set for the computer: it had to mark the lesion within one lesion dimension of its actual location.
For the entire series of films, the sensitivity of the general radiologists was 70% and that of the experts was 81%. The CAD system had a sensitivity of 80% and averaged 3.3 false-positive calcifications and 0.7 false-positive mass markings per case. Notably, CAD performed as well with digitized copies as with the original films.
Particular attention was paid to the 14 cancers that were identified by fewer than half of the radiologists. Of these cancers, 36% were found by the radiologists and 64% by the CAD system. Four of these difficult lesions were apparent on a single view, and only 23% of the radiologists identified them, whereas CAD found all four, with 3.5 false-positive calcification and 0 false-positive mass markings.
For several years, a multicenter study has been trying to define the mammographic features of cancers that are overlooked at screening. A series of 1,083 consecutive cases of cancer discovered on screening mammograms was collected from 13 centers, of which 427 had prior studies available. Expert mammographers examined these pairs of films and found 286 in which some suspect change was visible on the earlier films. These films, mixed with mammograms of normal breasts and of breasts with benign disease, were then viewed by panels of radiologists, who were asked to identify any lesions that they would have acted on. The number of radiologists who found particular lesions actionable was recorded, and the lesions were grouped into two categories: a high-consensus group (ie, at least three of the five radiologists identified the cancer as an actionable lesion) and a low-consensus group. There were 115 high-consensus lesions, about which two questions were asked: how many of them were identifiable by CAD? And what changed to make them more visible the following year?
Among the 115 high-consensus lesions, the CAD system marked 89. Thus, the original radiologists had a false-negative rate of 21% that might have been reduced by 77% if CAD had been available.4
Of these high-consensus lesions, 30% were calcifications, of which half were worrisome (ie, clustered or pleiomorphic). The CAD system marked 86%. The remainder of the lesions were masses, 40% of which had worrisome features; ie, spiculated or irregular margins. Here, CAD identified 73% of the cancers. The most common reasons radiologists overlooked either type of lesion were dense breasts (34%) and distracting lesions (44%). Thus, errors in detection were seen with both calcifications and masses, and CAD identified most (77%) of the lesions that were missed the first time.5
The Learning Curve
The radiologist needs to learn how to use a CAD system-not just how to manipulate the machine and the software, but how to make use of what the system reveals.
“You as the radiologist are still driving the interpretation,” cautions Robyn L. Birdwell, MD, associate professor and director of the residency program in radiology, Stanford University Medical School, and one of the investigators in the multicenter study. “You look carefully at the films and make your own decisions, and then look at what CAD has found. Once those marks are seen, you look again at those areas on the original mammogram.”
Schmidt likens this situation to Rod Serling’s Twilight Zone television series from the 1950s, where items were always “submitted for your approval.”
There have been objections that CAD will add to medical costs and patient stress by inducing the recall of far more women than at present for further examinations. Experience suggests this does not happen.
“For a short time, using CAD may increase your recall rate,” Destounis acknowledges, “but only by a few percent. Once the learning curve has passed, you find that you can discount many of the marks, for example, calcifications that are clearly vascular, not parenchymal.”
Schmidt notes that CAD also has different benefits according to the expertise of the user. “The performance of people who are less skilled is boosted more than that of people who are more skilled, because obviously, it is a lot tougher to improve the performance of experts,” he points out. “It is in general radiology that you expect the most benefit from CAD: it should enable people to perform more consistently and more like experts.”
At the same time, CAD cannot be expected to substitute for radiologist expertise. “A computer can help a pilot fly an airplane, but the pilot has to have certain basic skills before he or she can benefit,” Schmidt cautions.
The CAD systems might find a role in the credentialing and training of mammographers. Schmidt is working to establish a laboratory where mammographers can be tested.
“That can be a revelation,” he says wryly. “There are some people who should not be doing mammography, and they need to realize this. We have totally ignored the role of the observer for many years in our studies of mammography benefit, but assessing performance is now being done in all of radiology. Mammography is a good test case, because it poses only one question: is cancer present, or is it not?”
Whatever one’s views of electronic colleagues, no one doubts the productive capacity of CAD.
“The machine does not get tired, it does not lose focus, and it does not get bored,” Destounis points out. “It does not have a patient waiting for a biopsy, and nobody pages it to answer a telephone call. It is consistent, always marking the same things. As it gets harder to find mammographers, CAD’s appeal will grow.”
For Academic Centers Only?
New imaging technology always is faced with the question of whether it can be translated to the community from the academic medical centers. In mammography, CAD may be able to make this move, as shown in a prospective study published last year by Timothy W. Freer, MD, and Michael J. Ulissey, MD, of the Women’s Diagnostic and Breast Health Center in Plano, Texas.6? They interpreted 12,860 screening mammograms with and without CAD and compared the performance of the radiologist with that of the computer. The recall rate increased from 6.5% to 7.7%, and there was no change in the positive predictive value for biopsy (38%). However, there was a 19.5% increase in the number of cancers detected, and the proportion of cancers found when they were in stage 0 or I rose from 73% to 78%. Those investigators concluded that CAD could be beneficial in a community practice in that it improved the detection of low-stage cancers without an unacceptable increase in the recall rate.
What is CAD’s role today in facilities that have it in place?
“We are no longer comfortable reading mammograms without it,” Freer avers. “All breast cancers are eventually detected. That is not the point. Our job in screening mammography is to pick them up in a stage where there is a good chance of curing them. What we want to do is move the time of detection back a year or two from what is possible with conventional mammography, and CAD may help us do that.”
Perhaps the largest impediment to the widespread use of CAD is, as expected, money.
“Breast centers are usually struggling financially,” Freer observes. “Many of them are closing or are losing money and cannot enlarge their programs to meet the demand for services. If a vendor approaches a center like that and says, ‘We can make your work better with our technology, but it will cost you $200,000,’ what is the center going to say?
“We are looking forward to providing CAD to every patient we see, but right now, the technology is not being reimbursed uniformly,” Freer continues. “Medicare has assigned a CPT code as of January 1, but there is always a lag between Medicare approval and what the private carriers do. Right now, some will give you a few dollars for using CAD, some will give you $20, and others will give you nothing. In order to provide the study, we ask all our patients if they want a CAD review, for which they will pay an extra $20. Right now, 80% of our patients are willing to pay to have CAD done.”
Moberg K, Bjurstam N, Wilczek B, Rostgard L, Egge E, Muren C. Computed assisted detection of interval breast cancers. Eur J Radiol. 2001;39:104-110.
Zheng B, Ganott MA, Britton CA, et al. Soft-copy mammographic readings with different computer-assisted detection cuing environments: preliminary findings. Radiology. 2001;221:633-640.
Judith Gunn Bronson, MS, is a contributing writer for Decisions in Axis Imaging News.
- Destounis SV, DiNitto P, Willison K, Logan-Young WW, Bonaccio E, Zuley ML. Can computer-aided detection (CAD) decrease the false negative rate of consensus double read screen mammography? [abstract 998]. Radiology. 2001;221:P471?472.
- Logan-Young WW, Bonaccio E, Destounis SV, et al. Computer-aided diagnosis (CAD) of screening mammography compared with diagnosis by radiologists [abstract 1003]. Radiology. 2001;221:P473.
- Schmidt RA, Newstead GM, Linver MN. Potential of CAD (computer-assisted diagnosis) to aid radiologists in the detection of frequently missed breast cancers [abstract 1001]. Radiology. 2001;221:P472.
- Warren Burhenne LJ, Wood SA, D’Orsi CJ, et al. Potential contribution of computer-aided detection to the sensitivity of screening mammography. Radiology. 2000;215:554?562.
- Birdwell RL, Ikeda DM, O’Shaughnessy KF, Sickles EA. Mammographic characteristics of 115 missed cancers later detected with screening mammography and the potential utility of computer-aided detection. Radiology. 2001;219:192?202.
- Freer TW, Ulissey MJ. Screening mammography with computer-aided detection: prospective study of 12,860 patients in a community breast center. Radiology. 2001;220:781?786.