Over the years, the “C” and the “A” in CAD have always stood for computer aided or computer assisted. The “D” has meant many things. In the beginning, when engineers sought quicker blueprints, it meant drafting. When broader applications arrived, it meant design generally, and now with radiology it has come to mean detection and, in a narrower sense, diagnosis. The “D” in a less literal way might also stand for diligence and even defense, as in defense from a lawsuit.
Like it or not, computer-aided detection is upon us. Some mammography patients are demanding CAD examinations now, and breast centers are using the technology as a marketing tool. There is justification for this, it appears, though not the 100% reliability patients probably have in mind when they ask for CAD scans.
In plain film mammography, where competing CAD products contend for the hospital or breast center dollar, the utility of CAD is well documented. It has FDA (Food and Drug Administration) approval and Medicare reimbursement.
With digital mammography, CAD is in its infancy, primarily because digital mammography is rare. For anything but the busiest centers, the digital scanners themselves are difficult to cost-justify. One CAD system for? digitized x-ray or CR (computed radiography) lung scans has received FDA approval, but there is no reimbursement for it yet. For other digital imaging uses, especially for CT lung cancer scans and virtual colonoscopies, CAD systems are in the research and development stage and have yet to receive FDA sanction. Researchers say early results are encouraging. Some researchers even argue that in the next half-century radiologists as they are considered today will have become extinct, with all kinds of routine examinations always read first by computers.
If that sounds fanciful, consider the impact CAD is already having. Some studies of CAD application to mammography find as much as a 22% increase in the detection of breast cancer versus when the CAD is not used. That is the figure mentioned by Edward Lipsit, MD, following his radiology group’s assessment of the literature. Of course, just how much detection a CAD adds will depend not just on the CAD but on the skill of the radiologists doing the original radiograph interpretations. Many experts say a CAD will help those of limited skill more than it will aid expert mammographers. Still, even experts are putting in CAD systemsfor a number of reasons.
Lipsit is a mammography/ultrasound specialist and president of Washington Radiology Associates (WRA) PC, a 22-member group that operates seven outpatient radiology facilities in and around the District of Columbia. Lipsit says WRA first tried out a CAD system about two-and-a-half years ago and then “backed off” on installing it. “We thought the algorithm [the software instructions to the scanner] was going to improve, and we didn’t know how we would pay for it at that time,” he notes.
By July 2002, Lipsit and his colleagues had changed their minds. They installed five CAD systems. What precipitated the change were the favorable CAD detection results others were reporting. “I don’t want to minimize the importance of having top people in the field publish data confirming what you intuitively thought to be true about CAD,” Lipsit says.
WRA reads a significant amount of mammography studies, about 80,000 screening and diagnostic procedures per year, and the CAD is used on all of them routinely, Lipsit says. He says it is too early for WRA to have generated in-house efficacy data. “I can’t imagine reading without it,” he says. “We all have seen instances of observational oversights where the CAD brought to our attention both benign and malignant lesions.”
The process is to have the mammographers read the radiographs first, and then to check the CAD markers to confirm or add to the primary interpretations, says Lipsit. He fondly calls CAD “a spell-check for mammography.” He says it is not a double-read because the radiologist must check the CAD printouts or look at the monitors and add that data into the interpretative equation. He suggests CAD may be better than a human double-read “where the second person may rely on the first and the first may rely on the second.” How much mammographers may rely on CAD scanners to save them from missing cancers is something known only to the individual doctor. Lipsit says his colleagues only look at CAD after the film. “Human nature might short circuit the process if you looked at CAD first,” he says. Read first, CAD second seems to be the ubiquitous procedure in mammography where CAD has been deployed.
CAD scanners are essentially taught to seek patterns on film involving shape and light density. The CAD printouts are marked variously, but most CAD systems distinguish masses and calcifications. “Even the best mammographers have been humbled on occasion,” says Lipsit. “Every now and then, we see things the CAD didn’t pick up, but there are far more cases the other way.”
Everyone running CAD mammography systems agrees that the scanners tend to be too sensitive, marking too many spots that turn out to be false positives. But Lipsit says mammographers can generally quickly overrule the CAD on most of these markings. “This is not a short cut or a panacea to interpretation,” Lipsit says. “An inexperienced person the CAD will overwhelm. For us, false positives were a fear, but our call-back and biopsy rates are very similar to those we had without CAD.”
At the Elizabeth Wende Breast Clinic in Rochester, NY, human double-reading is the protocol, a luxury most breast clinics cannot afford. Kathy Willison, RT(R)M, director of clinic development, says the facility does about 350 mammograms daily. About 240 of these are screening examinations and the rest diagnostic. The Wende Clinic did a large prospective study after it installed CAD in spring 2000 to determine if the machine could replace a second read, says Willison. “We looked at more than 19,000 examinations” that had been interpreted before and after CAD, she explains. “What we found was that CAD could not replace the second reader. But it did increase the overall sensitivity of the double-read by 7% of cases. We found 90 cancers. The radiologists detected 93% and the CAD prompted 7% overlooked by both [human] readers. Despite the fact that 77% of the undetected cancers in the trial were DCIS [ductal carcinoma in situ] or less than 10 mm invasive cancers, all of the [additional] cancers that the CAD prompted were invasive; all but one were masses. This counters the criticism that CAD only aids in the detection of calcium, usually an indicator of DCIS, thought by some to be less likely to advance.”
The Wende Clinic now uses CAD routinely for mammography patients. “We would have let those six cancers walk out the door without the help of CAD,” says Willison. But the clinic continues double- reading because the CAD missed some cancers, mostly masses or increased densities that the radiologists identified. “The CAD missed 23%,” says Willison. Overlooked masses might be due to the way CAD systems are designed to search for tumors. The emphasis in design is on detecting small, early tumors. Radiologists are expected to see large lesions.
Willison says false positives are a problem with CAD that everyone is expected to improve as new algorithms are deployed. “The CAD marked an average of 2.3 marks per patient, which in our [prospective] study was about 40,000 marks. Only about 360 of those marks were cancer that we knew of at the time.” Radiologists were able to discount most of the extraneous marks, she says. “The CAD only impacted the recall rate by about 2%. Our positive predictive valuethe number out of those recalled who had cancerwas unchanged with or without CAD, but we err on the side of sensitivity.”
At breast clinics or centers where single reading is the only interpretation, CAD systems are being installed to act as a double-reader and a legal fail-safe.
Jeanne Marcoux is administrative manager of the Alice Viola Breast Care Center in North Providence, RI. The breast center does 15 to 20 mammograms daily plus about 25 per week that are brought by courier from a sister facility for interpretation. All the films are run through a CAD scanner. The scanner has been in only since June, but some patients are already asking for it, Marcoux says. She says the radiologists are enthusiastic. “Before they had CAD, they did a double-reading. Their goal in getting the CAD was to be able to stop double-reading.”
Shireen Braner, PA, RT(R)(M)(QM), is manager of the Kettering Breast Evaluation Center in Kettering, Ohio. The center has three outlets in this suburb of Dayton.
Braner was at Kettering in the mid 1990s when CAD developers used Kettering films and data to help define their algorithm. Braner worked with these programmers to teach the machine to recognize malignancies. It was not until June 2001 that Kettering got its first CAD. It still has just the one, although Braner says a second is in next year’s budget.
Through this August, the Kettering center has run about 17,000 studies through its CAD. The CAD is “part of our standard of care,” says Braner. She says a retrospective assessment of CAD indicates “everything the radiologists found, the CAD found, every significant thing.” She says the CAD does pick up things the radiologists disallow “and vice versa,” although she adds, “The radiologists are not exactly forthcoming about when the CAD saves them.”
Braner also points out what all those who use CAD for mammography know, that the CAD scanners cannot read specialty studies shot at unusual angles, including some scans of extremely large-breasted patients. The machines are not programmed for this sort of interpretation.
Virtually all of those using CAD for mammography say the impact of the machines on the practice itself has been next to nothing as far as disruptions in work flow are concerned. There is some slowdownusually only a few secondswhile radiologists check the CAD monitor or the paper printouts. Patients might experience slight delays. Overall, the processing of cases does not seem to be much altered.
What has been changed by CAD, and sometimes unexpectedly so, is the nontechnical staffing required. The CAD scanner must be fed the film studies. They have to be loaded into the CAD and then unloaded and arranged for radiologists to interpret. This takes staff time.
Jeanne Brotzky is director of women’s health services at Memorial Regional Hospital in Hollywood, Fla. The 650-bed hospital is home base for five mammography sites that process about 36,000 studies annually; about 25,000 of those are done at a breast center at the hospital itself. Memorial has two CAD units, one at the main facility and a second at a smaller hospital.
Brotzky says the CAD systems have performed beyond expectations, enabling the center to make quicker diagnoses. But she says they have also put a load on the center as far as FTEs (full-time employees) are concerned. Brotzky says that between the two CADs she has added 1.5 FTEs to load them. “They haven’t slowed our radiologists’ time for interpretations,” she adds.
At the Yuma Regional Medical Center in Yuma, Ariz, the director of diagnostic imaging is Frank Barby, CRA, MBA. Barby says the 257-bed hospital conducts about 15,000 mammography examinations per year, all of which are sent through a single CAD. Barby says the CAD has proved itself on detection, but he says the added staffing expense was unanticipated.
“The minimum time to run the film through the CAD is 5 minutes,” he reports. “Multiply that by 50 per daythat’s 4.2 hours. We did have to hire one minimum wage person.”
Barby says his hospital’s modest CAD cost about $50,000. High-capacity scanners reportedly cost as much as $160,000. Barby says reimbursements quickly put Yuma’s CAD into the black. But he worries that may have already changed. “Sometime after the first of this year, the [Medicare] fiscal intermediary stopped paying,” he says. “They’ll pay for the mammography, but they won’t pay for the CAD. I just found that out. We don’t know yet why they stopped.”
No one else interviewed for this story mentioned a cessation in Medicare payments. They said Medicare routinely pays them for CAD. But whether private payors reimburse is a hit-and-miss affair. Many reported mixed results with marketing and education campaigns aimed at getting private insurance reimbursement.
At WRA in the nation’s capital, Lipsit says many payors have followed Medicare in reimbursing for the procedure. Nonetheless, because it is using CAD for all patients, WRA requires patients themselves to pay a modest fee of $25 per mammogram, Lipsit says. “We sent brochures to patients so they would understand.” He says patients have not balked because “they realize they get better service.” Lipsit says his group has not used CAD as a profit center but has attempted only to cover its costs on using CAD for every screening and diagnostic procedure it does.
Yuma’s Barby says that even if CAD runs in the red, it will still be worthwhile because of the added cancer detection. He also sees it as protection against lawsuits. “The price tag is security to help you against having an undiagnosed cancer,” he says. “Mammography is the number-one reason radiologists get sued.”
The Clinical Frontier
With the exception of one CAD scanner for chest radiographs that has FDA approval, the CAD technologies for colon cancer and CT lung screens are still in the developmental stage. But they are far enough along that researchers can already see a pay-off.
Judy Yee, MD, is a diagnostic radiologist who is researching CT colon CAD scanning as part of an alpha trial at the San Francisco Veterans Administration Medical Center. Yee says results are promising. “In the long run for the future of colonography, I see CAD as being helpful.”
She says the CAD colon scanners are not so good at picking up flat or smaller lesions, but are good at larger lesions. She says she can increase sensitivity on the machines but that to do so increases the false-positive rate. “With very high sensitivity you have low specificity. You have to rework the algorithms.” She says that so far she has CAD-scanned about 200 colon patients. “Yes, the CAD has found lesions that would not have been detected otherwise.”
But Yee says there is more work to be done to refine the CAD systems before they are put on the market. “What has surprised me is that interpretation times have not decreased as much as you would theoretically think, and in some cases CAD has increased interpretation time through false positives. I would say we’re probably another 2 to 3 years away to get the algorithms developed.”
Pablo Delgado, MD, is an associate professor of radiology and director of the radiology program at the University of Missouri-Kansas City (UMKC). He is also director of radiology residency at the affiliated St. Luke’s Hospital. Delgado is beta testing a CAD system for CT lung scans. He says 300 to 400 patients have had their CT images run through the CAD system since it was installed in June.
“Right now with CAD, it is overly sensitive but not entirely specific,” he reports. “It’s very good at anything abnormal but not the specifics. There are a lot of false positives.” Delgado says the jury is still out on “what the specific role of the CAD is going to be. So far, the answer is there will probably be a role for it.”
One advantage of CAD may turn out to be standardized measurements, because radiologists’ measurements of the same lesion vary, Delgado suggests. He says a “controversial” use of CAD might be as independent screener to quickly look for nodules. But there are too many variables in interpretation for CAD to replace the radiologist any time soon, he says. He suggests that CAD may also put doctors in the position of “chasing lab values” when CAD picks up a tiny nodule or many scattered tiny nodules. “Try to put a needle in that. If there are many, what do you do? Do you have to take the whole lung? The patient will say, But, Doc, I have a nodule.'”
Delgado says many radiologists are of limited enthusiasm about CAD: “They’re not comfortable with a computer dictating findings to them.” Conversely, he says, “There’s a lot of interest from the clinicians’ point of view, which indicates they’re frustrated with radiologists’ variable findings.”
Matthew Freedman, MD, is an associate professor of oncology at the Georgetown University Medical Center, Washington, DC. He is also clinical director of the medical school’s ISIS (imaging science and information systems) research center.
Freedman and 15 or more colleagues have been researching and helping design several uses for CAD systems, including lung CT analysis. In one trial of cases selected because lung lesions were difficult to see, the CAD showed a 15% improvement over radiologists in detecting cancers in the 9 to 14 mm range, Freedman says. He says the radiologists are helping to create a CAD algorithm to detect these problem cases.
Freedman says early tests indicate CAD scanners could pick up a range of diseases like pneumonia, emphysema, and SARS, not just lung cancer. He says CAD systems? could also be designed to pick up metastases because “it’s very important to know when the cancer has come back.”
In one study, Freedman and his colleagues compared theoretical combinations of human/human and human/machine first and second readings to see which combinations were most successful at identifying lesions.1 “Among the radiologists, there was one superstar,” he reports. “If we combined that superstar with the other radiologists, they would beat the machine. There were two other radiologists who together were almost as good as the human combined with the machine. In all of the 256 other combinations, the radiologists paired with the machine did better than the radiologists paired with another radiologist.”
Freedman says today’s CAD systems are just the start of computerized detection and diagnosis from images by computers. “I do not think my great-grandchildren will know what a radiologist is,” he says. “By that time, CAD and bioinformaticsknowing for instance that a nodule is metabolically activeis going to replace not just radiologists but many specialists. That computer information will be fed to generalist physicians. The future of CAD will be involved in every aspect of imaging the human body.”
UMKC’s Delgado foresees a similar explosion in CAD deployment. “The sheer volume increase in imaging interpretation will require it,” he says. “Each study of 10 years ago is 350 images today, plus multislice CT. There is a data explosion and a decreasing number of radiologists. Radiologists will need all the new tools they can get to process this data overload.”
George Wiley is a contributing writer for Decisions in Axis Imaging News.
- Freedman et al. Detection of lung cancer on chest radiographs algorithm performance vs radiologist performance by size of cancer. SPIE Medical Imaging. 2001;4319:150-159.