If it’s a new stereo system you want, a new refrigerator you need or a new car you envision in your driveway, you’re likely to seek out opinions comparing features, performance and reliability from your friends who made recent, similar purchases. You also might consult evaluations published by the nonprofit Consumers Union in its Consumer Reports for a more independent view.
As institutional versions of the electronics-appliance-automobile-buying public, to whom do hospitals and other healthcare organizations turn when they are looking to compare medical devices and equipment?
Whether it’s new syringes they want, new surgical instruments they need or a new digital radiography (DR) system they envision in their emergency department, how do radiology departments go about ferreting out information on features, performance and reliability?
“We are like a third party between the hospitals and the manufacturers,” explains Jason Launders of ECRI (founded as the Emergency Care Research Institute), headquartered in Plymouth Meeting, Pa. “Hospitals come to us. They say, ‘Of all the devices out there, which should we buy?’ If a hospital wants to build a new department, it might come to us and ask, ‘What equipment do we need to buy?’”
ECRI, a nonprofit health services research agency that prides itself on its objective, independent and technically competent product evaluations, last year scrutinized a number of flat-panel DR systems. And healthcare entities fortunate enough to have more than one system in-house, through casual observations, have realized outcomes of their own.
At the same time, some researchers question what they consider too-early comparisons, pointing out that installations are few, technological aspects are immature and manufacturers have yet to engineer their detectors for truly practical use in the clinical environment.
Testing … testing
Launders, a senior project officer at ECRI, spent May through August 2000 evaluating computed radiography (CR) and DR systems. (The CR report was published in August 2001, while the DR report is still awaiting publication.) Flat-panel detector manufacturers Hologic Inc. (Bedford, Mass.), GE Medical Systems (GEMS of Waukesha, Wis.) and Philips Medical Systems North America (Bothell, Wash.) participated in the ECRI trials, with each choosing one system and clinical installation or facility to be evaluated.
Swissray International Inc. (New York) and Canon Medical Systems (Irvine, Calif.) were not included in Launders’ study, but both corporations have signaled a willingness to participate in future studies, Launders notes. Other companies — Cares Built Inc. (Keyport, N.J.); Imaging Dynamics Corp. (Calgary, Alberta, Canada); Advanced Instrument Development (AID of Melrose Park, Ill.), which distributes the Oy Imix (Tampere, Finland) system in the United States; and Wuestec Medical Inc. (Mobile, Ala.) — did not have product available at the time of his study, he says.
By Launders’ account, he collected a series of test exposures “under as closely matched conditions as possible on each system” while using a phantom for low-contrast detectability, a quality closely related to detective quantum efficiency (DQE). He also interviewed the test sites about their experiences: radiographic practice, for example, patient throughput, service issues, training, problems, diagnostic image quality, and HIS (hospital information system) and RIS (radiology information system) integration.
As a matter of background, the Hologic detector, also called a “direct” detector by the vendor, uses amorphous selenium as its photoconductor. The selenium, layered onto an array of thin-film transistors (TFT) that serve as little switches, directly converts X-ray photons into an electrical charge. Direct detectors claim nearly perfect modulation transfer function (MTF) numbers and promote those numbers as a measure of their effectiveness. MTF measures image spatial frequency.
The GEMS’ and Philips’ detectors, or “indirect” detectors, employ cesium iodide as a phosphor or scintillator material that captures the X-ray and converts it to light. The light is then detected by an array of thin-film diodes (TFD) that converts the light to electrical charge. Indirect detectors garner higher rates of DQE than direct detectors and so tout those numbers as proof of their capability. DQE takes into account the MTF and the Noise Power Spectrum (NPS) or noise response.
And the envelope, please?
“When you cross-compare, with averaging, the numbers came out about the same for the DR technologies,” Launders pronounces. “There’s not much difference between them.”
Launders says he is unable to provide specifics pending ECRI publication of his report. Yet he offers this overall assessment of the study’s results.
“If you are thinking of different-sized detail on your image, if you are looking at high-contrast, fine-bone detail, Hologic has got an advantage there,” he submits. “At what we call the low-contrast, large details — if you’re looking at the big picture as it were — the GE and Philips technology is actually better than Hologic’s. And that’s because it has better imaging characteristics for that size of image.”
Launders presents this example.
Draw lots of small, thin lines. That’s high spatial frequency. The higher the spatial frequency, the smaller the detail you can see. Now draw a few, thick lines. That’s low spatial frequency. All imaging devices have good DQE at low spatial frequencies — the thick lines. However, DQE trails off toward the higher spatial frequencies — the thin lines.
“When GE quotes their DQE figures, they quite often concentrate on the low-end, low-spatial frequency DQE which, when I said that was better for the large area, that’s what I was alluding to,” he expounds.
“At that low spatial frequency, the Hologic DQE is below the GE DQE, but the lines cross over as they rise. The Hologic DQE actually gets higher than the GE. It still reduces, it just doesn’t reduce as fast, so the Hologic device is better at detecting smaller details. When you look at the raw MTF, which is Hologic’s way of doing it, they look good all the way across.”
Tale of two detectors, Part I
At Mt. Auburn Hospital (Cambridge, Mass.), Radiology Manager Dean DeMaster has overseen the installation of two flat-panel detectors: a Hologic Epex, in August 2000, and a Philips Digital Diagnost room — a room that may not have a digital detector on initial delivery but is able to be upgraded to digital. The table bucky in Mt. Auburn’s Digital Diagnost room was upgraded in September 2000.
DeMaster came by his DR systems as part of a construction project that renovated several hospital floors and provided money for some new equipment. The Hologic unit found a home in ambulatory care, one floor above the radiology department. The Philips room is part of the emergency department (ED), across from radiology.
In addition, Mt. Auburn has two more detectors on order: an upright for the Philips room containing the upgraded table and a second digital table bucky upgrade for a second Philips room, also in the ED.
When Mt. Auburn began planning its renovation in 1997, it investigated and liked the selenium-based technology, convinced that it would have “inherently better resolution because it didn’t have the light scattering, didn’t have that step,” DeMaster remembers. Due to manufacturing delays, numerous industry mergers and acquisitions, and the fallout from those transactions, however, the hospital ended up with a retrofit unit in ambulatory care that evolved — eventually — into its current Epex system.
During the time he was keeping tabs on the developments affecting his ambulatory care unit, DeMaster also prepared to open his ED. He knew the Philips detector was not commercially available in the United States, yet the fact that the company’s Digital Diagnost rooms could be upgraded “in the field” appealed to DeMaster, who figured he would operate the rooms with CR while awaiting the flat-panel upgrade. Unfortunately, production delays followed the Philips order, derailing Mt. Auburn’s digital table bucky upgrade until last year.
Despite the hospital’s initial preference for selenium-based detector technology, DeMaster admits, “there is nothing inherent right now that would make us choose one over the other. We made a conscious choice, but it wasn’t dictated by side-by-side comparisons and choosing from multiple systems that were available at the same time. Even though we made a conscious choice for the selenium-based technology, our actual install was determined by who could provide equipment at the time we needed it to go forward with the project.”
It just so happens that the Hologic detector handles a majority of the hospital’s orthopedic cases, but DeMaster chalks that up to its location in ambulatory care, complete with back, cast and podiatry clinics.
The radiology manager insists that he is not able to compare the productivity of the two installations because the Philips room is incomplete without its upright bucky. (That upgrade was expected in September.) He estimates the ambulatory-room by Hologic performed 10,000 studies in the year since it was installed; the emergency room by Philips performed 26,000.
He awards points to the Epex for flexibility, but observes that Philips provides a 17-by-17-inch detector vs. Hologic’s 14-by-17-inch detector. “That means that if you want to change from portrait to landscape, on the Hologic detector — you have to rotate the detector itself 90 degrees,” he describes. “You can’t take a 17-by-17 image with the 17-by-17 detector, but you can electronically mask it in either direction, and that is a nice advantage for the techs.
“The Epex arm swings around and allows you to do cross-table work and it allows you to do tabletop work,” DeMaster remarks. “You have more versatility without having to go to a CR image to get a complete digital study on a patient.”
Not surprisingly, Mt. Auburn radiologists voiced their opinion of the DR images. And what they had to say landed like a bombshell.
“The biggest surprise is that the radiologists do not discern a difference in image quality,” DeMaster confesses, sounding stunned. “It surprised me very much that the radiologists did not express a preference for the DR images over the CR images because we know inherently that the DR detectors have better resolution.”
The DR devil is in the details
That radiologists aren’t bowled over by the quality of DR images doesn’t surprise Michael J. Flynn, Ph.D., one bit. Flynn, a physicist and bioscience professor with Henry Ford Hospital, part of the Henry Ford Health System (HFHS of Detroit, Mich.), maintains that it is a digital radiography system’s image-processing capability — not its direct or indirect approach — that makes the difference. And in his estimation, current image-processing sequences leave much to be desired.
“I think that it is both premature and potentially misleading to compare commercial systems at this time,” he proclaims. “I am very much a part of the belief that the processing of the raw acquired data from a DR or a CR system, for that matter, is extremely important in the perception of image quality presented to a radiologist. With the flat-panel systems, for the most part, the processing is immature and the key vendors that are coming into this market — if they are OEMs (original equipment manufacturers) using the Hologic Direct Radiography system — have not had enough time to put the processing part of the system in. GE has been very slow to apply other than the most basic tone scale or grayscale rendition processing on the system. So given that that is the state of the field, I think the images that people are looking at — they may be able to get some general impressions of sharpness and noise — but the processing that is being applied to it I don’t think has been developed to the end point that the user is going to be looking for. And as you attempt to compare one system against another, you may be observing differences not because the technology to acquire the image is better or worse, but because the vendor is further along in developing the presentation characteristics of the image.”
Henry Ford Hospital currently has no DR systems, although Flynn acknowledges that the hospital is in the process of bringing in a selenium-based flat panel for orthopedic use. He also notes that his research lab has an agreement with Eastman Kodak Co. (Rochester, N.Y.) aimed at optimizing image processing in CR and in the coming year, in DR. Overall, HFHS performs 1 million exams a year among its various facilities.
According to Flynn, once a detector captures an image, four generic processing steps turn that energy into a radiograph — exposure recognition, equalization, grayscale rendition for contrast, and edge restoration and noise reduction, both of which address high-frequency characteristics of the image.
Most companies possess some sort of proprietary image-processing software. Flynn suspects that incorporating those developments into their DR systems is a matter of money.
“There is a long list of things that have to be put in place to support the product and you just go step-by-step through them. You just have limited resources available to do it.”
Tale of two detectors, Parts II and III
At The Cleveland (Ohio) Clinic Foundation (CCF) and the University of California San Francisco (UCSF), meanwhile, DR image quality rates two thumbs up.
“Our experience with image quality on all of the manufacturers’ [systems] is that it is excellent,” states David W. Piraino, M.D. (left), staff physician and section head, computers and radiology, division of radiology at the Cleveland Clinic. “You find that the radiographic detail is very good, the ability to differentiate contrast — both soft tissue and bony contrast is good — and the ability to look at lung fields and other parts of the chest also is very good. What our studies would generally say — we didn’t test diagnostic accuracy, but we did look at subjective image quality — in our assessment it would be as good or better than plain radiographs.”
As an academic medical center with research agreements with both Canon Medical Systems and Siemens Medical Systems Inc. (Iselin, N.J.), the Cleveland Clinic has had DR chest units both companies for a couple of years. A Canon CXDI-11 is installed in cardiology and a Siemens Thorax FD is in CCF’s new cancer center. A separate Canon CXDI-22 system, installed about a year ago does double-duty as a clinical and a research unit.
Piraino indicates that the decision to divide the chest units between the bustling cardiology and oncology departments had nothing to do with their capabilities and everything to do with improving patients’ experience.
“We have chosen to distribute these to provide improvements in the patient experience, so we intentionally chose those areas where there was high volume. Having the patient spend less time in the radiology department may have a significant impact on their experience, so that was the cardiology and the oncology departments,” he says.
UCSF joins The Cleveland Clinic as one of two sites that took early delivery of a Canon CXDI-11 approximately two years ago. The California hospital’s other flat-panel is a GE Revolution XQ/i. Both are in clinical use, with the GE unit performing 30 to 50 studies a day. Figures for the Canon are not readily available.
“We have not had an image-quality complaint about either unit. They are both capable of high-quality images,” affirms Robert Gould, Sc.D., UCSF professor of radiology and bioengineering. “But truthfully, I don’t think anyone has done side-by-side comparisons on different manufacturers’ DR flat-panel systems. One of the reasons is that there are quite a number of chest units out there, but in actual numbers of DR rooms, there are not that many. We have on order a combination [DR room] from GE, and they haven’t been able to deliver it yet.”
Like The Cleveland Clinic, UCSF did not choose one manufacturer’s unit over the other’s because of any perceived advantages or disadvantages. UCSF wanted a dedicated chest unit — not just a detector, but also the entire front-end: an integrated X-ray generator, X-ray tube and the like. At the time of order, GE was best able to satisfy UCSF’s order for a complete package, Gould recalls.
The Cleveland Clinic sees more value in evaluating DR technology in general, as opposed to pitting one system against the other, Piraino informs. And those general appraisals have led Cleveland researchers to determine that the packaging of the technology — not necessarily the technology itself — is what needs work.
“You have this moderately large, heavy plate, it probably cost $100,000 or more, and with general radiography you have to take some unusual projections where you angle the plate, angle the tube, especially in orthopedics; sometimes with plain-film radiography we actually use curved cassettes,” he advises. “And the question is, ‘What’s a good configuration for a general radiographic room with a plate that would allow you to do 90 percent or 100 percent of what you now do in general radiographic rooms while protecting the plate and dealing with its weight?’ ”
Let’s get real
Launders also explored that question as part of his ECRI inquiry into DR detector differences. As he put it: “How are they put into clinical use? How are you going to engineer your device into a final product that is going to be used by technologists and others?”
Launders points out that the detector, often the size of a take-out pizza box but costing 10,000 times more than a $10 take-out meal, is packaged in a gantry. In his investigations of gantry positioning, the Hologic Epex outpaced the competition, and was singled out as the current best option for orthopedic work based on its ability to swing into various positions and its performance when imaging high-contrast, fine-bone detail.
“Hologic, Philips and GE can all put their devices into a standard bucky table,” he states. “The problem is, for GE and Philips, you have to buy two detectors. There is one in the table and one for upright work. If you are only going to do upright work, you need only one. If you are only going to do horizontal table work, you need only one in the table. But if you are going to do both, you need a separate detector in each.”
More than the sum of its parts
Gould, of UCSF, concedes that direct and indirect detectors differ from a physics standpoint and that the physics does reflect in image quality. Much like fellow scientist Flynn, however, he wonders whether dueling DR technology statistics are what entice customers to sign on the dotted line.
“I’m not saying there aren’t differences and they’re not important, but I am saying to the end-user: I don’t know how important they are,” he elaborates. “What you really want to figure out is: Does this take good images at a reasonable patient dose and give me reasonable throughput at a cost I can afford? You go look at the images and you talk to your friends and see if the system is functioning on a regular basis and not breaking all the time,” and which vendor can deliver in a timely manner. “That’s how the decision is made — not so much that the DQE is 80 percent vs. 75 percent.”
“I think it very much depends on the center, but I think that for physicians and administrators who are making purchasing decisions, it is unlikely that [DQE and MTF measurements] are a strong factor,” Flynn testifies.
“In our own work, we’ve done these measurements now for seven or eight years, but for many years we refused to publish data that directly compared results from different systems. We were sensitive about offending manufacturers; we were more concerned about getting scientifically valid results,” he adds. “It’s only been in the last year that we have published work where direct comparisons were made. And our intent there was more to focus on direct vs. indirect technology as opposed to vendor vs. vendor.”
But Launders’ ECRI report does focus on GE vs. Hologic vs. Philips. It also comments on the overall state of DR development.
The technology underlying flat panels will not change, for example, while the expense of manufacturing any kind of DR system may affect its viability — be it flat-panel or CCD-based (charged couple device) technology.
“I think they are rapidly maturing, particularly the companies which rely on these devices: I’m thinking Swissray and Hologic. That is their main product line, so they are very mature,” he opines. “The GE, the Philips, the Siemens: That’s a different issue. They are not quite a mature product yet. I don’t think they have worked out what the customers want in terms of a usable product. The underlying technology I think is fine. That is not going to go any further. For example, the GE product has a 200-micron pixel size. Philips and Hologic have about 140 microns. They are fine for most work.
“If you want anything finer,” he concludes, “then use a film.” 
