Validating new technologies demands rigorous clinical investigations, with solid statistics that demonstrate product benefits.

In an era of health care reform, medical equipment is being scrutinized more and more. This critical eye on over-utilization, particularly for imaging modalities, should prompt companies to develop medical technologies that demonstrate clear and significant clinical benefits with both patient care and cost-effectiveness improvement. This requires a commitment to rigorous clinical investigations, with solid statistics to reveal product benefits in a simple, straightforward, and convincing way.

Claude Cohen-Bacrie

A Case in Point: Diagnostic Breast Ultrasound Systems

Breast ultrasound provides a particularly good demonstration of this idea as the effectiveness of a diagnosis relies on both sensitivity (the ability to detect cancer) and specificity (the assessment of false alarm rates) of the exam.

Breast ultrasound plays a major role in breast cancer diagnosis as it complements palpation or mammography since it characterizes the detected lesions. Diagnoses of breast lesions today are based on a BIRADS? score that rates the probability of malignancy. Key morphological features are used to classify them: BIRADS 2: 100% benign; BIRADS 3: < 2% probability of malignancy; BIRADS 4: 3-94% probability of malignancy; BIRADS 5: >94% probability of malignancy.

Ultrasound is a particularly interesting imaging tool because of its very good negative predictive value, or ability to classify a lesion as benign (BIRADS 2). This is essential in breast exams, as one wants to be certain no cancers are missed. However, ultrasound lacks specificity. If a lesion is BIRADS 4 or more, the lesion has to undergo a biopsy; the majority of these biopsies (about 8/10) are benign. Biopsies involve important economic cost in terms of procedures and time, and can provoke trauma and anxiety for the patient. We, manufacturers of ultrasound equipment today, should focus our technical innovation on providing tools to increase diagnostic specificity and demonstrate this benefit with statistical significance.

Testing Clinical Benefits: A Rigorous Approach and Managing Complexities

Our approach to testing the clinical benefits for our ultrasound system was to complement ultrasound imaging with a new mode of imaging tissue called shear wave elastography. This technology assesses local tissue stiffness and displays the information through a colorized image overlay on a conventional ultrasound image. Our theory was that shear wave elastography could complement ultrasound imaging and define an improved BIRADS model with better specificity. Because of complex statistics, a good management strategy was required to turn these statistics into easily understood information to facilitate clinical acceptance and translate the theory into practice, as an applicable diagnostic tool.

Multicenter Clinical Evaluation: Gathering Worldwide Input

To test our theory, data from 17 sites around the world was gathered in a massive effort involving 2,300 patients in a multicenter clinical evaluation. The first aim of the study was to determine a model, which could better assess the probability of the malignancy of lesions. This was done using a regression function that was calculated retrospectively on 1,000 cases. Each lesion studied had a conventional BIRADS assessment as well as an assessment of several morphological features from a shear wave elastography image.

Next, to improve the specificity of the test without compromising the sensitivity, a cutoff point was chosen for the value of the regression function and lesions falling above this cutoff were classified as requiring a biopsy. Therefore, any given lesion could theoretically be assessed calculating its explicit probability of malignancy. Although statistically valuable, this calculation was not a practical approach for radiology.

Our goal was to find simple rules based on ultrasound and shear wave elastography morphological feature assessments to classifying a lesion as probably benign, suspicious, or highly suspicious of malignancy. To accomplish this, each individual ultrasound and shear wave elastography feature was evaluated for its ability to predict malignancy. The best performing features would be good candidates to define a new BIRADS model.

Through a top-down and bottom-up approach, we were able to derive the probability of malignancy as a function of the ultrasound and shear wave elastography image parameters and define simple rules to give a BIRADS score for the lesion through an assessment of both ultrasound and shear wave elastography morphological features.

The complexity of this exercise was about making these two opposite approaches reach the same end point and define a feature-based score definition, which corresponds to ranges of probability of malignancies. Pragmatism, compromise, and a priori management were essential to reach this goal.

The rigor of this clinical investigation has been key to reaching our goal of demonstrating the clinical benefits of shear wave elastography. Today, the diagnostic impact of new products and innovations has to be tailored to specific clinical questions. This requires important statistical proof of clinical benefits and puts the burden on the industry to provide this information to validate their clinical innovation.


Claude Cohen-Bacrie is cofounder and scientific director of SuperSonic Imagine, Aix-en-Provence, France. Previously, he held senior scientist positions at Philips Research France and Philips Research USA. He is the recipient of 10 patents in the field of medical imaging and makes regular contributions to key scientific conferences in the field of ultrasound.