The promise, mandate and controversy over clinical decision support software. Part 1 in a two-part series. 

By Teri Yates

Teri Yates, Accountable Radiology Advisors

Teri Yates, Accountable Radiology Advisors

There have been a variety of mechanisms employed by payors in recent years to control imaging utilization and costs, including prior authorization, radiology benefit management, and bundling payment for multiple examinations performed on the same day. In parallel with these payor-driven strategies, initiatives launched by professional medical societies also have been deployed to promote more appropriate use of diagnostic imaging. These include campaigns such as Image Gently to reduce unnecessary radiation exposure in children, or Choosing Wisely to encourage patients to question and avoid imaging procedures with no clear clinical benefit.

All imaging providers have been impacted to varying degrees by these programs, and there is more change on the horizon in the form of mandated use of clinical decision support (CDS) software. CDS is a tool to educate providers at the point-of-order about the clinical appropriateness of their proposed use of diagnostic imaging. The CDS software, which may be integrated with the electronic health record order entry system or accessed through a stand-alone web portal, analyzes details of the patient’s condition to determine if the radiology procedure ordered is indicated. This determination of appropriateness is based on the specific guidelines that are employed in the system; in circumstances where an order is not appropriate, the software also provides the ordering physician with recommendations about what alternatives are indicated given the clinical scenario.

CMS Mandate Based on a Failed Experiment

Beginning on January 1, 2017, the Centers for Medicare and Medicaid Services (CMS) will require as a condition of payment that providers consult appropriate use criteria via CDS before ordering high cost, high tech imaging procedures for Medicare beneficiaries in outpatient settings.1 Because CMS does not have a prior authorization requirement for imaging, the implementation of CDS has the potential to significantly reduce imaging utilization. Providers experienced in both the benefits and challenges of implementing CDS point out that the devil will be in the details.

Keith Hentel, MD

Keith Hentel, MD

“Decision support has the potential to be extremely helpful, but done wrong, it can be just as harmful,” said Keith Hentel, MD, chief of emergency/musculoskeletal imaging and vice-chairman for radiology clinical operations at New York Presbyterian Hospital. New York Presbyterian was one of the participants in the 2-year Medicare Imaging Demonstration Project (MID), which sought to evaluate the baseline appropriateness of advanced imaging utilization and the impact of CDS on physician ordering between 2011 and 2013.2 “When we rather enthusiastically began participation in the MID, we turned on clinical decision support for a long list of CPT codes and decided that what was good for Medicare patients would be good for all patients and deployed it for everyone,” said Hentel. “The experience was brutal, to say the least.”

Hentel shares that one cause of the hospital’s frustration was the lack of a robust integration between the CDS software and the EHR software version that was in use at the time, which necessitated significant entry of additional information by the referring physician during the ordering process. Another problem was the high percentage of examinations ordered for which no recommendation about appropriateness was provided after the effort to input the information was invested.

New York Presbyterian’s experiences with the MID were not unique, and in its report to Congress on the results, the RAND Corporation noted that only 35% of 139,757 imaging orders placed during the project could be rated for appropriateness based on the evidence utilized.2 This included the American College of Radiology (ACR) Appropriateness Criteria as well as guidelines from other medical specialty societies. RAND also found that there were some improvements in the quality of orders over time, but that other factors (like increased percentages of unrated orders) negated the apparent positive impact of CDS in the project. The report essentially concluded that CDS annoyed referring physicians but didn’t produce a clear improvement in the appropriateness of imaging orders.

Moving Past the MID

While these results might give many hospitals and lawmakers pause about the value of CDS, Congress had already mandated its future use as part of the Protecting Access to Medicare Act (enacted April 1, 2014) by the time RAND presented its findings that CDS did not improve imaging appropriateness during the MID. Proponents are quick to point out, however, that CDS technology has come a long way since the project.

National Decision Support Company (NDSC) is the exclusive distributor of the ACR’s guidelines for appropriate imaging for use in CDS applications. Bob Cooke, vice-president of marketing for NDSC, explains that the impact of CDS today is tremendously different than it was just a few years ago. “The MID was a good representation of the state of the technology at that time, but the integration and criteria mapping with the EHR systems was not ideal. Additionally, less than half of the available clinical scenarios were included in the MID, leading to a high level of unsecured events.”

Keith Dreyer, DO, PhD

Keith Dreyer, DO, PhD

The depth and volume of the ACR Appropriateness Criteria content has also grown since the MID, according to Keith Dreyer, DO, PhD, vice chairman of radiology at Massachusetts General Hospital and chair of the ACR Informatics Commission. “MID used 2009 content. This was paper criteria and not designed for deployment into EHR. Since then, we have spent an incredible amount of time to develop the criteria in a manner that is amenable to being mapped into EHR.”

According to Dreyer, the ACR is also working with other specialty societies like the American College of Cardiology to improve and expand the Appropriateness Criteria, and believes that there are few genuine conflicts in guidance about how and when to use imaging. “I’ve looked at 100 guidelines from 45 different societies. All of the areas that overlap are very similar in their message.”

Dreyer asserts that the ACR content has now reached the “good enough” level. “What we are seeing in the field is over 90% coverage of the clinical scenarios, not just for the exams included in the CMS mandate, but inclusive of all high cost imaging examinations. The question is, do you need to get to 100% to have an acceptable solution for providers? It appears from the market that a 90% coverage of clinical conditions is quite effective.”

Content Is the Critical Issue

Though Dreyer asserts that the Appropriateness Criteria covers 90% of the clinical scenarios, the work of others suggests that the remaining areas may also be quite important. W. Christopher Baughman, MD, is the associate director of clinical informatics for the MetroHealth System in Cleveland. While planning his institution’s implementation of clinical decision support, which will go live this month, he encountered what he characterized as “gaps” in the ACR content. Baughman describes the problems they faced when working on the first clinical scenario they hoped to target with the effort.

“We are implementing decision support in a very gradual manner to help ordering physicians get used to the new workflow before it is more broadly applied in 2017. We decided to first focus on CT for pulmonary embolism [PE] because there is excellent evidence supporting the use of the modified Well’s criteria to calculate a pretest probability for acute pulmonary embolism.” When making the determination to implement CDS for this indication, Baughman also considered the fact that several of the major professional societies (including the ACR) have recommended against imaging patients with a low pretest probability for pulmonary embolism in their Choose Wisely recommendations.

As part of the planning process, he spoke with members of the medical staff at MetroHealth to determine whether local best practices might conflict with the available evidence. On pulmonary embolism, Baughman said that there wasn’t really any disagreement. “The use of the modified Wells’ criteria in conjunction with D-dimer results or a PE rule-out criteria (PERC) score was supported by representatives of the ER, Pulmonology, and the Hospitalists.”

When investigating how to act upon that consensus, Baughman determined that the ACR Select content would not meet their needs. “We found that the ACR Select content did not allow for calculation of a Wells’ score, so we decided to not use it for that study. Instead we used Medicalis to build custom content that would allow for calculation of a pretest probability of PE based on clinical history, signs and symptoms, and D-dimer results. The system also allowed us to build separate logic pathways to address pregnant patients based on recommendations from the American Thoracic Society as the use of the Wells’ criteria and D-dimer has not been validated in pregnancy.”

The thoroughness of the ACR content is at issue because soon it could be deemed the standard by which CDS is implemented for Medicare beneficiaries. In what may be the most critical element of the law’s implementation, CMS must release its determination of what constitutes acceptable appropriate-use criteria (AUC) by November 15, 2015. Any AUC that CMS selects must meet the following specifications:

  • Criteria must be developed or endorsed by national professional medical specialty societies or provider-led entities
  • Criteria must be scientifically valid and evidence-based
  • Criteria must be based on published studies that are reviewable by stakeholders
Bob Cooke, Vice President of Marketing, NDSC

Bob Cooke, Vice President of Marketing, NDSC

While it is possible that CMS could restrict the AUC to the ACR criteria, spokespeople for the ACR and NDSC indicated that access to other criteria is also important. Cooke said, “Our experience [at NDSC] has been that there is absolutely a need for institutions to adapt their CDS to local requirements and needs. …We believe that the ultimate implementation of the legislation should also accommodate locally created criteria provided that the methodology used to create content is transparent and verifiable.” Dreyer echoed this position, saying, “I think that the way that this should play out is that anybody should be able to use any guidelines that they prefer, provided they meet a criteria of national acceptability. The CDS system should allow providers to select which guidelines they want to use.”

One potential challenge for hospitals implementing CDS is how to keep the evidence as current as possible. CMS will review and potentially modify the AUC on an annual basis. It is likely that without a more frequent mechanism to update the criteria, in some institutions CDS will drive different utilization of imaging services among Medicare beneficiaries than in other patient populations.

For example, Hentel recently read an article published in a peer-reviewed journal about the most effective use of head CT in emergency department patients presenting with syncope and dizziness. The researchers had determined that most scans for these indications were negative, and identified a small subset of patients for whom imaging is most likely to be of benefit.3

Hentel wrote a new decision support rule for New York Presbyterian immediately after reading the article, and plans to bring it to a multidisciplinary committee for approval and implementation within weeks. He sees the ability to immediately act upon information like this right away as one of the most valuable aspects of CDS. “If you are going to go to the trouble to implement decision support, then you should use it to improve your practice every way you can. The fact that I could create this rule 3 hours after learning new information about best practices is very powerful.”

The Least Expensive CDS Option: Free      

The legislative requirement includes a provision that CMS must ensure there is at least one free form of CDS available to referring providers, which will likely take the form of a stand-alone web portal that providers can use to enter clinical information and get advice on appropriateness in return. While not optimal in terms of workflow, this type of solution will enable facilities without EHR (or the capability to integrate CDS with their existing EHR) to still comply with the mandate.

The ACR advocated for a free option, and plans to make a web-based CDS portal available at no charge to users. Dreyer explains why the ACR will give away what they also charge for providing through NDSC. “NDSC was a solution to achieve integration between the Appropriateness Criteria and EHR systems, and was not intended to control access to the information. We have invested millions in developing the AC, and we want it to be available to help people make the right decisions about imaging.”

The ACR has not provided the date that the complementary CDS portal will become available; however, all CDS vendors must gain approval for their solutions prior to April 1, 2016.

Getting Ready for the Mandate

Even If the experts have some consternation about how the MID was structured, those concerns haven’t dampened their optimism about the positive impact that CDS will have in the future. Hentel indicates that CMS is performing due diligence with the MID convener groups to ensure the future impact is meaningful, and he encourages hospitals to adopt the technology. “I’m a big believer in clinical decision support, and I believe it is good to be on the front side of the curve when it comes to implementation.”

Cooke’s position is that CDS is the right thing to do, regardless of what is mandated by CMS. “We have to remember what’s most important. By implementing CDS, it is an important opportunity for radiologists to guide appropriate care for patients rather than leaving those decisions up to payors that are not involved in the care of the patient. CDS is a tremendous opportunity for radiologists to participate in improving care and outcomes for the patient.”


To find out about lessons learned from institutions that have already implemented CDS, look for Part 2 of this article coming soon.

Teri Yates is the Founder and Principal Consultant for Accountable Radiology Advisors, a consulting practice that helps radiologists and hospitals deliver services of higher value. You may contact her directly at [email protected] or visit for more information.



  1.  Protecting Access to Medicare Act of 2014. (2014, April 1). Protecting Access to Medicare Act of 2014. Accessed February 28, 2015.
  2.  RAND Corporation. Medicare Imaging Demonstration Final Evaluation: Report to Congress. Santa Monica, Calif: RAND Corporation; 2014.
  3.  Mitsunaga MM, Yoon HC. Head CT scans in the emergency department for syncope and dizziness. Am J Roentgenol. 2015;204:24-28.