|Sharon E. Antiles, MPH
Speech-recognition (SR) technology has revolutionized report generation in radiology. Children’s Hospital Boston (CHB) successfully implemented SR in October 2002 and currently achieves a departmental use rate of 97%. The technology led to a 53% reduction in report turnaround time and anticipated annual transcription savings of $250,000.
Despite the known benefits of SR, it has met resistance in many facilities due to concerns over recognition accuracy, the learning curve, and the time required to dictate reports. CHB performed a cost-benefit analysis and concluded that the benefits of SR outweighed those of the conventional system. The goal was to make SR technology successful in a busy pediatric hospital.
CHB is a 300-bed pediatric teaching institution. The radiology department is staffed by attending radiologists, fellows, and residents. It conducts approximately 150,000 examinations per year, with an average report turnaround time (before SR) of 1.5 days. Annual transcription costs are approximately $250,000 and increase at a rate of 2% per year.
CHB considered using SR in order to minimize transcription costs and improve service to referring clinicians. Although CHB is the only pediatric teaching institution in the Boston area, many other hospitals and medical centers provide top-notch pediatric care with a high level of service. The radiology department was losing patients to other facilities that use SR and provide patient results within hours. Service is an important driver of radiology referrals, and CHB determined that SR technology, in conjunction with a picture archiving and communications system (PACS), would help the radiology department deliver results more quickly.
|Charles K. Hornberger, MBA
A 1999 attempt to deploy SR was unsuccessful for many reasons. The technology was not robust enough to be used in a busy clinical practice. Recognition accuracy was poor, macros and templates did not work as expected, no dedicated support team was created to implement the system, and no formal training plan was developed. A few SR workstations were located in prime reading locations, but few radiologists were willing to take the time to create the speech profile that was required to use the system. The vendor was unable to resolve many problems and did not maintain a strong presence during implementation.
Two years passed before the department was willing to retry SR. Radiology leadership made a concerted effort to transition to new technologies, such as PACS and speech-recognition, with the goal of being fully digital (with SR reporting) by spring 2003.
Radiology administrators needed to present a strong case to gain funding for SR technology. To develop the case, they analyzed the current state of the department, determined implementation requirements, and justified the acquisition financially. The department developed a detailed project plan with an achievable timeline (covering hardware delivery, software installation, interface development and testing, and training). The hospital quickly realized the potential benefits of SR, with an anticipated net present value of $297,022 and a breakeven point of 1.6 years. In years 1, 2, and 3 through 5, the department expected to achieve SR-use levels of 30%, 50%, and 80%, respectively.
Once the department had secured funding, it began to investigate potential vendors. Major technical conferences provided insight into current technologies; vendors demonstrated SR system features, allowing product comparison. Vendors of the two systems that seemed most suitable made on-site presentations. Their systems were evaluated based on the same criteria, which were weighted based on their relative importance to functionality and clinical care. The evaluation indicated that one technology was clinically superior to the other, and this was confirmed by consulting with sites that used both systems.
|Melissa R. Weis
An on-site team was created to plan all stages of the implementation and to be available after implementation for sustained support. This team was developed as an entity separate from the hospital’s information systems department (ISD). The team reports to the radiology director of informatics, who reports to the radiology vice-chair of informatics. There are four subteams: radiology information system (RIS)/SR, PACS, technical services, and web development. The three-member RIS/SR team is responsible for installation of the SR application, interface testing between the RIS and SR system, application training and support, and tracking performance. The technical service team is responsible for all hardware, including building PCs, installing sound cards, and deploying the workstations. The SR vendor handled the off-site configuration of the servers.
CHB radiologists actively participated in the planning, decision-making, and implementation processes, which led to a higher level of system acceptance. The RIS/SR team met bimonthly with the radiologist division heads in order to understand what aspects of their work flow should be preserved and how best to integrate the new technology into each area. Meetings focused on education about the new technology, other sites that were using the system successfully, and information gained from sites with poor implementations. Radiologists participated in the vendor evaluation and analysis and viewed on-site demonstrations. Renovation discussions also took place. The radiologists provided significant input on the direction of the project. The RIS/SR team began negotiations with the SR vendor to select hardware, negotiate pricing, and schedule implementation.
The vendor and CHB ISD recommended a primary server for production and a fail-over server for testing and backup. The failover server has the same specifications as the primary server. The department also ordered 23 PCs with flat-panel monitors and handheld microphones.
CHB renovated the ultrasound, body CT and MRI, neuroradiology, and main reading-room (where plain films are read) to optimize the environments for the SR and PACS implementations. Areas of focus included ergonomics, sound paneling, carpeting, and task lighting. Interventional radiology and nuclear medicine already had environments that were suitable for both systems.
Robust interfaces are needed for effective communications between systems. CHB routes the majority of its Health Level 7 communications through an interface engine. A bidirectional interface was configured so that the RIS would send orders to, and receive results from, the SR system. In addition, radiologists are able to review and sign all reports in either the SR system or the RIS. Once the RIS receives a report from the interface engine, it sends a copy of the report to the SR system. The databases are synchronized, and both systems display the same report information (Figure 1).
|Figure 1. Illustration describes interfaces required for effective communications between HIS, RIS, EMR, and PACS.
During extensive interface testing, the team communicated progress and anticipated transition dates to the department. The system received a significant amount of attention. Promotional posters were prominently displayed in all reading areas. Mouse pads were created outlining how to use SR. Presentations were given to the technologists and radiologists illustrating how the technology would change their work flow. Because examination completion in the RIS triggers orders to be sent to SR, the importance of completion was reinforced. Anticipated reductions in report turnaround times were emphasized, along with the need for radiologists to review reports prior to final verification.
Unsuccessful SR implementations lack cohesive, well-planned training programs. Typically, the vendor comes on-site for 1 to 2 weeks for user training. Radiologists are trained and may begin using the system; however, once the trainers leave, on-site support is usually sparse, and many sites do not have the resources to support the system adequately.
|Robert T. Bramson, MD
CHB chose to train its own radiologists. By providing training and support, the team was able to gain user confidence in its knowledge of the system. Because CHB residents/fellows rotate on a 3-month to 2-year basis, the vendor would have needed to come onsite frequently to provide continual training; this would have become very expensive. Instead, the vendor provided all system administrator training for the team that planned the training and rollout of SR at CHB.
Radiologist training was held 1 to 2 weeks before the transition date. Training sessions lasted 1.5 hours, with 30 minutes slated for voice enrollment and one hour for application training. Since users can only absorb a certain amount of information in one sitting, follow-up training sessions that included more advanced skills were held after 1 to 2 weeks of clinical SR use.
CHB’s goal was to reach high levels of SR use at the initial stages of the implementation. Radiologists’ initial impressions and feedback set the stage for the remainder of the deployment. A phased implementation by modality best suited the clinical environment. The RIS/SR team spent 3 to 4 weeks working with each division. The gastrointestinal (GI)/genitourinary (GU) area was first because of its relatively low examination volume, limited hours of operation, and small pool of radiologists who regularly rotated through the area. The low examination volume allowed the team to test the effectiveness of the interfaces and make minor modifications without significantly affecting the department’s work flow.
The team chose ultrasound as the second area for SR implementation. The examination volume is approximately five times greater than in GI/GU, and many of the radiologists had already been trained by rotating through the GI/GU area. In addition, ultrasound has a consistent flow of patients throughout the day and a slower pace than plain-film radiology.
|S.T. Treves, MD
As high levels of SR use were being achieved in GI/GU and ultrasound, the team set up SR workstations in the main reading room and the online reading area. Plain films are read in these two areas, accounting for 80% of the department’s examination volume. The SR stations were set up just prior to the start of the new resident rotation in January 2003. After the residents’ SR training, there was a significant increase in SR use, with the residents dictating nearly 100% of their reports using SR.
Transition dates for neuroradiology, body CT, and body MRI followed in January through March 2003. The rollouts became much easier as the majority of the staff completed training.
A continuous level of support helped increase SR use rates. During the initial stages of implementation, a member of the RIS/SR team was assigned to the designated reading room to work with each of the radiologists as they dictated their cases. This support helped the radiologists gain confidence by reinforcing the proper commands to use, the appropriate placement of the microphone, the best method for creating and inserting macros and templates, and the easiest way to add new words to the system’s vocabulary. The RIS/SR team was present in each reading area for 3 to 4 weeks, with spot checks on the area being held later. In addition, pager numbers were posted at every workstation so that the radiologists could page the RIS/SR team with questions or problems.
Off-hours support for nights, weekends, and holidays was critical. The team developed a rotating on-call coverage program that has two tiers: a primary person on call and a manager on call. The person on call varies weekly. When users need additional support, they page the primary person on call. If the primary person on call does not respond within 15 minutes, the user pages the manager on call.
A site may have a robust support plan, but systems inevitably have expected and unexpected downtimes. CHB has regularly scheduled monthly and quarterly downtimes. In addition, there may be other unexpected downtimes that could affect the RIS, network, interface engine, or SR servers. Consequently, contingency plans are needed in order to keep the department operational. To maintain access during downtimes, CHB has not removed its conventional dictation systems from the reading rooms. However, if SR use declines significantly, the team plans to remove the microphones from the conventional dictation devices to inhibit their use. This will allow the hardware and connections to remain in place; during downtimes, the microphones can be deployed quickly.
During a downtime, if radiologists cannot access orders in SR, they have two choices: revert to the conventional dictation system or create temporary orders in the SR system (which can be matched with the correct orders once all problems have been resolved). The matching can be done either manually or automatically. In the event of network problems, radiologists can also revert to the conventional dictation system to transmit information by telephone. One question that frequently arises is whether the conventional dictation system will eventually be removed; because of its importance during network downtimes, the department decided not to eliminate it.
Statistical analysis allows CHB to view trends in data and to measure progress. The department focuses on SR use statistics (by department, division, and radiologist) weekly and monthly. As seen in Figure 2, the department data show an increasing use trend, maintaining a mean use level from October 2003 to March 2004 of 96.8%.
|Figure 2. Departmental use of speech recognition.
Charting SR-use data by radiologist show the department its major SR contributors. Radiologists with low use levels can also be targeted for additional training and/or assistance. During FY03, CHB achieved transcription savings of $162,000. FY04 numbers through March 2004 show transcription savings of more than $133,000, with total projected FY04 savings of $250,000 (Figure 3).
Report turnaround time was measured from examination completion to report finalization. Reports excluded from this analysis were those dictated more than 24 hours after examination completion and reports with a total turnaround time of more than 5 days. These restrictions exclude approximately 15% of reports. Report turnaround time was separated into two categories: reports dictated by staff (only) and reports dictated by fellows and residents with staff review. Pre-SR times were tabulated for the year beginning in September 2001, with the transition to SR taking place in October 2002. SR times were measured from April 2003 through March 2004 in order to capture data once the SR-use rate had reached a plateau.
|Figure 3. Monthly savings.
The staff report turnaround time dropped from 1.2 days to 0.2 days using SR, a reduction of 83.3% (Figure 4). After SR implementation, a greater number of reports were finalized within 24 hours, increasing from a pre- to post-SR mean of 69.6% to 92.2%, respectively. Turnaround times for reports made by fellows and residents and reviewed by staff radiologists dropped from 1.5 days to 0.6 days, a reduction of 60% (Figure 5). Similarly, 61.7% of reports before SR and 81.7% of reports after its implementation were finalized within 24 hours of examination completion.
When CHB embarked on this project, it expected 30% departmental SR use in year 1, 50% use in year 2, and 80% use in years 3 through 5. In year 1, it achieved a use rate of up to 96%, exceeding its estimate by 220%. CHB currently maintains a use rate of 97+% and continues to find ways to improve the existing system. The department attributes its success to the thorough rollout and continued support provided by the RIS/SR team. CHB’s next area of focus will be upgrading to the next SR version and the integration of SR and PACS.
|Figure 4. Staff report turnaround time.
|Figure 5. Fellow and resident report turnaround time with staff review.
Sharon E. Antiles, MPH, is the RIS-SR manager, Children’s Hospital Boston.
Charles K. Hornberger, MBA, is an RIS-SR application specialist at the hospital.
Melissa R. Weis is an RIS-SR application specialist at the hospital.
John Speziale is director of radiology information systems at the hospital.
Robert T. Bramson, MD, is executive vice-chairman and associate radiologist-in-chief at the hospital.
S.T. Treves, MD, is chief of the division of nuclear medicine and the vice-chairman of radiology information systems at the hospital.