J. Marc Overhage, MD, PhD

Practicing medicine means tracking the care that is provided to patients and the outcomes that are achieved, with the aim of improving the care of future patients. Inherent in this is the concept of learning, particularly from the physician’s own patients and practice. Every patient has something new to teach the clinician.

How can a physician routinely generate valid data from his or her practice in order to learn? How can he or she incorporate medical knowledge into a practice, not just from the medical literature, but from colleagues who treat similar patients in the same institution? Health care is information science (Brent James written communication, April 1995).

Clinical information systems (CIS) have demonstrated the potential to improve the efficiency and quality of health care. This improvement results from greater availability of clinical data and medical knowledge at the point of care. Clinicians make better decisions when they have access to complete, organized patient data, and their efficiency is enhanced when (in addition to having ready access to the data at the point of care) they are alerted to important information as it becomes available. Greater availability of medical knowledge addresses deficits such as limited adoption of criteria for the appropriate use of tests, 1 physicians’ limited risk-estimation capacity, 2-4 difficulty sustaining interventions (such as the review of utilization by senior physicians), 5,6 and problems delivering feedback in the timely fashion that is likely to be most effective. 5

The ways that a CIS can improve the efficiency and quality of health care fall into six categories:

  • it makes organized patient data readily and reliably available;
  • it can process patient data and remind clinicians of steps that should be taken for the patient, as well as warning them of interventions that are being used, but should not be;
  • it improves the efficiency of many health care processes, making them faster and more reliable;
  • it can make medical knowledge more readily available;
  • it makes measuring the effectiveness of clinicians’ activities easier; and
  • it facilitates clinical research.

Data Availability

A CIS makes organized patient data readily and reliably available. The quantities of patient data available to clinicians, and the pace at which they are expected to review and integrate these data, continue to increase. This increase has driven the need for faster, more reliable access to data. In traditional care settings, patient charts are not available when needed 28% of the time. 1-3 Often, they do not contain the needed information (or they contain erroneous information) even when they are available. Physicians spend 33% to 50% of their time managing information; nurses, 50%.7,8 By improving data availability and presenting data in an organized fashion, a CIS can reduce unnecessary testing in the emergency department 9,10 and other settings. 11-15 Having too much (or poorly organized) information is little better than having insufficient information. 16,17 Recent developments in information technology and clinical information standards allow nearly all patient data, including radiographic images and their interpretations, to be integrated. 18 These systems also improve the continuity of care and simplify transferring a patient’s care from one provider to the next.

CIS can remind and alert clinicians. There is evidence of the misuse, overuse, and underuse of medical interventions. The publication of To Err is Human by the Institute of Medicine 19 increased the visibility of what are perceived as medical errors, primarily misuse. Computerized physician-order entry (CPOE) systems that provide clinical decision support can substantially reduce medication error rates and improve the quality and efficiency of medication use. Researchers 4 have also demonstrated that clinical decision support can result in major improvements in rates for antibiotic-associated adverse drug events and can decrease costs. Researchers 5-8 have also demonstrated, in a series of studies, that nosocomial infection rates can be reduced using decision support. More recently, a study9 found that CPOE reduced serious medication errors by 55% in a randomized controlled trial. In another time-series study, 23 they found that even a simple CPOE system reduced overall medication error rates by 83%. There may be even a greater opportunity to improve care by reducing errors of omission. For example, there is an extensive body of evidence that a CIS can improve the use of preventive services. Several systematic reviews 20,22-24 summarize many of the studies of clinical decision support (CDS) systems in outpatient settings. The most commonly tested effects included cancer-screening compliance rates, 21,25 vaccination rates, 26-29 blood-pressure measurements, 30-32 use of laboratory tests, 27,33-36 prenatal-screening rates, 37,38 and medication-monitoring rates. 39-41

The evidence for overuse and underuse is equally compelling. An area of research called small-area-variation analysis has consistently demonstrated that the way in which care is delivered varies dramatically, not only from region to region and town to town, but from physician to physician (and even from day to day for the same physician). All these variations cannot represent optimal use of resources. Differences in care delivery that can be explained not by differences in patient characteristics but, rather, by differences in the availability of clinical resources (MRI, inpatient beds, and outpatient surgical centers) and local clinical practices are troubling. Variation destroys trust between providers and patients, causes payors to question practices, creates waste through inefficiencies, and increases the risk of errors. A number of factors account for this variation.

The complexity of care is one factor. The human mind can only include about seven factors in decision making due to limitations on how many factors can be held in short-term memory. 42,43

A second factor is the lack of clear evidence to guide practice. The presence of this variation has led to the evidence-based medicine movement, in which physicians are encouraged to make clinical decisions based on evidence available in the literature, rather than on their opinions or those of others. We know that expert opinion is a poor substitute for evidence. In the early 1980s, Eddy, a surgeon and mathematician, convened groups of experts in various fields in an effort to define correct clinical practices. What was astounding was that these experts not only failed to agree, but also could reach conclusions that were diametrically opposed. When they were asked to estimate the risk of death for a certain surgical procedure, for example, their estimates ranged from 3% to 95%. For other questions, the range was even wider. Unfortunately, despite the frequent need for information (with two questions generated for every three patients seen), practitioners have only 30 minutes per week to review evidence. The need for information is only going to expand, with the amount of medical knowledge estimated to double in size every 19 years, and to quadruple within most practitioners’ lifetimes. Despite this rapid growth, physicians still practice in an information vacuum. In several careful analyses,44-46 clear evidence could be found in the literature for only 20% of the decisions made by clinicians; half of that evidence showed that they chose the wrong things to do.

More recently, Andrew Balas, MD (written communication, March 2001) combined data from a number of studies to demonstrate that it takes 17 years for 14% of research knowledge to be put into clinical care. Physicians also adopt unproven methods in their efforts to do what is best for patients. 47

A third factor resulting in variation is physicians’ dependence on subjective judgment. Many clinicians treasure this subjectivity, calling it clinical judgment or the art of medicine, but they cannot learn what is needed to improve practices and patients’ outcomes when what they do varies based on subjective impressions. There is no doubt that decisions need to be highly tailored to the individual patient, but this tailoring needs to consistent.

The fourth factor is human error. Physicians simply make mistakes. As early as 1976, McDonald stated, “I conclude that though the individual physician is not perfectible the system of care is and that the computer will play a major part in the perfection of future care systems.” 35

Improvements in the care process, and consequent improvements in patient outcomes, should be expected from the use of CDS systems because physicians control 75% of all health care costs, bear primary responsibility for quality, and are the focal point for information collection. They perform these functions in a difficult environment; they must synthesize medical knowledge with the patient’s data and integrate the impact of the costs and benefits of diagnostic and treatment options in order to decide on an intended course of action. Carrying out these intentions may be hindered by a variety of barriers. Competing priorities, such as dealing with the patient’s presenting symptoms (usually the primary task of an outpatient encounter), distract attention from secondary tasks such as delivering preventive care or managing chronic diseases. A system for organizing, processing, and presenting relevant information at critical times may reduce physicians’ mental workloads and direct attention to tasks that they might otherwise overlook. 48

Clinicians follow a variety of decision-making strategies 49 (including hypothesis testing, 50 scripts, 51 and heuristics 52 ), rather than using reasoned models of clinical decision making, such as the threshold model of Pauker and Kassirer. 53 Information systems can help design complex models and can provide other information relevant to the clinical context (patient, setting, disease, test, and drug). 54 The primary role of a CDS system is to bring precisely the right information to the clinician’s attention in a highly usable format at exactly the right moment. This information includes not just the patient’s data, but medical knowledge.

Medical knowledge is expanding rapidly. Of all the randomized controlled trials indexed in MEDLINE, 1% were published between 1966 and 1971 and 49% were published between 1991 and 1995. 55 During 2000, more than 8,000 new citations were added to MEDLINE each week, but these accounted for only about 40% of biomedical publications worldwide. 56 Both the primary medical literature and secondary literature (such as guidelines and textbooks) are increasingly available electronically. More important, highly summarized, constantly updated medical knowledge is beginning to become available in the form of Internet information products.

The information-related reasons for inappropriate resource utilization can be addressed by combining a CPOE system used by physicians with a computerized data-review and reminder system that provides needed information when decisions are made, makes suggestions, and challenges orders when a potential problem is found. Specifically, using the computer to provide feedback and reminders to physicians is reliable and inexpensive, compared to the manual review of practices. In addition, order entry is immediately generalizable to all physicians. Once in place, it requires little maintenance and can be continued indefinitely. Most important, CPOE allows immediate feedback to physicians at the time that they order tests. To be optimally effective, an intervention should occur as close in time to the target event as possible, and it should be constructive and nonjudgmental. 57 Computerized feedback is ideal in both regards. Because physicians use their unique identification numbers to access the system, it is possible to track an individual physician’s behavior before and after an intervention designed to affect such behavior.

Improving Efficiency

CPOE reduces the time that elapses before patients admitted to the hospital receive the first dose of their medications. 58 Reducing this time may improve outcomes in patients with pneumonia and other conditions. A CIS can consistently and rapidly alert clinicians to abnormal results; this can lead to improvements in patient-care processes. 59

Cost reductions are the result of reducing medical errors and adverse events, of recommending the use of equally effective (but less costly) alternative interventions, of reducing the use of inappropriate tests, and of decreasing the ordering of redundant tests. Diagnostic tests account for a large share of total health care expenditures, and critics charge that they appear to have little effect on treatment. 60-62 Physicians are often unaware of the costs of these diagnostic tests. Further, they seldom know the probability of a positive test result. 63,64 They have no clear plan for using a test result to inform their therapeutic decisions.

There are also substantial costs attributable to information transfer in health care; some estimates of its cost are as high as 17% of all health care expenditures. A few health care delivery systems have demonstrated cost savings of 10% by installing CIS.

A CIS makes it easier to measure the effectiveness of what clinicians do. Providers want to deliver high-quality care. That is why they started practicing medicine in the first place. Measuring quality without automated tools is time consuming and labor intensive, yet the focus on lowering costs while maintaining or improving quality demands routine measurement. 65 Interventions to reduce costs and improve quality may also be most successful if they are focused at the level of individual decisions, yet are nonintrusive (a difficult combination to achieve). Fortunately, information technologies can help with both quality measurement and quality improvement. For quality measurement, information systems represent an inexpensive way to collect information on all patients, rather than the sample often required for chart review. Furthermore, such information can be grouped readily in different ways and manipulated. Perhaps most important, this information can be used directly to improve quality: for example, to contact all eligible patients who have not undergone a given preventive measure.

A CIS facilitates nearly all phases of clinical research. There are broad applications of CIS for clinical research. 66,67 If the clinical content is rich, the number of subjects eligible for prospective trials can be accurately estimated, assisting researchers greatly in study design and planning (as well as recruiting). Once subjects have been recruited, CDS systems can assist clinicians in following study protocols. Collection and abstraction of the necessary data are also greatly simplified by CIS. Extensive use has been made of CIS for retrospective studies, as well. 68 For example, data in the medical record system of the Regenstrief Institute for Health Care, Indianapolis, were recently used to carry out a retrospective study that demonstrated a relationship between treatment with erythromycin and pyloric stenosis in infants. 69


CIS have the potential to improve the quality and efficiency of health care by improving access to clinical data and medical knowledge, as well as by providing clinical information support that combines these data and this knowledge to help clinicians provide better care for their patients. CIS are necessary-but not sufficient, alone-to achieve any of these benefits. Changes in other aspects of the health care delivery system, including compensation models, are equally important.

J. Marc Overhage, MD, PhD is senior scientist, Regenstrief Institute for Health Care, Indianapolis, and associate professor of medicine, Indiana University School of Medicine, Indianapolis.


  1. Grossman RM. A review of physician cost-containment strategies for laboratory testing. Med Care. 1983;21:783-802.
  2. Poses RM, Cebul RD, Collins M, et al. The accuracy of experienced physicians’ probability estimates for patients with sore throats. JAMA. 1985;254:925??”929.
  3. Hlatky M, Botvinick E, Brundage B. Diagnostic accuracy of cardiologists compared with probability calculations using Bayes’ rule. Am J Cardiol. 1982:49;1927??”1931.
  4. Fryback DG, Thornbury JR. Evaluation of a computerized Bayesian model for diagnosis of renal cyst vs. normal variant from excretory urogram information. Invest Radiol. 1976;7:102-111.
  5. Eisenberg JM, Williams SV. Cost containment and changing physicians’ practice behavior. Can the fox learn to guard the chicken coop? JAMA. 1981;246:2195??”2201.
  6. Soumerai SB, Avorn J. Efficacy and cost-containment in hospital pharmacotherapy: state of the art and future directions. Milbank Q. 1984;62:447??”474.
  7. Wipf JE, Fihn SD, Callahan CM, Phillips CM. How residents spend their time in clinic and the effects of clerical support. J Gen Intern Med. 1994;9:694??”696.
  8. Smith DM, Martin DK, Langefeld CD, Miller ME, Freedman JA. Primary care physician productivity: the physician factor. J Gen Intern Med. 1995;10:495??”503.
  9. Wilson GA, McDonald CJ, McCabe GP. The effect of immediate access to a computerized medical record on physician test ordering : a controlled clinical trial in the emergency room. Am J Public Health. 1982;72:698-702.
  10. Stair TO. Reduction of redundant laboratory orders by access to computerized patient records. J Emerg Med. 1998;16:896-897.
  11. Neumann PJ, Parente ST, Paramore LC. Potential savings from using information technology applications in health care in the United States. Intl J Technol Assess Health Care. 1996;12:425-435.
  12. Hammond KW, Prather RJ, Dave VV, King CA. A provider-interactive medical record system can favorably influence costs and quality of medical care. Comput Biol Med. 1990;20:267-279.
  13. Kian L, Stewart M, Bagby C, Robertson J. Justifying the cost of a computer-based patient record. Healthcare Financial Management. 1995:58-67.
  14. Evans JC, Hayashi AM. Implementing on-line medical records. Document Management. 1994;Sept/Oct:12-17.
  15. Tierney WM, McDonald CJ, Martin DK, et al. Computerized display of past test results: effect on outpatient testing. Ann Intern Med. 1987;107:569-574.
  16. Svensson E, Angelborg-Thanderz M, Sjoberg L, Osson S. Information complexity??”mental workload and performance in combat aircraft. Ergonomics. 1977;40:362-380.
  17. TRW uses AI tools to control system problems. Aviation Week and Space Technology. 1986:79-81.
  18. Overhage JM, Aisen A, Barnes M, Tucker M, McDonald CJ. Integration of radiographic images with an electronic medical record. Proc AMIA Symp. 2001;513-617.
  19. Institute of Medicine. To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999.
  20. Haynes RB, Walker CJ. Computer-aided quality assurance. A critical appraisal. Arch Intern Med. 1987;147:1297??”1301.
  21. Banks NJ, Palmer RH, Kane NM, Braun OP, Feldstein ML, Harrington AM. Implementation and evaluation of a computerized reminder system in ambulatory care. Proceedings of the Annual Symposium on Computer Applications in Medical Care. 1988; 2:753??”757.
  22. Balas EA, Austin SM, Mitchell JA, Wigman E. The clinical value of computerized information services: a review of 98 randomized clinical trials. Arch Fam Med. 1996;5:271??”278.
  23. Johnston ME, Langton KB, Hayes B, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research. Ann Intern Med. 1994;120:135??”142.
  24. Sullivan F, Mitchell E. Has general practitioner computing made a difference to patient care? A systematic review of published reports. BMJ. 1995;311:848??”852.
  25. Chambers CV, Balaban DJ, Carlson BL, Ungemack JA, Grasberger DM. Microcomputer-generated reminders: improving the compliance of primary care physicians with mammography screening guidelines. J Fam Pract. 1989;3:273??”280.
  26. Brimberry R. Vaccination of high-risk patients for influenza: a comparison of telephone and mail reminder methods. J Fam Pract. 1988;26:397??”400.
  27. McDonald CJ, Hui SL, Smith DM, Tierney WM, Cohen SJ, Weinberger M, McCabe GP. Reminders to physicians from an introspective computer medical record: a 2-year randomized trial. Ann Intern Med. 1984;100:130??”138.
  28. Ornstein SM, Garr DR, Jenkins RG, Rust PF, Arnon A. Computer-generated physician and patient reminders: tools to improve population adherence to selected preventive services. J Fam Pract. 1991;32:82??”90.
  29. Soljak MA, Handford S. Early results from the Northland immunisation register. N Z Med J. 1987;100:244??”246.
  30. Barnett GO, Winickoff RN, Morgan MM, Zielstorff RD. A computer-based monitoring system for follow-up of elevated blood pressure. Med Care. 1983;21:400??”409.
  31. Burling TA, Marotta J, Gonzalez R et al. Computerized smoking cessation program for the worksite: treatment outcome and feasibility. J Consult Clin Psychol. 1989;5: 619??”622.
  32. Rogers JL, Haring OM, Goetz JP. Changes in patient attitudes following the implementation of a medical information system. QRB. 1984;10(3):65??”74.
  33. Rogers JL, Haring OM, Watson RA. Automating the medical record: emerging issues. Proceedings of the Annual Symposium on Computer Applications in Medical Care. 1979;3:255??”263.
  34. Berwick DM, Coltin KL. Feedback reduces test use in a health maintenance organization. JAMA. 1986;255:1450??”1454.
  35. McDonald CJ. Protocol-based computer reminders, the quality of care and the nonperfectibility of man. N Engl J Med. 1976;295:1351??”1355.
  36. McDonald CJ, Wilson GA, McCabe GP. Physician response to computer reminders. JAMA. 1980;244:1579??”1581.
  37. Morgan M, Studney DR, Barnett GO, Winickoff RN. Computerized concurrent review of prenatal care. QRB. 1978;4(9):33??”36.
  38. Hofmeyr GJ, Pattinson R, Buckley D, Jennings J, Redman CW. Umbilical artery resistance index as a screening test for fetal well-being: randomized feasibility study. Obstet Gynecol. 1991;78:359??”362.
  39. McDonald CJ. Use of computer to detect and respond to clinical events: its effect on clinician behavior. Ann Intern Med. 1976;84:162??”167.
  40. Schrezenmeir J, Achterberg H, Bergeler J. Controlled study on the use of hand-held insulin dosage computers enabling conversion to and optimizing of meal-related insulin therapy regimens. Life Support Systems. 1985;3(suppl):561??”567.
  41. White KS, Lindsay A, Pryor TA, Brown WF, Walsh K. Application of a computerized medical decision-making process to the problem of digoxin intoxication. J Am Coll Cardiol. 1984;4:571??”576.
  42. Miller GA. The magic number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev. 1956;63:81-97.
  43. Morris AH. Developing and implementing computerized protocols for standardization of clinical decisions. Ann Intern Med. 2000;132:373-383.
  44. Williamson JW, Goldschmidt PG, Jillson IA. Medical Practice Information Demonstration Project: Final Report Office of the Assistant Secretary of Health, DHEW. Baltimore, Md: Policy Research Inc; 1979.
  45. Institute of Medicine. Assessing Medical Technologies. Washington, DC: National Academy Press; 1985:5.
  46. Ferguson JH. Research on the delivery of medical care using hospital firms: proceedings of a workshop. Med Care. 1991;29(suppl):JS1-JS2.
  47. Chassin MR, Galvin RW. The urgent need to improve health care quality: Institute of Medicine national roundtable on health care quality. JAMA. 1998;280:1000-1005.
  48. Powsner SM, Wyatt JC, Wright P. Medical records: opportunities and challenges of computerisation. Lancet. 1998;352:1617??”1622.
  49. Wyatt J. Use and sources of medical knowledge. Lancet. 1991;338:1368??”1373.
  50. Elstern AS, Shulman LS, Sprafka SA. Medical Problem Solving. Cambridge, Mass: Harvard University Press; 1978.
  51. Patel V, Evans D, and Green G. Biomedical knowledge and clinical reasoning. In: Evans D, Patel V, eds. Cognitive Science in Medicine. Cambridge, Mass: MIT Press; 1986
  52. McDonald CJ. Medical heuristics: the silent adjudicators of clinical practice. Ann Intern Med. 1996;124:56??”62.
  53. Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med. 1980;302:1109??”1117.
  54. McDonald CJ, Overhage JM, Tierney WM, Abernathy GR, Dexter PR. The promise of computerized feedback systems for diabetes care. Ann Intern Med. 1996;124:170??”174.
  55. Chassin MR. Is health care ready for Six Sigma quality? Milbank Q. 1998;76:565-591.
  56. National Library of Medicine. Fact sheet: MEDLINE. Available at: http://www.nlm.nih.gov/pubs/factsheets/medline.html. Accessed January 9, 2002.
  57. Restuccia JD. The effect of concurrent feedback in reducing inappropriate hospital utilization. Med Care. 1982;20:46??”62.
  58. National Library of Medicine. Fact sheet: MEDLINE. Available at: http://www.nlm.nih.gov/pubs/factsheets/medline.html. Accessed January 9, 2002.
  59. Restuccia JD. The effect of concurrent feedback in reducing inappropriate hospital utilization. Med Care. 1982;20:46??”62.
  60. Tierney WM, Miller ME., Overhage JM, McDonald CJ. Physician inpatient order writing on microcomputer workstations. Effects on resource utilization. JAMA. 1993;269:379-383.
  61. Hubbel FA, Greenfield S, Tyler JL, Chetty K, Wyle FA. The impact of routine admission chest x-ray films on patient care. N Engl J Med. 1985;312:209??”212.
  62. Tierney WM, McDonald CJ, Hui SL, Martin DK. Computerized predictions of abnormal test results. JAMA. 1988;259:1194??”1198.
  63. . Tierney WM, Miller ME, McDonald CJ. The effects on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med. 1990;322:1499??”1504.
  64. Rhyne RL, Gehlbach SH. Effects of an educational feedback strategy on physician utilization of thyroid function panels. J Fam Pract. 1979;8:1003??”1007.
  65. Bates DW, Pappius E, Kuperman GJ, et al. Measuring and improving quality using information systems. Medinfo. 1998;2:A14??”A15.
  66. Tierney WM, McDonald CJ. Practice databases and their uses in clinical research. Stat Med. 1991;10:541-557.
  67. Tierney WM, Miller ME, Hui SL, McDonald CJ. Practice randomization and clinical research: the Indiana experience. Med Care. 1991;29:JS57-JS64.
  68. Measuring quality, outcomes, and cost of care using large databases. Proceedings of the 6th Regenstrief Conference. Ann Intern Med. 1997;127:665-774.
  69. Mahon BE, Rosenman MB, Kleiman MB. Maternal and infant use of erythromycin and other macrolide antibiotics as risk factors for infantile hypertrophic pyloric stenosis. J Pediatr. 2001;139:380-384.