A new implicit review instrument for measuring quality of care delivered to pediatric patients in the emergency department
BMC Emergency Medicine - 2007
Tóm tắt
There are few outcomes experienced by children receiving care in the Emergency Department (ED) that are amenable to measuring for the purposes of assessing of quality of care. The purpose of this study was to develop, test, and validate a new implicit review instrument that measures quality of care delivered to children in EDs. We developed a 7-point structured implicit review instrument that encompasses four aspects of care, including the physician's initial data gathering, integration of information and development of appropriate diagnoses; initial treatment plan and orders; and plan for disposition and follow-up. Two pediatric emergency medicine physicians applied the 5-item instrument to children presenting in the highest triage category to four rural EDs, and we assessed the reliability of the average summary scores (possible range of 5–35) across the two reviewers using standard measures. We also validated the instrument by comparing this mean summary score between those with and without medication errors (ascertained independently by two pharmacists) using a two-sample t-test. We reviewed the medical records of 178 pediatric patients for the study. The mean and median summary score for this cohort of patients were 27.4 and 28.5, respectively. Internal consistency was high (Cronbach's alpha of 0.92 and 0.89). All items showed a significant (p < 0.005) positive correlation between reviewers using the Spearman rank correlation (range 0.24 to 0.39). Exact agreement on individual items between reviewers ranged from 70.2% to 85.4%. The Intra-class Correlation Coefficient for the mean of the total summary score across the two reviewers was 0.65. The validity of the instrument was supported by the finding of a higher score for children without medication errors compared to those with medication errors which trended toward significance (mean score = 28.5 vs. 26.0, p = 0.076). The instrument we developed to measure quality of care provided to children in the ED has high internal consistency, fair to good inter-rater reliability and inter-rater correlation, and high content validity. The validity of the instrument is supported by the fact that the instrument's average summary score was lower in the presence of medication errors, which trended towards statistical significance.
Từ khóa
Tài liệu tham khảo
Donabedian A: The quality of care. How can it be assessed?. Jama. 1988, 260 (12): 1743-1748. 10.1001/jama.260.12.1743.
Chamberlain JM, Patel KM, Pollack MM: The Pediatric Risk of Hospital Admission score: a second-generation severity-of-illness score for pediatric emergency patients. Pediatrics. 2005, 115 (2): 388-395. 10.1542/peds.2004-0586.
Chamberlain JM, Patel KM, Pollack MM, Brayer A, Macias CG, Okada P, Schunk JE: Recalibration of the pediatric risk of admission score using a multi-institutional sample. Ann Emerg Med. 2004, 43 (4): 461-468. 10.1016/j.annemergmed.2003.08.001.
Gorelick MH, Lee C, Cronan K, Kost S, Palmer K: Pediatric emergency assessment tool (PEAT): a risk-adjustment measure for pediatric emergency patients. Acad Emerg Med. 2001, 8 (2): 156-162.
Hendricks C: The PEAT scale: an EMS tool. Emerg Med Serv. 2004, 33 (11): 47-8, 50, 52 passim.
Dans PE, Weiner JP, Otter SE: Peer review organizations. Promises and potential pitfalls. N Engl J Med. 1985, 313 (18): 1131-1137.
Goldman RL: The reliability of peer assessments of quality of care. Jama. 1992, 267 (7): 958-960. 10.1001/jama.267.7.958.
Goldman RL: The reliability of peer assessments. A meta-analysis. Eval Health Prof. 1994, 17 (1): 3-21. 10.1177/016327879401700101.
Hulka BS, Romm FJ, Parkerson GR, Russell IT, Clapp NE, Johnson FS: Peer review in ambulatory care: use of explicit criteria and implicit judgments. Med Care. 1979, 17 (3 Suppl): i-vi, 1-73.
Kahn KL, Rogers WH, Rubenstein LV, Sherwood MJ, Reinisch EJ, Keeler EB, Draper D, Kosecoff J, Brook RH: Measuring quality of care with explicit process criteria before and after implementation of the DRG-based prospective payment system. Jama. 1990, 264 (15): 1969-1973. 10.1001/jama.264.15.1969.
Rubenstein LV, Kahn KL, Reinisch EJ, Sherwood MJ, Rogers WH, Kamberg C, Draper D, Brook RH: Changes in quality of care for five diseases measured by implicit review, 1981 to 1986. Jama. 1990, 264 (15): 1974-1979. 10.1001/jama.264.15.1974.
Rubin HR, Rogers WH, Kahn KL, Rubenstein LV, Brook RH: Watching the doctor-watchers. How well do peer review organization methods detect hospital care quality problems?. Jama. 1992, 267 (17): 2349-2354. 10.1001/jama.267.17.2349.
Smith MA, Atherly AJ, Kane RL, Pacala JT: Peer review of the quality of care. Reliability and sources of variability for outcome and process assessments. Jama. 1997, 278 (19): 1573-1578. 10.1001/jama.278.19.1573.
Caplan RA, Posner KL, Cheney FW: Effect of outcome on physician judgments of appropriateness of care. Jama. 1991, 265 (15): 1957-1960. 10.1001/jama.265.15.1957.
Hayward RA, McMahon LF, Bernard AM: Evaluating the care of general medicine inpatients: how good is implicit review?. Ann Intern Med. 1993, 118 (7): 550-556.
California Rural Health Policy Council. [http://www.ruralhealth.oshpd.state.ca.us/faq.htm]
[http://www.cms.hhs.gov/data/download/default.asp]
[http://bhpr.hrsa.gov/shortage/]
Rubin HR, Rand Corporation.: Guidelines for structured implicit review of the quality of hospital care for diverse medical and surgical conditions. A Rand note ; N-3066-HCFA. 1990, Santa Monica, Calif. , Rand, x, 47 p.-
Hulka BS: Quality of ambulatory care: an exploration of the discrepancy between explicit process criteria and performance. 1980, Hyattsville, Md , U.S. Dept. of Health Education and Welfare Public Health Service Office of Health Research Statistics and Technology National Center for Health Services Research
Kahn KL, Rand Corporation., United States. Health Care Financing Administration.: Structured implicit review for physician implicit measurement of quality of care : development of the form and guidelines for its use. A Rand note ; N-3016-HCFA. 1989, Santa Monica, CA , Rand, xvii, 85 p.-
Rubenstein LV: Structured implicit review of the medical record : a method for measuring the quality of inhospital medical care and a summary of quality changes following implementation of the Medicare prospective payment system. A Rand note ; N-3033-HCFA. 1991, Santa Monica, CA (1700 Main Street, P.O. Box 2138, Santa Monica 90407-2138) , RAND, xi, 52 p.-
Bates DW, Boyle DL, Vander Vliet MB, Schneider J, Leape L: Relationship between medication errors and adverse drug events. J Gen Intern Med. 1995, 10 (4): 199-205. 10.1007/BF02600255.
Kaushal R, Bates DW, Landrigan C, McKenna KJ, Clapp MD, Federico F, Goldmann DA: Medication errors and adverse drug events in pediatric inpatients. Jama. 2001, 285 (16): 2114-2120. 10.1001/jama.285.16.2114.
NCC MERP Taxonomy of Medication Errors. 1998, Rockville, MD , NCC MERP
Shrout PE, Fleiss JL: Intraclass Correlations - Uses in Assessing Rater Reliability. Psychological Bulletin. 1979, 86 (2): 420-428. 10.1037/0033-2909.86.2.420.
Fleiss JL: Statistical methods for rates and proportions. 1981, New York , Wiley, xviii, 321 p.-2d
Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33 (1): 159-174. 10.2307/2529310.
Cronbach LJ: Coefficient alpha and the internal structure of tests.Psychometrika. 1951, 16: 297-324. 10.1007/BF02310555.
Nunnally JC, Bernstein IH: Psychometric theory. McGraw-Hill series in psychology. 1994, New York , McGraw-Hill, xxiv, 752 p.-3rd
Folli HL, Poole RL, Benitz WE, Russo JC: Medication error prevention by clinical pharmacists in two children's hospitals. Pediatrics. 1987, 79 (5): 718-722.
Kozer E, Scolnik D, Macpherson A, Keays T, Shi K, Luk T, Koren G: Variables associated with medication errors in pediatric emergency medicine. Pediatrics. 2002, 110 (4): 737-742. 10.1542/peds.110.4.737.
Raju TN, Kecskes S, Thornton JP, Perry M, Feldman S: Medication errors in neonatal and paediatric intensive-care units. Lancet. 1989, 2 (8659): 374-376. 10.1016/S0140-6736(89)90548-5.
Selbst SM, Fein JA, Osterhoudt K, Ho W: Medication errors in a pediatric emergency department. Pediatr Emerg Care. 1999, 15 (1): 1-4.
Ashton CM, Kuykendall DH, Johnson ML, Wray NP: An empirical assessment of the validity of explicit and implicit process-of-care criteria for quality assessment. Med Care. 1999, 37 (8): 798-808. 10.1097/00005650-199908000-00009.
Weingart SN, Mukamal K, Davis RB, Davies DT, Palmer RH, Cahalane M, Hamel MB, Phillips RS, Iezzoni LI: Physician-reviewers' perceptions and judgments about quality of care. Int J Qual Health Care. 2001, 13 (5): 357-365. 10.1093/intqhc/13.5.357.
Hofer TP, Bernstein SJ, DeMonner S, Hayward RA: Discussion between reviewers does not improve reliability of peer review of hospital quality. Med Care. 2000, 38 (2): 152-161. 10.1097/00005650-200002000-00005.
Hampers LC, Faries SG: Practice variation in the emergency management of croup. Pediatrics. 2002, 109 (3): 505-508. 10.1542/peds.109.3.505.
Hampers LC, Trainor JL, Listernick R, Eddy JJ, Thompson DA, Sloan EP, Chrisler OP, Gatewood LM, McNulty B, Krug SE: Setting-based practice variation in the management of simple febrile seizure. Acad Emerg Med. 2000, 7 (1): 21-27.
Schweich PJ, Smith KM, Dowd MD, Walkley EI: Pediatric emergency medicine practice patterns: a comparison of pediatric and general emergency physicians. Pediatr Emerg Care. 1998, 14 (2): 89-94. 10.1097/00006565-199804000-00001.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-227X/7/13/prepub