Temporal and Geographic variation in the validity and internal consistency of the Nursing Home Resident Assessment Minimum Data Set 2.0
Tóm tắt
The Minimum Data Set (MDS) for nursing home resident assessment has been required in all U.S. nursing homes since 1990 and has been universally computerized since 1998. Initially intended to structure clinical care planning, uses of the MDS expanded to include policy applications such as case-mix reimbursement, quality monitoring and research. The purpose of this paper is to summarize a series of analyses examining the internal consistency and predictive validity of the MDS data as used in the "real world" in all U.S. nursing homes between 1999 and 2007. We used person level linked MDS and Medicare denominator and all institutional claim files including inpatient (hospital and skilled nursing facilities) for all Medicare fee-for-service beneficiaries entering U.S. nursing homes during the period 1999 to 2007. We calculated the sensitivity and positive predictive value (PPV) of diagnoses taken from Medicare hospital claims and from the MDS among all new admissions from hospitals to nursing homes and the internal consistency (alpha reliability) of pairs of items within the MDS that logically should be related. We also tested the internal consistency of commonly used MDS based multi-item scales and examined the predictive validity of an MDS based severity measure viz. one year survival. Finally, we examined the correspondence of the MDS discharge record to hospitalizations and deaths seen in Medicare claims, and the completeness of MDS assessments upon skilled nursing facility (SNF) admission. Each year there were some 800,000 new admissions directly from hospital to US nursing homes and some 900,000 uninterrupted SNF stays. Comparing Medicare enrollment records and claims with MDS records revealed reasonably good correspondence that improved over time (by 2006 only 3% of deaths had no MDS discharge record, only 5% of SNF stays had no MDS, but over 20% of MDS discharges indicating hospitalization had no associated Medicare claim). The PPV and sensitivity levels of Medicare hospital diagnoses and MDS based diagnoses were between .6 and .7 for major diagnoses like CHF, hypertension, diabetes. Internal consistency, as measured by PPV, of the MDS ADL items with other MDS items measuring impairments and symptoms exceeded .9. The Activities of Daily Living (ADL) long form summary scale achieved an alpha inter-consistency level exceeding .85 and multi-item scale alpha levels of .65 were achieved for well being and mood, and .55 for behavior, levels that were sustained even after stratification by ADL and cognition. The Changes in Health, End-stage disease and Symptoms and Signs (CHESS) index, a summary measure of frailty was highly predictive of one year survival. The MDS demonstrates a reasonable level of consistency both in terms of how well MDS diagnoses correspond to hospital discharge diagnoses and in terms of the internal consistency of functioning and behavioral items. The level of alpha reliability and validity demonstrated by the scales suggest that the data can be useful for research and policy analysis. However, while improving, the MDS discharge tracking record should still not be used to indicate Medicare hospitalizations or mortality. It will be important to monitor the performance of the MDS 3.0 with respect to consistency, reliability and validity now that it has replaced version 2.0, using these results as a baseline that should be exceeded.
Tài liệu tham khảo
Morris JN, Hawes C, Fries BE, Phillips CD, Mor V, Katz S, Murphy K, Drugovich ML, Friedlob AS: Designing the national resident assessment instrument for nursing homes. Gerontologist. 1990, 30 (3): 293-307.
Mor V: A comprehensive clinical assessment tool to inform policy and practice: applications of the minimum data set. Med Care. 2004, 42 (4 Suppl): III50-59.
Zinn J, Feng Z, Mor V, Intrator O, Grabowski D: Restructuring in response to case mix reimbursement in nursing homes: a contingency approach. Health Care Manage Rev. 2008, 33 (2): 113-123.
Fries BE, Schneider DP, Foley WJ, Gavazzi M, Burke R, Cornelius E: Refining a case-mix measure for nursing homes: Resource Utilization Groups (RUG-III). Med Care. 1994, 32 (7): 668-685. 10.1097/00005650-199407000-00002.
Zimmerman DR, Karon SL, Arling G, Clark BR, Collins T, Ross R, Sainfort F: Development and testing of nursing home quality indicators. Health Care Financ Rev. 1995, 16 (4): 107-127.
Werner RM, Konetzka RT, Kruse GB: Impact of public reporting on unreported quality of care. Health Serv Res. 2009, 44 (2 Pt 1): 379-398. 10.1111/j.1475-6773.2008.00915.x.
Harris Y, Clauser SB: Achieving improvement through nursing home quality measurement. Health Care Financ Rev. 2002, 23 (4): 5-18.
Hawes C, Mor V, Phillips CD, Fries BE, Morris JN, Steele-Friedlob E, Greene AM, Nennstiel M: The OBRA-87 nursing home regulations and implementation of the Resident Assessment Instrument: effects on process quality. J Am Geriatr Soc. 1997, 45 (8): 977-985.
Welton JM: Implications of Medicare reimbursement changes related to inpatient nursing care quality. J Nurs Adm. 2008, 38 (7-8): 325-330.
Ossip-Klein DJ, Karuza J, Tweet A, Howard J, Obermiller-Powers M, Howard L, Katz P, Griffin-Roth S, Swift M: Benchmarking implementation of a computerized system for long-term care. Am J Med Qual. 2002, 17 (3): 94-102. 10.1177/106286060201700304.
Resnick HE, Manard BB, Stone RI, Alwan M: Use of electronic information systems in nursing homes: United States, 2004. J Am Med Inform Assoc. 2009, 16 (2): 179-186. 10.1197/jamia.M2955.
Rantz MJ, Petroski GF, Madsen RW, Mehr DR, Popejoy L, Hicks LL, Porter R, Zwygart-Stauffacher M, Grando V: Setting thresholds for quality indicators derived from MDS data for nursing home quality improvement reports: an update. Jt Comm J Qual Improv. 2000, 26 (2): 101-110.
Hawes C, Morris JN, Phillips CD, Mor V, Fries BE, Nonemaker S: Reliability estimates for the Minimum Data Set for nursing home resident assessment and care screening (MDS). Gerontologist. 1995, 35 (2): 172-178.
Morris JN, Nonemaker S, Murphy K, Hawes C, Fries BE, Mor V, Phillips C: A commitment to change: revision of HCFA's RAI. J Am Geriatr Soc. 1997, 45 (8): 1011-1016.
Mor V, Angelelli J, Jones R, Roy J, Moore T, Morris J: Inter-rater reliability of nursing home quality indicators in the U.S. BMC Health Serv Res. 2003, 3 (1): 20.-10.1186/1472-6963-3-20.
Schnelle JF, Cadogan MP, Yoshii J, Al-Samarrai NR, Osterweil D, Bates-Jensen BM, Simmons SF: The minimum data set urinary incontinence quality indicators: do they reflect differences in care processes related to incontinence?. Med Care. 2003, 41 (8): 909-922. 10.1097/00005650-200308000-00005.
Schnelle JF, Wood S, Schnelle ER, Simmons SF: Measurement sensitivity and the Minimum Data Set depression quality indicator. Gerontologist. 2001, 41 (3): 401-405.
Simmons SF, Cadogan MP, Cabrera GR, Al-Samarrai NR, Jorge JS, Levy-Storms L, Osterweil D, Schnelle JF: The minimum data set depression quality indicator: does it reflect differences in care processes?. Gerontologist. 2004, 44 (4): 554-564.
Morris JN, Fries BE, Mehr DR, Hawes C, Phillips C, Mor V, Lipsitz LA: MDS Cognitive Performance Scale. J Gerontol. 1994, 49 (4): M174-182.
Mor V, Branco K, Fleishman J, Hawes C, Phillips C, Morris J, Fries B: The structure of social engagement among nursing home residents. J Gerontol B Psychol Sci Soc Sci. 1995, 50 (1): P1-P8.
Morris JN, Fries BE, Morris SA: Scaling ADLs within the MDS. J Gerontol A Biol Sci Med Sci. 1999, 54 (11): M546-553. 10.1093/gerona/54.11.M546.
Gambassi G, Landi F, Lapane KL, Sgadari A, Mor V, Bernabei R: Predictors of mortality in patients with Alzheimer's disease living in nursing homes. J Neurol Neurosurg Psychiatry. 1999, 67 (1): 59-65. 10.1136/jnnp.67.1.59.
Intrator O, Zinn J, Mor V: Nursing home characteristics and potentially preventable hospitalizations of long-stay residents. J Am Geriatr Soc. 2004, 52 (10): 1730-1736. 10.1111/j.1532-5415.2004.52469.x.
Rahman AN, Applebaum RA: The nursing home Minimum Data Set assessment instrument: manifest functions and unintended consequences--past, present, and future. Gerontologist. 2009, 49 (6): 727-735. 10.1093/geront/gnp066.
Housen P, Shannon GR, Simon B, Edelen MO, Cadogan MP, Jones M, Buchanan J, Saliba D: Why not just ask the resident?. J Gerontol Nurs. 2009, 35 (11): 40-49. 10.3928/00989134-20091001-01.
Simmons SF, Schnelle JF: The identification of residents capable of accurately describing daily care: implications for evaluating nursing home care quality. Gerontologist. 2001, 41 (5): 605-611.
Intrator O, Berg K, Hiris J, Mor V, Miller S: The residential history file: studying nursing home residents' long term care histories. HSR.
Intrator O, Hiris J, Berg K, Miller SC, Mor V: The Residential History File: Studying Nursing Home Residents' Long-Term Care Histories(*). Health Serv Res. 2010
Gambassi G, Landi F, Peng L, Brostrup-Jensen C, Calore K, Hiris J, Lipsitz L, Mor V, Bernabei R: Validity of diagnostic and drug data in standardized nursing home resident assessments: potential for geriatric pharmacoepidemiology. SAGE Study Group. Systematic Assessment of Geriatric drug use via Epidemiology. Med Care. 1998, 36 (2): 167-179. 10.1097/00005650-199802000-00006.
Schroll M, Jonsson PV, Mor V, Berg K, Sherwood S: An international study of social engagement among nursing home residents. Age Ageing. 1997, 26 (Suppl 2): 55-59.
Hirdes JP, Frijters DH, Teare GF: The MDS-CHESS scale: a new measure to predict mortality in institutionalized older people. J Am Geriatr Soc. 2003, 51 (1): 96-100. 10.1034/j.1601-5215.2002.51017.x.
Dijk PTMv, Mehr DR, Ooms ME, Madsen R, Petroski G, Frijters DH, Pot AM, Ribbe MW: Comorbidity and 1-Year Mortality Risks in Nursing Home Residents. Journal of the American Geriatrics Society. 2005, 53 (4): 660-665. 10.1111/j.1532-5415.2005.53216.x.
Cortina J: What is Coefficient Alpha? An Examination of Theory and Applications. Journal of Applied Psychology. 1993, 78 (1): 98-104. 10.1037/0021-9010.78.1.98.
Fries BE, Simon SE, Morris JN, Flodstrom C, Bookstein FL: Pain in U.S. nursing homes: validating a pain scale for the minimum data set. Gerontologist. 2001, 41 (2): 173-179.
Koehler M, Rabinowitz T, Hirdes J, Stones M, Carpenter GI, Fries BE, Morris JN, Jones RN: Measuring depression in nursing home residents with the MDS and GDS: an observational psychometric study. BMC Geriatr. 2005, 5: 1-10.1186/1471-2318-5-1.
Leary R, Farley D: Surviving Medicare's IPPS changes: what you need to know. Healthc Financ Manage. 2007, 61 (12): 48-55.
Ouslander JG: The Resident Assessment Instrument (RAI): promise and pitfalls. J Am Geriatr Soc. 1997, 45 (8): 975-976.
Schnelle JF, Bates-Jensen BM, Levy-Storms L, Grbic V, Yoshii J, Cadogan M, Simmons SF: The minimum data set prevalence of restraint quality indicator: does it reflect differences in care?. Gerontologist. 2004, 44 (2): 245-255.
Schnelle JF: Can nursing homes use the MDS to improve quality?. J Am Geriatr Soc. 1997, 45 (8): 1027-1028.
Hawes C, Phillips CD, Mor V, Fries BE, Morris JN: MDS data should be used for research. Gerontologist. 1992, 32 (4): 563-564.
Schnelle JF, Bates-Jensen BM, Chu L, Simmons SF: Accuracy of nursing home medical record information about care-process delivery: implications for staff management and improvement. J Am Geriatr Soc. 2004, 52 (8): 1378-1383. 10.1111/j.1532-5415.2004.52372.x.
Simmons SF, Lim B, Schnelle JF: Accuracy of minimum data set in identifying residents at risk for undernutrition: oral intake and food complaints. J Am Med Dir Assoc. 2002, 3 (3): 140-145. 10.1016/S1525-8610(04)70456-9.
Wu N, Miller SC, Lapane K, Roy J, Mor V: The quality of the quality indicator of pain derived from the minimum data set. Health Serv Res. 2005, 40 (4): 1197-1216. 10.1111/j.1475-6773.2005.00400.x.
Roy J, Mor V: The effect of provider-level ascertainment bias on profiling nursing homes. Stat Med. 2005, 24 (23): 3609-3629. 10.1002/sim.2215.
Grabowski DC, Feng Z, Intrator O, Mor V: Recent trends in state nursing home payment policies. Health Aff (Millwood). 2004, Suppl Web Exclusives: W4-363-373.
Fries BE, Cooney LM: Resource utilization groups. A patient classification system for long-term care. Med Care. 1985, 23 (2): 110-122. 10.1097/00005650-198502000-00002.
Zinn JS, Mor V, Intrator O, Feng Z, Angelelli J, Davis JA: The impact of the prospective payment system for skilled nursing facilities on therapy service provision: a transaction cost approach. Health Serv Res. 2003, 38 (6 Pt 1): 1467-1485. 10.1111/j.1475-6773.2003.00188.x.
Mor V: Defining and measuring quality outcomes in long-term care. J Am Med Dir Assoc. 2007, 8 (3 Suppl 2): e129-137. 10.1016/j.jamda.2006.12.014.
Mukamel DB, Spector WD: Nursing home costs and risk-adjusted outcome measures of quality. Med Care. 2000, 38 (1): 78-89. 10.1097/00005650-200001000-00009.
Mor V, Berg K, Angelelli J, Gifford D, Morris J, Moore T: The quality of quality measurement in U.S. nursing homes. Gerontologist. 2003, 43 (Spec No 2): 37-46.
Gruneir A, Miller SC, Feng Z, Intrator O, Mor V: Relationship between state medicaid policies, nursing home racial composition, and the risk of hospitalization for black and white residents. Health Serv Res. 2008, 43 (3): 869-881. 10.1111/j.1475-6773.2007.00806.x.
Mor V, Gruneir A, Feng Z, Grabowski DC, Intrator O, Zinn J: The Effect of State Policies on Nursing Home Resident Outcomes. J Am Geriatr Soc. 2011
Morris JN, Mor V, Goldberg RJ, Sherwood S, Greer DS, Hiris J: The effect of treatment setting and patient characteristics on pain in terminal cancer patients: a report from the National Hospice Study. J Chronic Dis. 1986, 39 (1): 27-35. 10.1016/0021-9681(86)90104-9.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/11/78/prepub