Methodological considerations for estimating policy effects in the context of co-occurring policies
Tóm tắt
Understanding how best to estimate state-level policy effects is important, and several unanswered questions remain, particularly about the ability of statistical models to disentangle the effects of concurrently enacted policies. In practice, many policy evaluation studies do not attempt to control for effects of co-occurring policies, and this issue has not received extensive attention in the methodological literature to date. In this study, we utilized Monte Carlo simulations to assess the impact of co-occurring policies on the performance of commonly-used statistical models in state policy evaluations. Simulation conditions varied effect sizes of the co-occurring policies and length of time between policy enactment dates, among other factors. Outcome data (annual state-specific opioid mortality rate per 100,000) were obtained from 1999 to 2016 National Vital Statistics System (NVSS) Multiple Cause of Death mortality files, thus yielding longitudinal annual state-level data over 18 years from 50 states. When co-occurring policies are ignored (i.e., omitted from the analytic model), our results demonstrated that high relative bias (> 82%) arises, particularly when policies are enacted in rapid succession. Moreover, as expected, controlling for all co-occurring policies will effectively mitigate the threat of confounding bias; however, effect estimates may be relatively imprecise (i.e., larger variance) when policies are enacted in near succession. Our findings highlight several key methodological issues regarding co-occurring policies in the context of opioid-policy research yet also generalize more broadly to evaluation of other state-level policies, such as policies related to firearms or COVID-19, showcasing the need to think critically about co-occurring policies that are likely to influence the outcome when specifying analytic models.
Tài liệu tham khảo
Abouk, R., Pacula, R.L., Powell, D.: Association between state laws facilitating pharmacy distribution of naloxone and risk of fatal overdose. JAMA Int. Med. 179(6), 805–811 (2019). https://doi.org/10.1001/jamainternmed.2019.0272
Ali, M.M., Dowd, W.N., Classen, T., Mutter, R., Novak, S.P.: Prescription drug monitoring programs, nonmedical use of prescription drugs, and heroin use: evidence from the national survey of drug use and health. Addict. Behav. 69, 65–77 (2017). https://doi.org/10.1016/j.addbeh.2017.01.011
Angrist, J.D., Pischke, J.R.-S.: Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press, Princeton (2009)
Blanchard, J., Weiss, A.J., Barrett, M.L., McDermott, K.W., Heslin, K.C.: State variation in opioid treatment policies and opioid-related hospital readmissions. BMC Health Serv. Res. 18(1), 971 (2018). https://doi.org/10.1186/s12913-018-3703-8
Buchmueller, T.C., Carey, C.: The effect of prescription drug monitoring programs on opioid utilization in Medicare. Am. Econ. J. Econ. Policy 10(1), 77–112 (2018). https://doi.org/10.1257/pol.20160094
Callaway, B., Sant’Anna, P.: Difference-in-Differences with multiple time periods. J. Econmet. 225(2), 200–230 (2021)
Castillo-Carniglia, A., Gonzalez-Santa Cruz, A., Cerda, M., Delcher, C., Shev, A.B., Wintemute, G.J., Henry, S.G.: Changes in opioid prescribing after implementation of mandatory registration and proactive reports within California’s prescription drug monitoring program. Drug Alcohol Depend. 218, 108405 (2021). https://doi.org/10.1016/j.drugalcdep.2020.108405
De Chaisemartin, C., d'Haultfoeuille, X.: Two-way fixed effects and differences-in-differences estimators with several treatments. National Bureau of Economic Research Working Paper. No. 29734. (2022)
Chan, N.W., Burkhardt, J., Flyr, M.: The effects of recreational marijuana legalization and dispensing on opioid mortality. Econ. Inq. 58(2), 589–606 (2020). https://doi.org/10.1111/ecin.12819
Cochrane, D., Orcutt, G.H.: Application of least squares regression to relationships containing auto-correlated error terms. JASA 44(245), 32–61 (1949). https://doi.org/10.1080/01621459.1949.10483290
Dave, D., Deza, M., Horn, B.P.: Prescription Drug Monitoring Programs, Opioid Abuse, and Crime (Working Paper 24975). Retrieved from Cambridge, MA (2018). https://www.nber.org/papers/w24975
Encinosa, W., Bernard, D., Selden, T.M.: Opioid and non-opioid analgesic prescribing before and after the CDC’s 2016 opioid guideline. Int. J. Health Econ. Manag. (2021). https://doi.org/10.1007/s10754-021-09307-4
Goodman-Bacon, A.: Difference-in-differences with variation in treatment timing. J. Econmet. 225(2), 254–277 (2021)
Griffin, B.A., Schuler, M.S., Patrick, S.W., Schell, T., Morral, A., Smart, R., Stuart, E.A.: Moving beyond the classic difference-in-differences model: a simulation study comparing statistical methods for estimating effectiveness of state-level policies. BMC Med. Res. Methodol. 21(279) (2021)
Ji, X., Haight, S.C., Ko, J.Y., Cox, S., Barfield, W.D., Zhang, K., Li, R.: Association between state policies on improving opioid prescribing in 2 states and opioid overdose rates among reproductive-aged women. Med. Care 59(2), 185–192 (2021). https://doi.org/10.1097/MLR.0000000000001475
Kennedy-Hendricks, A., Richey, M., McGinty, E.E., Stuart, E.A., Barry, C.L., Webster, D.W.: Opioid overdose deaths and Florida’s crackdown on pill mills. Am. J. Public Health 106(2), 291–297 (2016). https://doi.org/10.2105/AJPH.2015.302953
Kilby, A.: Opioids for the Masses: Welfare Tradeoffs in the Regulation of Narcotic Pain Medications. Massachusetts Institute of Technology, Cambridge (2015)
Kuo, Y.F., Raji, M.A., Chen, N.W., Hasan, H., Goodwin, J.S.: Trends in opioid prescriptions among Part D Medicare recipients from 2007 to 2012. Am. J. Med. 129(2), 221 e221–230 (2016). https://doi.org/10.1016/j.amjmed.2015.10.002
Lee, B., Zhao, W., Yang, K.C., Ahn, Y.Y., Perry, B.L.: Systematic evaluation of state policy interventions targeting the US opioid epidemic, 2007–2018. JAMA Netw. Open 4(2), e2036687 (2021). https://doi.org/10.1001/jamanetworkopen.2020.36687
Li, Y., Li, L.: Propensity score analysis methods with balancing constraints: a Monte Carlo study. Stat. Methods Med. Res. 30(4), 1119–1142 (2021)
Mallatt, J.: The effect of prescription drug monitoring programs on opioid prescriptions and heroin crime rates (Working Paper 1292) (2018). Retrieved from https://ssrn.com/abstract=3050692
Matthay, E.C., Gottlieb, L.M., Rehkopf, D., Tan, M.L., Vlahov, D., Maria Glymour, M.: What to do when everything happens at once: analytic approaches to estimate the health effects of co-occurring social policies. Epidemiol. Rev. 43(1), 33–47 (2021a). https://doi.org/10.1101/2020.10.05.20205963
Matthay, E.C., Hagan, E., Joshi, S., Tan, M.L., Vlahov, D., Adler, N., Maria Glymour, M.: The revolution will be hard to evaluate: how co-occurring policy changes affect research on the health effects of social policies. Epidemiol. Rev. 43(1), 19–32 (2021b)
Mauri, A.I., Townsend, T.N., Haffajee, R.L.: The association of state opioid misuse prevention policies with patient- and provider-related outcomes: a scoping review. Milbank Q. 98(1), 57–105 (2020). https://doi.org/10.1111/1468-0009.12436
McClellan, C., Lambdin, B.H., Ali, M.M., Mutter, R., Davis, C.S., Wheeler, E., Kral, A.H.: Opioid-overdose laws association with opioid use and overdose mortality. Addict. Behav. 86, 90–95 (2018). https://doi.org/10.1016/j.addbeh.2018.03.014
McInerney, M.: The Affordable Care Act, Public Insurance Expansion and Opioid Overdose Mortality. University of Connecticut, Department of Economics, Working papers: 2017–23 (2017). Retrieved from http://web2.uconn.edu/economics/working/2017-23.pdf
Pacula, R.L., Smart, R.: Medical Marijuana and Marijuana Legalization. Annu. Rev. Clin. Psychol. 13, 397–419 (2017). https://doi.org/10.1146/annurev-clinpsy-032816-045128
Paulozzi, L.J., Kilbourne, E.M., Desai, H.A.: Prescription drug monitoring programs and death rates from drug overdose. Pain Med. 12(5), 747–754 (2011). https://doi.org/10.1111/j.1526-4637.2011.01062.x
Pauly, N.J., Slavova, S., Delcher, C., Freeman, P.R., Talbert, J.: Features of prescription drug monitoring programs associated with reduced rates of prescription opioid-related poisonings. Drug Alcohol Depend. 184, 26–32 (2018). https://doi.org/10.1016/j.drugalcdep.2017.12.002
Phillips, E., Gazmararian, J.: Implications of prescription drug monitoring and medical cannabis legislation on opioid overdose mortality. J. Opioid Manag. 13(4), 229–239 (2017). https://doi.org/10.5055/jom.2017.0391
Puac-Polanco, V., Chihuri, S., Fink, D.S., Cerda, M., Keyes, K.M., Li, G.: Prescription drug monitoring programs and prescription opioid-related outcomes in the United States. Epidemiol. Rev. 42(1), 134–153 (2020). https://doi.org/10.1093/epirev/mxaa002
Rutkow, L., Chang, H.Y., Daubresse, M., Webster, D.W., Stuart, E.A., Alexander, G.C.: Effect of Florida’s prescription drug monitoring program and pill mill laws on opioid prescribing and use. JAMA Int. Med. 175(10), 1642–1649 (2015). https://doi.org/10.1001/jamainternmed.2015.3931
Schell, T., Cefalu, M., Griffin, B.A., Smart, R., Morral, A.: Changes in firearm mortality following the implementation of state laws regulating firearm access and use. Proc. Natl. Acad. Sci. u. s. a. 117(26), 14906–14910 (2020)
Schell, T., Griffin, B., Morral, A. Evaluating Methods to Estimate the Effect of State Laws on Firearm Deaths: A Simulation Study. RR-2685-RC. Santa Monica, CA: RAND Corporation (2019)
Schuler, M.S., Heins, S.E., Smart, R., Griffin, B.A., Powell, D., Stuart, E.A., Stein, B.D.: The state of the science in opioid policy research. Drug Alcohol Depend. 214, 108137 (2020). https://doi.org/10.1016/j.drugalcdep.2020.108137
Snell-Rood, C., Willging, C., Showalter, D., Peters, H., Pollini, R.A.: System-level factors shaping the implementation of "hub and spoke" systems to expand MOUD in rural areas. Subst. Abus., 1–17 (2020). https://doi.org/10.1080/08897077.2020.1846149
Sun, L., Abraham, S.: Estimating dynamic treatment effects in event studies with heterogeneous treatment effects. J. Economet. 225(2), 175–199 (2021)
U.S. Department of Labor. Bureau of Labor Statistics (2019). Retrieved from https://www.bls.gov/2019
Wing, C., Simon, K., Bello-Gomez, R.A.: Designing difference in difference studies: best practices for public health policy research. Annu. Rev. Public Health 39, 453–469 (2018). https://doi.org/10.1146/annurev-publhealth-040617-013507
Yarbrough, C.R.: Prescription drug monitoring programs produce a limited impact on painkiller prescribing in Medicare Part D. Health Serv. Res. 53(2), 671–689 (2018). https://doi.org/10.1111/1475-6773.12652