BMC Medical Informatics and Decision Making

Interoperable phenotyping algorithms, needed to identify patient cohorts meeting eligibility criteria for observational studies or clinical trials, require medical data in a consistent structured, coded format. Data heterogeneity limits such algorithms’ applicability. Existing approaches are often: not widely interoperable; or, have low sensitivity due to reliance on the lowest common denominator (ICD-9 diagnoses). In the Scalable Collaborative Infrastructure for a Learning Healthcare System (SCILHS) we endeavor to use the widely-available Current Procedural Terminology (CPT) procedure codes with ICD-9. Unfortunately, CPT changes drastically year-to-year – codes are retired/replaced. Longitudinal analysis requires grouping retired and current codes. BioPortal provides a navigable CPT hierarchy, which we imported into the Informatics for Integrating Biology and the Bedside (i2b2) data warehouse and analytics platform. However, this hierarchy does not include retired codes. We compared BioPortal’s 2014AA CPT hierarchy with Partners Healthcare’s SCILHS datamart, comprising three-million patients’ data over 15 years. 573 CPT codes were not present in 2014AA (6.5 million occurrences). No existing terminology provided hierarchical linkages for these missing codes, so we developed a method that automatically places missing codes in the most specific “grouper” category, using the numerical similarity of CPT codes. Two informaticians reviewed the results. We incorporated the final table into our i2b2 SCILHS/PCORnet ontology, deployed it at seven sites, and performed a gap analysis and an evaluation against several phenotyping algorithms. The reviewers found the method placed the code correctly with 97 % precision when considering only miscategorizations (“correctness precision”) and 52 % precision using a gold-standard of optimal placement (“optimality precision”). High correctness precision meant that codes were placed in a reasonable hierarchal position that a reviewer can quickly validate. Lower optimality precision meant that codes were not often placed in the optimal hierarchical subfolder. The seven sites encountered few occurrences of codes outside our ontology, 93 % of which comprised just four codes. Our hierarchical approach correctly grouped retired and non-retired codes in most cases and extended the temporal reach of several important phenotyping algorithms. We developed a simple, easily-validated, automated method to place retired CPT codes into the BioPortal CPT hierarchy. This complements existing hierarchical terminologies, which do not include retired codes. The approach’s utility is confirmed by the high correctness precision and successful grouping of retired with non-retired codes.

Chronic pain has emerged as a disease in itself, affecting a growing number of people. Effective patient-provider communication is central to good pain management because pain can only be understood from the patient’s perspective. We aimed to develop a user-centered tool to improve patient-provider communication about chronic pain and assess its feasibility in real-world settings in preparation for further evaluation and distribution. To identify and prioritize patient treatment goals for chronic pain, strategies to improve patient-provider communication about chronic pain, and facilitate implementation of the tool, we conducted nominal group technique meetings and card sorting with patients with chronic pain and experienced providers (n = 12). These findings informed the design of the PainAPP tool. Usability and beta-testing with patients (n = 38) and their providers refined the tool and assessed its feasibility, acceptability, and preliminary impact. Formative work revealed that patients felt neither respected nor trusted by their providers and focused on transforming providers’ negative attitudes towards them, whereas providers focused on gathering patient information. PainAPP incorporated areas prioritized by patients and providers: assessing patient treatment goals and preferences, functional abilities and pain, and providing patients tailored education and an overall summary that patients can share with providers. Beta-testing involved 38 patients and their providers. Half of PainAPP users shared their summaries with their providers. Patients rated PainAPP highly in all areas. All users would recommend it to others with chronic pain; nearly all trusted the information and said it helped them think about my treatment goals (94%), understand my chronic pain (82%), make the most of my next doctor’s visit (82%), and not want to use opioids (73%). Beta-testing revealed challenges delivering the tool and summary report to patients and providers in a timely manner and obtaining provider feedback. PainAPP appears feasible for use, but further adaptation and testing is needed to assess its impact on patients and providers. This study was approved by the University of New England Independent Review Board for the Protection of Human Subjects in Research (012616–019) and was registered with ClinicalTrials.gov (protocol ID: NCT03425266) prior to enrollment. The trial was prospectively registered and was approved on February 7, 2018.

Systematic Reviews (SRs) are an essential part of evidence-based medicine, providing support for clinical practice and policy on a wide range of medical topics. However, producing SRs is resource-intensive, and progress in the research they review leads to SRs becoming outdated, requiring updates. Although the question of how and when to update SRs has been studied, the best method for determining when to update is still unclear, necessitating further research. In this work we study the potential impact of a machine learning-based automated system for providing alerts when new publications become available within an SR topic. Some of these new publications are especially important, as they report findings that are more likely to initiate a review update. To this end, we have designed a classification algorithm to identify articles that are likely to be included in an SR update, along with an annotation scheme designed to identify the most important publications in a topic area. Using an SR database containing over 70,000 articles, we annotated articles from 9 topics that had received an update during the study period. The algorithm was then evaluated in terms of the overall correct and incorrect alert rate for publications meeting the topic inclusion criteria, as well as in terms of its ability to identify important, update-motivating publications in a topic area. Our initial approach, based on our previous work in topic-specific SR publication classification, identifies over 70% of the most important new publications, while maintaining a low overall alert rate. We performed an initial analysis of the opportunities and challenges in aiding the SR update planning process with an informatics-based machine learning approach. Alerts could be a useful tool in the planning, scheduling, and allocation of resources for SR updates, providing an improvement in timeliness and coverage for the large number of medical topics needing SRs. While the performance of this initial method is not perfect, it could be a useful supplement to current approaches to scheduling an SR update. Approaches specifically targeting the types of important publications identified by this work are likely to improve results.

Women of low socioeconomic status (SES) diagnosed with early stage breast cancer are less likely to be involved in treatment decisions. They tend to report higher decisional regret and poorer communication. Evidence suggests that well-designed encounter decision aids (DAs) could improve outcomes and potentially reduce healthcare disparities. Our goal was to evaluate the acceptability and feasibility of encounter decision aids (Option Grid, Comic Option Grid, and Picture Option Grid) adapted for a low-SES and low-literacy population. We used a multi-phase, mixed-methods approach. In phase 1, we conducted a focus group with rural community stakeholders. In phase 2, we developed and administered a web-based questionnaire with patients of low and high SES. In phase 3, we interviewed patients of low SES and relevant healthcare professionals. Data from phase 1 (n = 5) highlighted the importance of addressing treatment costs for patients. Data from phase 2 (n = 268) and phase 3 (n = 15) indicated that using both visual displays and numbers are helpful for understanding statistical information. Data from all three phases suggested that using plain language and simple images (Picture Option Grid) was most acceptable and feasible. The Comic Option Grid was deemed least acceptable. Option Grid and Picture Option Grid appeared acceptable and feasible in facilitating patient involvement and improving perceived understanding among patients of high and low SES. Picture Option Grid was considered most acceptable, accessible and feasible in the clinic visit. However, given the small sample sizes used, those findings need to be interpreted with caution. Further research is needed to determine the impact of pictorial and text-based encounter decision aids in underserved patients and across socioeconomic strata.

Patients and clinicians expect patient decision aids to be based on the best available research evidence. Since 2005, this expectation has translated into a quality dimension of the International Patient Decision Aid Standards. We reviewed the 2005 standards and the available literature on the evidence base of decision aids as well as searched for parallel activities in which evidence is brought to bear to inform clinical decisions. In conducting this work, we noted emerging and research issues that require attention and may inform this quality dimension in the future. This dimension requires patient decision aids to be based on research evidence about the relevant options and the nature and likelihood of their effect on outcomes that matter to patients. The synthesis of evidence should be comprehensive and up-to-date, and the evidence itself subject to critical appraisal. Ethical (informed patient choice), quality-of-care (patient-centered care), and scientific (evidence-based medicine) arguments justify this requirement. Empirical evidence suggests that over two thirds of available decision aids are based on high-quality evidence syntheses. Emerging issues identified include the duties of developers regarding the conduct of systematic reviews, the impact of comparative effectiveness research, their link with guidelines based on the same evidence, and how to present the developers’ confidence in the estimates to the end-users. Systematic application of the GRADE system, common in contemporary practice guideline development, could enhance satisfaction of this dimension. While theoretical and practical issues remained to be addressed, high-quality patient decision aids should adhere to this dimension requiring they be based on comprehensive and up-to-date summaries of critically appraised evidence.

Shared decision-making in rheumatoid arthritis (RA) care is a priority among policy makers, clinicians and patients both nationally and internationally. Demands on patients to have basic knowledge of RA, treatment options, and details of risk and benefit when making medication decisions with clinicians can be overwhelming, especially for those with limited literacy or limited English language proficiency. The objective of this study is to describe the development of a medication choice decision aid for patients with rheumatoid arthritis (RA) in three languages using low literacy principles. Based on the development of a diabetes decision aid, the RA decision aid (RA Choice) was developed through a collaborative process involving patients, clinicians, designers, decision-aid and health literacy experts. A combination of evidence synthesis and direct observation of clinician-patient interactions generated content and guided an iterative process of prototype development. Three iterations of RA Choice were developed and field-tested before completion. The final tool organized data using icons and plain language for 12 RA medications across 5 issues: frequency of administration, time to onset, cost, side effects, and special considerations. The tool successfully created a conversation between clinician and patient, and garnered high acceptability from clinicians. The process of collaboratively developing an RA decision aid designed to promote shared decision making resulted in a graphically-enhanced, low literacy tool. The use of RA Choice in the clinical encounter has the potential to enhance communication for RA patients, including those with limited health literacy and limited English language proficiency.

Heart failure is one of the leading causes of hospitalization in the United States. Advances in big data solutions allow for storage, management, and mining of large volumes of structured and semi-structured data, such as complex healthcare data. Applying these advances to complex healthcare data has led to the development of risk prediction models to help identify patients who would benefit most from disease management programs in an effort to reduce readmissions and healthcare cost, but the results of these efforts have been varied. The primary aim of this study was to develop a 30-day readmission risk prediction model for heart failure patients discharged from a hospital admission. We used longitudinal electronic medical record data of heart failure patients admitted within a large healthcare system. Feature vectors included structured demographic, utilization, and clinical data, as well as selected extracts of un-structured data from clinician-authored notes. The risk prediction model was developed using deep unified networks (DUNs), a new mesh-like network structure of deep learning designed to avoid over-fitting. The model was validated with 10-fold cross-validation and results compared to models based on logistic regression, gradient boosting, and maxout networks. Overall model performance was assessed using concordance statistic. We also selected a discrimination threshold based on maximum projected cost saving to the Partners Healthcare system. Data from 11,510 patients with 27,334 admissions and 6369 30-day readmissions were used to train the model. After data processing, the final model included 3512 variables. The DUNs model had the best performance after 10-fold cross-validation. AUCs for prediction models were 0.664 ± 0.015, 0.650 ± 0.011, 0.695 ± 0.016 and 0.705 ± 0.015 for logistic regression, gradient boosting, maxout networks, and DUNs respectively. The DUNs model had an accuracy of 76.4% at the classification threshold that corresponded with maximum cost saving to the hospital. Deep learning techniques performed better than other traditional techniques in developing this EMR-based prediction model for 30-day readmissions in heart failure patients. Such models can be used to identify heart failure patients with impending hospitalization, enabling care teams to target interventions at their most high-risk patients and improving overall clinical outcomes.

Low back pain (LBP) is a common condition made up of a variety of anatomic and clinical subtypes. Lumbar disc herniation (LDH) and lumbar spinal stenosis (LSS) are two subtypes highly associated with LBP. Patients with LDH/LSS are often started with non-surgical treatments and if those are not effective then go on to have decompression surgery. However, recommendation of surgery is complicated as the outcome may depend on the patient’s health characteristics. We developed a deep learning (DL) model to predict decompression surgery for patients with LDH/LSS. We used datasets of 8387 and 8620 patients from a prospective study that collected data from four healthcare systems to predict early (within 2 months) and late surgery (within 12 months after a 2 month gap), respectively. We developed a DL model to use patients’ demographics, diagnosis and procedure codes, drug names, and diagnostic imaging reports to predict surgery. For each prediction task, we evaluated the model’s performance using classical and generalizability evaluation. For classical evaluation, we split the data into training (80%) and testing (20%). For generalizability evaluation, we split the data based on the healthcare system. We used the area under the curve (AUC) to assess performance for each evaluation. We compared results to a benchmark model (i.e. LASSO logistic regression). For classical performance, the DL model outperformed the benchmark model for early surgery with an AUC of 0.725 compared to 0.597. For late surgery, the DL model outperformed the benchmark model with an AUC of 0.655 compared to 0.635. For generalizability performance, the DL model outperformed the benchmark model for early surgery. For late surgery, the benchmark model outperformed the DL model. For early surgery, the DL model was preferred for classical and generalizability evaluation. However, for late surgery, the benchmark and DL model had comparable performance. Depending on the prediction task, the balance of performance may shift between DL and a conventional ML method. As a result, thorough assessment is needed to quantify the value of DL, a relatively computationally expensive, time-consuming and less interpretable method.