ERROR! No headcode.htm file found.


Professional Education

  • Doctor of Philosophy, University of Pittsburgh (2019)
  • Master of Science, University of Pittsburgh (2014)
  • Bachelor of Science, Capital Medical University (2012)

Stanford Advisors


All Publications

  • Learning from Past Respiratory Failure Patients to Triage COVID-19 Patient Ventilator Needs: A Multi-Institutional Study. Journal of biomedical informatics Carmichael, H., Coquet, J., Sun, R., Sang, S., Groat, D., Asch, S. M., Bledsoe, J., Peltan, I. D., Jacobs, J. R., Hernandez-Boussard, T. 2021: 103802


    BACKGROUND: Unlike well-established diseases that base clinical care on randomized trials, past experiences, and training, prognosis in COVID19 relies on a weaker foundation. Knowledge from other respiratory failure diseases may inform clinical decisions in this novel disease. The objective was to predict 48-hour invasive mechanical ventilation (IMV) within 48 hours in patients hospitalized with COVID-19 using COVID-like diseases (CLD).METHODS: This retrospective multicenter study trained machine learning (ML) models on patients hospitalized with CLD to predict IMV within 48 hours in COVID-19 patients. CLD patients were identified using diagnosis codes for bacterial pneumonia, viral pneumonia, influenza, unspecified pneumonia and acute respiratory distress syndrome (ARDS), 2008-2019. A total of 16 cohorts were constructed, including any combinations of the four diseases plus an exploratory ARDS cohort, to determine the most appropriate cohort to use. Candidate predictors included demographic and clinical parameters that were previously associated with poor COVID-19 outcomes. Model development included the implementation of logistic regression and three ensemble tree-based algorithms: decision tree, AdaBoost, and XGBoost. Models were validated in hospitalized COVID-19 patients at two healthcare systems, March 2020-July 2020. ML models were trained on CLD patients at Stanford Hospital Alliance (SHA). Models were validated on hospitalized COVID-19 patients at both SHA and Intermountain Healthcare.RESULTS: CLD training data were obtained from SHA (n=14,030), and validation data included 444 adult COVID-19 hospitalized patients from SHA (n=185) and Intermountain (n=259). XGBoost was the top-performing ML model, and among the 16 CLD training cohorts, the best model achieved an area under curve (AUC) of 0.883 in the validation set. In COVID-19 patients, the prediction models exhibited moderate discrimination performance, with the best models achieving an AUC of 0.77 at SHA and 0.65 at Intermountain. The model trained on all pneumonia and influenza cohorts had the best overall performance (SHA: positive predictive value (PPV) 0.29, negative predictive value (NPV) 0.97, positive likelihood ratio (PLR) 10.7; Intermountain: PPV, 0.23, NPV 0.97, PLR 10.3). We identified important factors associated with IMV that are not traditionally considered for respiratory diseases.CONCLUSIONS: The performance of prediction models derived from CLD for 48-hour IMV in patients hospitalized with COVID-19 demonstrate high specificity and can be used as a triage tool at point of care. Novel predictors of IMV identified in COVID-19 are often overlooked in clinical practice. Lessons learned from our approach may assist other research institutes seeking to build artificial intelligence technologies for novel or rare diseases with limited data for training and validation.

    View details for DOI 10.1016/j.jbi.2021.103802

    View details for PubMedID 33965640

  • Learning from past respiratory infections to predict COVID-19 Outcomes: A retrospective study. Journal of medical Internet research Sang, S. n., Sun, R. n., Coquet, J. n., Carmichael, H. n., Seto, T. n., Hernandez-Boussard, T. n. 2021


    In the clinical care of well-established diseases, randomized trials, literature and research are supplemented by clinical judgment to understand disease prognosis and inform treatment choices. In the void created by a lack of clinical experience with COVID-19, Artificial Intelligence (AI) may be an important tool to bolster clinical judgment and decision making. However, lack of clinical data restricts the design and development of such AI tools, particularly in preparation of an impending crisis or pandemic.This study aimed to develop and test the feasibility of a 'patients-like-me' framework to predict COVID-19 patient deterioration using a retrospective cohort of similar respiratory diseases.Our framework used COVID-like cohorts to design and train AI models that were then validated on the COVID-19 population. The COVID-like cohorts included patients diagnosed with bacterial pneumonia, viral pneumonia, unspecified pneumonia, influenza, and acute respiratory distress syndrome (ARDS) from an academic medical center, 2008-2019. Fifteen training cohorts were created using different combinations of the COVID-like cohorts with the ARDS cohort for exploratory purpose. Two machine learning (ML) models were developed, one to predict invasive mechanical ventilation (IMV) within 48 hours for each hospitalized day, and one to predict all-cause mortality at the time of admission. Model performance was assessed using the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We established model interpretability by calculating SHapley Additive exPlanations (SHAP) scores to identify important features.Compared to the COVID-like cohorts (n=16,509), the COVID-19 hospitalized patients (n=159) were significantly younger, with a higher proportion of Hispanic ethnicity, lower proportion of smoking history and fewer comorbidities (P <0.001). COVID-19 patients had a lower IMV rate (15.1 vs 23.2, P=0.016) and shorter time to IMV (2.9 vs 4.1, P <0.001) compared to the COVID-like patients. In the COVID-like training data, the top models achieved excellent performance (AUV > 0.90). Validating in the COVID-19 cohort, the best performing model of predicting IMV was the XGBoost model (AUC: 0.826) trained on the viral pneumonia cohort. Similarly, the XGBoost model trained on all four COVID-like cohorts without ARDS achieved the best performance (AUC: 0.928) in predicting mortality. Important predictors included demographic information (age), vital signs (oxygen saturation), and laboratory values (white blood count, cardiac troponin, albumin, etc.). Our models suffered from class imbalance, that resulted in high negative predictive values and low positive predictive values.We provided a feasible framework for modeling patient deterioration using existing data and AI technology to address data limitations during the onset of a novel, rapidly changing pandemic.

    View details for DOI 10.2196/23026

    View details for PubMedID 33534724

  • Phenotyping severity of patient-centered outcomes using clinical notes: A prostate cancer use case LEARNING HEALTH SYSTEMS Bozkurt, S., Paul, R., Coquet, J., Sun, R., Banerjee, I., Brooks, J. D., Hernandez-Boussard, T. 2020

    View details for DOI 10.1002/lrh2.10237

    View details for Web of Science ID 000548944700001

  • A Qualitative Exploration of Patients' Experiences with Lifestyle Changes After Sleeve Gastrectomy in China. Obesity surgery Yu, Y., Burke, L. E., Shen, Q., Imes, C. C., Sun, R., Groth, S., Zhang, W., Kalarchian, M. A. 2020


    PURPOSE: Sleeve gastrectomy (SG) is the most widely used surgical treatment for severe obesity worldwide. Individuals who have undergone SG usually need to change lifestyle behaviors as a response to the anatomical changes imposed by SG, and patients need to sustain lifestyle changes for long-term surgical success. Little is known about how patients experience and manage lifestyle changes following SG. In China, where SG comprises over 70% of bariatric surgical procedures, there have been no reports addressing this issue. This study aimed to describe individuals' experiences related to lifestyle changes after SG in China.MATERIALS AND METHODS: Semi-structured interviews were conducted at the Shanghai Huashan Hospital in China with adults who had undergone SG between 2012 and 2018. Two independent researchers used an interpretive thematic approach to analyze transcripts for themes and sub-themes.RESULTS: Interviews (N=15) revealed three major themes of participants' experiences with postoperative lifestyle changes: advantages outweigh disadvantages; developing self-management strategies (i.e., adopting new behaviors and developing habits, continuing self-monitoring, focusing on health over weight, staying determined); and experiencing culture-specific difficulties in adherence to follow-up visits and lifestyle recommendations.CONCLUSION: The data from this study provided a rich description of the postoperative experiences of patients in China. Participants reported that surgical benefits supersede the surgery-related side effects, and participants were able to develop self-management strategies in order to achieve success. However, personal and social barriers, such as the challenges of applying postoperative dietary guidelines into daily practice, may impede patients making and sustaining recommended behavioral changes.

    View details for DOI 10.1007/s11695-020-04653-7

    View details for PubMedID 32385666

  • Reporting of demographic data and representativeness in machine learning models using electronic health records. Journal of the American Medical Informatics Association : JAMIA Bozkurt, S. n., Cahan, E. M., Seneviratne, M. G., Sun, R. n., Lossio-Ventura, J. A., Ioannidis, J. P., Hernandez-Boussard, T. n. 2020


    The development of machine learning (ML) algorithms to address a variety of issues faced in clinical practice has increased rapidly. However, questions have arisen regarding biases in their development that can affect their applicability in specific populations. We sought to evaluate whether studies developing ML models from electronic health record (EHR) data report sufficient demographic data on the study populations to demonstrate representativeness and reproducibility.We searched PubMed for articles applying ML models to improve clinical decision-making using EHR data. We limited our search to papers published between 2015 and 2019.Across the 164 studies reviewed, demographic variables were inconsistently reported and/or included as model inputs. Race/ethnicity was not reported in 64%; gender and age were not reported in 24% and 21% of studies, respectively. Socioeconomic status of the population was not reported in 92% of studies. Studies that mentioned these variables often did not report if they were included as model inputs. Few models (12%) were validated using external populations. Few studies (17%) open-sourced their code. Populations in the ML studies include higher proportions of White and Black yet fewer Hispanic subjects compared to the general US population.The demographic characteristics of study populations are poorly reported in the ML literature based on EHR data. Demographic representativeness in training data and model transparency is necessary to ensure that ML models are deployed in an equitable and reproducible manner. Wider adoption of reporting guidelines is warranted to improve representativeness and reproducibility.

    View details for DOI 10.1093/jamia/ocaa164

    View details for PubMedID 32935131

  • Phenotyping severity of patient-centered outcomes using clinical notes: A prostate cancer use case. Learning health systems Bozkurt, S. n., Paul, R. n., Coquet, J. n., Sun, R. n., Banerjee, I. n., Brooks, J. D., Hernandez-Boussard, T. n. 2020; 4 (4): e10237


    A learning health system (LHS) must improve care in ways that are meaningful to patients, integrating patient-centered outcomes (PCOs) into core infrastructure. PCOs are common following cancer treatment, such as urinary incontinence (UI) following prostatectomy. However, PCOs are not systematically recorded because they can only be described by the patient, are subjective and captured as unstructured text in the electronic health record (EHR). Therefore, PCOs pose significant challenges for phenotyping patients. Here, we present a natural language processing (NLP) approach for phenotyping patients with UI to classify their disease into severity subtypes, which can increase opportunities to provide precision-based therapy and promote a value-based delivery system.Patients undergoing prostate cancer treatment from 2008 to 2018 were identified at an academic medical center. Using a hybrid NLP pipeline that combines rule-based and deep learning methodologies, we classified positive UI cases as mild, moderate, and severe by mining clinical notes.The rule-based model accurately classified UI into disease severity categories (accuracy: 0.86), which outperformed the deep learning model (accuracy: 0.73). In the deep learning model, the recall rates for mild and moderate group were higher than the precision rate (0.78 and 0.79, respectively). A hybrid model that combined both methods did not improve the accuracy of the rule-based model but did outperform the deep learning model (accuracy: 0.75).Phenotyping patients based on indication and severity of PCOs is essential to advance a patient centered LHS. EHRs contain valuable information on PCOs and by using NLP methods, it is feasible to accurately and efficiently phenotype PCO severity. Phenotyping must extend beyond the identification of disease to provide classification of disease severity that can be used to guide treatment and inform shared decision-making. Our methods demonstrate a path to a patient centered LHS that could advance precision medicine.

    View details for DOI 10.1002/lrh2.10237

    View details for PubMedID 33083539

    View details for PubMedCentralID PMC7556418

  • A Longitudinal Examination of Patient Portal Use on Glycemic Control among Patients with Uncontrolled Type 2 Diabetes. Diabetes research and clinical practice Sun, R. n., Burke, L. E., Korytkowski, M. T., Saul, M. I., Li, D. n., Sereika, S. M. 2020: 108483


    This study examined glycemia level over a 2-year period between portal users and non-users.This retrospective cohort study used data from electronic health records (EHRs) of a large academic medical center and its ancillary patient portal. A total of 15,528 patients with uncontrolled type 2 diabetes mellitus (T2DM) were included. Using propensity score matching (PSM), portal users and non-users were balanced on demographic and clinical characteristics. Mixed-effects polynomial regression modeling was employed to evaluate the HbA1c change over time between groups.The patient sample was 85.9% (13,333) white and 52.5% (7375) male. On average, patients were 62.8 (SD, 11.7) years old and with obesity (mean BMI: 34.27.2 kg/m2) with uncontrolled T2DM (initial HbA1c: 8.51.5%). After PSM, portal users (n=4924) and non-users (n=4924) were matched on all variables except for the insurance. The mixed-effects modeling showed a nonlinear decrease of HbA1c in both groups over time. A significant interaction was observed with a greater decline, followed by a smaller rise of HbA1c in portal users than non-users.The use of the patient portal was significantly associated with a lower HbA1c. This finding supports patient portals as a promising tool for improving clinical outcomes in patients with uncontrolled T2DM.

    View details for DOI 10.1016/j.diabres.2020.108483

    View details for PubMedID 33038473

  • Outcomes of a Problem-Solving Medication Management Intervention for Informal Caregivers WESTERN JOURNAL OF NURSING RESEARCH Erlen, J. A., Sereika, S. M., Sun, R., Tamres, L. K., Tang, F., Lingler, J. H. 2019; 41 (11): 1642?57
  • Military Teen App Engagement, Safety, Commercialization, and Marketing CIN-COMPUTERS INFORMATICS NURSING Puskar, K. R., Sun, R., Gleeson, A., Rivetti, D., Nichols, D., Khan, N., McFadden, T. 2019; 37 (6): 285?88
  • Psychometric Evaluation of the Barriers to Healthy Eating Scale: Results from Four Independent Weight Loss Studies OBESITY Sun, R., Rohay, J. M., Sereika, S. M., Zheng, Y., Yu, Y., Burke, L. E. 2019; 27 (5): 700?706

    View details for DOI 10.1002/oby.22414

    View details for Web of Science ID 000466236800001

  • Sleep quality and medication management in family caregivers of community-dwelling persons with memory loss APPLIED NURSING RESEARCH Sun, R., Sereika, S. M., Lingler, J. H., Tamres, L. K., Erlen, J. A. 2019; 46: 16?19

    View details for PubMedID 30853069

  • Outcomes of a Problem-Solving Medication Management Intervention for Informal Caregivers. Western journal of nursing research Erlen, J. A., Sereika, S. M., Sun, R., Tamres, L. K., Tang, F., Lingler, J. H. 2019: 193945919825844


    Older adults with memory loss often require assistance from caregivers to manage their medications. This study examined the efficacy of a problem-solving-based intervention focused on caregiver medication management, problem solving, self-efficacy, and daily hassles. Caregiver health-related quality of life (HRQoL) and patient health care utilization were secondary outcomes. Totally, 83 patients (age 79.98.8 years) and their informal caregivers (age 66.912 years, female 69.9%, White 85.5%) were randomized; data collection occurred at baseline, 8, 16, and 24 weeks. Linear mixed modeling showed significant decreases in medication deficiencies which were sustained over time. No significant changes in caregiver problem solving, daily hassles, or patient health care utilization occurred between groups or over time. In addition, caregiver self-efficacy and mental HRQoL decreased in both groups. Physical HRQoL decreased in the intervention group, yet increased in the usual care group. Future research should investigate these outcomes in larger and more diverse samples.

    View details for PubMedID 30729881

  • Use of a Patient Portal for Engaging Patients with Type 2 Diabetes: Patterns and Prediction. Diabetes technology & therapeutics Sun, R. n., Burke, L. E., Saul, M. I., Korytkowski, M. T., Li, D. n., Sereika, S. M. 2019


    Patient portals empower patients by providing access to their health information and facilitating communication with care providers. This study aimed to examine the usage patterns of a patient portal offered as part of an electronic health record and to identify predictors of portal use among patients with type 2 diabetes (T2DM).A 2-year retrospective cohort study was performed using outpatient data from the healthcare system and its patient portal. Demographic and clinical data from 38,399 T2DM patients were analyzed. Descriptive statistics were used to summarize portal usage patterns. Binary logistic regression was employed to examine predictors and two-way interactions associated with portal use.Almost one-third of patients (n=12,615; 32.9%, 95% CI:[32.38%, 33.32%]) had used the portal for a mean 2.51.9 years prior to the study period. Portal use was higher on weekdays than weekends (P<0.001). An increase in portal use was observed in response to email reminders. A nonlinear relationship between age and portal use was observed and depended on several other predictors (Ps<0.05). Patients living in more rural areas with low income were at lower odds to use the portal (P=0.021), and this finding also applied to non-whites with low income (P<0.001). More chronic conditions and a higher initial HbA1c value were associated with portal use (P=0.014).The patient portal usage remained relatively stable over the 2-year period. A combination of factors is associated with an individual's patient portal use. Patient engagement in portal use can be facilitated through a proactive approach by healthcare providers.

    View details for DOI 10.1089/dia.2019.0074

    View details for PubMedID 31335206

  • Military Teen App: Engagement, Safety, Commercialization, and Marketing. Computers, informatics, nursing : CIN Puskar, K. R., Sun, R. n., Gleeson, A. n., Rivetti, D. n., Nichols, D. n., Khan, N. n., McFadden, T. n. 2019; 37 (6): 285?88

    View details for DOI 10.1097/CIN.0000000000000544

    View details for PubMedID 33055492

  • Subjective versus Objective Measures of Sleep: Results From an Ecological Momentary Assessment Study Imes, C. C., Kline, C. E., Mendez, D. D., Sun, R., Yang, Y., Chasens, E. R., Rathbun, S. L., Burke, L. E. LIPPINCOTT WILLIAMS & WILKINS. 2019
  • Psychometric Evaluation of the Relapse Situation Efficacy Questionnaire in a Weight Loss Treatment Study AMERICAN JOURNAL OF HEALTH BEHAVIOR Yu, Y., Shiffman, S., Rohay, J. M., Ma, Q., Sereika, S. M., Sun, R., Burke, L. E. 2018; 42 (4): 80?89


    Objectives We assessed the psychometric properties of the Relapse Situation Efficacy Questionnaire - Weight (RSEQ-W) including internal consistency reliability, criterion-related validity (concurrent and predictive validity) and construct validity (convergent validity and factor analysis). Methods We administered the RSEQ-W at baseline, and at 6 and 12 months in a 12-month prospective behavioral weight loss study. Spearman correlations were used to examine the convergent and concurrent validity; multivariate linear regression was used to assess the predictive validity; exploratory factor analysis was conducted using principal component analysis. Results The sample (N = 148) was 90.5% women and 81.1% white with a mean body mass index of 34.1 4.6 kg/m2. The RSEQ-W showed good internal consistency (Cronbach's ? = .95) and convergent validity (r = .69). PCA results revealed that the 31 items can be factored into 6 components negative affect, positive affect, social occasions, low focus, activity andlack of energy. Conclusion These results provide preliminary support for the reliability and validity of the RSEQ-W. Future work needs to apply RSEQ-W in studies with larger and more diverse samples and also consider adding more items to the factor lack of energy.

    View details for DOI 10.5993/AJHB.42.4.8

    View details for Web of Science ID 000437980800008

    View details for PubMedID 29973313

  • Electronic Messaging Use via a Patient Portal Is Associated with Primary Care Utilization in Patients with Type 2 Diabetes Sun, R., Sereika, S. M., Saul, M., Korytkowski, M. T., Li, D., Burke, L. E. AMER DIABETES ASSOC. 2018
  • Patient Portal Use in Diabetes Management: Literature Review. JMIR diabetes Sun, R. n., Korytkowski, M. T., Sereika, S. M., Saul, M. I., Li, D. n., Burke, L. E. 2018; 3 (4): e11199


    Health information technology tools (eg, patient portals) have the potential to promote engagement, improve patient-provider communication, and enhance clinical outcomes in the management of chronic disorders such as diabetes mellitus (DM).The aim of this study was to report the findings of a literature review of studies reporting patient portal use by individuals with type 1 or type 2 DM. We examined the association of the patient portal use with DM-related outcomes and identified opportunities for further improvement in DM management.Electronic literature search was conducted through PubMed and PsycINFO databases. The keywords used were "patient portal*," "web portal," "personal health record," and "diabetes." Inclusion criteria included (1) published in the past 10 years, (2) used English language, (3) restricted to age ?18 years, and (4) available in full text.This review included 6 randomized controlled trials, 16 observational, 4 qualitative, and 4 mixed-methods studies. The results of these studies revealed that 29% to 46% of patients with DM have registered for a portal account, with 27% to 76% of these patients actually using the portal at least once during the study period. Portal use was associated with the following factors: personal traits (eg, sociodemographics, clinical characteristics, health literacy), technology (eg, functionality, usability), and provider engagement. Inconsistent findings were observed regarding the association of patient portal use with DM-related clinical and psychological outcomes.Barriers to use of the patient portal were identified among patients and providers. Future investigations into strategies that engage both physicians and patients in use of a patient portal to improve patient outcomes are needed.

    View details for PubMedID 30401665

    View details for PubMedCentralID PMC6246970

  • Student Perceptions of Classic and Game-Based Online Student Response Systems. Nurse educator Wang, W. n., Ran, S. n., Huang, L. n., Swigart, V. n. 2018


    Online student response systems (OSRSs), driven by the Internet and cell phone technology, provide a free, easily accessible method to increase student engagement, facilitate active learning, and provide learners and teachers with instant feedback about learning progress.This article describes undergraduate nursing students' use of 2 OSRSs and their perceptions of the impact of the tools on class participation and engagement.Students used their own mobile phones or computers to access 2 types of OSRSs: a classic and a game-based OSRS.Students indicated that both systems increased participation and engagement. The game-based OSRS was favored over the classic OSRS. The potential for use of the game-based OSRS for assessing rapid-answer fact-based knowledge and the classic OSRS for assessing more complex learning tasks is discussed.Nurse educators are encouraged to consider integrating online response system technology into their classroom teaching.

    View details for PubMedID 30130268

  • Handling Internet-Based Health Information: Improving Health Information Web Site Literacy Among Undergraduate Nursing Students JOURNAL OF NURSING EDUCATION Wang, W., Sun, R., Mulvehill, A. M., Gilson, C. C., Huang, L. L. 2017; 56 (2): 110?14


    Patient care problems arise when health care consumers and professionals find health information on the Internet because that information is often inaccurate. To mitigate this problem, nurses can develop Web literacy and share that skill with health care consumers. This study evaluated a Web-literacy intervention for undergraduate nursing students to find reliable Web-based health information.A pre- and postsurvey queried undergraduate nursing students in an informatics course; the intervention comprised lecture, in-class practice, and assignments about health Web site evaluation tools. Data were analyzed using Wilcoxon and ANOVA signed-rank tests.Pre-intervention, 75.9% of participants reported using Web sites to obtain health information. Postintervention, 87.9% displayed confidence in using an evaluation tool. Both the ability to critique health Web sites (p = .005) and confidence in finding reliable Internet-based health information (p = .058) increased.Web-literacy education guides nursing students to find, evaluate, and use reliable Web sites, which improves their ability to deliver safer patient care. [J Nurs Educ. 2017;56(2):110-114.].

    View details for DOI 10.3928/01484834-20170123-08

    View details for Web of Science ID 000398044500008

    View details for PubMedID 28141885

  • Is There An Association Between Comorbid Conditions and Self-Efficacy for Weight Loss Among Participants in Weight Loss Intervention Study? Sun, R., Imes, C., Yu, Y., Danford, C., Goode, R., Hu, L., Ma, Q., Loar, I., Mendez, D., Sereika, S., Zheng, Y., Burke, L. LIPPINCOTT WILLIAMS & WILKINS. 2016: E105?E106

Stanford Medicine Resources: