Prediction Model Performance for Binary Outcomes in R

Interested in learning how to evaluate the performance and reliability of prediction models? Assessing model performance is a critical step in determining whether a prediction model is accurate, trustworthy, and suitable for research or clinical applications. Through live demonstrations and hands-on exercises in R, participants will gain practical experience using widely accepted methods to evaluate binary outcome prediction models. This workshop introduces key concepts and techniques for assessing prediction model performance, including discrimination, calibration, and internal validation. Participants will learn how to generate and interpret common performance metrics and create clear, publication-ready visualizations to communicate their findings effectively.

This session is ideal for researchers, graduate students, and analysts interested in developing and evaluating prediction models. Basic familiarity with R and regression modeling concepts is recommended.

By the end of this session, participants will be able to:

1. Develop prediction models for binary outcomes and generate predicted probabilities in R.
2. Assess model discrimination using the C-statistic (AUC) and receiver operating characteristic (ROC) curves.
3. Evaluate model calibration using calibration plots and calibration statistics.
4. Perform internal validation using resampling techniques such as bootstrapping and cross-validation.
5. Create publication-ready figures, tables, and summaries to effectively communicate prediction model performance.

Details: Any preparatory work for the session can be found on its information page. This virtual workshop will be recorded and shared on the same page, and discoverable via the Sherman Centre's Online Learning Catalogue.

Certificate Eligibility: This workshop is eligible for the Sherman Centre's certificate program. For more information, visit scds.ca/certificate-program. It is also eligible for the Canadian Certificate for Digital Humanities. To learn more, visit ccdhhn.ca or contact scds@mcmaster.ca.