Report
Kyoto University School of Public Health Short Course from November 30th to December 4th, 2020 (17:00-20:00 JST)
Introduction to Methods for Developing Clinical Prediction Models

Speakers:
Prof. Georgia Salanti, Professor, Institute of Social and Preventive Medicine (ISPM), University of Bern, Switzerland.
Dr. Orestis Efthimiou, Postdoctoral Research Fellow, Institute of Social and Preventive Medicine (ISPM), University of Bern, Switzerland.

Opening Remarks:
Opening remarks were made by Dr. Ayako Kohno from Internationalization Promotion Office (IPO), School of Public Heath, Kyoto University (KUSPH). After that, Prof. Toshiaki Furukawa from the Departments of Health Promotion and Human Behavior and Clinical Epidemiology, KUSPH, introduced the speakers.

Summary:
This was a 5-day Short Course lectures by Prof. Salanti and Dr. Efthimiou, teaching basics concepts and methods for clinical prediction modelling. We discussed various topics on prediction modeling, and we did practical exercises using R software.

At the beginning of the course, Prof. Salanti explained the course objectives. She explained the definition and type of the models, model development, and model validation with concrete examples. After Dr. Efthimiou gave an introduction to statistical methods for predictive modeling, he taught us that type of predictive model depends on the outcome type and not the covariates. He explained that linear regression is typically used for continuous variables, whereas logistic regression is used for binary outcomes. Cox regression is used for survival analysis when we are interested to see the effects of predictors (hazard ratios). He also mentioned that parametric survival models offer a better choice for building prediction models for survival outcomes.

Dr. Efthimiou introduced the concepts of underfitting and overfitting in prediction modeling. Overfitting is an issue, especially when we use highly complicated models and/or the sample size is small. He explained that models applying very complex models can lead to overfitting, if one is not careful. Overfitting can be solved by shrinking or penalizing the model. Shrinkage is needed especially in small datasets and datasets with many predictors. Other methods to prevent overfitting are having larger datasets incorporating external knowledge or including fewer parameters.

All the modern estimation methods include tuning parameters that control for complexity aiming to reach the sweet spot in the bias-variance tradeoff. Overfitting will produce models that perform very well in the training data creating optimism in the model performance but are invalid for the new data. Assessing and correcting for optimism can be achieved using bootstrapping techniques.

Prof. Salanti discussed methods for accounting for missing data and also methods for incorporating non-linear terms in the models using splines. She also presented methods for the evaluation of a model’s performance and methods for validating a prediction model. Dr. Efthimiou discussed some advanced methods for prediction modeling, including modern machine learning methods.

Lastly, Prof. Salanti discussed the results of practical on model performance, and explained the methods of decision curve analysis (DCA). After that, three participating students form KUSPH presented their prognostic research projects and received feedback from Prof. Salanti and Dr. Efthimiou.

We had active Q & A sessions with questions related to the topics on all five days.

Special Thanks to:
Prof. Toshi Furukawa, Dean, Professor of
Kyoto University School of Public Health for inviting Prof. Salanti and Dr. Efthimiou.

Organized by:
Ayako Kohno, Program-specific Assistant Professor
Internationalization Promotion Office (IPO),
Kyoto University School of Public Health

Staff:
Mai Takeshita, DrPH. candidate, Dept of Health Informatics
Swati Mittal, MPH candidate, Dept of Health Informatics