Read on Twitter
Several questions on Cross Validated asked about interpretation of estimated model coefficients in a binary logistic regression model, so I thought I would write a thread on some issues to bear in mind when performing this interpretation. Buckle up for the ride! #rstats
To keep things simple, let's say we fit a binary logistic regression model which predicts admission status in grad school (admit) from gre score for students selected at random from some target population of students. Here, admit = 1 if student is admitted and 0 otherwise.
In R, this binary logistic regression model can be fitted with the commands:
mydata <- read.csv(" https://stats.idre.ucla.edu/stat/data/binary.csv")
model <- glm(admit ~ gre, data = mydata,
family = binomial(link="logit"))
summary(model)
mydata <- read.csv(" https://stats.idre.ucla.edu/stat/data/binary.csv")
model <- glm(admit ~ gre, data = mydata,
family = binomial(link="logit"))
summary(model)
Now the fun begins - the results produced by the model can be interpreted on 3 different scales:
1. The log-odds scale;
2. The odds scale;
3. The probability scale.
So, for interpretation purposes, we need to be clear on what scale we picked.
1. The log-odds scale;
2. The odds scale;
3. The probability scale.
So, for interpretation purposes, we need to be clear on what scale we picked.
The log-odds scale is perhaps the most difficult to understand but it's what comes out from the model summary produced by R. It's an additive (points) scale:
Each additional GRE point is associated with an increase of 0.003582 points in the log-odds of admission to grad school.
Each additional GRE point is associated with an increase of 0.003582 points in the log-odds of admission to grad school.
The effect of gre on the log-odds of admission is assumed to be linear & that's why we can quantify it using a slope coefficient for gre:
library(effects)
effect_link<-Effect("gre", mod = model)
plot(effect_link, type="link",
main = "gre effect plot\\n(log odds scale)")
library(effects)
effect_link<-Effect("gre", mod = model)
plot(effect_link, type="link",
main = "gre effect plot\\n(log odds scale)")
Some people would find the (multiplicative) odds scale more intuitive to work with than the (additive) log-odds scale. To shift to it, just exponentiate the coefficient of gre reported by R in its model summary:
exp(0.003582) = 1.003588
exp(0.003582) = 1.003588
The exponentiated coefficient of gre represents a multiplicative increase in odds of admission associated with an additional GRE point: the odds of admission increase by a multiplicative factor of 1.003588 for each additional GRE point.
It's often easier to express a multiplicative increase on the odds scale in terms of a percentage increase:
(exp(0.003582) - 1) x 100% = (1.003588 - 1) x 100% = 0.36%
The odds of admission increase by 0.36% for each additional GRE point. This is a really small increase.
(exp(0.003582) - 1) x 100% = (1.003588 - 1) x 100% = 0.36%
The odds of admission increase by 0.36% for each additional GRE point. This is a really small increase.
The last scale we could use for our interpretation purposes is the probability scale. Because the effect of gre is assumed linear on the log-odds scale but nonlinear on the probability scale, we need to visualize what it looks like on the probability scale first.
library(effects)
effect_response <- Effect("gre", mod = model)
plot(effect_response, type="response",
main = "gre effect plot\\n(probability scale)")
effect_response <- Effect("gre", mod = model)
plot(effect_response, type="response",
main = "gre effect plot\\n(probability scale)")
Then we can describe the effect of gre on the probability of admission in qualitative terms:
The GRE score has a positive, nonlinear effect on the probability of admission.
The GRE score has a positive, nonlinear effect on the probability of admission.
In particular, the probability of admission is estimated by our model to be around 0.1 for those with a GRE score in the low 200s, 0.2 for those with a GRE score of 500 and slightly below 0.5 for those with a GRE score of 800.
Even if we include additional predictors in our model, the same scale considerations will hold for our interpretation. Which scale we use depends on the audience. Generally, the log odds scale is the hardest to understand and the probability scale is the easiest. The end!
