Interaction Terms Are Tricky Little Devils
So, I saw this image:
And I got to thinking. When you see something like this, where the data seems to follow totally different shapes depending on a single factor, in linear regressions we start looking for an interaction term.
What is an interaction term? If y is predicted by x and z, we have a couple of possible equations. We have y= x + z, we have y= x*z, and we have y= x + z + x*z. x*z is an interaction term.
And so this got me thinking. Maybe fat is simply colinear with the real cause of health problems. Or maybe it’s an interaction term that’s getting all the blame.
Of course, first I needed to figure out if that was even possible in statistics. So I fired up R, and generated some data:
data=data.frame(x=rep(0,50),z=rep(0,50),y=rep(0,50))
c=1
for (i in 1:50) {
data$x[i] = c+rnorm(1,0,10)
data$z[i]= 2*c-4+rnorm(1,0,50)
data$y[i]= .1*data$x[i]*data$z[i]+rnorm(1,0,1) + data$x[i]
c = c+1
}
This is 50 points of random data, where y is predicted by an x*z interaction term, and by x alone. But if circumstances are just right, z is gonna look guilty.
I started out by measuring correlation:
> cor(data)
x z y x 1.0000000 0.3797703 0.6577556 z 0.3797703 1.0000000 0.7615119 y 0.6577556 0.7615119 1.0000000
As you can see, z and y look a bit more related than x and y. So I made up some linear models:
lmx= lm(y~x, data=data)
lmz=lm(y~z, data=data)
lmxz=lm(y~x+z, data=data)
lmxzi=lm(y~x+z+x:z, data=data)
I checked out their summaries. In lmx, lmz, and lmxz, all the predictors were highly significant. Then, on lmxzi, a miracle occurred. The p-value on the stand-alone z-term was 0.898. That’s very, very insignificant. I just might have done it.
So far, I’d shown half of my hypothesis. You could make a highly significant term drop out by including an interaction term. But was z alone a better predictor than x alone? And was the interaction model better than a single-predictor model?
To answer these questions, I calculated the Akaike Information Criterion on each model. I won’t go into terribly much detail, but it’s a number to describe how good a model is. The lower the number, the better the model. The AIC on lmx was 646, and the AIC on lmz was 631. lmz was the best single-predictor model. In comparison lmxz and lmxzi had AICs of 609 and 128 respectively.
I did it. The data I generated had z as the best single predictor, despite the fact that z showed up in the best model only as an interaction term. And the best model? It blew the competition right out of the water.
So watch out for interaction terms. They’re tricky little devils.
