Maybe I'm known in astronomy for being both a machine-learning developer and a machine-learning skeptic. I hope so! Anyway, I love linear regression, because it has a lot of the power of bigger ML models, but it's easy to implement and to understand. And yet!
Today Kate Storey-Fisher (NYU) and I looked at her code to predict galaxy properties given dark-matter-halo properties in a set of n-body simulations. We are doing very simple regressions but the condition numbers of the matrices are blowing up and some of our answers don't look great. And this is generic: Many linear-regression models are messed up by condition numbers and numerical linear algebra, and it is hard to diagnose, and it is hard to treat. And if linear regression is hard—and hard for us—why do I believe anything that inovolves 42 layers of fully-connected RELU network?
No comments:
Post a Comment