As my loyal reader knows, I am violating all of my principles and engaging in a linear-regression version of a symbolic regression project to test how ideas in dimensional analysis (or units equivariance) might impact machine learning. I have been struggling to get a sparse regression working, because when problems get large, optimizing combined L1 and L2 losses can be sticky and tricky. But Soledad Villar (JHU) saved me today by pointing out that in the over-parameterized regime (when you have more terms in your linear regression than data), you can do a sparse regression with a cleverly designed linear program! Woah, we coded it up and it Just Worked (tm)! We can get the exactly correct total mechanical energy expression in our toy problem with a very small amount of training data, far less than we needed when we were using L2 as our objective. Far less.
No comments:
Post a Comment