2018-06-01

regularized regression, self-calibration of spectroscopists

Christina Eilers (MPIA) and I went off the reservation today and implemented a L1-regularized linear regression method for our spectroscopic-parallax project. This permitted us to consider using the full spectrum as a feature vector, and not just derived quantities. That is, it obviated a lot of our feature engineering! But we also discovered a massive bug: We had been using the uncertainties where we should have had our inverse variances! That's been done before. But when we made these changes, we got better results; it looks like we might be able to beat 10-percent distances with a little more work.

Andy Casey (Monash) and Natalie Hinkel (Vanderbilt) showed me the self-calibration results they have from the Hypatia Catalog. The results are beautiful! They have affine translations between labels in one survey into labels in another survey, and de-noised labels for all surveys. It is cool! Much more needs to be done. But a great start, and very promising for answering some of my questions about accuracy and precision.

At my data-group group meeting, each participant had a short time interval to explain a figure they are working on. The range of subjects shown was amazing! And we learned that having everyone talk for a well-defined pre-set short time is better than having a few people talk for an undefined amount of time. That's a win.

No comments:

Post a Comment