writing down the linear regression GP relationship

My loyal reader knows that I have been working on the fundamentals of linear regression for a bit now. Today I did some writing on this topic. Last week, Soledad Villar (JHU) and I got the point that we could write down a specific case where the limit of infinite features in a particular, carefully designed linear regression becomes exactly a Gaussian Process with a particular, carefully chosen kernel. I understand how to generalize this result partially, but not completely: Apparently this will work in an infinity of different bases, with an infinity of different weighting functions or kernels. My goal is to write something pedagogical and useful for practitioners.

human aspects of data analysis

We had a fun data group meeting today, in which we discussed many human aspects of data analysis (like asking questions in talks and seminars, and sharing work when it is in pre-publication status). I spoke about the connection between Gaussian processes and linear fitting with enormous numbers of basis functions; there is a limit in which they become identical, which is awesome. Group meeting was followed by a conversation with Storey-Fisher about what we are going to work on next: Pulsar timing? Looking for anomalies in large-scale structure? Intensity mapping?

talking about writing

It was a low-research day today. But I did have a great conversation with Viviana Acquaviva (CUNY) about the textbook she is writing on machine learning for advanced undergraduates in the natural sciences.

a Gaussian process is a limit of linear regression

Today Soledad Villar (JHU) and I completed a problem I've had open for literally years (I think I first worked on it in AstroHackWeek 2017): Does a linear fit become a Gaussian process when the number of components (parameters) goes to infinity? The answer is yes! But you have to choose your features very carefully, and take the limit (to infinite features) sensibly. But if you meet those conditions it works, and the kernel function for the GP becomes a Fourier transform of the squares of the amplitudes of the features. That is, the kernel function in real space is the Fourier transform of the power spectrum in fourier space. There are many details I don't yet understand, but we got it working both theoretically (on paper) and numerically (on the computer).

writing philosophy about EPRV

Lily Zhao (Yale), Megan Bedell (Flatiron), and I are working on a project to look at stellar spectral variations in extreme-precision radial-velocity spectroscopy, with EXPRES data. Do these stellar spectral variations tell you anything about stellar noise that distort radial-velocity measurements? This project is very specific and technical, but it connects to some deep ideas in velocity measurement:

In principle you can only precisely measure radial-velocity changes in a star, never the precise absolute or systemic radial velocity. But this precision argument depends on having a constant spectrum. If the spectrum varies, there is no rock to stand on. So this project requires some philosophical backing, I think. I tried to write some of that down this morning. I love stuff like this!

rebooting MySpace

Some people of a certain age will know what it means when you say that MySpace is dead. But the statement is wrong! Today Jason Hunt (Flatiron) rebooted an old project by Price-Whelan and mine called MySpace. The motivation for our reboot: The upcoming ESA Gaia EDR3.

The idea is to figure out how the velocity-space structure (the moving groups, as it were) in the local disk varies with position, with a data-driven model, and then interpret the variations with position in terms of dynamical properties of the Milky Way. Hunt's innovation is to apply this same procedure to simulations as well as data, and use the output to classify the velocity substructure (classify as in: Does it come from resonances or disrupting clusters or what?). We discussed some of the math and optimization involved. Because we phrase this as the fitting of an expansion.

Kate submits her first first-author paper!

We submitted Kate Storey-Fisher's (NYU) paper on estimating the correlation function to the AAS Journals (probably ApJ, but they decide now, not us). I am so excited. It's been a great project and it has beautiful results and—if we can get this method adopted—we will save future missions and projects a lot of compute time. (And therefore reduce their carbon footprints!)

[Note added later: Here is the manuscript.]

big, huge linear regressions

I spoke (remotely) at CCA today about linear regression (fitting linear models for the purposes of prediction), when the linear regressions have huge numbers of parameters. Yes huge: More than the number of data points! It turns out that even though you can thread the data perfectly—your chi-squared will be exactly zero—you can still make good predictions for held-out data. That surprised the crowd, which, in turn, surprised me: Many in this crowd use Gaussian processes and deep learning, both of which have these properties: More parameters than data, can fit any training data perfectly, and yet still make good, non-trivial predictions on held-out data.

My slides are here. Should I write something about all this?

Bode's-Law noise

When we think about finding extra-solar planets from the reflex motions they imprint into stellar radial-velocity data, we think about the problem of noise: There is shot noise, there are spectrograph-calibration offsets, there are imprints of the atmosphere, there is surface convection on the star and asteroseismic modes, there is magnetic activity, flaring, and so on! It's a mess. But there's also noise from other, unmodeled and undiscovered planets. That is, the other things orbiting the star, other than the planet of interest.

Today, Winston Harris (MTSU), Megan Bedell (Flatiron) and I came up with a plan for inserting this planetary-system noise into Harris's simulations of radial-velocity data. The question arose: What periods to use for the planets? And Bedell suggested that we adapt the Titius–Bode law! Hilarious. This gives us an extra-solar system architecture that makes sense, and simultaneously trolls anyone reading our paper.

[Insert here obligatory objection to naming things after people.]

finishing a paper is hard!

I spent research time today with Kate Storey-Fisher on the final details in her new paper on a continuous-function estimator for the 2-point correlation function. It removes binning from the estimation (all current estimators bin), and makes the results far less computationally costly to interpret.

CCA leadership retreat

This afternoon, David Spergel (Flatiron) and the CCA group leaders (and some friends) had a retreat to discuss long-term mission and plans. Given the global sitch, this retreat was only partially in-person. We discussed the plans of all the groups, and whether there is a coherent, cross-cutting mission statement that could be adopted by the full CCA. I think there is! We also discussed the structure of the organization, and how we want to be organized in the future. Not research, maybe? But in support of research, in the long run.

spectral-shape influences on radial-velocity measurements

As my loyal reader knows, I have been interested in finding out how radial-velocity measurements of stars (for, say, exoplanet discovery) are affected by shape changes in the stellar spectrum. Today Lily Zhao (Yale) had a breakthrough: She did regression of residuals away from a constant-spectrum fit to radial-velocity data, and showed that the residuals can be used to predict the radial velocity! That is, she can show that stellar spectrum shape predicts measured radial velocity, over and above the expected Doppler shift. And she did this with proper cross-validation, so the result looks solid. I'm stoked! I need to write down some theory this weekend.

radial-velocity vs photometric variability at short periods

Jonah Goldfine (NYU) is looking, with Adrian Price-Whelan (Flatiron) and me, at short-period binary star systems in the NASA TESS data. We find that most of the short-period binaries that Price-Whelan finds in the APOGEE radial-velocity data have interesting variability in their photometry in the TESS data. Today we compared light curves folded on the Price-Whelan period found by The Joker with light curves folded on the period found with the Lomb-Scargle periodogram. There are lots of stars where The Joker gets the period better than the light curve, which is surprising, since we are validating with the light curve! There are so many kinds of variability to consider. Goldfine is going to start with ellipsoidal variations, I think.

forwards vs backwards predictions; astrometry from rv

Gaby Contardo (Flatiron) and I are asking whether you can predict the next data point in a stellar light curve from the last N data points or whether you can post-dict (is that a word?) the previous data point in a light curve from the next N. The results are surprisingly rich. She showed some of them in Stars and Exoplanets Meeting today.

In that same meeting I showed my attempt to measure the astrometry (celestial position and proper motion) of a star from the radial-velocity variations you see on an Earth-bound observatory. It is surprisingly precise! But not as precise as direct astrometric measurements.

funding astrometry.net as an open-source project

Dustin Lang (Perimeter) called me today and alerted me to this NASA funding call related to open-source projects. He argued that we need to take Astrometry.net/ to the next level. I agree! So we kicked around project and development ideas and vowed to take a stab at a letter of intent.