We submitted Kate Storey-Fisher's (NYU) paper on estimating the correlation function to the AAS Journals (probably ApJ, but they decide now, not us). I am so excited. It's been a great project and it has beautiful results and—if we can get this method adopted—we will save future missions and projects a lot of compute time. (And therefore reduce their carbon footprints!)
[Note added later: Here is the manuscript.]
I spoke (remotely) at CCA today about linear regression (fitting linear models for the purposes of prediction), when the linear regressions have huge numbers of parameters. Yes huge: More than the number of data points! It turns out that even though you can thread the data perfectly—your chi-squared will be exactly zero—you can still make good predictions for held-out data. That surprised the crowd, which, in turn, surprised me: Many in this crowd use Gaussian processes and deep learning, both of which have these properties: More parameters than data, can fit any training data perfectly, and yet still make good, non-trivial predictions on held-out data.
My slides are here. Should I write something about all this?
When we think about finding extra-solar planets from the reflex motions they imprint into stellar radial-velocity data, we think about the problem of noise: There is shot noise, there are spectrograph-calibration offsets, there are imprints of the atmosphere, there is surface convection on the star and asteroseismic modes, there is magnetic activity, flaring, and so on! It's a mess. But there's also noise from other, unmodeled and undiscovered planets. That is, the other things orbiting the star, other than the planet of interest.
Today, Winston Harris (MTSU), Megan Bedell (Flatiron) and I came up with a plan for inserting this planetary-system noise into Harris's simulations of radial-velocity data. The question arose: What periods to use for the planets? And Bedell suggested that we adapt the Titius–Bode law! Hilarious. This gives us an extra-solar system architecture that makes sense, and simultaneously trolls anyone reading our paper.
[Insert here obligatory objection to naming things after people.]
I spent research time today with Kate Storey-Fisher on the final details in her new paper on a continuous-function estimator for the 2-point correlation function. It removes binning from the estimation (all current estimators bin), and makes the results far less computationally costly to interpret.
This afternoon, David Spergel (Flatiron) and the CCA group leaders (and some friends) had a retreat to discuss long-term mission and plans. Given the global sitch, this retreat was only partially in-person. We discussed the plans of all the groups, and whether there is a coherent, cross-cutting mission statement that could be adopted by the full CCA. I think there is! We also discussed the structure of the organization, and how we want to be organized in the future. Not research, maybe? But in support of research, in the long run.
As my loyal reader knows, I have been interested in finding out how radial-velocity measurements of stars (for, say, exoplanet discovery) are affected by shape changes in the stellar spectrum. Today Lily Zhao (Yale) had a breakthrough: She did regression of residuals away from a constant-spectrum fit to radial-velocity data, and showed that the residuals can be used to predict the radial velocity! That is, she can show that stellar spectrum shape predicts measured radial velocity, over and above the expected Doppler shift. And she did this with proper cross-validation, so the result looks solid. I'm stoked! I need to write down some theory this weekend.
Jonah Goldfine (NYU) is looking, with Adrian Price-Whelan (Flatiron) and me, at short-period binary star systems in the NASA TESS data. We find that most of the short-period binaries that Price-Whelan finds in the APOGEE radial-velocity data have interesting variability in their photometry in the TESS data. Today we compared light curves folded on the Price-Whelan period found by The Joker with light curves folded on the period found with the Lomb-Scargle periodogram. There are lots of stars where The Joker gets the period better than the light curve, which is surprising, since we are validating with the light curve! There are so many kinds of variability to consider. Goldfine is going to start with ellipsoidal variations, I think.
Gaby Contardo (Flatiron) and I are asking whether you can predict the next data point in a stellar light curve from the last N data points or whether you can post-dict (is that a word?) the previous data point in a light curve from the next N. The results are surprisingly rich. She showed some of them in Stars and Exoplanets Meeting today.
In that same meeting I showed my attempt to measure the astrometry (celestial position and proper motion) of a star from the radial-velocity variations you see on an Earth-bound observatory. It is surprisingly precise! But not as precise as direct astrometric measurements.
Dustin Lang (Perimeter) called me today and alerted me to this NASA funding call related to open-source projects. He argued that we need to take Astrometry.net/ to the next level. I agree! So we kicked around project and development ideas and vowed to take a stab at a letter of intent.
Today was a board meeting for the Terra Hunting collaboration. It was an important meeting, because in it, we more-or-less approved adding a laser-frequency comb to the required hardware of the project. This increases our budget, and brings in new partners. But I'm a huge fan (or I should say “And I'm a huge fan”), because our new partners will be awesome, and, just as important, as Lily Zhao, Megan Bedell, and I have shown, the LFC is great for constructing a fully data-driven, non-parametric calibration method for the spectrograph.
Calibration paper to appear on arXiv soon!
In our paper on factoring products of Gaussiasn, we recommend using the matrix inversion lemma (and we should have recommended the matrix determinant lemma too) to speed calculations. This weekend I wrote the most efficient numpy code I could to implement the log-Gaussian formulae in that paper. Here were my rules: Never construct a zero matrix or a sparse matrix using diag() or anything else that makes a container of mainly zeros. Never use inv(), only solve(). Always store and re-use elements of the calculation, especially those shared between the inversion and determinant lemmas. In the end I got a factor of thousands in speed over naive implementations, which themselves were hundreds of times faster than just constructing the matrices and operating on them.
That was fun, and a rare moment of pure coding for me.
As my loyal reader knows, our radial-velocity companion-search code called The Joker is a brute-force least-square fitting (plus some Bayes) of an elliptical orbit at every possible period, orientation, and phase. In astrophysics, the Lomb–Scargle periodogram is a workhorse tool that, under the hood, is a brute-force least-square fitting of a sinusoid at every possible period and phase. So these two ideas are fundamentally incredibly similar. And indeed, today Winston Harris (MTSU) demonstrated quantitatively that these are the same, and will become identical as eccentricities go to zero. That's interesting, because if we are doing companion search and we don't mind making the approximation that eccentricities are zero, the fitting of sinusoids (even at arbitrary phase, because: trig identities) is way, way faster than the fitting of Kepler functions.
Today Juna Kollmeier (Carnegie) convened a tiny meeting with Dalal, Percival, and me to discuss the next generation of cosmology missions and projects. We wandered around metrics, around data-analysis methods (forward modeling?), and new hardware, but didn't come up with much specific to say, yet. But Kollmeier is right to be thinking forward, because the landscape is changing and the most interesting objects of cosmological research are evolving on time-scales shorter than project execution. Of course that's always the case for interesting disciplines!
We're in the process of finishing a paper for submission to AISTATS. I spent time on the weekend and Monday working on figures, and I basically failed. But luckily Teresa Huang (JHU) picked it up and made really nice figures that show the double-descent phenomena in regression, the susceptibility of methods to “data poisoning” attacks, and the good effects of well-chosen regularizations.
Adam Wheeler (Columbia) and I went through points in his (very constructive) referee report on his paper using data-driven methods to find stars with abundance anomalies. We also discussed what to do next in this line of research. We are both leaning towards even more data-driven things, like finding parts of the spectrum that do and don't predict other parts of the spectrum. Information-theory-like things.
This weekend, I did something that Teresa Huang (JHU) has been suggesting I do for a while in our project on linear regression: Plot the null prediction. That is, how much better do our regression methods do than a trivial prediction of zero, no matter what, for every data point? It turns out that her intuitions were correct: There are many settings for our regression problems in which the null prediction does better than any data-driven regression! What gives? If the training data are noisy, it is possible for any training to move your predictions in the worse direction from the null prediction. Hahaha, that's something we have to think about.
Today I hid in an undisclosed location and worked on a submission for AISTATS, which is due next week. The paper is with Teresa Huang (JHU) and Soledad Villar (JHU) and it is about regression and robustness thereof, as measured by prediction accuracy and sensitivity to adversarial attacks.
My loyal reader knows that Adrian Price-Whelan (Flatiron) and I are trying to finish a paper in which we image the orbital tori in the phase-space of the Milky Way using chemical abundances of stars. We are using a very simplistic, toy model of the galaxy mass distribution, which then generates orbits, which trace the tori. In the process of writing this up, we realized that there must be a fully data-driven, flexible model for the orbital tori that we could use. In this formulation we would find the best closed-3-torus foliation of 6-dimensional phase space, and then only after finding these tori, interpret them in terms of the force law and hence the mass model. Interesting thought whether there are useful math tools that do this.
Winston Harris (MTSU) and I looked at the output of The Joker and the Lomb-Scargle periodogram on his fake exoplanet radial-velocity data. They maybe look similar. It makes sense, since they are both doing a simple likelihood-based linear fit of periodic functions.
I spent some time putting together a notebook to test the idea that Josh Winn (Princeton) had that the Solar System barycentric radial-velocity correction could be indicative of the stellar astrometry for the very brightest stars, where precise astrometry is (strangely) hard. It should be possible at some level, since the barycentic correction does depend on the astrometry! I love the idea of doing astrometry by taking radial-velocity measurements.
I joined Hans-Walter Rix's group meeting at MPIA today by telecon so I could see Anthony Brown (Leiden) present the plans for the ESA Gaia eDR3 data release. It's going to be great, with proper motions improving by a factor of two (in standard error; a factor of four in information content). But full DR3 isn't until 2022, so we don't get low-res spectra, high-res spectra, and other such goodies for a while. Now, how to get ready for eDR3? What projects to concentrate on?
I talked to Jason Hunt (Flatiron) this afternoon about things we might do with the ESA Gaia early-DR3 data (December 2020), which will be similar to DR2 but with improved parallaxes and proper motions. In 2022 we'll get the complete DR3, which contains lots of qualitatively new data and measurements. But in the meantime, eDR3 is great for projects that we did—or tried to do—in DR2 but were limited by proper-motion precisions (especially, since proper motions get better by a lot). One such project is work Hunt did with Bovy on velocity substructure in the disk. So I pitched to him the MySpace project that Price-Whelan and I formulated a while back. I have an intuition that it should work well with the new data.
I got some weekend research time in, which was fun: When I work on the weekends, I try to make it things that I want to do, rather than things I have to do. I wrote code to predict a point in a NASA Kepler light curve from the preceding points and then from the following points. And it appears that the two predictions disagree, as expected, at gaps in the light curve (where there is a stretch of missing data) and at discontinuities in the light curve (like those created by small stellar flares). So it is interesting! And just linear regression of course (that's my brand). Now: Can we do science with it?
This was originally inspired by a comment by Bernhard Schölkopf (MPI-IS), years ago, about whether we can predict the past from the future better than we can predict the future from the past, which might have relationships to causal inference. I'm enjoying thinking about the philosophical aspects.
Today was a busy day. The first half of the day was the Terra Hunting Science Meeting, day 3. We substantially narrowed down our target-selection criteria, simplifying in the process. Tim Naylor (Exeter) ran a great meeting.
The second half of the day was me giving a physics colloquium at UC Merced. I spoke about Chemical Torus Imaging, which is our new name for the project code-named Chemical Tangents, with Price-Whelan. I started at the beginning, with Kepler and Newton, to motivate the idea that we can do better inferences if we can see the orbits.
Today Gaby Contardo (Flatiron) and I discussed the following problem, which (I find) is hard to even formulate: The red-giant branch (above the red clump) is supposed to be a place where most stars are “going up” (getting brighter and bigger) and a minority of stars are “going down”, heading back towards the red clump. And yet, all the stars on the red-giant branch look very similar spectrally (though not asteroseismically!). Can we see in the spectra (I'm thinking APOGEE here), in a purely data-driven way (that is, without a supervisory training set), that the red-giant branch is a folded locus that projects to a simple line, and not just a simple line? This paper by Ting, Hawkins, & Rix gives me hope that we can.
I'm interested in all this because it gets at what can be just observed in the data, as compared to what we project onto the data from theory. This project is a toy, in some sense, because we do know about the asteroseismic differences, and we can use them, without being theoretically supervised.
