empirical template RVs

As my loyal reader knows, I am finishing up an old paper on information theory, radial velocities, and measurement. Today I wrote the code that produces radial-velocity measurements using cross-correlations with an empirical template that is built from the data themselves. I learned a lot doing this! One thing I learned is that edge effects matter: It is important to either design your template, or else design your method, such that as you redshift or blueshift the template, lines that enter or leave the spectral range don't mess up your radial-velocity measurements. In principle this is all handled correctly by correct statistics, but in practice it requires attention!

what are the responsibilities of letter writers?

The Astronomical Data Group Meeting at Flatiron is a remarkable weekly meeting. It is always impressive. Contributions are categorized into categories like words, statistics, visualization, and feelings and so on. But many of the most interesting conversations have come from the feelings category! Today was no exception. Alex Gagliano (UIUC) asked about the ethical responsibilities of letter writers (as in: letters of recommenation) to applicants, and the responsibilities the other way. It led to a great conversation about many things that we all think about but are not written down, taught, or explicitly discussed in our field. What are we doing? The union of the feelings-categorized conversations we have had in group meeting would make an admirable textbook!

linear programming FTW

As my loyal reader knows, I am violating all of my principles and engaging in a linear-regression version of a symbolic regression project to test how ideas in dimensional analysis (or units equivariance) might impact machine learning. I have been struggling to get a sparse regression working, because when problems get large, optimizing combined L1 and L2 losses can be sticky and tricky. But Soledad Villar (JHU) saved me today by pointing out that in the over-parameterized regime (when you have more terms in your linear regression than data), you can do a sparse regression with a cleverly designed linear program! Woah, we coded it up and it Just Worked (tm)! We can get the exactly correct total mechanical energy expression in our toy problem with a very small amount of training data, far less than we needed when we were using L2 as our objective. Far less.

Gaia DR3 plans: much better distances; unicorns

Katie Breivik (Flatiron) and I convene a weekly meeting at Flatiron for NYC-area astronomers to discuss and work on things we are going to do with the ESA Gaia DR3 and SDSS-V data. Today I showed people what Price-Whelan (Flatiron), Eilers (MIT), and I are planning on doing with the 10⁸ (!) very low-resolution Gaia BpRp spectra, which will be released in DR3 this summer: We plan on making a much-improved version of this method for estimating spectrophotometric distances to stars but covering the whole HR diagram and with better overall data and precision (we hope). In the meeting today we discussed ways to validate our distances (which is hard!) and we discussed what interesting side things might appear as part of the project. One category, that Dalcanton (Flatiron) liked, is this: Since our method involves comparing stars to other stars in their neighborhoods in spectrophotometric space, we will automatically and unintentionally find unicorn stars that are unlike any others. Those will be interesting for follow-up.

I broke everything

My loyal reader knows that Soledad Villar (JHU) and I are trying to get a paper on units-equivariance (machine learning that exactly obeys the symmetries of dimensional analysis) ready in time for the ICML deadline. Maybe not a good idea! Anyway, right now none of our numerical experiments are working all that well and today I broke one even further. We have several different, equivalent forms for unit-equivariance. My code had been written for one, and then we wrote the paper for a different one. So I updated my code and now nothing works. The old and new codes aren't identical, so it makes sense that they differ. But it used to work. Argh. I can use version control to go backwards, but the problem is: Then the code won't be consistent with the framework we advocate in the paper.

making fake RV data

I spent the day hacking away in notebooks to make fake but realistic radial-velocity data, and to test that maximum-likelihood and cross-correlation are good methods for measuring the radial velocities themselves, given the data. In cross-correlation the issues are subtle: If the templates are not normalized absolutely correctly, things can go badly wrong. But I can now empirically justify my theoretical claim that cross-correlation (with suitsbly normalized templates) saturates the bounds guaranteed by maximum-likelihood estimation.

This, by the way, is not new: I, and many others, have done all this previously. My previous work here was with Megan Bedell (Flatiron), and indeed I am hacking around to resurrect some of our past results.

more data is always better, right?

In Astronomical Data Group Meeting at Flatiron today, Lily Zhao (Flatiron) asked a great question about how to combine radial-velocity data: Imagine you have a very high-resolution echelle-type spectrograph with many spectral orders. You get a stellar radial-velocity out of every order (or, in some cases, out of many small spectral patches in every order. Now: How to combine those radial-velocity measurements into one, true radial velocity measurement? Obviously you just do an inverse-variance weighted average, no? Well, no! It doesn't work right. There are bad orders, and the only way to know is to see that they make the measurements worse in some end-to-end variance sense. So what to do? How do you empirically determine how to do the combination of data? This problem is simultaneously trivial and impossible. It's a great subject of discussion, and one I've mentioned here previously.

the bolometric correction of Vega is not zero

Wow I went Inside Baseball (tm) today on magnitudes. Magnitudes were originally designed to be purely observational, purely relative measurements, relative to a standard star. But then once you go off-course and define absolute magnitudes (gasp) and then even farther off and define bolometric magnitudes (double gasp), the system is no longer relative to anything! It depends on (imprecise) distance estimates and (wrong) stellar models. So you get many absurdities, like that Vega is not zero in any Vega-relative absolute or bolometric magnitude system. I think maybe we've lost our way! And not because magnitudes are obtuse; apparent magnitudes and colors make lots of sense! We lost our way when we tried to extend this to cover luminosities of various kinds.

weighing the disk with the Snail

Today we had a presentation from Axel Widmark (NBI), who has traced the spiral structure in the Snail (tm), which is the phase spiral in the vertical dynamics discovered by Antoja and others in the ESA Gaia data. He uses the trace to estimate the density of the Milky Way disk. Widmark's approach reminds me of old-school work on stream tracks (by, for instance, Johnston and others, but also maybe me?): He traces the spiral ridge-line and asks for it to represent a frequency dependence on vertical energy. That's a good idea! And it seems to work. And it motivated me to think more about methods that—rather than just trace a linear locus—perform full distribution-function fitting in a time-independent potential.

why does anyone need a bolometric correction?

As I mentioned yesterday, and against my better judgement, I am writing a long pedagogical document on magnitudes and how they work. I got to the bolometric-correction part and I realized that I didn't understand exactly why they are defined and how they are used. So I had a long think. The answer is: Theoretical models of stars are better at predicting the total bolometric luminosity output (energy per time) coming from a star than they are at predicting the detailed spectrum. Thus it makes sense to separate the belief about the total luminosity from the belief about the luminosity in any one band.

the magnitude system: go big or go home

I spent some time on the weekend and today writing in my various writing projects. Actually, I am obliged to do this: I have a two-paragraph-per-day New Year's resolution. One thing I am (stupidly) writing is a note about how apparent magnitudes work, in order to (ultimately) explain the bolometric correction to physicists. This latter concept is confusing because when you measure through a limited band, you don't see the whole stellar spectrum, and yet the bolometric correction can be positive or negative or zero. Today as I was writing in this document, I realized that I have to do it all: Absolute magnitudes, distance moduli, color excesses, and so on, if I am going to reach my intended audience. I expanded the outline of the paper and I feel good about where it's going. That said, shouldn't I be doing research?

detecting shifts with cross-correlations

Lily Zhao (Flatiron) has been looking at the variability of the SDSS-IV BOSS spectrograph calibration frames, in the thought that if we can understand them well enough, we might be able to substantially reduce the observing overheads for the robotic phase of SDSS-V. To start, we are doing simple cross-correlations between calibration images. After all, the dominant calibration changes from exposure to exposure appear to be shifts. But these cross-correlations have weird features in them that I don't understand: In general if you are cross-correlating two images that are very similar, the cross-correlation function should look very symmetric, I think? But Zhao's cross-correlation functions look asymmetric in weird ways. We ended our session puzzled.

other people's code

I spent much of the day understanding Other People's Code (tm). One piece of code is legacy code from Aaron Dotter (Harvard) that computes magnitudes from flux densities. I was looking at it to confirm my frantic writing of yesterday. Another piece of code is the code by Weichi Yao (NYU) that implemented our group-equivariant machine learning. I was looking at it to see if we can modify it to impose units equivariance (dimensional scaling symmetries). I think we can, and I confirmed that it Just Works (tm).

magnitudes are (logarithmic) ratios of signals

A couple of weeks ago I had intense arguments with Belokurov (Cambridge) and Farr (Stony Brook) about the definition of a photometric bandpass and a photometric magnitude. And a few months ago I had long conversations with Breivik (Flatiron) about the bolometric correction. For some reason these things took over my mind today and I couldn't stop myself from starting a short pedagogical note about how magnitudes are related to spectral energy distributions. It's not trivial! Indeed, the integral isn't the integral you naively think it might be, because most photometric systems count photons (they don't integrate energy). People often say that a magnitude is negative-2.5 times the log of a flux. But that's not right! It is negative-2.5 times the log of a ratio of signals measured in two experiments.

building dimensionless monomials

I got stuck today on a problem that seems trivial but is in fact not at all trivial: Given N inputs, how to make all possible dimensionless monomials of those N inputs, at (or less than) degree d. For our purposes (which are the units-equivariant machine-learning projects I have with Soledad Villar, JHU), a dimensionless monomial of degree d is a product of integer powers of the inputs, in which those powers can be positive or negative, such that the dimensions cancel out completely, and for which the the max (or sum or some norm) of the absolute values of the powers is ≤d. We have a complete basis of dimensionless monomials, such that any valid dimensionless monomial can be expressed as a monomial of the basis monomials. Because of this, the dimensionless monomials can be seen as the vertices of a Bravais lattice, technically. The problem is just to traverse the entire lattice within some kind of ball. Why is this hard? Or am I just dull? I feel like there are fill algorithms that should do this correctly.