finding emission lines (and other oddities) in hot stars

I showed my robust spectral decomposition (dimensionality reduction) and residuals to the MPIA Binaries group today. There was much useful feedback (including that my H-gamma was actually H-delta; embarassing!). One comment was that the model isn't truly a causal separation between star and lines, so there will be some mean lines in the star model; lines aren't entirely outliers. That's true! The group suggested that I iterate to remove stars with lines from the training set.

After the meeting, I implemented some of that, but problems like this have a pathology: If you carefully remove stars with high residuals at some wavelength, then the training data will be deficient, or low, at that wavelength. And then the model will go lower, and then more stars will have excess at that wavelength and: Disaster. So when I implemented, I required a 2-sigma deviation, and I removed both high and low outliers. I don't know if this will work, but I am testing now.

young stars in SDSS-V

Over the last two weeks, I built a new robust dimensionality-reduction method called Robust-HMF. This method is a hammer, looking for a nail. This week, Hans-Walter Rix (MPIA) suggested that I use the method on young stellar objects observed in SDSS-V BOSS spectra. I did that, and I found hundreds of young stars with narrow H-alpha emission lines. It turns out that the Robust-HMF method does really well on these data; it can fit all the absorption lines in the stars, plus all the wacky continuum shapes generated by a combination of instrumental effects and dust attenuation. That is, I can run the method on more-or-less raw data, provided that it has been shifted to rest frame (and the raw SDSS-V pipelines do that pretty well on stars like these).

Anyway, I don't know what to say about my results quantitatively yet, but I find hundreds of emission-line stars, and my equivalent-width sensitivity is excellent. Who wants this sample?

The Cannon and El Cañon

At the end of the day I got a bit of quality time in with Danny Horta (Flatiron) and Adrian Price-Whelan (Flatiron), who have just (actually just before I met with them) created a new implementation of The Cannon (the data-driven model of stellar photospheres originally created by Melissa Ness and me back in 2014/2015). Why!? Not because the world needs another implementation. We are building a new implementation because we plan to extend out to El Cañon, which will extend the probabilistic model into the label domain: It will properly generate or treat noisy and missing labels. That will permit us to learn latent labels, and de-noise noisy labels.

Happy birthday, Rix

Today was an all-day event at MPIA to celebrate the 60th birthday (and 25th year as Director) of Hans-Walter Rix (MPIA). There were many remarkable presentations and stories; he has left a trail of goodwill wherever he has gone! I decided to use the opportunity to talk about measurement, which is something that Rix and I have discussed for the last 18 years. My slides are here.

I've been very lucky with the opportunities I've had to work with wonderful people.

divide by your selection function, or multiply by it?

With Kate Storey-Fisher (San Sebastián), Abby Williams (Caltech) is working on a paper about large-angular-scale power, or anisotropy, in the distribution of quasars. It is a great subject; we need to estimate this power in the context of a very non-trivial all-sky selection function. The tradition in cosmology is to divide the data by this selection function. But of course you shouldn't manipulate your data. Instead, you could multiply your model by the selection function. You can guess which one I prefer! In fact you can do either, as long as you weight the data in the right way in the fit. I promised to write up a few words and equations about this for Williams.

auto-encoder for calibration data

Connor Hainje (NYU) is looking at whether we could build a hierarchical or generative model of SDSS-V BOSS spectrograph calibration data, such that we could reduce the survey's per-visit calibration overheads. He started by building an auto-encoder, which is a simple, self-supervised generative model. It works really well! We discussed how to judge performance (held-out data) and how performance should depend on the size of the latent space (I predict that it won't want a large latent space). We also decided that we should announce an SDSS-V project and send out a call for collaboration.

[Note added later: Contardo (SISSA) points out that an autoencoder is not a generative model. That's right, but there are multiple definitions of generative model; only one of which is that you can sample from it. Another is that it is a parameterized model that can predict the data. Another is that it is a likelihood function for the parameters. But she's right: We are going to punk parts of the auto-encoder into a generative model in the sense of a likelihood function.]

Terra Hunting Fall Science Meeting, day 2

So many good things happened in the meeting today! Highlights were presentations by Niamh O'Sullivan (Oxford) Ben Lakeland (Exeter) who showed amazing results running models of stellar variability on data from the Sun. O'Sullivan can see that the sun goes through many different phases of spots, granulation, and super-granulation. She finds these by fitting Gaussian processes of certain forms. Related: Suzanne Aigrain (Oxford) showed that even in very gappy data, the GP fits are unbiased, whereas naive use of periodograms is biased!

Lakeland showed that super-granulation can in principle be modeled in the Solar time series, and maybe the tiniest hint that when he corrects for super-granulation well, the RV variability might be even lower than at times at which there is no super-granulation in play at all. Does super-granulation suppress other kinds of variability?

I'm very optimistic—between Liang yesterday, Zhou's work at Flatiron, and these presentations—that we will be able to mitigate many difficult sources of stellar variability. I was inspired to outline a conceptual paper on why or how this is all going to work.

double periodogram

Cole Johnston (Leuven) is in New York this week. We discussed the problem of finding oscillation modes in the photometry of stars in the presence of a large, binary-induced periodicity. What he kind-of wants is a simultaneous fitting of a flexible periodic function plus a periodogram. We did some experiments (very promising!) and discussed the elements that will come together to make this all happen. The final method will look like a double fourier transform, in which one frequency grid gets the periodic part, and the other grid gets the rest of the modes and noise.

radical papers I want to write (or will never write)

I have to finish my NSF proposal with Mike Blanton (NYU), so naturally I am in procrastination mode. Here are three papers I wish I would write. Maybe I should post them on my ideas blog:

Occam's Razor is wrong: This paper, co-authored with Jennifer Hill (NYU), would be about the fact that, in the real, observed world, the simplest explanation is always wrong or at least incomplete.

Causation is just causality: This paper, maybe co-authored with David Blei (Columbia) or Bernhard Schölkopf (MPI-IS) or Hill, shows that you don't need to have free will in order to have cogent causal explanations of data. That is, you don't need to phrase causality in terms of predictions for counter-factual experiments that you might have chosen to do.

You don't ever want evidence: This paper shows that any time you are computing the Bayesian evidence—what I call the fully marginalized likelihood (fml)—you are doing the wrong integral and solving the wrong problem. For both practical and theoretical (principled) reasons.

predicting spectra from spectra

Saakshi More (NYUAD) came into my office during office hours today to ask about possible data science projects in physics. I pitched to her predicting ESA Gaia RVS spectra from Gaia XP spectra, and vice versa. Has anyone done that? In one direction, you have to predict high resolution detail from low-resolution input; in the other direction, you have to predict a wide wavelength range from narrow input. It seems like perfect for something like a linear auto-encoder (at least for a small patch of the color–magnitude diagram; non-linear for a large patch). Later in the day I talked to Gaby Contardo and she said: If you want to go simple, how about nearest neighbor? Good idea!

toning down my language

I spent travel time (at airports and on airplanes) working on the title, abstract, and introduction of the forthcoming paper with Andy Casey (Monash) about combining visit spectra into mean spectra. This was mainly about me changing the tone from “You are all doing it wrong!” to a tone more like “Here's a way to think about it, and the consequences thereof.” After all, no method is the best for all situations and cases. Our method is best for situations where the individual visit spectra are barely sampled or under-sampled.

biases from machine learning

Today I gave a talk (with these slides) at a meeting in Denver for the NSF initiative Harnessing the Data Revolution. I spoke about the necessity and also the dangers of using machine-learning methods in scientific projects. I brought up two very serious possible biases. The first is that if emulators are used to replace simulations, and they can't be easily checked (because the simulation requirements are too expensive), the emulators will lead to a confirmation-bias problem: We will only carefully check the emulations if they lead to results that we don't like! The second bias I raised is that if we perform joint analyses on objects (stars, say) that have been labeled (with ages, say) by a machine-learning regression, there will in general be strong biases in those joint analyses. For example, the average value of 1000 age labels for stars labeled by a standard ML regression will not be anything like an unbiased estimate of the true average age of those stars. These biases are very strong and bad! That said, I also gave many example locations where using machine learning methods is not just okay but actually intellectually correct, in areas of instrument calibration, foregrounds, and other confounders.

The question period was great! We had 25 minutes of questions and answers, which ranged across a very wide set of topics, including statistics, experimental design, and epistemology.

Bayesian evidence?

Kate Storey-Fisher, Abby Williams, and I spent some time discussing unpublished work that relies heavily on calculations of the Bayesian evidence. Bayesian evidence—what I call the “fully marginalized likelihood”—relates to the volume of the posterior in parameter space. It is generally extremely sensitive to the width of the prior pdf, since if you are comparing two models with different parameterizations, the numbers you get depend on how you normalize or scale out the units of those parameter-space volumes. Indeed, you can get any evidence ratios you want by tuning prior pdf widths. That's bad if you are trying to conclude something, scientifically! Bayesian inference is only principled, imho, when you can quantitatively state the prior pdf that correctly describes your beliefs, prior to seeing the new data. And even then, your evidence is special to you; any other scientist has to recompute from scratch.

representation of flexible functions

Emily Griffith (Colorado) and I met today to look at replacing a spline interpolation function deep inside some of our code with a Fourier series. The idea is that we need a flexible function of one variable, and we were using a spline of a set of control points, but (for many reasons) we wanted to change to a sum of sines and cosines. The code work was a mess! The small change hits a lot of places inside our model, which is our K-process data-driven nucleosynthetic model. This same problem appears in the new version of wobble by Matt Daunt (NYU). I love flexible functions, but it's hard to implement them in a properly abstracted way. That is, it is hard to write a model so that you can just swap in a Fourier series or a Gaussian process where you used to have an interpolation of control points.

machine-learning theory and practice

Today I got invited to be on a panel discussion (hosted by Soledad Villar of JHU) with Alberto Bietti (Flatiron) about the theory and practice of machine learning. It was great! We talked about why ML works for scientific applications, and Bietti said something (obvious maybe) that I loved: Maybe ML only works because of properties of the data. That is, maybe when we are analyzing ML methods we are looking in the wrong place, and we should be analyizing the data to which they are successfully applied? I made fun of interpretation in ML, and that led to interesting comments from both Bietti and the audience. Several audience members suggested taking something more like a causal approach to interpretation: How does the method work under interventions or in conditional situations? That's interesting; it isn't what a physicist would consider interpetation, but it might be sufficient in many cases.

is a periodic signal in a time series statistically significant?

I had conversations with Nora Eisner (Flatiron) and Abby Shaum (CUNY) today about how we report the significance of a signal we find in a time series. In particular a periodic signal. It's an old, unsolved problem, with a lot of literature. And various hacks that are popular in the exoplanet community (and binary-star community!). My position is very simple: Since all methods for determining significance are flawed, and since when you fit a signal you have to estimate also an uncertainty on that signal's parameters, the simplest and most basic test of significance is the significance with which you measure the amplitude of the proposed signal. That is, if the amplitude is well measured, the signal is real. Of course there are adversarial data sets I can make where this isn't true! But that's just a restatement of the point that this is an unsolved problem. For deep reasons!

an alternative to the L–S periodogram

Following some experiments and rants over the last few days with Nora Eisner (Flatiron), I wrote down today an algorithm for a hacky replacement of the Lomb–Scargle periodogram. This periodogram method has various bad pathologies, the worst of which is that it presumes that there is exactly one frequency that fully generates the data. If there are two, the assumptions are broken and the good properties are lost.

Not that my alternative has any good properties! It is like the radio interferometry method called CLEAN: It involves iteratively identifying frequencies and fitting them out. It's terrible. But it might be better than some of the wacky hacks that people do right now in the asteroseismology community.

elusive quasar dipole

There should be an imprint of the kinematic dipole observed in the cosmic microwave background in any cosmological tracer: The dipole is set by the Solar System barycentric velocity relative to the local Hubble flow, and that same velocity should imprint a dipole on anything cosmological. I have been working on this in part because it is a good measurement to make with Quaia, and in part because the dipole in the quasars is controversial. I have many thoughts, but I will save them for later.

Anyways, Abby Williams (NYU) has been working on making this measurement, and her dipole amplitude and direction depend on what we hold fixed and what we vary (in particular selection-function components), and they also depend on what sky region we use. None of this is surprising; the selection function has a strong dipole in it, and it is not known precisely. But then I don't understand how the studies published previously have such good error bars. Maybe they didn't consider the various different fitting regimes?

an insight about machine learning

Gaby Contardo (SISSA) completed her visit to Heidelberg today. Over coffee this morning she delivered a very simple, but very nice insight about machine learning outputs. Apologies that this is very Inside Baseball:

As I like to emphasize, you can't really average (or do any populations inferences with) a collection of labels delivered by a discriminative ML method run on a collection of objects. Think: Finding the mean age of a cluster, where each star in the cluster got an age estimate from a discriminative ML method trained on stars with known ages. This is because the discriminative ML methods output something very akin to posterior quantities, and if you average a bunch of posterior estimates, you are multiplying in a prior times itself many times; eventually the prior dominates the inference (in many cases).

Contardo's point: If what you want is a label for a collection of objects, like that mean age, you should train on collections of objects. That is, make a training set where you have sets of N stars, labeled by mean age. Then this model can be applied to a new collection of stars and deliver a mean age estimate! Haha, brilliant. And correct. And consistent with the rules of inference.

mapping the image plane of a spectrograph

I had a phone conversation about wavelength-calibrating the multi-object APOGEE instrument with Karlo de Leon (NYU) today. He has arc images from each night, and line lists for the arc lamps. But before even using the arc lamps, I recommended that he try to find a model for the 2D images that is an outer product of 1D functions: One is the intensity as a function of wavelength from the arc lamp, and the other is the intensity as a function of slit position from the fibers on the slithead.

The thing we realized in the call is that the coordinate system is right when the image is well described as the outer product of these two functions, warped according to that coordinate system! Okay that's nice, now what will the residuals look like? One issue is that there are cosmic rays, hot pixels, and so on. Another issue is that there will be some vignetting that violates the strict outer-product model. We'll address these issues once we get close.