what constitutes an exoplanet discovery?

Over the last few days, Megan Bedell (Flatiron) and I have been discussing the criteria that go into a detection of an exoplanet, or what things need to be true for a detection or discovery to be considered made. We are thinking of discoveries with the Terra Hunting Experiment or other radial-velocity surveys. We identified three-ish condtions:

One condition is that the amplitude be significantly different from zero; that is, the null is ruled out. Another is that the planet be characterizable: that is, the orbital parameters can be estimated to some level of precision. A third is that the planetary explanation of the signal be preferred at some confidence over other qualitatively different explanations, like stellar variabilities, stellar rotation, or signals faked by beats from other planets.

I presented these criteria at Astronomical Data Group meeting, and Dan Foreman-Mackey (Flatiron) said that he disagreed with every aspect of it. But we didn't (yet) find out why.

how to show spectra to astronomers?

OMG I posted a paper to arXiv today! It should appear on Monday.

In the morning, I met with Villar (NYU) and Huang (NYU) to discuss Huang's paper on adversarial attacks against machine-learning regression methods in astronomical contexts. We spent a lot of the call discussing how to display the results in such a way that the astronomers will understand them and see how strange they are. Our main target is spectroscopic methods, so this involves displaying spectra.

finished a paper!

I didn't have a great day today. So I took the last few hours of the day and tried to finish the final to-do items on my paper with Price-Whelan (Flatiron) and Leistedt (Imperial) on multiplying Gaussians. Yes, multiplying together Gaussians. How can I have 13 pages of things to say about that? But I actually did finish, and we might submit to arXiv tomorrow (for appearance on Monday). This is my first first-author paper in a long time (and it isn't a research contribution, really).

almost nothing

Today was an almost-nothing research day. I did speak to Eilers about her projects, and Villar about double descent and organizing our results. Price-Whelan about writing. But almost nothing else today.

double descent is a thing

Yesterday in the Flatiron Astronomical Data Group weekly meeting, I showed the crew something called double descent: When you are training a model and the number of data points you have (number of training-set objects) approaches the number of features you have (number of pixels in your image, say) then regression models often blow up. That is, you get better answers with fewer training objects when the number approaches certain values. This is a highly discussed issue in machine learning (and math and statistics; it's like some kind of phase transition) but hasn't really hit the domains (like astronomy) very much yet. The crew was surprised so today I made a tiny colab notebook to demonstrate it for a polynomail fit. It's amusing!

how to design figures and captions?

Lily Zhao (Yale) gave Megan Bedell (Flatiron) and I a tour of the figures in our nascent paper on spectrograph calibration. For each figure we asked: Does this figure tell a part of our story? And: Is that point clear in both the figure and the caption? My philosophy is that most of your readers are not going to curl up with a hot cocoa on the sofa and luxuriously read your paper; most of your readers are going to skim your paper on their phones, while waiting in line to pick up a coffee. So you want the paper to be comprehensible on a skim. That skim probably involves settling on figures and captions, so these should tell the whole story of the paper, or as best they can. Of course I don't always follow my own advice here, but I should.

homework problem

This week Megan Bedell (Flatiron) assigned me a homework problem, which is to work out the information theory (or whatever) that will tell us how sensitive a particular radial-velocity survey is to exoplanets. I started to write this down as a problem and a solution today. It might evolve into a paper if we have enough to say!

search and characterization

I had a wide-ranging conversation with Ana Bonaca (Harvard) about many things. One thing we discussed is the relationship between what astronomers call search and what astronomers call characterization. We were talking about this in the context of asteroseismology signals. But it is there for exoplanet searches, binary-star searches, and even the Higgs boson. There are certain respects in which search and characterization are very similar: Most search methods involve parameter estimation over some grid of models (periods and phases, say, estimating amplitudes, or masses, estimating coupling parameters). But there are certain respects in which they are very different too: Search involves harsh approximations and cheats for performance; characterization involves high-quality inference with baroque models. Anyway, it is interesting to think about these worlds and ideas and how much astronomy has changed since I was a wee astronomer.

page layout, the assumptions of linear models

I spent way more of my research time today than I should have working on the page layout of the paper I am currently writing with Price-Whelan and Leistedt. My goal is to make a document that works well either printed on paper, or read on a computer screen, or read on a phone. And a document that includes footnotes and figures, which the user can see without endless scrolling up and down the document. One reason I am so against typesetting two-column (as many astronomy papers do) is that it is very challenging to read on a phone, at least current-generation phones.

I also spent some time today reading in the Agresti book on linear models. Although this book is a bible for linear models and generalized linear models, it states at the very outset (yes, in Chapter 1) that every single method in every chapter of the entire book will presume that there is no noise in the features, only noise in the labels. (Think: You are predicting noisy labels y given features x.) That assumption—no noise in x—is fine, but it is violated in every important example I know in every single application of any kind of linear model (or nonlinear model of course) in every area, academic and commercial.

Other than that, it's a good assumption.

how to write the (very mathematical) method section?

Eilers (MIT) started the day with a discussion of how, specifically, to modify our paper given Friday's discussion of potential perturbations and realism in steady-state dynamical models. It is obvious that a mixture of rigidly rotating fixed patterns can mock up what we want for our potential perturbation, but then computing each of these is not simple, since resonances occur. So what to do to mock up something that is more like a continuous integral over an infinite family such perturbations, which would have no such pathologies? We have various hacks in mind, but there are trade-offs between having more complete math and having more discursive discussion about the math. I usually prefer the latter. But if part of our audience is graduate students trying to reproduce the result, it might make sense to get more math in there.

invert, solve, and least-squares

One of the things I throw into seminars and papers these days is about numerical linear-algebra operators. My advice is: Never use the invert() function to invert a matrix! Why not? Because you almost never want the inverse, you want the inverse applied to a vector. So when you are tempted to do dot( inv(A), x ) you should instead do solve( A, x ). The first gets a machine-precision (if you are lucky) inverse of A and applies it to x. The latter gets a machine-precision (if you are lucky) estimate of A-inverse applied to x. You see the difference?

Well, all of that got complexified today when I learned (from Soledad Villar NYU) about lstsq(). This does the same as solve() (and more) but it is zero-safe. Meaning: It gives good answers even when the A matrix has zero eigenvalues. I think my advice may be evolving.

true confessions of dynamicists

There was an hour-long conversation this morning among the coauthors of Eilers et al. We discussed inconsistencies in the model as described in that paper. It was a wide-ranging discussion! There are no self-consistent, steady-state, non-axisymmetric models for disks! So we are building models in which a symmetric disk is stirred or modified by a rigid potential. Our issues are all about how to make this model useful for understanding the Milky Way, which is certainly not being stirred by a rigid, rotating, m=2 potential perturbation! There was quite a range of positions on the call, but I think pragmatism won the day. We all left a bit confused.

noisy power spectra

Ana Bonaca (Harvard) and I continued today our discussion of the determination of nu-max and delta-nu in asteroseismic observations of stars. We find that there are cases in which it is obvious-ish to an asteroseismic expert which peaks in the spectrum are the forest of lines, so delta-nu can be read, but in which our probabilistic model for the spectrum sees no strong evidence for that delta-nu. The asteroseismologists know what they are doing, so I suspect either a bug or a think-o. But the spikiness of power spectra is concerning. It's hard to know what's believable. Especially to a newcomer like me. And noise in a power spectrum is famously structured.

funding! and interpolation!

Kate Storey-Fisher (NYU) got great news today. Her NASA FINESST proposal was funded! So she gets paid for the next three years. And she did it herself. Funding is one of the most challenging, confusing, disheartening, and complicated parts of this job. I don't like it, but it's reality. Congratulations, Storey-Fisher

On my student-research call, I worked on interpolating complex numbers with Avery Simon (NYU). We have a set of phase transforms of two very similar images, which return, for components, complex amplitudes (or amplitudes and phases, if you like). Now the question is: How to interpolate between these images, or their phase transforms? We want to test the hypothesis (put forward in computer-vision research like this) that interpolating in complex phase is way better in many respects than interpolating naively.

debugging copy pasta

It has been nice, during the quarantine, to be doing some coding again. Of course my coding is very amateurish these days. I am not obeying good code practices! For one, I am working in Jupyter notebooks, which are very bad for good code practice: They encourage cut-and-paste and discourage building structures and modules. Today I found many bugs in my linear-regression statistics and information-theory code. Most were from repeated code blocks. Any time I am writing the same code twice, I am almost certainly making a mistake!