machine-learning theory and practice

Today I got invited to be on a panel discussion (hosted by Soledad Villar of JHU) with Alberto Bietti (Flatiron) about the theory and practice of machine learning. It was great! We talked about why ML works for scientific applications, and Bietti said something (obvious maybe) that I loved: Maybe ML only works because of properties of the data. That is, maybe when we are analyzing ML methods we are looking in the wrong place, and we should be analyizing the data to which they are successfully applied? I made fun of interpretation in ML, and that led to interesting comments from both Bietti and the audience. Several audience members suggested taking something more like a causal approach to interpretation: How does the method work under interventions or in conditional situations? That's interesting; it isn't what a physicist would consider interpetation, but it might be sufficient in many cases.

is the world lagrangian?

My day started with a long and very fun conversation with Monica Pate (NYU) about conservation laws in classical physics. As we all know, conservation laws are related to symmetries; each symmetry of the laws of physics creates a conservation law. Or does it? Well, it's a theorem! But it's a theorem when the laws of physics are lagrangian (or hamiltonian). That is, every symmetry of a hamiltonian system is associated with a conservation law in that system. So I asked: How do we know if or whether the world is lagrangian or hamiltonian? How could we know that? My best guess is that we know it because of these very conservation laws! The situation is complex.

planets forming in a disk

At the end of last week I had a great conversation with Valentina Tardugno (NYU) and Phil Armitage (Flatiron) about how planets form. I spent the whole weekend thinking about it: If a few planets are forming in a proto-planetary disk, there are all sorts of interactions between the planets and the disk, and the planets and each other, and the disk with itself. You can think of this (at least) two different ways:

You can think of planets which are interacting not just directly with one another, but also with a disk, and with each other as mediated by that disk. This is the planet-centric view. In this view, the planets are what you are tracking, and the disk is a latent object that makes the planets interact and evolve.

Alternatively, you can think of the disk, with planets in it. In this disk-centric view, the planets are latent objects that modify the disk, creating gaps and spiral waves.

Both views are legitimate, and both have interesting science questions. We will explore and see where to work. I am partial to the planet-centric view: I want to know where planetary systems come from!

planning your science

I had two interactions today that made me think seriously about big-picture and design things. I like design language: How do you design your whole research program, and how do you design individual projects so they fit into it. One interaction was in the Astronomical Data Meeting at Flatiron, where Vivi Acquaviva (CUNY) talked about the intersection between what you are good at, what is important, and what brings you joy. That's a hard intersection to find. Or way too easy; I am not sure. The other interaction was a conversation with Jiayin Dong (Flatiron), who is thinking about faculty job applications and the like. How to talk about your research in terms of the next decade instead of the next year?

One comment that is frequently made by Hans-Walter Rix (MPIA) is that he feels like most early-career (and even mid-career) people spend too much time doing their science and not enough time planning and justifying their science. It is important to be able to answer “why” questions about your research, and in the medium term it helps all your projects.

lost

I got really lost with respect to research today. In almost all of my projects I am supposed to be mentoring postdocs and students. Today various blocks came up that interfered with that mentoring. And then I found that I had nothing sensible to work on! Of course that isn't true: I have literally a dozen projects in a mature state waiting on final work from me. But I couldn't figure out how to work on any of them. Research is hard. At the end of the day, Andy Casey (Monash) helped me out by giving me some very specific jobs to do.

uncertainty estimation for regression outputs

Most methods for performing regressions don't provide natural uncertainties. Some do, of course! But few deliver uncertainties you will believe. I discussed these issues with Contardo (SISSA) today, in the context of our project to (confidently) find infrared excesses around boring old main-sequence stars. One option is to look at the performance on held-out data. But then you have to decide how to aggregate this information in a way that is relevant for each object in your sample: They probably don't all have the same uncertainty! Another option is to look at the variation of prediction across training sets. That's good! But it requires that you have lots of training data. In this case, we do, so that's where we are at right now.

regressions for point clouds

I spent my research time today writing in a document that proposes (and demonstrates) some methods for performing machine-learning-style regressions, but where the input objects (features) are variable-size point clouds. Contributions also from Villar (JHU) and Gebhard (MPI-IS). I spent way too long working out the terminology and notation, and I am still wrong.

is a periodic signal in a time series statistically significant?

I had conversations with Nora Eisner (Flatiron) and Abby Shaum (CUNY) today about how we report the significance of a signal we find in a time series. In particular a periodic signal. It's an old, unsolved problem, with a lot of literature. And various hacks that are popular in the exoplanet community (and binary-star community!). My position is very simple: Since all methods for determining significance are flawed, and since when you fit a signal you have to estimate also an uncertainty on that signal's parameters, the simplest and most basic test of significance is the significance with which you measure the amplitude of the proposed signal. That is, if the amplitude is well measured, the signal is real. Of course there are adversarial data sets I can make where this isn't true! But that's just a restatement of the point that this is an unsolved problem. For deep reasons!

teeny tiny cosmological simulations.

Connor Hainje (NYU) is looking at this paper by Chen et al which uses a machine-learning regression to interpolate between cosmological simulation outputs at different cosmological epochs. To build an end-to-end pipeline for testing ideas, he has been running 32-cubed cosmological simulations. These might be the smallest simulations run since the 1980s! But, interestingly, he is finding that the interpolation isn't working great. Is this because it is harder to train a regression on a small simulation than it is on a large simulation? Is a small simulation less predictable or less interpolate-able? It's expensive to find out!

gradients of unit vectors

When you work in a curvilinear coordinate system, and you need to take gradients or tensor derivatives of scalar, vector, and tensor functions, the gradients of the unit vectors appear in your expressions. The unit vectors have gradients because, in a curvilinear coordinate system, they have orientations that depend on position. I gestured and imagined and guessed these derivatives for a spherical coordinate system by thinking geometrically. I got strange expressions I didn't believe. Then, today, I checked them by painstakingly taking derivatives, and my intuitive derivatives turned out to be exactly correct?

kinematic dipole and dust

I had a long conversation with Kate Storey-Fisher (NYU) and Abby Williams (Caltech) about the dipole in the Quaia catalog caused by the kinematic motion of the Solar System barycenter with respect to the cosmic rest frame. Williams has found that the amplitude of the dipole we get depends very strongly on how we account for dust in our sample. There is currently a controversy about the amplitude of the dipole seen in WISE quasars. We now think that it is possible that the measured amplitude is a strong function of how dust is corrected for in the sample? We designed new tests for next week.