wrote like the wind; frequentist vs Bayes on sparsity

My goal this year in Heidelberg is to move forward all writing projects. I didn't really want to start new projects, but of course I can't help myself, hence the previous post. But today I crushed the writing: I wrote four pages in the book that Rix (MPIA) wants me to write, and I got more than halfway done with a Templeton Foundation pre-proposal that I'm thinking about, and I partially wrote up the method of the robust dimensionality reduction that I was working on over the weekend. So it was a good day.

That said, I don't think that the iteratively reweighted least squares implementation that I am using in my dimensionality reduction has a good probabilistic interpretation. That is, it can't be described in terms of a likelihood function. This is related to the fact that frequentist methods that enforce sparsity (like L1 regularization) don't look anything like Bayesian methods that encourage sparsity (like massed priors). I don't know how to present these issues in any paper I try to write.

why study astrophysics?

I spent the day with Neige Frankel (CITA), working on various projects. One of the things we discussed was her slides for an upcoming talk. I made the following blanket statement; is it true? There are only two ways to ultimately justify a subject of study in astrophysics. Either it will tell us something important about fundamental physics (think: dark matter, initial conditions of the Universe, or nucleosynthesis, say), or else it will tell us something about our origins (formation of our Galaxy, occurrence of rocky, habitable planets, origin of life, say). I am not entirely sure this is right, but I can't currently think of much in the way of counter-examples. I guess one other justification might be that we are developing technologies that will help people in other areas (CCDs, spacecraft attitude management, or machine learning, say).

partial differential equations

I am trying to write a proposal to fund the research I do on machine-learning theory. The proposal is to work on ocean dynamics. It's a great application for the things we have done! But it's hard to write a credible proposal in an area that's new to you. Interdisciplinarity and agility is not rewarded in the funding system at present! At least I am learning a ton as I write this.

grant proposals

There is a non-wrong view of academic science that it is all about applying for funding, and evaluating the proposals of others for funding. That's all I did today (evaluated proposals for a foreign funding program; I submitted my own proposal to the NSF yesterday).

other kinds of machine learning

Astronomy is very focused on machine learning in the sense of regression and classification, but machine learning can do many other things. In addition, machine learning is a sub-field of machine intelligence, which is broader. I started today working on a proposal for the NSF (to be written with Mike Blanton, NYU) in which we propose using other kinds of machine learning and machine intelligence, and apply them earlier in the scientific process (like at operations and calibration) instead of at the end (like at source classification and labeling).

four kinds of emulators

I wrote in a draft grant proposal related to machine-learning emulators today. I wrote about five different kinds of emulators. Yes I think there are five qualitatively distinct kinds. Here they are:

Full replacement

The most extreme—and most standard—kind of emulator is one that simply replaces the full input–output relationship of the entire simulation. Thus if the simulation starts with initial conditions and boundary conditions, and ends with a final state (after an integration), the full-replacement emulator would be trained to learn the full relationship between the initial and boundary conditions and the final state. A full-replacement emulator is a complete, plug-in replacement for the simulator.

Integrator

Simulation run times generally scale linearly with the number of time steps required to execute the integration. A set of emulators can be trained on a set of snapshots of the simulation internal state at a set of times that is much smaller than the full set of integration time steps. Each emulator is trained to learn the relationship between the internal state of the simulation at one time tA and the internal state of the simulation at a later time tB, such that the emulator can be used to replace the integrator during the time interval from tA to tB. A set of such emulators can be used to replace part or all of the integration performed by the simulator.

Resolution translator

Simulation run times generally scale with the number of grid points or basis functions in the representations of the state. Thus the simulator gets faster as resolution is reduced. An emulator can be trained to learn the relationship between a low-resolution simulation and a matched high-resolution simulation. Then a high-resolution simulation can be emulated by running a fast low-resolution simulation and applying the learned translation.

Physics in-painter

In most physical systems, there are coupled physics domains with different levels of computational complexity. For example, in cosmology, the pure gravitational part of the simulation is relatively low in computational cost, but the baryonic part—the atoms, photons, ram pressures, magnetic fields—is very high in computational cost. The simulator gets faster as physics domains, or equations, or interaction terms, are dropped. An emulator can be trained to learn the relationship between a simulation with some physics dropped and a matched full simulation. Then a full-physics simulation can be emulated by running a partial-physics simulation and applying the learned in-painting of the missing physics.

Statistics generator

In many contexts, the goal of the simulation is not to produce the full state of the physical system, but only certain critical statistics, such as the two-point correlation function (in the case of some cosmology problems). In this case, there is no need to emulate the entire simulation state. Instead, it makes sense to train the emulator to learn only the relationship between the initial and boundary conditions of the simulation and the final statistics of particular interest.

building trust in emulators

I started writing in a possible grant proposal (that would be in collaboration with others) about the trustworthiness of machine-learning emulators. Emulators are systems that learn the input–output relationship of a computationally expensive simulation and produce (or speed the computation of) new simulation outputs, reducing total computational requirements for a given number of simulations. These are so important now that the ESA Euclid and Simons Observatory data-analysis plans crucially involve emulation.

The issue is: How do we trust that the emulators are giving good outputs? There is no obvious way to test them, except by comparing to held-out training data. But in large-scale structure contexts, no amount of held-out data can test the enormous input data space. I don't know how we will ever trust such systems (and damn do we need to!), but I have some ideas about how to improve the situation. One involves enforcing physics symmetries on the emulators. Another involves running adversarial attacks on them.

getting ready for the job market

I had a great conversation with Kate Storey-Fisher (NYU) today about her preparations for the academic job market. We talked about places, applications, proposals, and so on. In particular, we spent time talking about the structure of a good job proposal, which I think involves lots of scales from very very big picture down to very specific ideas for particular shovel-ready projects. We also talked about what makes Storey-Fisher unique on the market and how to talk about that uniqueness in the application. I think an odd thing about applications is that you have to narrate your work—which usually is a set of random and contingent projects—like it is a scientific program with coherence. This is odd, but not really irrelevant, since the ability to narrate it well shows an ability to make connections and see themes.

generalized flat-relative extraction

I asked, in the Astronomical Data Group meeting at Flatiron, about the method of spectral 2D-to-1D extraction known as flat-relative optimal extraction. It's genius, and simple, but it makes strong assumptions about the spectrograph. I asked how we might improve it. And I think I maybe have a plan. The idea (which was thrown out by Megan Bedell) is to make the spectral representation something continuous, and evaluate it individually at every pixel, not just once per column of the detector. This should improve extraction. And it is relevant to the NASA proposal I am writing with Matt Daunt.

information loss

I wrote words today about how information is being lost in radial-velocity-spectrograph data-analysis pipelines at the stage of going from 2D spectra to 1D spectra. I am proposing to NASA (with Matt Daunt, NYU) to fix these problems! This is important, in my opinion, but I have to admit that it is not currently considered the tall pole in EPRV.

exoplanet roadmaps, plans, and surveys

Inspired by research by Matt Daunt (NYU), I looked at the various reports, presentations, and papers that have been written by NASA panels, committees, and projects about the tall poles and engineering gaps in the exoplanet research ecosystem. Why? Writing a proposal, of course! Daunt and I are proposing to work very close to the metal in radial-velocity work, so we are looking at the critical infrastructure that's close to the metal.

writing like mad about Astrometry.net

It's crunch time this weekend on Dustin Lang (Perimeter) and my proposal for the NASA Open-Source Tools, Frameworks, and Libraries call. I spent a lot of quality time this weekend cranking out words. After doing some literature review, we find that Astrometry.net is used in a huge number of projects, from NASA missions to cosmic-ray detectors to (of course) amateur astrophotography workflows. That's exciting, and relevant to our proposal. One of the great things about the NASA call is that it requires us to think about project management, community building, and collaboration policies. That is good; it will help our project immensely.

writing proposals is hard!

Today I took a serious shot at getting words down in my upcoming NASA proposal for open-source tools, frameworks, and libraries. This is a new call to support development and maintainance of open-source projects that are aligned with NASA science missions (yay open science and NASA!). Dustin Lang (Perimeter) and I are proposing to support Astrometry.net, which is used in multiple NASA missions, including SOFIA and SPHEREx. It is hard to put together a full proposal; writing a proposal is comparable in intellectual scope to writing a scientific paper! And it must be done on deadline, or not at all.

NSF center proposal

I spent some time today discussing a possible NSF Center on real-time data analysis with Ashley Villar (Columbia) and Tyler Pritchard (NYU), based on the wide-ranging grass-roots interest we found in time-domain astrophysics we have discovered in NYC this pandemic. NSF Centers are big projects!

Gaia EDR3

Today ESA Gaia EDR3 dropped! It was a fun day; the data are more precise and less noisy! I'm involved in a few different projects with the data. With Hunt (Flatiron) and Price-Whelan (Flatiron) I am looking at the local velocity-space structure in the disk, and seeing if we can classify features by looking at how they vary spatially around the Solar position. With Eilers (MIT) I am going to update our spectrophotometric distance estimates to APOGEE luminous red giants. With Bonaca (Harvard) I will find out if we can improve the kinematics and orbit identification of stellar streams. None of these projects got very far today, but we did make this visualization!

a selection function for Gaia

This morning we had a call to begin the GaiaUnlimited project, which is a multi-institution collaboration to make a useful selection function for the ESA Gaia Catalog and Mission. The idea is: Gaia produces catalogs, but it has no deliverable mask or selection probability, so it is not possible to use the catalogs for certain (maybe most?) statistical purposes without additional information. We are going to try to construct that information for the community. Today we kicked off this project, and discussed the scope. We decided that all observational selections are in, but the three-dimensional dust map in the Milky Way is out!

After the call, Rix (MPIA) and I decided that we have to find some good example projects that make use of the selection function but don't need the dust map, because we want to be customers for the project as well as owners.

By the way, this project was started at a #GaiaSprint!

BOSS bright limit and Gaia parallax quality

Today I finished my open-fiber proposal, with Adrian Price-Whelan (Flatiron). We discovered that one of the oddities that we discovered yesterday—to wit, that there are no spectra of very luminous red giants—comes from an interaction between any sensible parallax signal-to-noise cut on the ESA Gaia data, and the bright limit on the SDSS visible spectrographs. Brutal! We have to select in some way that doesn't make the signal-to-noise cut (since the bright limit is unavoidable). I have ideas (one of which is in this paper), but I didn't have time to implement them before we had to submit the proposal. Oh well! We will get opportunities to update our target lists later.

more open fiber proposal

My research time today was spent writing in my SDSS-V open-fiber proposal. Adrian Price-Whelan (Flatiron) got and selected the relevant ESA Gaia data, and we did experiments with boxelization of the color-magnitude diagram. We are finding that there are (to my surprise) many SDSS-II, SDSS-III, and SDSS-IV spectra of the stars in between the main sequence and the white dwarf sequence (home of CVs, stripped stars, and low-metallicity stars). Thousands! But to my equally large surprise, there are almost no optical spectra of the most luminous giant stars. What gives?

open-fiber proposal

The SDSS-V project uses robot fiber positioners to take millions of short (15-ish minutes per visit) spectra in the visible and infrared. Because of the geometric constraints of the fiber positioners, and the targeting, there will be many, many unusued fibers—meaning, many opportunities to add additional spectroscopic targets! The project issued an internal call for proposals for the open fibers. Today I spent time writing one, which is very simple: It is to fill out the unobserved parts of the ESA Gaia color-magnitude diagram, but targeting stars for spectroscopy that do not already have a nearby star with a spectrum. The word “nearby” implies a resolution (how nearby?). The proposals are due in a few days and we still don't know exactly what our resolution should be! Also, do we treat variable stars differently from non-variable stars? We have work to do!

funding astrometry.net as an open-source project

Dustin Lang (Perimeter) called me today and alerted me to this NASA funding call related to open-source projects. He argued that we need to take Astrometry.net/ to the next level. I agree! So we kicked around project and development ideas and vowed to take a stab at a letter of intent.