I had a great (weekly) call with Andy Casey (Monash) today. We discussed many things, including a hilarious idea for April Fools' that Matt Daunt (NYU) and I conceived. But one of my action items out of the meeting is to start a short note on how to avoid ever shifting and stacking your data (think spectra taken at different times of year, so at different Doppler shifts relative to the Earth). How can you combine data without doing any interpolation of the data? The answer is simple: You forward-model your data with a model that (by optimization) becomes the average of those data. (And the optimization is often closed-form.) Then you only ever shift the model, and you don't have to deal with shifting noise arrays or mask arrays, or anything else. I'll try to write a stub this weekend.
2022-09-29
2022-09-19
Dagstuhl, day 1
Today was day 1 of Machine Learning for Science: Bridging Data-driven and Mechanistic Modeling at Schloss Dagstuhl. The first day was mainly about applications of machine learning, in Earth science, livestock management, astrophysics (dark matter), cells, and mechanical engineering. I had many thoughts and realizations. Here are a few random ones:
The problems that appear in Earth science, and the data types, are very similar to those that appear in astrophysics! But in Earth science, biology is a big driver of global processes, and there is no good mechanistic model for (say) how plants grow and take up carbon. The world is filled with mobile phones, with good cameras, and the methods we could could be employing to be doing science in a distributed way are way, way under-used. Cells are incredibly complicated. The mechanistic model involves literally thousands of individual processes. Like our model for the cell is as complicated as our model for the entire Earth system (which, by the way, depends on cells!), or even more complicated.
In the areas of the cell and the Earth, a theme was that the investigators want to preserve the causal structure we believe, and just use the machine learning to replace one tiny piece, with a data-driven model. Related: You can think of the machine learning as an effective theory for something (a sub-part of the problem) that doesn't work well from first principles. That's a good idea!
2022-09-06
continuous representations of stellar spectra
Matt Daunt (NYU) and I had lunch today and discussed various things. One is the idea that we need (for technical reasons of data analysis) to make and use continuous representations of spectra. That is, we need to represent our spectra of stars such that they can be exactly and losslessly interpolated to any (sufficiently fine) grid of points. The ESA Gaia Mission XP spectra have this property: They are represented as polynomial basis functions, which confuse and surprise everyone. They are hard! But there are many other choices. For example, if the spectra are represented with b-spline basis coefficients, the coefficients “look like” just a set of flux values at wavelengths (so traditionalists are happy), but in fact they are the parameters of a continuous model that can be interpolated losslessly to any grid.
2022-03-01
MIT visit
I spent the day at MIT today. I learned a huge amount! One highlight was that Rob Simcoe (MIT) showed me the hardware in his lab, and we discussed trade-offs between software and hardware in instrument design. Another was talking to a great group of graduate students over lunch. My talk was about the ontology and epistemology of machine learning. My slides are here. At dinner, Deepto Chakrabarty (MIT) encouraged me to complete my pedagogical note on bolometric magnitudes and so on.
2022-02-27
epistemology and adversarial attacks
I am trying to prepare slides for a new talk (meaning: no recycled slides!) that I will be giving at MIT in the coming week. I am trying to boil down what's so troubling about machine learning in the natural sciences. I realized that what's so troubling (to me) is that the only standard of truth for a machine-learning model is how it performs on a finite set of held-out test data (usually drawn from the same distribution as the training data). That is nothing like the standard of truth for scientific models or theories! So, for example, a successful adversarial attack against a method is seen as a strange feature, when it comes to industrial machine-learning contexts. But I think that a successful adversarial attack completely invalidates a machine-learning method used in (many contexts in) natural science: It shows that the model isn't doing what we expect it to be doing. I hope to be able to articulate all this before Tuesday.
2022-01-14
detecting shifts with cross-correlations
Lily Zhao (Flatiron) has been looking at the variability of the SDSS-IV BOSS spectrograph calibration frames, in the thought that if we can understand them well enough, we might be able to substantially reduce the observing overheads for the robotic phase of SDSS-V. To start, we are doing simple cross-correlations between calibration images. After all, the dominant calibration changes from exposure to exposure appear to be shifts. But these cross-correlations have weird features in them that I don't understand: In general if you are cross-correlating two images that are very similar, the cross-correlation function should look very symmetric, I think? But Zhao's cross-correlation functions look asymmetric in weird ways. We ended our session puzzled.
2021-12-23
sailing upwind; lift ratios; sail-to-keel ratio
In writing up a description of some of my physics-of-sailing results, I realized some things about sailboat design. For deep reasons you want the ratio of the area of the sail to the area of the keel to be roughly the ratio of the density of water to the density of air, or 700. That flows from the point that the sail and the keel have symmetric roles in sailing. But if you set this ratio to 700, you can only sail upwind if the sail lift ratio (the ratio of useful sail force to drag force) is very high. Since it is hard to make this lift ratio high on a commercial boat, the alternative is to make the sail smaller (relative to the keel). Looking at the data I can find on real sailboats, most have okay lift ratios but small sail-to-keel ratios (smaller than 700 anyway), so that they can sail quickly upwind. The cost of these design choices is that you can't go downwind faster than the wind. If you want to be able to sail both downwind faster than the wind—and also upwind—you have to have amazing sail and keel lift ratios.
2021-07-13
how much calibration data does a spectrograph need?
There was an SDSS-V operational telecon today, in which we discussed the plans for the first year of data, and how those plans should depend on, or be conditional on, what we learn in the commissioning phase. One of the most important things about SDSS-V is that it is robotic and fiber-fed, so we can move fast and do time domain things. But how fast? This depends, in turn, on how much calibration data we need as a function of time. I proposed that we could ask how well we can synthesize calibration data in one telescope configuration at one time, given calibration data from other telescope configurations at other times. This is not unlike the approach we took to calibrating EXPRES. So of course it was put back to me: Design the experiments that will answer this question! The main issue is that the BOSS spectrogaphs hang off the back of the pointing, tracking telescope.
2021-06-28
SDSS-V MWM target selection
Today Jennifer Johnson (OSU) crashed the weekly Manhattan-area SDSS-V discussion meeting to give us the current state of Milky Way Mapper (a component of SDSS-V) target selection. It was a great discussion, because there are many, many target categories, and many of them are interesting to Manhattan-area locals. For me the most impressive thing about the meeting was that Johnson could answer almost any question from anyone on any of the literally dozens of target categories! It was a tour de force as they say. And we learned a lot. One of my goals with this meeting (which was started and is operated by Katie Breivik, Flatiron) is to increase excitement in Manhattan for SDSS-V and Johnson did that admirably.
2021-06-13
a model for sailing (yes, sailing)
I've had a lifetime of conversations with Hans-Walter Rix (MPIA) about the point that you could in principle sail with a sailboat with flat sails: Nothing about the curvature of the sails is integral or required by sailing. The curvature helps, but isn't necessary. I have had another lifetime of conversations with Matt Kleban (NYU) about the point that sailing depends on the relative velocity between the air and the water, and this leads to some hilarious physics problems involving sailing on rivers in zero wind (it's possible because a flowing river is moving relative to the dead air).
These worlds collided this weekend because—inspired by a twitter conversation—I finally built a proper ram-pressure model of a flat-sail, flat-keel sailboat and got it all working. It's sweet! It sails beautifully. Much more to say, but question is: Is there a paper to write?
2021-06-10
information theory at Cambridge
Today I gave a colloquium at the University of Cambridge. My slides are here. I spoke about how to make precise measurements, how to design surveys, and how to exploit structure in noise. It's a rich set of things, and most of the writing about information theory in astronomy is only in the cosmology domain. Time to change that, maybe? It is also the case that the best book about information and inference ever written was written in Cambridge! So I was bringing coals to Newcastle, ish!
2021-06-03
Dr Lily Zhao
I had the great honor to be on the PhD committee of Lily Zhao (Yale), who defended her dissertation today. It was great and remarkable. She has worked on hardware, calibration, software, stellar astrophysics, and planets. Her seminar was wide-ranging, and the number and scope of the questions she fielded was legion. She has already had a big impact on extreme precision radial-velocity projects, and she is poised to have even more impact in the future. One of the underlying ideas of her work is that EPRV projects are integrated hardware–software systems. This idea should inform everything we do, going forward. I asked a million technical questions, but I also asked questions about the search for life, and the astronomical community's management and interoperation of its large supply of diverse spectrographs. In typical Zhao fashion, she had interesting things to say about all these things.
2021-05-05
microscopy of fiber robots
Conor Sayres (UW) and I continued today our discussion of the data-analysis challenges associated with the SDSS-V focal-plane system (the fiber robots). Today we discussed the microscopy of the robots. Sayres has images (like literally RGB JPEG images) from a setup in which each fiber robot is placed into a microscope. From this imaging, we have to locate the three fibers (one for the BOSS spectrograph, one for the APOGEE spectrograph, and one for a back-illumination system used for metrology, all relative to the outer envelope of the robot arm. And do this for 300 or 600 robots. The fibers appear as bright, resolved circles in the imaging, but on a background that has lots of other detail, shading, and variable lighting. This problem is one that comes up a lot in astrophysics: You want to measure something very specific, but in a non-trivial image, filled with other kinds of sources and noise. We discussed options related to matched filtering, but we sure didn't finish.
2021-04-14
how to calibrate fiber robots?
Today we had the second in a series of telecons to discuss how we get, confirm, adjust, and maintain the mapping, in the SDSS-V focal planes (yes there are two!) between the commands we give to the fiber-carrying robots and the positions of the target stellar images. It's a hard problem! As my loyal reader might imagine, I am partial to methods that are fully data-driven, and fully on-sky, but their practicality depends on a lot of prior assumptions we need to make about the variability and flexibility of the system. One thing we sort-of decided is that it would be good to get together a worst-case-scenario plan for the possibility that we install these monsters and we can't find light down the fibers.
2021-03-04
why make a dust map? and Bayesian model elaboration
Lauren Anderson (Carnegie) and I had a wide-ranging conversation today. But part of it was about the dust map: We have a project with statisticians to deliver a three-dimensional dust map, using a very large Gaussian-process model. Right now the interesting parts of the project are around model checking and model elaboration: How do you take a model and decide what's wrong with it, in detail. Meaning: Not compare it to other models (that's a solved problem, in principle), but rather, compare it to the data and see where it would benefit from improvement.
One key idea for model elaboration is to check the parts of the model you care about and see if those aspects are working well. David Blei (Columbia) told us to climb a mountain and think on this matter, so we did, today. We decided that our most important goals are (1) to deliver accurate extinction values to stellar targets, for our users, and (2) to find interesting dust structures (like spiral arms) if they are there in the data.
Now the challenge is to convert these considerations into posterior predictive checks that are informative about model assumptions. The challenge is that, in a real-data Bayesian inference, you don't know the truth! You just have your data and your model.
2021-01-11
integrated hardware–software systems
I had a wide-ranging conversation today with Rob Simcoe (MIT) about connections between my group in New York and his group in Cambridge MA. He does complex hardware. We do principled software. These two things depend on each other, or should! And yet few instruments are designed with software fully in mind, in the sense of making good, non-trivial trades between hardware costs and software costs. And also few software systems are built with deep knowledge of the hardware that produces the input data. So there are synergies possible here.
2020-09-28
Terra Hunting Science Meeting, day 1
Today was the first day of the Terra Hunting science meeting. We made great progress on target selection, which was our primary goal. We decided that we should only consider stars that have sufficient visibility from the observatory and sufficient brightness to deliver sufficient photons to provide precisely enough measured radial-velocity variations over a decade to meet our planet-detection goals. That is, we should make the parent sample of our final selection to be stars where we at least have enough photons to detect an Earth. That is an obvious and simple point, but this was the first meeting where we really clearly identified it, and started to figure out how that flows down to a target list. It turns out that there aren't huge numbers of stars that even meet this strict requirement. And of course we spent lots of time talking about all the reasons that we won't get photon-limited radial-velocities, so our final target list must be much more restricted, probably.
2020-09-21
writing linear algebra operators and implementations
I spent my research time this morning typing math into a document co-authored by Ana Bonaca (Harvard) about a likelihood function for asteroseismology. It is tedious and repetitive to type the math out in LaTeX, and I feel like I have done the nearly-same typing over and over again. And, adding to the feeling of futility: The linear-algebra objects we define in the paper are more conceptual than operational: In many cases we don't actually construct these objects; we avoid constructing things that will make the execution either slow or imprecise. So in addition to typing these, I find myself also typing lots of non-trivial implementation advice.
2020-07-15
asteroseismic likelihood
Ana Bonaca (Harvard) and I discussed two important successes today (both of them due to Bonaca): We found the issue in our code that caused there to be frequency structure on frequency scales smaller than what you might call the “uncertainty-principle” limit of the inverse of the duration of the survey. It was literally a bug, not a think-o. Which is reassuring. And we do have likelihoods that seem to peak (at least locally) at sensible values of delta-nu, the large frequency difference, in the parlance of asteroseismology. So now there are questions of how to improve performance, and how Bayesian to be, as it were. Frequentism is easier in some respects, harder in others. Lots of engineering to think about!
2020-07-04
experimental design for exoplanet discovery
I spent time today writing a new paper stub (as is my wont). I re-framed the project Bedell (Flatiron) and I are working on in planet detection into a project on experimental (survey or project) design. After all, if you want to perform experimental design trades, you need a system that can tell you what you could or would observe or detect. So these questions: How do you discover an exoplanet? and How do I design my survey? are very intimately related. It's just a stub; we'll see if Bedell likes it.