Today was an all-day event at MPIA to celebrate the 60th birthday (and 25th year as Director) of Hans-Walter Rix (MPIA). There were many remarkable presentations and stories; he has left a trail of goodwill wherever he has gone! I decided to use the opportunity to talk about measurement, which is something that Rix and I have discussed for the last 18 years. My slides are here.
I've been very lucky with the opportunities I've had to work with wonderful people.
On Monday of this week, Shirley Ho (Flatiron) gave a talk at NYU in which she mentioned the unreasonable effectiveness of pre-training a neural network: If, before you train your network on your real (expensive, small) training data, you train it on a lot of (cheap, approximate) pre-training data, you get better overall performance. Why? Ho discussed this in the context of PDE emulation: She pre-trains with cheap PDEs and then trains on expensive PDEs and she gets way better performance than she does if she just trains on the expsensive stuff.
Why does this work? One interesting observation is that even pre-training on cat videos helps with the final training! Ho's belief is that the pre-training gets the network understanding time continuity and other smoothness kinds of things. My conjecture is that the pre-training teaches the network about (approximate) diffeomorphism invariance (coordinate freedom). The cool thing is that these conjectures could be tested with interventions!
I spent the day with Juna Kollmeier (CITA) talking about epistemology, physical cosmology, and project management (especially academic management). I found myself saying to her the following argument (which I have not seen written down anywhere): Imagine that our Universe is hamiltonian (or lagrangian; it doesn't matter for these purposes). And imagine that our Universe is a simulation being run inside some bigger universe, which is also hamiltonian.
If our Universe is being observed in any sense by any system in that bigger universe, then there ought to be a loss of unitarity in our Universe. That is, there should be a violation of Liouville's theorem, or a violation of key conservation laws, or an information sink. And there is! At black hole horizons, there is an information paradox: Information that goes in never comes back (an evaporating black hole evaporates thermally, or so we think). Thoughts?
I had a meeting this morning with Megan Bedell (Flatiron) about our dormant paper about information theory and extreme-precision radial-velocity measurements. We see the paper a bit differently (is it about methods or is it about concepts?), but we were able to re-state a scope with which we are both happy. We assigned tasks (Bedell writing and me coding, mainly), and promised to make progress before next week. It is very, very, very hard to finish a paper! Especially when all authors are above some seniority, where they spend most of their time with others. I would love to get a lot more personal coding time!
Today was day two of my visit to University of Florida. I had many interesting discussions. One highlight was with Dhruv Zimmerman, who wants to infer big labels (non-parametric functions of time) from small features (a few bands of photometry). That's my kind of problem! We discussed different approaches, and we discussed possible featurizations (or dimensionality reductions) of the labels. I also pitched an information-theoretic analysis. If there's one thing I've learned in the last few years, it is that you shouldn't be afraid to solve problems where there are fewer data than parameters! You just have to structure the problem with eyes wide open.
After many more (equally interesting) discussions, the day ended with Sarah Ballard's group out at a lovely beer garden. We discussed the question: Should students be involved in, and privy to, all the bad things with which we faculty interact as academics, or should we protect students from the bad things? You can imagine my position, since I am all about transparency. But the positions were interesting. Ballard pointed out that in an advisor–student relationship, the student might not feel that they can refuse when the advisor wants to unload their feelings! That power asymmetry is very real. But Ballard's students (Chance, Guerrero, Lam, Seagear) said that they want to understand the bad things too; they aren't in graduate school just to write papers (that comment is for you, Quadry!).
I had conversations with Nora Eisner (Flatiron) and Abby Shaum (CUNY) today about how we report the significance of a signal we find in a time series. In particular a periodic signal. It's an old, unsolved problem, with a lot of literature. And various hacks that are popular in the exoplanet community (and binary-star community!). My position is very simple: Since all methods for determining significance are flawed, and since when you fit a signal you have to estimate also an uncertainty on that signal's parameters, the simplest and most basic test of significance is the significance with which you measure the amplitude of the proposed signal. That is, if the amplitude is well measured, the signal is real. Of course there are adversarial data sets I can make where this isn't true! But that's just a restatement of the point that this is an unsolved problem. For deep reasons!
Connor Hainje (NYU) is looking at this paper by Chen et al which uses a machine-learning regression to interpolate between cosmological simulation outputs at different cosmological epochs. To build an end-to-end pipeline for testing ideas, he has been running 32-cubed cosmological simulations. These might be the smallest simulations run since the 1980s! But, interestingly, he is finding that the interpolation isn't working great. Is this because it is harder to train a regression on a small simulation than it is on a large simulation? Is a small simulation less predictable or less interpolate-able? It's expensive to find out!
In the exoplanet (and, before that, eclipsing-binary) communities, transit-timing variations are described in terms of a quantity called O−C (pronounced “oh minus sea”), which is the difference between the observed transit time and the “computed” transit time. Right now, Abby Shaum (CUNY) and I are using this terminology in our manuscript about phase variations in coherent pulsators with companions, at the behest of Keaton Bell (CUNY). Okay fine! But O−C has this terrible property, which is that the C part depends on the period or frequency you assume. You can completely change the appearance or morphology of an O−C plot just by slightly tweaking the period. And there is no true period of course! There is just whatever estimates you can make. Which are, in turn, affected by what you use to model the O−C. So it is absolutely awful in every way. Not a stable observable, people! Not even identifiable.
I had a conversation with Vedant Chandra (Harvard) today about how catalogs are used, and how that relates to how they are built. We started off by arguing about how principled one should be about doing a populations inference. Too abstract! So Chandra moved us in the pragmatic direction: Let's look at a very specific inference and see what matters about it. We decided to look at the distances to distant clusters in the ESA Gaia data: How do your inferences depend on the number of stars you use, the signal-to-noise ratios of those stars, and whether your individual-star measurements are maximum-likelihood or obtained by consideration of a posterior pdf? That should answer questions, and set up some concrete points of discussion.
My day started with a conversation with Wolfgang Brandner (MPIA), who asked me how to figure out the information content of ESA Gaia RVS spectra, but in a data-driven way. He wants to avoid the theoretical models at first; that is, he wants to figure out how precisely the spectra contain temperature and metallicity and age information without having temperatures, metallicities, and ages that we believe. One approach is to compare to other data that are sensitve to temperature, metallicity, and age: If the RVS spectra can predict those data, then (conditioned on assumptions) they must contain information about temperature, metallicity, and age. This is similar to questions of risk (or expected error in prediction) in machine-learning contexts.
There were discussions this week at University of Warwick about the Terra Hunting Experiment strategy and likely detection capability. Various take-homes include that we need to mitigate lots of stellar noise, and that we care deeply about the covariance (as a function of separation in time) of adjacent measurements. I advocated that we split our ten-year survey into two or three surveys, of varying length. In the first, we learn about the stars, and in the last, we go to town on the very most promising targets. There was general agreement that this is a good idea. But now we need a very specific plan for what this means. As my loyal reader knows, in my view, the decisions must be based on repeatable operations, so that we have some hope of learning statistical things about populations in the end.
Today Chirag Modi (Flatiron) gave a really great lunchtime talk about new technologies in cosmology and inference or measurement of cosmological parameters. He beautifully summarized how cosmology is done now (or traditionally): Make summary statistics of the observables, make a theory of the summary statistics, make up a surrogate likelihood function for use in inference, measure covariance matrices to use in the latter, and go. He's trying to obviate all of these things by using the simulations directly to make the measurements. He has nice results in forward modeling of the galaxy field, and in simulation-based inferences. Many interesting things came up in his talk, including the idea that I have discussed over the years with Kate Storey-Fisher (NYU) of enumerating all possible cosmological statistics! So much interesting stuff in the future of large-scale structure.
On Fridays, Kate Storey-Fisher (NYU) organizes a small meeting to discuss her projects on dark-matter halos using equivariant scalar objects constructed from n-body simulation outputs. Today we included Yongseok Jo (Flatiron), who has worked on building tools to paint galaxies onto dark-matter-only n-body simulations. We discussed joint projects, and conceptual issues about mass-assembly histories. In particular, I am interested in how we can predict formation histories of dark-matter halos from the galaxy contents alone, or infer the dark matter distribution in phase space from the stellar distribution in phase space. I love these projects, because they combine growth of structure, gravitational dynamics, galaxy formation, and machine learning.
Long ago, Adrian Price-Whelan (Flatiron) and I and others built The Joker, which is a Monte Carlo method (but not a MCMC method) for dealing with the Kepler problem. It exploits the fact that some parameters are linear, and some are nonlinear. This week, Lawrence Peirson (Stanford) is visiting Flatiron to generalize this point. Peirson's point is that the trick we use for linear parameters can be used for any parameters that have smooth, unimodal-ish posteriors. We just have to add some linearization and some optimization. So we are working on writing that down. And coding it up.
Along the way, Peirson found another linear parameter in The Joker, so we can now make it way, way faster. That's awesome!
I spent time in an undisclosed location on the weekend writing in the discussion section of my draft with Megan Bedell (Flatiron) on information theory and extreme precision radial-velocity measurement. I think my language is a bit loose when I write on vacation!
I had conversations today with both Hans-Walter Rix (MPIA) and Kate Storey-Fisher (NYU) about the upcoming ESA Gaia quasar sample. We are trying to make somewhat realistic mocks to test the size of the sample, the computational complexity of things we want to do, the expected signal-to-noise of various cosmological signals, and the expected amplitude and spatial structure of the Gaia selection function. We have strategies that involve making clean samples with a lognormal mock, and making realistic samples (but which have no clustering) using the Gaia EDR3 photometric sample (matched to NASA WISE).
Today Yucheng Zhang (NYU) defended his PhD. He used SDSS eBOSS large-scale structure samples to test gravity on large scales, and also made forecasts for measuring the non-Gaussiany parameter fnl and other very-large-scale-structure measurements in upcoming surveys. Beautiful work and a very nice defense. In the question period, Kate Storey-Fisher (NYU) asked Zhang about his possible forecasts for the upcoming ESA Gaia sample of 6.4 million quasars. Zhang has not considered this sample yet (almost no cosmologists have!) but he said that he does have the technology to make predictions for it. His intuition is that it would be great for measuring baryon acoustic feature and fnl. We plan to take Zhang out to lunch to discuss in the near future!
I wrote words today about how information is being lost in radial-velocity-spectrograph data-analysis pipelines at the stage of going from 2D spectra to 1D spectra. I am proposing to NASA (with Matt Daunt, NYU) to fix these problems! This is important, in my opinion, but I have to admit that it is not currently considered the tall pole in EPRV.
Sarah Pearson (NYU) is working on modeling a stellar stream (disrupted satellite galaxy) around an external galaxy. The goal is to figure out what observables are most critical, and what properties of the host galaxy are most strongly constrained by a good model. That is, information theory. Pearson showed beautiful results today to Adrian Price-Whelan (Flatiron) and me: She can show that the mass of the galaxy's dark-matter halo is covariant with velocity gradients along the stream. Those would be hard to measure but not impossible. One high-level objective is to understand what would be the scientific merit of a big program with new imaging data and follow-up spectroscopy.
I spoke with Abby Williams (NYU) and Kate Storey-Fisher (NYU) today about Williams's forecasts for measuring cosmological-scale gradients in the large-scale structure. We came up (many moons ago) with approximate scalings with survey volume, the number of tracers, and the amplitude of the clustering. Some of these are obeyed by Williams's results and some aren't! What gives? We think it might have to do with the occupation number of the modes. If the number density of tracers is high, the clustering precision depends on volume, not galaxy number density.
