new methods for cosmology

Kate Storey-Fisher (NYU) and I had a lunch-time conversation about her project to replace the standard large-scale-structure correlation-function estimator with a new estimator that doesn't require binning the galaxy pairs into separation bins. It estimates continuous functions. We discussed how to present such a new idea to a community that has been using the same binned estimator since the 1990s, and even before that they were only marginally different. That is, the change Storey-Fisher proposes is the biggest change to correlation-function estimation since it all started, in my (somewhat not humble) opinion.

But this creates a problem: How to convince the cosmologists that they need to learn new tricks? We have many arguments, but which one is strongest? We no longer need to bin, and binning is sinning! Or: We can capture more functional variation with fewer degrees of freedom, so we reduce simulation requirements! Or: We can restrict the function space to smooth functions, so we regularize away unphysical high-frequency components! Or: We get smaller uncertainties on the clustering at every scale! Or: We can make our continuous function components be similar to derivatives of the correlation function with respect to cosmological parameters and therefore create clustering statistics that are close to Fisher-optimal given the data and the model!

Writing methodological papers is not easy.

#wetton18, day 1

Today was the first day of the Wetton Workshop at Oxford. There were many interesting talks from all over the map, but with a goal at understanding how we make sure that we stay open to unexpected discoveries, even as we make more and more targeted data sets and experiments. One theme that emerged is that of systematics: As you push data harder and harder&mdashin cosmology or exoplanet search or anything else—you become more and more sensitive to the details of your hardware and electronics and selection and so on. This led to a discussion of end-to-end simulation of data sets to understand how hardware issues enter and to see if we understand the hadware.

That's important! But I think there is an equally important aspect to this: If we don't take our data with sufficient heterogeneity, we can't learn certain things. For example, if you take all LSST exposures at 15 seconds, you never test the shutter, never test linearity of the detector, never find out on what time scales the PSF is changing, and so on. For another, if you take all the Euclid imaging survey on a regular grid, you never get cross-calibration information from one part of the detector to another, nor can you find certain kinds of anisotropies in the detector or the point-spread function. If we are going to saturate the bounds, we are going to need to take science data in many, many configurations.

Here are the slides from the public talk I gave at the end of the day. Note my digs at press-release artists' conceptions. I think we should be honest about what we do and don't know!

a day at MIT

I spent the day today at MIT, to give a seminar. I had great conversations all day! Just a few highlights: Rob Simcoe and I discussed spectroscopic data reduction and my EPRV plans. He agreed that, in the long run, the radial-velocity measurements should be made in the space of the two-d pixel array, not extracted spectra. Anna Frebel and I discussed r-process stars, r-process elements, and chemical-abundance substructure in the Galaxy Halo. We discussed the immense amount of low-hanging fruit coming with Gaia DR2. I had lunch with the students, where I learned a lot about research going on in the Department. In particular Keaton Burns had interesting things to say about the applicability of spectral methods in solving fluid equations in some contexts. On the train up, I worked on the theoretical limits of self-calibration: What is the Cramér–Rao bound for flat-field components given a self-calibration program? This, for Euclid.

Euclid and MCMC

I did some work on the (NYC) long weekend on two projects. In the first, I built some code to make possible observing strategies for the in-flight self-calibration program for ESA Euclid. Stephanie Wachter (MPIA) contacted me to discuss strategies and metrics for self-calibration quality. I wrote code, but realized that I ought to be able to deliver a sensible metric for deciding on dither strategy. This all relates to this old paper.

On Monday I discussed our nearly-finished MCMC paper with Dan Foreman-Mackey (Flatiron) and we decided to finish it for submission to the AAS Journals. I spent time working through the current draft and reformatting it for submission. There is lots to do, but maybe I can complete it this coming week?

latent variable models, weak lensing

The day started with a call with Bernhard Schölkopf (MPI-IS), Hans-Walter Rix (MPIA), and Markus Bonse (Darmstadt) to discuss taking Christina Eilers's (MPIA) problem of modeling spectra with partial labels over to a latent-variable model, probably starting with the GPLVM. We discussed data format and how we might start. There is a lot of work in astronomy using GANs and deep learning to make data generators. These are great, but we are betting it will be easier to put causal structure that we care about into the latent-variable model.

At Cosmology & Data Group Meeting at Flatiron, the whole group discussed the big batch of weak lensing results released by the Dark Energy Survey last month. A lot of the discussion was about understanding the covariances of the likelihood information coming from the weak lensing. This is a bit hard to understand, because everyone uses highly informative priors (for good reasons, of course) from prior data. We also discussed the multiplicative bias and other biases in shape measurement; how might we constrain these independently from the cosmological parameters themselves? Data simulations, of course, but most of us would like to see a measurement to constrain them.

At the end of Cosmology Meeting, Ben Wandelt (Flatiron) and I spent time discussing projects of mutual interest. In particular we discussed dimensionality reduction related to galaxy morphologies and spatially resolved spectroscopy, in part inspired by the weak-lensing discussion, and also the future of Euclid.

reading the basics

Today we decided that the newly-christened Astronomical Data Group at Flatiron will start a reading group in methods. Partially because of the words of David Blei (Columbia) a few weeks ago, we decided to start with BDA3, part 1. We will do two chapters a week, and also meet twice a week to discuss them. I haven't done this in a long time, but we realized that it will help our research to do more basic reading.

This week, Maggie Lieu (ESA) is visiting Justin Alsing (Flatiron) to work (in part) on Euclid imaging analysis. We spent some time discussing how we might build a training set for cosmic rays, asteroids, and other time-variable phenomena in imaging, in order to train some kind of model. We discussed the complications of making a ground-truth data set out of existing imaging. Next up: Look at what's in the HST Archive.

crazy space hardware

I spent today at JPL, where Leonidas Moustakas (JPL) set up for me a great schdule with various of the astronomers. I met the famous John Trauger (JPL), who was the PI on WFPC2 and deserves some share of the credit for repairing the Hubble Space Telescope. I discussed coronography with Trauger and various others. I learned about the need for coronographs to have two (not just one) deformable mirror to be properly adaptive. With Dimitri Mawet (Caltech) I discussed what kind of data set we would like to have in order to learn in a data-driven way to predictively adapt the deformable mirrors in a coronograph that is currently taking data.

With Eric Huff (JPL) I discussed the possibility of doing weak lensing without ever explicitly measuring any galaxies—that is, measuring shear in the pixels of the images of the field directly. I also discussed with him the (apparently insane but maybe not) idea of using the Sun itself as a gravitational lens, capable of imaging continents on a distant, rocky exoplanet. This requires getting a spacecraft out to some 550 AU, and then positioning it to km accuracy! Oh and then blocking out the light from the Sun.

Martin Elvis (CfA) gave a provocative talk today, about the future of NASA astrophysics in the context of commercial space, which might drive down prices on launch vehicles, and drive up the availability of heavy lift. A theme of his talk, and a theme of many of my conversations during the day, was just how long the time-scales are on NASA astrophysics missions, from proposal to launch. At some point missions might start to take longer than a career; that could be very bad (or at least very disruptive) for the field.

ZTF; self-calibration; long-period planets

I spent today at Caltech, where I spoke about self-calibration. Prior to that I had many interesting conversations. From Anna Ho (Caltech) I learned that ZTF is going to image 15,000 square degrees per night. That is life-changing! I argued that they should position their fields to facilitate self-calibration, which might break some ideas they might have about image differencing.

With Nadia Blagorodnova (Caltech) I discussed calibration of the SED Machine, which is designed to do rapid low-resolution follow-up of ZTF and LSST events. They are using dome and twilight flats (something I said is a bad idea in my colloquium) and indeed they can see that they are deficient or inaccurate. We discussed how to take steps towards self-calibration.

With Heather Knutson (Caltech) I discussed long-period planets. She is following up (with radial velocity measurements) the discoveries that Foreman-Mackey and I (and others) made in the Kepler data. She doesn't clearly agree with our finding that there are something like 2 planets per star (!) at long periods, but of course her radial-velocity work has different sensitivity to planets. We discussed the possibility of using radial-velocity surveys to do planet populations work; she believes it is possible (something I have denied previously, on the grounds of unrecorded human decision-making in the observing strategies).

In my talk I made some fairly aggressive statements about Euclid's observing strategies and calibration. That got me some valuable feedback, including some hope that they will modify their strategies before launch. The things I want can be set or modified at the 13th hour!

a new kind of photometric redshift

Boris Leistedt came by for an hour to discuss his progress on huge-survey photometric redshift determination. His method is truly novel: It is not a template-based method in the traditional sense, but it is also not a machine-learning method of the train-and-test variety (supervised regression, like Random Forest or equivalent): It uses the causal and noise model of a template method, but fits for the details of every one of the set of templates and also the luminosity functions for the galaxies generated by each template. This makes it possible to build a photometric redshift system entirely from the data but that can be trained with a very small amount of spectroscopy. But most importantly, the model can be trained with with a training set that does not have the same flux or redshift distribution as the bulk of the sources. That is, his method is perfectly matched to the future of imaging surveys. We planned the first paper.

#AAAC, day 2, classification by discriminability

The morning started with the second day of the #AAAC meeting. Steve Kahn (Stanford) talked to us about the relationships among the superficially similar projects LSST, Euclid, and WFIRST. The argument is that they are highly complementary. I didn't really disagree, but it is not obvious that we as a community would be willing to spend a lot of money on WFIRST if we knew that LSST and Euclid are definitely going forward. I asked pointed questions and hope to follow up. Since WFIRST can do so many things, maybe it should slightly re-prioritize given the context?

In the afternoon, I talked with Amit Singer (Princeton), who was pretty adamant that the stuff I am doing on single-photon imaging is stupid and a waste of time! Late in the day, based on a comment by Greengard, Jeremy Magland and I formulated an awesome new clustering (or unsupervised classification) algorithm: Define discriminability (of j from k) to be the empirical probability that a point from distribution j be closer to a neighbor in distribution j than to a point in distribution k. Now set the boundary (which could be an arbitrarily shaped surface) to maximize discriminability. Magland ended up getting pessimistic when we realized that it would be slow. But it is worth exploring.

JPL, day 1

I am visiting JPL this week, hosted by Leonidas Moustakas. I gave a seminar today, about our work with Kepler, with a focus on the noise modeling aspects of the project. In the rest of the day I had too many great and interesting conversations to list, but highlights were the following:

With Roland de Putter I discussed parameter estimation and inference in large-scale structure projects. We talked a bit about correlation functions and Gaussian kernels. He had some nice intuitions and scaling arguments about the Gaussianity of measurements of the power spectrum that helped me understand (and corrected some of my errors in) my thinking about my cosmological inference projects. He also helped me formulate the simplest possible demonstration that what is traditionally done (in, say BAO projects) to mock up a "likelihood function" is formally (and maybe substantially) wrong and can be replaced. We also talked about inferring the initial conditions of the Universe from what we see. I pitched a "deep learning" version of this problem that I love so much I will have to write it down on the ideas blog!

Francis-Yan Cyr-Racine (JPL) told me about an absolutely awesome project to understand the statistical effect of substructure on lenses without the requirement that the substructures be detected individually. That's right up my alley, and the alley of Brewer as well. He has broken the problem into parts, combining the weak parts into a Gaussian noise and doing the strong parts the hard way. Genius.

I argued for randomized (or other regular but clever) observing strategies for WFIRST and Euclid with Jason Rhodes (JPL) and the dark-matter and dark-energy group. We lamented the lack of good observing-strategy simulations for these projects, which would make such arguments quantitative, open, transparent, and efficient.

three talks

Three great talks happened today. Two by Jason Kalirai (STScI) on WFIRST and the connection between white dwarf stars and their progenitors. One by Foreman-Mackey on the new paper on M-dwarf planetary system abundances by Ballard & Johnson. Kalirai did a good job of justifying the science case for WFIRST; it will do a huge survey at good angular resolution and great depth. He distinguished it nicely from Euclid. It also has a Guest Observer program. On the white-dwarf stuff he showed some mind-blowing color-magnitude diagrams; it is incredible how well calibrated HST is and how well Kalirai and his team can do crowded-field photometry, both at the bright end and at the faint end. Foreman-Mackey's journal-club talk convinced us that there is a huge amount to do in exoplanetary system population inference going forward; papers like Ballard & Johnson only barely scratch the surface of what we might be doing.

Gaia status, Kepler calibration

Great conversations today with Brewer about sampling and inference, Knut Jahnke and Stefanie Wachter (MPIA) and Doug Finkbeiner (CfA) about Euclid calibration, and Jennifer Hill (NYU) about data science. But that's not all:

In the morning, Coryn Bailer-Jones (MPIA) gave a status update on the Gaia mission, which was launched in December. It is performing to spec in almost all respects. Bailer-Jones gave us straight talk on three issues they now face: The scattered light (from sources not in the field of view) is much higher than expected, reducing the magnitude limits for all accuracies and capabilities. There is something (probably water) affecting the total photometric throughput. It looks like this can be mitigated with occasional thermal cycling of the optics. The "fundamental angle" between the two telescopes seems to be varying with time with a larger amplitude than expected. This can be modeled, so long as it is understood properly. I think I speak for the entire astronomical community when I say that we can't wait for early data releases and wish the Gaia Collaboration the best of luck in addressing these challenges.

In the afternoon, Dun Wang, Foreman-Mackey, Schölkopf, and I worked through Wang's results on Kepler data-driven calibration. I am pretty excited about this project: Wang has shown that when we "train" the model on data not near (in time) to an injected transit, the data recalibrated with the model produces unbiased (on average) inferences about the transit. We assigned Wang next steps.

imaging survey strategy

I spent some time discussing with Stephanie Wachter (MPIA) the Euclid calibration strategy and survey requirements. I think that Euclid (and all surveys) should do something like the "strategy C" in the paper on survey strategy by Holmes et al. Indeed, I think that strategy C would not only lead to far better calibration of Euclid than any of their current survey strategy ideas, I think it would make for more uniform solid-angular coverage, better time-domain coverage, and greater robustness to spacecraft faults or issues.

interferometry, meet probability

Great morning chats with Jahnke (MPIA) about Euclid survey and calibration strategy and Kapala (MPIA) about comparing H-alpha emission to C+ emission in M31. After that, Malte Kuhlmann (MPI-IS) arrived to discuss generative probabilistic approaches to radio interferometry (think: replacing CLEAN). We discussed his compressed-sensing-inspired methods (which produce nicely regularized point estimates) and confronted him with some highly astronomical considerations: How do we create something that is not just righteous but also gets rightly adopted by a significant user base? And: How do we make it so that we can use the output of our code to more responsibly evaluate significance and more responsibly propagate uncertainty than what is currently industry-standard in radio astronomy? On the former, a key idea is that whatever we do has to fit into the standard radio-imaging workflow. On the latter, the key idea is that we need to exercise a justified likelihood function. I have a very good feeling about this project. Kuhlmann's background is in math and stats, which is a good background for bringing important new ideas into astrophysics. The day ended with a great talk by Watkins (MPIA) about omega Cen, the content of which also inspired a spirited discussion of globular clusters on the 17:48 bus.

finishing papers

So many papers are so close to being done and yet...! Today Holmes and I gave a final push on responding to referee on our self-cal paper. We just have to let all the code run through one more time, update the final numbers and submit (I very much hope). The flip side of all the work is that the paper is very much improved, thanks to a good referee and many comments from the community.

writing down my thoughts

After great conversations in Valchava, visions of sugar-plums and spike-slab priors have been dancing in my head. So I spent today (and much of the past weekend) writing furiously a couple of documents. The first is a white-paper regarding a justified probabilistic approach to radio interferometry data. This captures things my loyal reader has been reading about for the last few weeks, plus many new ideas coming from Valchava.

The second document is a white-paper regarding the possibility of calibrating the pixel-level flat-field map (that is, many millions of parameters) for an imaging camera using the science data alone (that is, no sky or dome flats). This latter project would work by looking at consistency among neighboring pixels given (and this requires assumptions) an observing program that observes without trying to put particular science objects in particular locations. One of the things I love about this project is that sky flats can be very misleading: The smooth sky illuminates the detector differently than a star does, because there are often unbaffled stray light paths and reflection paths. I have been thinking about these issues because they might be applicable to the Euclid mission, which currently plans to rely on internal LED illuminators for calibration information.

I am not sure why I write these white papers. They are a huge amount of work, and although they are great for organizing and recording my ideas, almost none of them ever results in a real publication.

self-calibration

I spent the weekend—in transit to Heidelberg and in the garden there—trying to spec out the least complicated possible project demonstrating that hierarchical inference beats averaging for weak-lensing shear-map measurement. Today I was at MPIA, my summer home, where I will be in residence for July and August. I showed off The Thresher to the PanSTARRS team at MPIA, and then spent the afternoon struggling with Python with Rory Holmes. We are very close to resubmitting our self-calibration paper, which shows that some imaging survey strategies are much better than others. Holmes and I also talked about the next projects, which involve more realistic survey strategies for Euclid and BigBOSS and also methods that will permit simultaneous self-calibration of the flatfield, of intra-pixel variations in sensitivity, of the astrometric mapping, and of the point-spread function.

calibrating Euclid and Stripe 82

I worked with Holmes and Foreman-Mackey on calibration projects. With Holmes, we are finishing his thesis chapter on survey strategies optimized for photometric self-calibration. His results set the strategy for the Euclid proposed deep field strategy. With Foreman-Mackey, we are trying to convince ourselves that our super-robust multi-exposure calibration model is doing a good job at setting the photometric zero-points for Stripe 82. In the late afternoon, Nicholas Martin (MPIA) gave a nice talk about work on M31 satellites by Crystal Brasseur (MPIA, deceased).

four survey strategies

Holmes and I made two new survey strategies for our self-calibration project, and improved the random strategy to make it far more homogenous (previously it was Poisson-distributed, now we made it more uniform without giving up randomness). We got everything together we need to write paper 1, except for the dependencies of the calibration quality on things like the density of standard stars, parameters of the noise model, and so-on.