THE meeting, day 1

Today was the first day of the Terra Hunting Experiment collaboration meeting. This project is to use HARPS3 for a decade to find Earth-like planets around Sun-like stars. The conversation today was almost entirely about engineering and hardware, which I loved, of course! Many things happened, too many to describe here. One of the themes of the conversation, both in session and out, is that these ultra-precise experiments are truly integrated hardware–software systems. That is, there are deep interactions between hardware and software, and you can't optimally design the hardware without knowing what the software is capable of, and vice versa.

One presentation at the meeting that impressed me deeply was by Richard Hall (Cambridge), who has an experiment to illuminate CCD detectors with a fringe pattern from an interferometer. By sweeping the fringe pattern across the CCD and looking at residuals, he can extremely precisely measure the effective centroid in device coordinates of every pixel center. That is impressive, and it is now known to be one of the leading systematics in extreme precision radial velocity. That is, we can't just assume that the pixels are on a perfect, regular, rectangular grid. I also worked out (roughly) a way that he could do this mapping with the science data, on sky! That is, we could self-calibrate the sub-pixel shifts. This is highly related to things Dustin Lang (Perimeter) and I did for our white paper about post-wheel Kepler.

black holes and weird pixel effects

In group meeting, Huppenkothen argued out the projects we discussed on Monday related to machine classification of black-hole accretion states of GRS 1915. We talked about all three levels of project: Using supervised methods to transfer classifications for a couple of years of data onto all the other years of data, using unsupervised methods to find out how many classes there plausibly are for the state, and building some kind of generative model either for state transitions or for literally the time-domain photon data. We discussed feature selection for the first and second projects.

Also at group meeting, Foreman-Mackey showed a new Earth-like exoplanet he has discovered in the Kepler data! Time to open our new Twitter (tm) account. He also showed that a lot of his false positives relate to un-discovered discontinuities in the Kepler photometry of stars. After lunch, we spent time investigating these and building (hacky, heuristic) code to find them.

Here are the symptoms of these events (which are sometimes called "sudden pixel sensitivity drops"): They are very fast (within one half-hour data point) changes to the brightness of the star. Although the star brightness drops, in detail if you look at the pixel level, some pixels brighten and some get fainter at the same time. These events appear to have signs and amplitudes that are consistent with a sudden change in telescope pointing. However, they are not shared by all stars on the focal plane, or even on the CCD. Insane! It is like just a few stars jump all at once, and nothing else does. I am confused.

Anyway, we now have code to find these and (in our usual style) split the data at their locations.

finding tiny planets, Kepler jumps, papers

Foreman-Mackey and I had a long and wide-ranging conversation about exoplanet search. He has search completeness in regimes of exoplanet period heretofore unexplored, and more completeness at small radii than anyone previously, as far as we can tell. However, his search still isn't as sensitive as we would like. We are doing lots of hacky and heuristic things ("ad hockery" as Jaynes and Rix both like to say), so there is definitely room for improvement. All that said, we might find a bunch of smaller and longer-period planets than anyone before. I am so stoked.

In related news, we looked at a Kepler star that suffered an instantaneous change in brightness. We went back to the pixel level, and found the discontinuity exists in every pixel in the star's image, but the discontinuity has different amplitudes including different signs in the different pixels. It is supposed to be some kind of CCD defect, but it is as if the star jumped in position (but its fellow stars on the CCD didn't). It is just so odd. When you do photometry at this level of precision, so many crazy thing appear.

Late in the day I caught up on reading and commenting on papers people have sent me, including a nice paper by Megan Shabram (PSU) et al on a hierarchical Bayesian model for exoplanet eccentricities, a draft paper (with my name on it) by Jessi Cisewski (CMU) et al on likelihood-free inference for the initial mass function of stars, a draft paper by Dustin Lang (CMU) and myself on principled probabilistic source detection, a paper by John Jenkins (Ames) et al on optimal photometry from a jittery spacecraft (think Kepler), and the draft paper by Melissa Ness et al on The Cannon.

Gaia RVS spectra

As I mentioned a few days ago, Mark Cropper (UCL) brought up the question of how to infer clean, high-resolution spectra from the Gaia RVS data, which are afflicted by distortions caused by charge-transfer inefficiency. I spent my small amount of work time today drafting a document with my position, which is that if you can forward-model the CTE (which they can) then spectral extraction is not hard.

GaiaCal, day two

A lot of the second day of #GaiaCal was about "benchmark stars". Indeed, my concerns about "truth" from yesterday got more severe: I don't think there is any possibility of knowing the "true" value of Fe/H or anything else for a star, so we should probably figure out ways of doing our science without making reference to such inaccessible things. One approach is to define the results of a certain set of procedures on a certain set of stars to be "truth" and then warp all models and methods to reproduce those truths. That's worth thinking about. During the day's proceedings, Blomme and Worley showed some incredible spectra and spectral results on the benchmarks, including abundance measurements for some 25 elements.

Along different lines, Cropper gave an astounding talk about the Gaia RVS spectral data reduction pipeline. It was astounding because it involves a complicated, empirical, self-calibrated model of charge-transfer efficiency and radiation damage, which is continuously updated as the mission proceeds. It also includes a spectral calibration (wavelength solution, astrometry) that is fit simultaneously with all spectral extractions in a complete self-calibration. Cropper asked the implicit question in his talk: "How do we extract CTE-undamaged spectra from CTE-damaged data?". I think I know how to answer that; resolved to write something about it.

Liu showed amazing empirical calibrations of the LAMOST spectra using SEGUE parameters and support vector machines. He has done a great job of tranferring the one system onto the other data, which gives hope for the benchmark plans that are emerging. Bovy gave a nice talk about ages (birth dates), chemical abundances, and orbital actions serving as stellar "tags" which can then be understood statistically and generatively.

Kepler issues

At breakfast, I went through with Barclay and Quintana (Ames) my list of all the effects that lead to variability in Kepler light-curves. These include intrinsic stellar variability, stellar variability from other stars that overlap the target, stellar variability transferred to the target by electronics issues. They include stellar proper motion, parallax, and aberration. They include variations in spacecraft temperature, pointing, and roll angle. And so on. The list is long! I am trying to make sure we understand what our pixel-level model covers and what it doesn't. I am spending a lot of my writing time on our data-driven pixel-level model getting the assumptions, capabilities, and limitations clearly specified.

the sub-pixel flat field

I took a risk up at Columbia's Pizza Lunch forum by talking about the Kepler flat-field. I also was exceedingly rude and talked through Price-Whelan's spot (he was supposed to follow me). I apologize! Well, you can't say I didn't try to bore the pants off of everyone: I talked about the (novel, and exciting to almost no-one other than me) result, published in our white paper, that it is possible to infer the properties of the flat field at higher than pixel resolution.

That is, the team (meaning, in this case, Lang) made simulated data with drifting stars, a PSF that varies slowly with position (and is well understood), and no prior knowledge about how the stars in the field are drifting. We find (meaning Lang finds) that he can simultaneously figure out the pointing of the satellite and the flat-field, even when the flat-field is both created and fit with models that have multiple sub-pixels per pixel. The reason it works is that as the star moves, it illuminates each pixel differently, and is therefore differently sensitive to the different parts of each pixel. It is not clear yet whether we can do this accurately enough to recover the Kepler sub-pixel flat-field, but damn I want to try. Unfortunately, we need lots of data taken in the two-wheel mode, and (as far as I know) they aren't yet taking any new data. Kepler: Please?

Save Kepler Day, insane imaging precision

Today was Save Kepler Day at Camp Hogg. Through a remarkable set of fortunate events, I had Barclay (Ames), Fergus (NYU), Foreman-Mackey (NYU), Harmeling (MPI-IS), Hirsch (UCL, MPI-IS), Lang (CMU), Montet (Caltech), and Schölkopf (MPI-IS) all working on different questions related to how might we make Kepler more useful in two-wheel mode. We are working towards putting in a white paper to the two-wheel call. The MPI-IS crew got all excited about causal methods, including independent components analysis, autoregressive models, and data-driven discriminative models. By the end of the day, Foreman-Mackey had pretty good evidence that the simplest auto-regressive models are not a good idea. The California crew worked on target selection and repurpose questions. Fergus started to fire up some (gasp) Deep Learning. Lang is driving the Tractor, of course, to generate realistic fake data and ask whether what we said yesterday is right: The loss of pointing precision is a curse (because the system is more variable) but also a blessing (because we get more independent information for system inference).

One thing about which I have been wringing hands for the last few weeks is the possibility that every pixel is different; not just in sensitivity (duh, that's the flat-field) but in shape or intra-pixel sensitivity map. That idea is scary, because it would mean that instead of having one number per pixel in the flat-field, we would have to have many numbers per pixel. One realization I had today is that there might be a multipole expansion available here: The lowest-order effects might appear as dipole and quadrupole terms; this expansion (if relevant) could make modeling much, much simpler

The reason all this matters to Kepler is that—when you are working at insane levels of precision (measured in ppm)—these intra-pixel effects could be the difference between success and failure. Very late in the day I asked Foreman-Mackey to think about these things. Not sure he is willing!

imaging models; save Kepler

I arrived at the MPI-IS in Tübingen to spend two days talking about image modeling with Schölkopf, Harmeling, and Kuhlmann. A lot of what we are talking about is the possibility of saving Kepler, where our big idea is that we can recover lost precision (from loss of pointing accuracy) by modeling the images, but we also talked about radio interferometry. On the Kepler front, we discussed the past Kepler data, the precision requirements, and the problems we will have in modeling the images. One serious problem for us is that because Kepler got its precision in part by always putting the stars in the exact same places in the CCD every exposure, we don't get the kind of data we want for self-calibration of the detector and the PSF. That's bad. Of course, the precision of the whole system was thereby made very good. In two-wheel mode (the future), the inevitably larger drift of the stars relative to the CCD pixels will be a curse (because the system won't be perfectly stable and stationary) but also a blessing (because we will get the independent information we need to infer the calibration quantities).

On the radio-interferometry front, we discussed priors for image modeling, and also the needs of any possible "customers" for a new radio-interferometry image-construction method. We decided that among the biggest needs are uncertainty propagation and quantification of significance. These needs would be met by propagating noise, either by sampling or by developing approximate covariance-matrix representations. In addition, we need to give investigators ways to explore the sensitivities of results to priors. We came up with some first steps for Kuhlmann.

In the afternoon, I spoke about data analysis in astrophysics to a group of high-school students interested in machine learning.

blind calibration

In the usual form of astronomical self-calibration—like what we did to get the flats in SDSS or what I wrote up with Holmes and Rix—you use repeated observations of the same stars at different focal-plane positions to determine the sensitivity as a function of focal-plane position. Today at computer-vision-meets-astronomy group meeting, Fadely showed that he has the potential to calibrate the sensitivity of a device using only the statistical properties of observed image patches. He doesn't need to even identify sources, let alone determine when the same source has been re-observed. It is still early days, so we don't know the final precision or how it depends on the number of images, number of sources in those images, PSF sampling, and so on, but it looks very promising. In particular, if we pair it with the more traditional self-calibration it could be extremely precise. Our goal: Obviate the taking of (always questionable) flat-field images. This is also something I enjoyed discussing two weeks ago with Suntzeff (TAMU) when I was out in Kansas. (He thought I was talking crazy.)

Gaia subtleties

En route back to New York, I read this paper by Bastian & Biermann (recommended by Anthony Brown) about the astrometric calibration issues that arise in the Gaia satellite because of the finite drift-scan integration time across the CCDs. There are several valuable insights in the paper, but the most amusing to me is that very rapidly variable objects will obtain signals in Gaia with a different effective mean time on the CCD and therefore be subject to a very slightly different spacecraft attitude model, in some sense. I don't think this will be problematic for many stars, but at Gaia precision, a lot of things matter. It is one of the many reasons I love interacting with the Gaia community.

Gaia attitude and residuals

I had long conversations with Anthony Brown (Leiden), Daniel Risquez (Leiden), and Giorgia Busso (Leiden) about Gaia data processing and catalog output. I learned a lot, not limited to: Risquez is looking at the changes in attitude modeling precision as the model complexity is changed, where the model complexity is set by the time spacing in knots of a bspline model. The Gaia attitude model is completely data-driven (it has no sense of torques or moments of inertia, it just knows about transit times of stars). It is also not trying to know the true attitude, but the effective attitude averaged over 4.4-second intervals (because that is the CCD drift-scan time). That leads to interesting subtleties and constraints, one of which is that knot spacings much smaller than 4.4 seconds (or maybe half that) can never be useful.

Busso is working on the charge-transfer-inefficiency model for the CCDs. As the CCDs get damaged, they develop traps which delay a small fraction of the charge in the CCD drift-scan. This slightly moves the stellar centers, by much less than a CCD pixel but much more than the required precision (and in a stellar-brightness-dependent way)! It sounds impossible, but because Gaia cuts through every star at 80 or more different angles at 80 or more different times, the magnitude-dependent and time-dependent CTI effects can in fact be modeled and fit to restore the precision of the instrument. The nice thing is that the collaboration hopes to be able to make an empirical model of the CTI and its evolution; that reassures me because the theoretical models of CTI are both young and simplistic.

Brown, who I think (looking from the outside) has had lots of great influence on the Gaia collaboration, told me that early data releases from Gaia are now more-or-less promised. That's a big and important thing; if you think about how much SDSS learned from its early (and often wrong) data releases and how much Hipparcos benefitted from its complete post-final-release reanalysis, early data release is essential to the production of the best possible catalog. Brown also intrigued me by saying that it was likely that the releases would include all the timing residuals for every star at every transit. This is exciting to me because these residuals could be used to create an approximate Gaia likelihood function as I have been trying to imagine for some time now.