Gaia subtleties

En route back to New York, I read this paper by Bastian & Biermann (recommended by Anthony Brown) about the astrometric calibration issues that arise in the Gaia satellite because of the finite drift-scan integration time across the CCDs. There are several valuable insights in the paper, but the most amusing to me is that very rapidly variable objects will obtain signals in Gaia with a different effective mean time on the CCD and therefore be subject to a very slightly different spacecraft attitude model, in some sense. I don't think this will be problematic for many stars, but at Gaia precision, a lot of things matter. It is one of the many reasons I love interacting with the Gaia community.

Bertin, McCracken, Wu

I spent an enjoyable day at the IAP in Paris, chatting with the local luminaries. It was great to see the man himself Bertin (IAP), with whom I talked about extreme deconvolution, the Tractor, and modeling point-spread functions. On the former, he strongly encouraged us to work on photometric redshifts for galaxies (we have already done quasars); he expressed a strong conviction that most methods being developed were not properly taking into account the photometric uncertainties or noise, and therefore doomed to underperform. We could do this easily; the way Bovy set up XDQSO, it is an easy swap-in replacement of training data. Does anyone out there want to do this? I would be very happy to consult, and I bet we could convince Bovy too. A successful project could have a big impact on next-generation weak lensing projects.

McCracken (IAP) and I talked about calibration and catalog generation in huge new surveys. McCracken made a nice point (preaching to the converted I should say) that the generation of catalogs is very closely related to the calibration and reduction of the data. You can't really separate these, in part because if the catalog is an approximation to the parameters of a maximum-likelihood model, its creation depends sensitively on understandings of the noise. McCracken asked me to distinguish the Tractor from the next generation of SExtractor and I think the key point is that in the Tractor we see all the information as parameters; we make no distinction between catalog quantities and calibration quantities. We can also very flexibly freeze or fit parameters as our knowledge or confidence changes. That said, we are also vapor-ware right now!

My ex-student Ronin Wu is now at CEA, so we met at IAP. We talked about getting her most interesting thesis chapter—a measurement of the molecular-hydrogen mass function using Spitzer IRS spectroscopic data—ready for publication. It is very close, but we have some strange results that suggest that different radiation-hardness indicators don't correlate well, and that the star-formation rate is not a strong function of molecular gas mass. Those need to be tracked down or understood.

interstellar medium

In transit to Paris, I worked on a problem given to me by Rix and Brent Groves (MPIA): Model the dust emission in M31 at the resolution of the highest-resolution Herschel maps, but constrained by all the maps. This is a perfect job for the Tractor but it requires creating new objects (think unresolved, emitting dust blobs). I didn't get very far, because it is a substantial generalization of what we have. What we have doesn't currently constrain the spectral energy distribution of any source very much a priori. That is, the Tractor doesn't yet have strong priors on emission mechanisms.

probabilistic interferometry

Rix and I spent part of the morning with Frank Bigiel (Heidelberg) talking about radio interferometry. In the upcoming ALMA world, people doing work on faint sources (deep fields) will want to get probabilistic information out of radio maps—information like confidence intervals on source brightness and existence. What they don't currently do in the radio world is write down a likelihood function for the scene; there is no probability of the read-out amplitudes and phases on baselines conditioned on the model for the scene on the sky. We discussed the possibility of creating that function; we spent (therefore) a lot of time talking about noise.

fast imaging; Kepler

I spent a bit of the morning interviewing Wolfgang Brandner (MPIA) about Astralux, their fast imaging camera (no longer Lucky Imaging camera since we are going to get everyone using The Thresher). We talked about how the fast cameras work; I learned that they have significant non-linearity and saturation can be a problem. The first has been corrected (how well, I wonder?) and the second we would need to take into account at Threshing. We also discussed the possibility (which is hard to deal with) that the same atmospheric variations that create a non-trivial PSF can also create plate-scale variations. Not sure whether to think about that; we do see some evidence for that in the data we have.

In the afternoon Hans Kjeldsen (Aarhus) gave a very nice seminar about Kepler. He showed that they can use helioseismology to determine the fraction of hydrogen burning that has completed inside a star; that is, as a clock! He showed nice results on exoplanets too, including surprising albedo measurements and a demonstration that multi-planet systems draw their planets from a different distribution than single-planet systems. That's nice. He said that in exchange for a mission extension, the Kepler team is giving away all the data with no proprietary period, starting soon. That might have a big impact on my activity in the next year!

self-calibration

I spent the weekend—in transit to Heidelberg and in the garden there—trying to spec out the least complicated possible project demonstrating that hierarchical inference beats averaging for weak-lensing shear-map measurement. Today I was at MPIA, my summer home, where I will be in residence for July and August. I showed off The Thresher to the PanSTARRS team at MPIA, and then spent the afternoon struggling with Python with Rory Holmes. We are very close to resubmitting our self-calibration paper, which shows that some imaging survey strategies are much better than others. Holmes and I also talked about the next projects, which involve more realistic survey strategies for Euclid and BigBOSS and also methods that will permit simultaneous self-calibration of the flatfield, of intra-pixel variations in sensitivity, of the astrometric mapping, and of the point-spread function.

Gaussian processes at UvA

I spent the afternoon in Amsterdam talking one-on-one with people about Gaussian processes. There are a lot of time-domain projects in astrophysics that involve both stochastic and quasi-periodic variability. Aside from eclipsing binaries, almost no variables are perfectly periodic. Indeed even eclipsing binaries are complicated if the members of the binary are themselves stochastically variable. Gaussian processes are among the simplest methods for modeling stochastically varying and quasi-periodically varying objects, but the simplicity is only visible once you have confronted some conceptual blocks! It is simultaneously very simple and not obvious that you can think of your entire data set as a single draw from an enormous, multi-variate Gaussian (or even better, a finite-dimensional sampling of an infinite-dimensional one). Once you make that leap—the leap of seing not each point as being a Gaussian draw but the whole data set as being a Gaussian draw—you get a huge amount of power to describe non-trivial phenomena in the time domain.

optics and data

I spent the afternoon at Leiden chatting with the locals. Among other things, I learned that Kenworthy (my host) has figured out how to use the real-time output from an adaptive-optics wavefront sensor to predict the point-spread function in the science data. This is a big deal. It means we could potentially have a model of the PSF before we even read out the data. The wavefront-sensing data are used to control the adaptive optics system but of course they also contains scientific information of great value. That point is simultaneously obvious and brilliant.

Labbé and I discussed alternatives to stacking—simple adding or averaging of data—when there are signals too weak to see in any one datum. He has written some nice things about this issue but also has the pragmatic view that doing a better job only makes sense if you get a better answer. We discussed contexts for making that point.

On the way to dinner, Kuijken described his approaches to weak lensing problems, which involve finding the shear map that distorts the image into one in which there is no net ellipticity. He made the nice point that it is much easier to show that a group of galaxies shows no net ellipticity than it is to quantitatively measure the departure from zero ellipticity. That's an interesting point and worth contemplating. He still has to deal with the point-spread function, which has not been lensed, so his methods are not trivial; we discussed the problem of the PSF because we have some disagreements there.

Gaia attitude and residuals

I had long conversations with Anthony Brown (Leiden), Daniel Risquez (Leiden), and Giorgia Busso (Leiden) about Gaia data processing and catalog output. I learned a lot, not limited to: Risquez is looking at the changes in attitude modeling precision as the model complexity is changed, where the model complexity is set by the time spacing in knots of a bspline model. The Gaia attitude model is completely data-driven (it has no sense of torques or moments of inertia, it just knows about transit times of stars). It is also not trying to know the true attitude, but the effective attitude averaged over 4.4-second intervals (because that is the CCD drift-scan time). That leads to interesting subtleties and constraints, one of which is that knot spacings much smaller than 4.4 seconds (or maybe half that) can never be useful.

Busso is working on the charge-transfer-inefficiency model for the CCDs. As the CCDs get damaged, they develop traps which delay a small fraction of the charge in the CCD drift-scan. This slightly moves the stellar centers, by much less than a CCD pixel but much more than the required precision (and in a stellar-brightness-dependent way)! It sounds impossible, but because Gaia cuts through every star at 80 or more different angles at 80 or more different times, the magnitude-dependent and time-dependent CTI effects can in fact be modeled and fit to restore the precision of the instrument. The nice thing is that the collaboration hopes to be able to make an empirical model of the CTI and its evolution; that reassures me because the theoretical models of CTI are both young and simplistic.

Brown, who I think (looking from the outside) has had lots of great influence on the Gaia collaboration, told me that early data releases from Gaia are now more-or-less promised. That's a big and important thing; if you think about how much SDSS learned from its early (and often wrong) data releases and how much Hipparcos benefitted from its complete post-final-release reanalysis, early data release is essential to the production of the best possible catalog. Brown also intrigued me by saying that it was likely that the releases would include all the timing residuals for every star at every transit. This is exciting to me because these residuals could be used to create an approximate Gaia likelihood function as I have been trying to imagine for some time now.

information theory and phase space

At Groningen (my first stop on my trip), I chatted with Robyn Sanderson (Groningen) about finding substructure in current SDSS-II and future Gaia stellar kinematic data. The idea we quickly worked out—which maybe is in the literature already—is to ask a general information-theory-like question about the phase-space distribution function in something like action space, and then tune potential parameters to make the answer to that question as informative as possible. As a straw man idea, I suggested the K–L divergence between the action distribution and a maximum entropy distribution with the same low-order moments. The thing I like about this plan is that it might not be very sensitive to the observational noise. That is, the scalar will be sensitive to it, but the optimum in parameter space might not. I said might. It's worth a try.

toy Gaia catalog

On my way to the Netherlands, and in preparation for chatting with the Leiden Gaia group, I brushed off and got working again my toy Gaia mission, in which a zero-dimensional spacecraft surveys a one-dimensional sky. The spacecraft is battered by angular-momentum-changing collisions and scans the one-d sky clockwise and counter-clockwise. The toy is designed to test ideas about fitting charge-transfer inefficiency models and spacecraft attitude models. Still not sure why I am doing either of those things; I suppose it relates at least slightly to my GALEX photon projects with Schiminovich.

stretching images for human viewing

What I call the "stretch"—the relationship between the data numbers in a data image and the byte values in a human-viewable JPEG or PNG file—is a constant problem for us, in The Thresher, in the Atlas, in the Tractor, and in Marshall's new Lens Zoo project. I spent time today with Foreman-Mackey writing hacky but very robust code for doing this on the fly on the in-progress status plots in The Thresher. The code is ugly! The problem is that you want to see the range of the data but you also want to see the noise level; at the same time, you don't know the noise level in advance and you can hit images (think clipped or sparse images) where it is quite hard to even determine the noise level. Foreman-Mackey and I solved this problem by making multiple noise estimates and using relationships among them to find the best one.

The Thresher

We had a big push on our post-lucky imaging pipeline. As part of this, we renamed it The Thresher and Foreman-Mackey made a sweet github-hosted web page for it. We are very close to release, and over lunch we discussed last things to do before writing paper zero. Paper zero will compare The Thresher to traditional lucky imaging, showing that we win in signal-to-noise at fixed resolution, and win in resolution at fixed signal-to-noise. No smoothing applied! We fit the best band-limited scene to all the data. At Camp Hogg, We Don't Throw Away Data (tm).

[By the way, in case you think a thresher throws away husks, you might be confusing a thresher with a winnower. Indeed, if I ruled the world, I would rename traditional lucky imaging pipelines winnowers.]

The Atlas

I spent a full day on the Sloan Atlas of Galaxies, working on galaxy surface-photometry measurements, sample selection, and plate layout. Damn, those galaxies are beautiful!

Gelman, photons

While the GALEX photons are winging their way to NYC by FedEx (yes, don't underestimate the bandwidth of a station wagon full of hard drives), Schiminovich, Foreman-Mackey, and I took advantage of an opportunity to spend some quality time with Andrew Gelman, stats blogger extraordinaire. We discussed, among other things, the problem of sampling in spaces of high degeneracy. The conversation ended with Gelman requesting that we get specific by writing a density function in C++ and he will fire it into STAN. This relates to our MCMC High Society challenge, though we will start with an easier problem. It also relates to the plan (hatched at CMML at NIPS) to create testbeds or challenge problems that are simple enough and documented such that they can be shared across disciplines.

In the rest of the day, we did our usual (that is, discuss everything under the Sun, including cosmological tests with Lam Hui, who we found at the coffee shop) and noted that brilliant work by the GALEX team has got us a lot further along our project of modeling the GALEX instrument response, spacecraft pointing model, and astronomical source variability using all the individual (that is, not co-added) photons.