in the space of the data!

Bird and I spent our last hours together in Heidelberg working out a plan and to-do list for his paper on the age–velocity relation in the Milky-Way disk. We planned tests and extensions, the scope of the minimal paper, and a visualization of the model. We worked out a way to express the model in the space of the raw-ish data, which is important: You can't assess what your model is doing in the space of the latent parameters; the data are the only things that exist (imho). That is, you have to project your model into the data space to assess where it is working and not working. And to decide whether you need to do more. That's old-school Roweis.

Fouesneau, Geha

Morgan Fouesneau (UW) arrived for a few days of paper-finishing intensity. He is trying to measure the unresolved (confused) light in young star clusters observed in the PHAT data on M31. We discussed the model and I asked for some diagnostics. They look good; the models seem to be working better than Fouesneau himself thought! Marla Geha (Yale) also showed up for some spectroscopic calibration consulting. We discussed the projects I had been doing with Roweis (and Bolton a bit) at the time of Roweis's death. She may want to give them a shot! We started by specifying some visualizations to make from her current calibrations. The idea is: If we can find invariants of the calibration, we can strongly regularize the calibration fits. Roweis and I were doing this in full generality, but the first thing to do is just look at the data.

cosmology meets machine learning

Today was the first day of the workshops at NIPS, and the day of the Cosmology Meets Machine Learning organized by a group led by Michael Hirsch (UCL and MPK Tübingen). What a day it was! The talks, by astronomers doing cosmology with sophisticated machine tools, were edutaining, with (among others) Lupton doing his best to pretend to be curmudgeonly (okay, he does have a point that some of the stuff I say is not all that practical), Starck showing amazing decompositions of Planck-like maps, and Refregier doing his best to alarm us about the difficulty of the cosmological weak lensing problem. In between these talks were shorts by the poster presenters; all good and all high bandwidth in their four-minute spots. A standout for me was Kaisey Mandel and his hierarchical probabilistic model for the type-Ia SNe, making the cosmological constraints more precise by hierarchically learning the priors over the nuisance parameters you need to marginalize out if you want to do things right!

While many left to ski, Marshall declared the afternoon break to be an un-workshop in which workshop topics self-assembled and self-organized. This evolved to two big un-workshops, one on probabilistic graphical models, with Iain Murray doing the heavy lifting, and one on blind deconvolution with Hirsch throwing down. Hirsch showed some devastating results in blind and non-blind deconvolution, including (in the style of Rob Fergus), outrageous ability to compensate for bad hardware or bad photography. Outrageous.

Despite all that, it was the PGM workshop with Murray that—and I am not exaggerating here—was possibly the most educational ninety minutes of my post-graduate-school life. After some introductory remarks by Murray, we (as a group) tried to build a PGM for Refregier and Bridle's weak-lensing programs. Marshall insisted we use the notation that is common in the field and keep it simple, Murray insisted that we do things that are not blantantly wrong, Stefan Harmeling provided philosophy and background, especially about the relationship between generative modeling and probabilistic modeling, Lupton tried to stay as curmudgeonly as he could, and at the end, Murray broke it all down. It wasn't just science, it was like we were starring in an HBO special about science. We realized that PGMs are very valuable for de-bugging your thinking, structuring the elements of your code, and, of course, making sure you write down not-wrong probability expressions. Aawww Yeah!

At the end of the day, Marshall moderated a (huge) panel, which covered a lot of ground. The crazy thing is that we had some important points of consensus, not limited to the following: (1) As a pair of overlapping communities, our best area of overlap is in structured, physics-informed probabilistic modeling. Many cosmologists are stuck on problems like these, many machine learners have good technology (things like sparse methods, online and stochastic methods, and sampling foo). Neil Lawrence pointed out that the machine learners got their Bayes from astronomers Gauss and Laplace. Now the astronomers are asking for it back. (2) We should be setting up some simple challenges and toy problems. These make it easy to draw machine learners into the field, and help us boil our issues down to the key ideas and problems. That's Murray's big point.

Hirsch, Bridle, Marshall, Murray, and everyone else: Thank you. Absolutely cannot understand why Sam Roweis wasn't there for it. I never really will.

How does a coronograph work?

As my loyal reader knows, Fergus and I have been working on data from Oppenheimer's (AMNH) 1640 coronograph. Fergus's model is a data-driven, empirical model of the speckle pattern as a function of wavelength, informed by—but not fully determined by—the physical expectation that the pattern should grow (in an angular sense) with wavelength. Fergus's model is very simple but at the same time competitive with the official data pipeline. Nonetheless, we had to make a lot of decisions about what we can and can't hold fixed, and what we can and can't assume about the observations. We resolved many of these issues today in a long meeting with Oppenheimer and Douglas Brenner (AMNH).

Complications we discussed include the following: Sometimes in the observing program, guiding fails and the star slips off the coronograph stop and into the field. That definitely violates the assumptions of our model! The spectrograph is operating at Cassegrain on the Palomar 200-inch, so as the telescope tracks, the gravitational load on the instrument changes continuously. That says that we can't think of the optics as being rigid (at the wavelength level) over time. When the stars are observed at significant airmass, differential chromatic refraction makes it such that the star cannot be centered on the coronograph stop simultaneously at all wavelengths. The planet or companions to which we are sensitive are not primarily reflecting light from the host star; these are young planets and brown dwarfs that are emitting their own thermal energy; this has implications for our generative model.

One more general issue we discussed is the obvious point made repeatedly in computer vision but rarely in astronomy that astronomical imaging (and spectroscopy too, actually) is a bilinear problem: There is an intensity field created by superposing many sources and an instrumental convolution made by superposing point-spread-function basis functions. The received image is the convolution of these two unknown functions; since convolution is linear, this makes the basic model bilinear—a product of two linear objects. The crazy thing is that any natural model of the data will have far more parameters than pixels, because the PSF and the scene both are (possibly) arbitrary functions of space and time. Astronomers deal with this by artificially reducing the number of free parameters (by, for example, restricting the number of basis functions or the freedom of the PSF to vary), but computer vision types like Fergus (and, famously, my late colleague Sam Roweis) aren't afraid of this situation. There is no problem in principle with having more parameters than data!

Sam Roweis Symposium

Today was the Sam Roweis Symposium at NIPS. I spoke, along with four other of Roweis's close collaborators. I learned a lot, especially how LLE and related methods work. It was a great session. One thing it all reminded me of is that the NIPS crowd is far more statistically and inferentially sophisticated than even the most sophisticated astronomers. It really is a different world.

In the morning before the Roweis symposium, two talks of note were by Martin Banks (Berkeley) and Josh Tenenbaum (MIT). Banks talked about the perceptual basis for photographic rules and concepts. The most impressive part of it, from my point of view, was that he explained the tilt-shift effect: If you limit the depth of field in an image, the objects being photographed appear tiny. The effect is actually quantitatively similar to binocular parallax, in the sense that the governing equation is identical. In binary parallax you measure distances relative to the separation of your eyes; in depth-of-field you measure distances relative to the size of your pupil entrance!

Tenenbaum talked about very general models, in which even the rules of the model are up for inference. He has beautiful demos in which he can get computers to closely mimic human behavior (on very artificial tasks). But his main point is that the highly structured models of the mind, including language, may be learned deeply; that is, it might not just be fitting parameters of a fixed grammar. He gave good evidence that it is possible that everything is learned, and noted that if the program is to be pursued, it needs to become possible to assign probabilities (or likelihoods) to computer programs. Some work already exists in this area.

faster iterations, but a whole lot more of them

On my last day at MPIA—what a great summer it has been—Tsalmantza and I raced an iterated gradient descent against a block-diagonal method for solving the matrix factorization problem we have been working on (I have given it many names, but I now realize that Roweis would have called it matrix factorization). We were both sure that gradient descent would crush because in our (not brilliant) R implementation, the gradient descent steps took hundreds of times less time to execute than the block-diagonal steps. But it turned out that gradient descent takes more than hundreds of times more iterations! So the block-diagonal method wins. We probably could beat both with conjugate gradient, but that will wait for another day. I have packing to do!

domains

Not much today. I worked on my writing projects, and got Roweis's domain name registrations—which include Astrometry.net—under control.

fiber mapper

I did some writing in three places: I worked on my Roweis reminiscence; I put together a straw man proposal for taking Bayesian exoplanet analysis to meta-analysis, with hyper-priors and simultaneous fitting of all planets; and I wrote an email to the SDSS-III BOSS collaboration about fiber-mapping (the determination of which fiber on the slit-head is plugged into which hole in the focal-plane plate). That email contained the following summary and introductory paragraphs of that email:

executive summary: If ever we needed to replace the fiber mapper system, there is a simple system we could build that maps all of the fibers with a single read of the science CCDs, obviating the need for a fiber-mapping hardware system that is independent of the science camera, and obviating the need for a large part of our cross-system meta-data maintenance. The system works by projecting non-degenerate patterns—a different one for each of a set of optical wavelengths—on the focal plane. The set of wavelengths "seen" down each fiber is then a unique (invertible) function of position.

historical note: On January 17th, at the memorial for Sam Roweis here in New York, Finkbeiner (Harvard), Eisenstein (Arizona), Lang (Princeton), Hogg (NYU), and Mierle (Google) had a late-night conversation about spectroscopy in which Mierle (engineer, worked on astrometry.net data structures and worlds-fastest kd tree) asked how the SDSS spectrographs work. After a few minutes of introduction, Finkbeiner challenged him with the fiber-mapping question and Mierle produced this idea, more-or-less. What I say below is based on refinements of the idea that happened around the table that night and in conversations after. I am sorry to be so slow to transmit this to the collaboration, but I guess since we have a working fiber-mapper, this is not urgent!

[Detailed proposals follow this in the email but are too boring to repeat here; email me if you want the full text.]

obituaries done, merger rates measured

I finished my two obituaries for Sam Roweis, one for our high school and one for the American Astronomical Society. You can read the submitted version of the AAS obituary here (PDF). I also spent some time writing in my much longer reminiscence, which I am hoping will be a document that captures what I learned from Roweis in a professional and personal way.

I also spent a long time talking to Tao Jiang, the student who is following up Masjedi's work on merger rates, but now in the full SDSS Main Sample, and thinking about what to do with SDSS-III. I am pretty excited about our results which have very high signal-to-noise.

pixel modeling and giving up

I spent the day talking with Bolton and Lang (and, variously, Blanton and Bovy) about modeling images and spectra. We principally debated what advantages you will or could get in data analysis by building good models of the raw data at the pixel level. We all have intuitions that it will be good, but we have few concrete examples. We assigned Bovy one task along these lines: Show that the SDSS astrometry of point sources could or would be much better if we re-centroid using a PSF fit rather than the (approximate) thing done by the pipelines at present.

In the afternoon, I made the difficult decision to drop my robotic telescope proposal for NYU Abu Dhabi and the NYU Global Campus. This is a great project, which we might resurrect, but I am feeling stretched too far and I need more time to understand my research program in the post-Roweis period. Argh.

obituaries, universes, spectrographs

[Just back from a few days of vacation.]

I attended an interesting and extremely interactive talk about the observability of universe–universe collisions by Tommy Levi (UBC). His calculations are approximate, but give a general idea about what would be observed. Interestingly, there is a cold spot in the WMAP map that could be what he predicts.

In the rest of the day I discussed next-generation spectroscopy projects with Adam Bolton (Utah) and worked on two obituaries for Sam Roweis.

Sam, numerical stability

I spent the train ride coming back from Queen's drafting an AAS obituary about Roweis. I decided also that I am going to count scientific reminiscence about Roweis as research for the purposes of this diary, lest I end up with nothing to say!

On the flight back to New York, I followed up an intuition floating around in my head that small changes in what we call the orbital parameters could make the code that converts between phase-space position and standard orbital elements much more numerically stable at edge cases. I found some nice things, and I am confident that there are large improvements to be made. Here's a trivial example: Instead of working with inclination of the orbit and the longitude of the ascending node, work with the sine of the first times the sine and cosine of the second. These two products are just the x and y components of the angular-momentum-direction unit vector. A well-known change is to work with the argument of perihelion plus the longitude of the ascending node and not with argument on its own, but I think I can show that an even better one is to work with the eccentricity times the sine and cosine of that angle sum. And so on. Not sure if we will analyze and implement all these before or after we finish our Holmes project.