coordinate freedom

Bernhard Schölkopf (MPI-IS) and I spent time drinking coffee this weekend. Among the many subjects we discussed was language around invariance, equivariance, covariance, coordinate freedom, and symmetry. These words! I have strong opinions, as my loyal reader might know. But during the conversation I had an epiphany in which I understood why Einstein called the symmetry of general relativity “general covariance”: He was probably keying off the mathematicians, who used covariance back then the way we use equivariance now.

I don't like the word “equivariance” at all! Why do we want to write the laws of physics in a rotationally-symmetric (or rotationally equivariant or orientation-free) way? There are two completely different reasons! One is that the laws of physics are observed to be rotationally invariant (which leads to conservation of angular momentum and so on). The other is the theoretical idea that the laws of physics can't depend on investigator choices about coordinates. These are completely different, and the latter is extremely strong. We debated whether there was something to write about all this somewhere.

cosmology, now with less approximation; RNADE

In the morning, Mike O'Neil (NYU), Iain Murray (Edinburgh), Malz, Vakili, and I met for coffee to talk cosmology and cosmological inference. We discussed the linear algebra of making full (non-approximate) likelihood calls for cosmic background inference, which includes determinant and solve calls for enormous (dense) matrices. A few days ago, Wandelt made the (perhaps obscure) comment to me that he did his Gibbs sampling in the CMB to avoid making determinant calls. I finally understood this: The determinant is big and expensive in part because it is like an integral or marginalization over the nuisance parameters (which are the initial conditions or phases). If you can compute the determinant, you get something you can think of as being a marginalized likelihood function. The Gibbs sampling is part of a system that does posterior sampling, so it does the marginalization but doesn't return the amplitude of the likelihood function. If you need a marginalized likelihood function, you can't do it with the Gibbs sampling. Murray once again made the simple point that the posterior (for the cosmological parameters) will in general be much easier to compute and represent than the likelihood function (the probability of the data given the cosmological parameters) because the former is a much smaller-dimensional thing. That's a deep point!

That deep point played also into his talk in the afternoon about his RNADE method (and code) to use deep learning to represent very complex densities, or estimate a density given a sampling. One of his application areas is a project to obtain posterior constraints on the mass of the Milky Way given (as data) the properties of the Magellanic Clouds. The theory in this case is represented by a large number of numerical simulations, but Murray wants a continuous density to do inference. RNADE is really impressive technology.

so many conversations!

Today was an all-talk day. I started off with Stumm and Foreman-Mackey for coffee, where we talked about various matters of only limited relation to astronomy. I chatted with yesterday's speaker Ruderman (Berkeley) about naturalness and the future of particle physics. Then Bob Lindner (Rutgers) arrived and we did an early lunch, at which I pitched improvements to radio astronomy image reconstruction methods. Lindner gave a mid-day talk on clusters observed in the radio and sub-millimeter to look at S-Z effect and star formation (and lensing). The afternoon was filled with our MCMC meeting, in which we battled Sven Kreiss about how they put error bars on parameters in the LHC ATLAS world. We learned something about the relationships between frequentist confidence intervals and the Bayesian equivalents in high dimensional parameter spaces. Awesome!

fun day

On my way to work I ran into Fergus, taking photos in preparation for a SIGGRAPH submission. I helped him out for an hour. Fouesneau (UW) made plots to evaluate the quality of his PHAT young stellar cluster fits. They look great; I think he has nailed this fitting. Next up: Do the fits in multiple bands and check that the color distribution is tighter than it used to be. At lunch Foreman-Mackey figured out that the dry-erase glass coffee table I am building (yes, building) could include a back-projected external monitor. That lost us some time with online shopping for compact, short-throw projectors. At MCMC meeting (Goodman, Hou, Foreman-Mackey, Fadely, and I) discussed combinatoric degeneracies and their ubiquity and hardness. The fact that you can reorder a set of planets or stars or whatever in your model and leave the likelihood unchanged is a real problem: You either fix this massive, perfect degeneracy in the prior (by enforcing order) or else live with it. Either way, it hurts performance in sampling with almost all known methods. Late in the day, as I was describing my dream of a data-driven modeling approach to APOGEE chemical abundances, Fadely had a brainstorm: We could consider random subsets of the spectrum to avoid unknown or uncharacterized data or noise issues. That made Foreman-Mackey immediately say random forest, a method I never thought I would find myself using. Over the next hour I became more and more convinced that random forest could do almost exactly what I need. There were many other breakthroughs in that conversation, including desiderata for the data-driven model and ways to test it or start out.

triaxial Galaxy?

At morning coffee at OCIW, David Law (UCLA) told us that he has evidence—from the morphology of the Sagittarius stream, mainly—that the Milky Way halo could be triaxial. This is pretty intriguing, because it is the generic prediction for dark-matter halos, as I understand is (substructure notwithstanding). He reminded me of the paradox that the Sagittarius stream must—in any model—be very old, which goes against the idea that galaxies are generically building up their mass through mergers.