I spent a very large part of the day today at the whiteboard in front of Charlie Epstein (Penn), Leslie Greengard, Jeremy Magland (SCDA), and Marina Spivak (SCDA). I presented my proposed solution to the problem of diffraction imaging of molecules in the limit of very few photons per exposure. We had a brief discussion of the physics, a very long discussion of the idea of solving this problem by optimizing a marginalized likelihood, and a brief discussion of its derivatives with respect to parameters in a representation. It was an incredibly useful session: The crowd found some mistakes on my part, and it forced me to clearly articulate how I think probabilistic inference works in these cases.
I think my proposed solution is palatable to both Bayesians and frequentists: In principle the frequentists should object to my marginalization over angles, but this is close to unassailable, because when the angles are generated by an isotropic process, they really do have a well-defined distribution that does not have to be described as a prior. That is, this integral is frequentist-safe. In principle the Bayesians should object to the fact that I am going to optimize the (marginalized) likelihood rather than fully sample the posterior, but even hard-core Bayesians recognize that sometimes you can't afford to do more than get the peak and its width!
Amusing notes from the chat: My definition of “principled” is very different from Leslie Greengard's! And the crowd was not as confident as I am that I can solve a problem where the intermediate steps to the answer require as much disk storage space as exists in all of facebook (note the reference to “1011 to 1017 numbers” on the board).
I was just coming to terms with logsumexp, and now I need to start thinking about sumlogsumexpsumlogsum
ReplyDeleteI can direct you to someone who will belligerently argue that anything short of full Bayesian inference is a waste of your time and you're an idiot for even think about it; if you've encountered him then you'll know who I mean!!
ReplyDeleteTwo serious comments: have you tried TMB (Template Model Builder). It's a package for building c++ code to compute likelihoods along with their first and second derivatives via autodiff, then you're free to plug this function into your favourite HMC, MCMC, Bayesian optimisation, Newton's method, etc. code. Importantly, it has automatic sparsity detection and can apply a Laplace approximation over any random effects you nominate (if appropriate). It also allows the user to nominate particular computations to be done in parallel (if appropriate). Who knows if it'd fall over when asked to scale up to this unusually large problem on the cloud.
Second, have you any interest in a GP technique for fast computation tailored to six dimensions? My colleague, Seth Flaxman (who's also worked with your collaborator Andrew Wilson), has such a technique and is looking for applications to which I thought: galactic dynamics (3 x space + 3 x velocity).