Boris Leistedt dropped in today and we discussed his methods to build a physically possible model of galaxy spectral energy distributions and therefore photometric redshifts, but with an exceedingly flexible model. His method is brilliant because it is entirely data-driven (no fixed templates) and yet it respects the physics of special relativity (the Doppler shift), which the machine-learning methods do not.
He made the amusing point that his method can be trained with a training set that contains literally a single galaxy with a spectroscopic redshift! That is, if you even have only one single redshift, you can put photometric redshifts (with, admittedly, large error bars) on all the other photometric galaxies! That is a property that no other data-driven method has. The point is that if you have multi-wavelength data on a single galaxy with a redshift, you can make rough predictions about how other galaxies would look at other redshifts.
His real breakthrough is the idea of using Gaussian processes to put priors on the spectral energy distributions (templates): If the SEDs are drawn from a Gaussian process, then all of the photometry (which consists of linear projections of the SEDs) is also drawn from a Gaussian process. We discussed the magic of all of this.
I also read a proposal by Daniela Huppenkothen, and wrote words in my inference-of-variances paper.