non-negative again, pixel-level self-calibration

Michael Hirsch (UCL & MPI-IS) arrived for two days at NYU. We talked about self-calibration and blind deconvolution. On the latter, I was arguing that many things people usually do in computer vision might not work for astronomy, because astronomers expect to be able to make measurements (especially flux measurements) on their processed images. Some computer vision methods break that, or make measurements highly biased. On that point, I did my usual disparage of non-negative. Like Schölkopf, Fergus disagreed: If we think the fundamental image-formation mechanism is non-negative, then non-negative is the way to go methodologically. I think there might be a problem if you impose non-negative but not, at the same time, other things that are similarly informative that you know about the imaging. Anyway, we left it that I would make a fake data set that obeys exactly the image formation model but still leads to badly biased results when standard blind deconvolution is applied to it. That would be a service to this endless argument.

We also thought more and argued more about the idea that Fadely's brain-dead model of tiny patches of SDSS imaging data could be used for self-calibration purposes. We have a rough plan, but we are still contemplating whether the calibration and the data model could be learned simultaneously.

1 comment:

  1. Isn't Eddington bias an example of this - the bias that causes you to overestimate the fluxes of sources near the few-sigma detection limit, because you detect more often the ones that have a positive statistical fluctuation. Effectively this happens because you are modeling the image as a sum of PSFs (or similar) down to some fraction of the detection limit, which is basically a type of deconvolution with non-negativity imposed. You could try deconvolving it as a sum of a nearly infinite number of positive PSFs but this is likely to go berserk unless you know the PSF really well. (This is effectively how radio's CLEAN works, but they have either calibration data for measuring the PSF or good self-calibration. We may have talked about that this summer.)

    It all depends on what you want to do with the data. If you just want to detect the location of sources down to something resembling a 3 or 5-sigma limit, then a non-negative model will work. But if you want to know the fluxes accurately, the same model will be significantly biased near the limit. See also endless accounts of flux de-boosting in sub-mm observations, where essentially all the sources are marginal enough to be biased.