I have been working on automated calibration and vetting of data for a while. At the same time, I have been telling people that the only way to know whether your data are okay is to do some real scientific investigation with those data. The logical conclusion of this is that if we want hard-core automated data vetting, we need robots that do science. I wrote about this in my contribution for the upcoming CESS 2009.
This all ties into our craziness about the
Theory of Everything or simultaneous modeling of all astronomical imaging. We have figured out—in principle—a way to do this fitting so that a robot can make
discoveries as it ingests new data; any data useful for a real discovery is clearly vetted at some level.