Today Kate Storey-Fisher (NYU) and I met with Mike Blanton (NYU) and Zhongxu Zhai (NYU) to discuss possible projects that Storey-Fisher and I have been talking about. We are thinking about trying to systematize (and pre-register) the search for anomalies in cosmological surveys. The idea (which is still vague) is to somehow lexicographically order all anomalies we could search for, and then search, such that we can keep exquisite track of the number of independent hypotheses we have checked.
Blanton and Zhai had some advice for us. One category of advice was around systematics: Anomalies and systematics in the data might appear similar! So we should think about anomalies that are somehow least sensitive to these systematics. One good thing is that we are working at the home of many of the tools that we need to make these assessments. Another category of advice was to think about what anomalies are motivated by questions of theory in the dark sector, in galaxy formation, or in the initial conditions. Theory-inspired (if not predicted) anomalies are more productive, in a scientific-literature sense, than randomly specified anomalies. We are close to being able to specify a project!