Adrian Price-Whelan and I met at Columbia to discuss various things. We looked at the new paper of Reid et al on distances; which appeared on twitter this month as an argument that the Milky Way has spiral structure. Although this paper is not really dishonest—it explains what it did (though with a few lacunae)—it is misleading and wrong in various ways. The most important: It is misleading because it is being used as evidence for spiral structure (its figure 5 is being tweeted around!). But it also shows (in its figure 6) that even if there was no evidence at all for spiral structure in the data, their analysis would find a spiral pattern in the posterior pdf and distance estimators! It is wrong because it (claims to) multiply together posteriors (rather than likelihoods). That is, it violates the rules of probability that I tried to set out clearly here. I try not to use the word "wrong" when talking about other people's work; I don't mean to be harsh! The team on this paper includes some of the best observational astrophysicists in the world. I just mean that if you want to do probabilistic data analysis, you should obey the rules, and clearly state what you can and cannot conclude from the data.
At lunch, Jeno Sokolowski (Columbia) spoke about accreting white dwarfs in orbit around red giant stars. I realized during her talk that we can potentially generate a catalog of enormous numbers of these from our work (with Anna Ho) on LAMOST.
If I understand the paper correctly, they appear to have assumed that their spiral arm model from Reid+ 14 (shown in Fig 1) has infinite precision (see the penultimate eqn on page 6). This goes into their prob_SA, which as Fig 6 shows is clearly responsible for much of the spiral structure in their results.
ReplyDeleteGiven that they do have some very very nice data I wonder what would happen if they propagated through the error on the spiral arm positions (or better still, built a proper model) and generally tidied things up.
Agreed - though they also assume that the spiral arms are very narrow, and the background model is low in amplitude.
Delete