Over the last few days, Megan Bedell (Flatiron) and I have been discussing the criteria that go into a detection of an exoplanet, or what things need to be true for a detection or discovery to be considered made. We are thinking of discoveries with the Terra Hunting Experiment or other radial-velocity surveys. We identified three-ish condtions:
One condition is that the amplitude be significantly different from zero; that is, the null is ruled out. Another is that the planet be characterizable: that is, the orbital parameters can be estimated to some level of precision. A third is that the planetary explanation of the signal be preferred at some confidence over other qualitatively different explanations, like stellar variabilities, stellar rotation, or signals faked by beats from other planets.
I presented these criteria at Astronomical Data Group meeting, and Dan Foreman-Mackey (Flatiron) said that he disagreed with every aspect of it. But we didn't (yet) find out why.