Today was the first day of dotastronomy, the meeting for astronomy and web and outreach and so-on, this time in Cambridge, MA. Stand-out talks included those by Stuart Lynn (Adler) on the Zooniverse and Elisabeth Newton (Harvard) about astronomy blogging in general (she mentioned this blog) and Astrobites in particular. Astrobites has been an incredible resource for astronomy, and it is carefully cultivated, edited, and managed. What a project!
In the afternoon we switched to unconference, some of which I skipped to attend a phonecon about Kepler data with the exoSAMSI crew, organized by Bekki Dawson (Harvard), who is effectively our leader. On that call, we discussed what everyone has been doing since exoSAMSI, which is quite a bit. Barclay (Ames) has been working on inferring the limb-darkening laws using transits as measuring tools. Quarles (Texas) has been searching the real-stars-with-injected-planets that we (read: Foreman-Mackey) made back at exoSAMSI, with some success. Foreman-Mackey and Angus have been searching for long-period systems with a fast Gaussian Process inside the search loop. We also spent some time talking about modeling the pixel-level data, since we at CampHogg have become evangelists about this. The SAMSI program, organized mainly by Eric Ford (PSU) has been incredibly productive and is effectively the basis for a lot of my research these days.
In my dotastro talk this morning, I mentioned the point that in "citizen science" you have to model the behavior of your citizens, and then generalized to "scientist science": If you are using data or results over which you have almost no control, you probably have to build a model of the behavior and interests and decision-making of the human actors involved in the data-generating process. In the afternoon, Lintott (Oxford) suggested that we find a simple example of this and write a short paper about it, maybe in an area where it is obviously true that your model of the scientists impacts your conclusions. That's a good idea; suggestions about how to do this from my loyal reader (you know who you are) are welcome.
Looking forward to reading that paper on April 1!
ReplyDeleteExoplanet searches could be a good place for this - any attempt to infer the overall number of exoplanets would presumably have to account for (a) publication bias and (b) preferential targeting of stars where it's easier to detect planets. Not a field I know much about though.
ReplyDeleteThe field you want is forensic science. The two big journals are JFS (Journal of Forensic Science) and FSI (Forensic Science International). You cannot properly interpret a paper in forensic science without knowing whether it was written by a group that leans toward the prosecutor side or toward the defense side.
ReplyDeleteWe read a forensic science paper in one of our lab meetings. I was shocked by the misleading statistics used by the authors for part of the results. My advisor explained (as I remember it) that this was because the authors were a prosecutor-focused lab and they did not want to publish something that could be used by the defense in a court case.
idea 1. A detection ratio, flux ratio, or luminosity function measurement where you have to fold in publication bias for detections, parametrized in some simple way. Do this for high redshift galaxies and you'll make a lot of friends.
ReplyDeleteidea 2. A meta analysis of the literature on some parameter, like the Hubble constant, where your model of the scientist allows for a bias towards publishing results that agree (or disagree!) with the current state of the literature, and you model this effect on both the reported value and the error bar. The model should include time evolution since the bias depends on current state. Ideally you would be able to decompose the convergence toward a consensus value into the amount of convergence explained by statistical power (or at least reported error bars) and the part coming from bias.
In the 1980s, I was interested in the radial distribution of Arp's QSO/galaxy associations as possible gravitational lensing cases (even did an ApJ letter on it). That was before I talked to him and he told me he started looking for UB-blue objects from the galaxy outward and stopped when he found one, unless the first one was interestingly close; then he looked for more. So that r^1/4-like distribution was much less interesting than I thought...
ReplyDelete