postdoc applications

There is a non-wrong view of the academic enterprise that it is entirely about getting hired, evaluating people for hire, and hiring. That's all I did today (okay the latter two, not the first).

refereeing can be very valuable

Christina Eilers (MIT) and I discussed our referee report today, on our paper on re-calibrating abundance ratios as measured by APOGEE to remove log-g-dependent systematics. The referee report came quickly! And it was very useful: The referee found an assumption that we are making that we had not explicitly stated in the paper. And this is important: As my loyal reader knows, I believe that a data-analysis paper is correct only insofar as it is consistent with its explicitly stated (and hopefully tested and justified) assumptions. So if a paper is missing an assumption, it is wrong!

massive revision of a manuscript

Teresa Huang's paper with Soledad Villar and me got a very constructive referee report, which led to some discoveries, which led to more discoveries, which led to a massive revision and increase in scope. And all under deadline, as the journal gave us just 5 weeks to respond. It is a really improved paper, thanks to Huang's great work the referee's inspiration. Today we went through the changes. It's hard to take a paper through a truly major revision: Everything has to change, including the parts that didn't change! Because: Writing!

my attitude towards referee responses

Early in the day, Adrian Price-Whelan (Flatiron) called to discuss his latest response to referee. The referee report is (as many are) extremely constructive and valuable; the paper will be much better for every change we make in response. We discussed our philosophies about responding to referees. My approach is the following:

Every referee comment, confusion, or question is a real problem with the paper. The huge value of a referee report or submitting to a refereed journal is that you have called in a close read of your paper by a professional astronomer. That's amazingly valuable. Any place where that referee's close read led to a confusion or a comment or a question is a place where (by example!) a professional astronomer might have a comment or a question or a confusion! Even if you think the referee misunderstands, the misunderstanding is, at least at some level, because you didn't explain something well enough in the paper. So I recommend making a change in the paper text in response to every referee comment. Every comment is a real issue with the paper.

quasar lifetimes

Today Christina Eilers (MPIA) gave a great colloquium talk at MPIA about the intergalactic medium, and how it can be used to understand the lifetime of quasars: Basically the idea is that quasars ionize bubbles around themselves, and the timescales are such that the size of the bubble tells you the age of the quasar. It's a nice and simple argument. Within this context, she finds some very young quasars; too young to have grown to their immense sizes. What explanation? There are ways to get around the simple argument, but they are all a bit uncomfortable. Of course one idea I love (but it sure is speculative) is the idea that maybe these very young quasars are primordial black holes!

In other research today (actually, I think this is not research according to the Rules), I finished a review of a book (a history of science book, no less) for Princeton University Press. I learned that reviewing a book for a publisher is a big job!

technology-enhanced distributed peer review

At Stars and Exoplanets Meeting today, Wolfgang Kerzendorf spoke about a novel idea for peer review (for telescope-time proposals, but it could be applied to funding proposals or paper refereeing too): When you submit a proposal, you are sent K proposals to review. And the reviews thus obtained are combined in a sensible way to perform the peer review. This approach is scalable, and connects benefit (funding opportunity) to effort (reviewing). That's a good idea, and crystallizes some things I have been trying to articulate for years.

Kerzendorf's contribution, however, is to make a technology that makes this whole problem simpler: He wants to use natural-language processing (NLP) to help the organizations match proposals to reviewers. He showed snippets from a paper that shows that a simple NLP implementation, looking for similarity between proposal texts and proposers' scientific literature, does a reasonable job of matching reviewers to proposals that they feel comfortable to review. This is a great set of issues, and connects also to the discussions in our community about blind reviewing.

so, you are writing a review report?

I sometimes work long days, but I try not to fill my weekend with working. This weekend, however, I had promised myself that I would take the notes from the SDSS-V review I led in Denver and turn them into a draft report for the project. I can't believe it, but I succeeded!

Are you going to be on a review panel? I have advice (unsolicited advice, which I try not to give, but after this weekend, I can't help myself):

Make sure you do lots of writing while in session. If you just listen and talk for the period of the review, you leave the review with nothing written, and then you have to reconstruct it from memory and your notes. Instead, schedule executive-session writing time during the review and come away from the review with everyone's notes and comments compiled into one jointly editable document. I learned this from Mike Hauser (formerly STScI), who chaired the Spitzer Oversight Committee for many years.

Act fast. If you don't write your report immediately, you will never write it. So kill your procrastination and write the hell out of it immediately. And then your panel members will be so shocked at your turnaround time, they will be inspired to act fast themselves. They will take the draft you write fast and turn it into a final version.

Be helpful and constructive. Think carefully about and (more importantly) discuss with the team precisely what they want out of the report and what they can do. Make sure you are answering the questions they want answered, and that the answers you give can be implemented usefully and without huge burden. Report from reviews are about the future not the past.

I love the SDSS-V Project and Collaboration and I want both to succeed. I very much hope that what we have written will help them meet their goals.

SDSS-V review, day 1

Today was day one of a review of the SDSS-V Multi-object spectroscopy systems. This is not all of SDSS-V but it is a majority part. It includes the Milky Way Mapper and Black-Hole Mapper projects, two spectrographs (APOGEE and BOSS), two observatories (Apache Point and Las Campanas), and a robotic fiber-positioner system. Plus boatloads of software and operations challenges. I agreed to chair the review, so my job is to lead the writing of a report after we hear two days of detailed presentations on project sub-systems.

One of the reasons I love work like this is that I learn so much. And I love engineering. And indeed a lot of the interesting (to me) discussion today was about engineering requirements, documentation, and project design. These are not things we are traditionally taught as part of astronomy, but they are really important to all of the data we get and use. One of the things we discussed is that our telescopes have fixed focal planes and our spectrographs have fixed capacities, so it is important that the science requirements both flow down from important scientific objectives, and flow down to an achievable, schedulable operation, within budget.

There is too much to say in one blog post! But one thing that came up is fundraising: Why would an institution join the SDSS-V project when they know that we are paragons of open science and that, therefore, we will release all of our data and code publicly as we proceed? My answer is influence: The SDSS family of projects has been very good at adapting to the scientific interests of its members and collaborators, and especially weighting those adaptations in proportion to the amount that people are willing to do work. And the project has spare fibers and spare target-of-opportunity capacity! So you get a lot by buying into this project.

Related to this: This project is going to solve a set of problems in how we do massively multiplexed heterogeneous spectroscopic follow-up in a set of mixed time-domain and static target categories. These problems have not been solved previously!

words on a plane

I spent time today on an airplane, writing in the papers I am working on with Jessica Birky (UCSD) and Megan Bedell (Flatiron). And I read documents in preparation for the review of the SDSS-V Project that I am leading over the next two days in a Denver airport hotel.

responding to referee

I spent the weekend finally finishing a response to referee on my paper on using spectroscopy and photometry to get precise stellar distances. It was a very constructive, helpful, and positive report, so it really is embarrassing that it took me this long to finish. But it's done and we will resubmit on Monday. Somehow in my dotage, it gets hard to do the things I must do for myself. I am motivated to meet my obligations to others, but it is hard to meet those that help primarily me.

Dr Lukas Henrich

It was an honor and a privilege to serve on the PhD defense committee of Lukas Heinrich (NYU), who has had a huge impact on how particle physicists do data analysis. For one, he has designed and built a system that permits re-use of intermediate data results from the ATLAS experiment in new data analyses, measurements, and searches for new physics. For another, he has figured out how to preserve data analyses and workflows in a reproducible framework using containers. For yet another, he has been central in convincing the ATLAS experiment and CERN more generally to adopt standards for the registration and preservation of data analysis components. And if that's not all, he has structured this so that data analyses can be expressed as modular graphs and modified and re-executed.

I'm not worthy! But in addition to all this, Heinrich is a great example of the idea (that I like to say) that principled data analysis lies at the intersection of theory and hardware: His work on ruling out supersymmetric models using ATLAS data requires a mixture of theoretical and engineering skills and knowledge that he has nailed.

The day was a pleasure, and that isn't just the champagne talking. Congratulations Dr. Heinrich!

refereeing, selecting, and planning

I don't think I have done good job of writing the rules for this blog, because I don't get to count refereeing. Really, refereeing papers is a big job and it really is research, since it sometimes involves a lot of literature work or calculation. I worked on some refereeing projects today for a large part of the day. Not research? Hmmm.

Also not counting as research: I worked on the Gaia Sprint participant selection. This is a hard problem because everyone who applied would be a good participant! As part of this, I worked on demographic statistics of the applicant pool and the possibly selected participants. I hope to be sending out emails next week (apologies to those who are waiting for us to respond!).

Late in the day I had a nice conversation with Stephen Feeney (Flatiron) about his upcoming seminar at Toronto. How do different aspects of data analysis relate? And how do the different scientific targets of that data analysis relate? And how to tell the audience what they want to know about the science, the methods, and the speaker. I am a big believer that a talk you give should communicate things about yourself and not just the Universe. Papers are about the Universe, talks are about you. That's why we invited you!