scripting web bots

I spent the day working on scripting a web bot to run a bunch of the "lost" SDSS images mentioned yesterday through our web service.

recovering lost SDSS data

One of the things we like to brag about with astrometry.net is that it can be used to recover data that have been lost or badly archived (that is, have been tagged with incorrect meta-data). We recovered SDSS run 2301, which is an engineering run of no particular import except that the astrometry for the run was never correctly determined (probably because the telescope was confused about where it was pointed at the start of the run). Today, some SDSS collaborators contacted me about SDSS run 6895, which goes through globular cluster M71. I spent the few research minutes I got today trying to massage the data into astrometry.net.

Note added next morning: When I got the first field of the SDSS data into JPEG form and got it into the system, it solved fast, and our system even said that the field contains M71.

Red stars are our object-detections and green stars are in the USNO-B. M71 is in the upper right corner.

results section of USNO-B paper

My loyal readers may have noticed that I have spent most of the last month writing rather than doing. It is annoying, but it ain't science if it ain't published! I finished a first draft of the results section of the USNO-B Catalog cleaning paper, making a complete first draft of the whole paper for the first time. Barron will visit in September for us to finish and submit it!

tarball

We are so close to having our Astrometry.net alpha code on the web as a tarball plus associated binary files. I spent a little time today testing the compilation and running of the tarball in preparation for making a tutorial.

test-driven halo measurements

I encouraged Zolotov to go with a test-driven development style in her work on the shape of the Milky Way and simulated galaxy stellar halos. When it is important to be not wrong, test-driven is very good. I have learned this the hard way!

statistics and language

At group meeting, Eyal Kazin (NYU) told us about measurements of the redshift-space, projected two-dimensional, and comoving-space correlation functions of LRGs in the current SDSS sample. He has a beautiful detection of the baryon acoustic feature even in the projected function! This means that it has great signal-to-noise in the overall data. After Kazin, I spoke about my statistical work on proper motions.

In the afternoon, much work was done (largely by email) on the language and writing of the USNO-B cleaning paper among Lang, Barron, and me. We made good progress on language and presentation issues.

submitted!

I finished the Masjedi, Hogg, and Blanton paper on the accretion rate for luminous red galaxies and submitted it to the Astrophysical Journal. It has a nice punchline: The accretion rate, even when integrating over a factor of 50 in merger mass ratios, is less than a few percent per Gyr, substantially lower than that predicted in CDM simulations, and much lower than many morphological measures of the merger or accretion rate. It will appear tomorrow as arXiv:0708.3240.

galaxy merging

With funding proposals and the beginning of term looming, I didn't get much time for research yesterday and today, but what I got was spent working through the literature on galaxy merging for the LRG accretion rate paper, which is so close to being done!

papers

I worked on the LRG accretion rate and USNO-B cleaning papers today.

dark matter halos

The only research I did on my week-long vacation off the grid was to draft a section for Peebles's and my synthesis of galaxy evolution in the cold-dark-matter paradigm. The section I drafted was on the predictions and observations of the dark-matter halos (or concentrations, virialized or otherwise) around galaxies and groups of galaxies. I think that weak and strong lensing, along with sensitive measurements of intragroup light, might put serious constraints on the cold dark matter model at small scales, because, depending on some issues of how baryons populate the halos, each of these provides some handle on the distribution of dark matter—radial distribution, large-scale anisotropy, and small-scale substructure—which is (apparently) a robust prediction of the model.

psf, M/L

I spent time yesterday (traveling for family reasons) and today working on incorporating non-trivial point-spread-function estimates into the proper-motion measurement code. I fear that the gaussian approximation to the point-spread function is affecting my results, at least slightly.

I also spent some time reading and commenting on a very nice draft paper by Sheldon on the mass-to-light ratios (as a function of scale) of virialized structures in the Universe, based on a comparison of cluster–galaxy cross-correlations and cluster–mass cross-correlations (based on statistical weak lensing).

My loyal readers will both be sad to learn that I go out of all internet contact for a week starting Monday, so there may not be much here.

merger rates

I spent the morning re-reading papers about merger rates, to improve the discussion in our merger-rate estimate paper. I was reminded of how stark the difference is between studies that estimate the merger rate morphologically (ie, by identifying probable merging galaxies by tidal features or asymmetries in their appearances) and studes that estimate it by close pairs. The morphological studies all get much higher rates, which says to me that either morphological asymmetries are raised by very small accretion events, or else that they last many dynamical times. The close-pair rates have the virtue that—if done correctly—they can provide a strict upper limit on the merger rate, so the fact that the close-pair studies come in lower means that the morphological studies are biased high.

constrained realizations, confirming the known

In among many non-research activities today, I managed to have a long and useful argument with Blanton and visiting student Abate (UCL) about constrained realizations, the local velocity field, and velocity correlation functions. I also spent some time measuring the proper motions of objects in the SDSS Southern Stripe with known proper motions.

brown dwarfs have higher proper motions than quasars!

Stop the presses! No, seriously, I showed today that I can measure the faint, very red sources in the SDSS Southern Stripe—sources so red that they must either be brown dwarfs or high-redshift quasars—with enough accuracy that I can separate them with some reliability on the basis of proper motion alone. The separation is not perfect, because there are some faint brown dwarfs with very small proper motions. However, the most widely separated brown dwarfs are also the most interesting, because they have the highest probabilities of being very low luminosity and therefore old and cold.

In other news, Lang, Mierle, and Roweis worked all night and got the first astrometry.net paper submitted.

debugging and tweaking

I spent most of my time yesterday (forgot to post!) and today continuing to add functionality to and debug the proper motion code. I decided that debugging requires that I have code to visually compare the stack of a set of epochs at zero proper motion and at some candidate non-zero proper motion, to understand, functionally, why I get some (clearly wrong) proper motions for some sources in real data, while not in the artificial data. I expect neighboring objects and noise issues (eg, underestimated noise) are affecting me. On the former (neighbors), the right thing to do is to simultaneously fit for all sources in the field!

The time not spent debugging was spent helping Lang, Mierle, Blanton, and Roweis finish the astrometry.net submission to Science. They are intent on submitting today. It is a lofty goal, and perhaps an achievable one.