python cartography

On the plane to Paris I worked out how programming/scripting in Python works. I can fully understand why the open-source side of the astronomical community is migrating to Python! Indeed, I was reading up on Python because we intend to make all of the astrometry.net software open-source and cross-platform, for the obvious reason that we want everyone to run it! Already Mierle is writing most of his astrometry.net code in Python.

My first Python project is going to be some simple celestial cartography; I also worked out on the plane all the equations I need for putting polygons and great and small circles on the sphere and projecting them onto the plane. First rule of astrometry and celestial cartography: Don't use RA, Dec pairs to locate objects, use unit vectors on the sphere. No coordinate singularities or discontinuities, and spherical trig turns into simple vector operations.

clustocentric morphologies

Quintero and I (read: Quintero) produced this plot today:

(Click to enlarge.) The galaxies shown are randomly selected galaxies in bins of clustocentric distance (scaled by virial radius) and color; the images have been rotated such that the (large) cluster that each galaxy is near is to the left. As I see it, one of these galaxies has a badly measured color (SDSS problem), and one is strongly asymmetric in a way that is likely related to the cluster. Most of the galaxies show no strong sign of the nearby cluster. Note that galaxies at 0 to 0.3 on the horizontal axis are likely falling in to the cluster now.

bright galaxies in SDSS

Wu showed me that most of the bright galaxies in the Virgo Cluster are not detected properly in the Sloan Digital Sky Survey! Admittedly, the SDSS software is optimized for 16–19 mag galaxies, but seriously, this is a disgrace! We have to decide whether we are going to punt on our Virgo Cluster update or else fix the SDSS catalog at the bright end (a huge job).

Quintero made some nice figures, but nothing was fully ready in time for this blog post!

visualization

Again my work was vicarious: I helped Morad with the plotting of PRIMUS spectra, and Quintero with the display of galaxy images. Also conversations with Kazin and Willman.

PRIMUS redshifts, galaxy morphologies

Vicariously I worked on PRIMUS redshifts with Morad, who is adapting Blanton's kcorrect code to fit redshifts to the ultra-low resolution spectroscopy we are taking with IMACS on Magellan. Why ultra-low? Because we can fit far more spectra into the field and take far more redshifts per hour than anyone else. We hope.

I worked also with Quintero on getting set up to look at galaxy morphologies in cluster infall regions. I am excited, because—unlike with many projects—we have no idea what we are going to find. Usually we have some clue.

I am also doing astrometry.net stuff, but today it occupied my brain at nice +19.

infall regions, GALEX

With Quintero, I worked on the possibility of finding evidence for galaxy transformations/evolution on infall to large galaxy clusters. We are looking at the morphologies of galaxies in the coordinate system defined by the nearest cluster.

I also started to re-build astrometric standards information relevant to solving GALEX (ie, ultraviolet) data blind. Apparently this problem is non-trivial, in that UV-bright stars are rare.

Wechsler

[Yesterday:] Risa Wechsler (Chicago) gave a nice talk in which she showed that a simple insertion of galaxies into subhalos (dark-matter concentrations within virialized dark-matter halos) according to the subhalo circular velocity (expected velocity of a test particle on a circular orbit in the subhalo) produced a halo occupation (number of galaxies per halo of a given mass) that nicely explains the correlation function of galaxies and the abundance of galaxy groups of a fixed multiplicity. It also explained the variations in the correlation function with scale, galaxy luminosity, and even redshift.

This may all sound extremely technical, but (a) up till now the determination of halo occupation has been very ad-hoc and a posteriori, and (b) until we have simulations of enormous boxes in which the small-scale structure actually looks like the galaxies we actually see, low-resolution, large simulations plus halo occupation is the only way to reasonably model measurements that the community expects to use for the next five years of high-precision cosmology.

I also did the usual group-interacting and a small bit of referee-responding.

environments

I discussed and worked on measurements of environments—the environments of galaxies and the environments of groups of galaxies. Risa Wechsler (Chicago) is visiting and she has a prediction about the relative clustering of old and young groups. Berlind confirms it (clustering is a strong function of apparent group age, based on galaxy colors). Blanton doesn't (looking at the densities of group environments). We need to resolve this.

Interestingly, there may be evidence that the color distribution in each of Berlind's groups is not consistent with a Poisson sampling of (ie, independent draws from) the color distribution of galaxies. To do this test correctly, you have to use not the color distribution of all galaxies, but rather the color distribution of galaxies in the same environments as those in the groups. There may be, truly, red groups and blue groups. This relates to an old problem we assigned to Masjedi, but never finished.

Also I have an old paper about the environments of post-starburst galaxies that I am re-working. Are the environmental measures affected by the 55 arcsec spectrograph constraint of SDSS? No, it turns out. How did I determine this? By re-running my analysis using only the roughly 40 percent of the sky which the spectrograph covers more than once.

astrometric standards data, visualizations

I moved SDSS and USNO-B1.0 astrometric standards information onto the disks of astrometry.net today, in order to get our tweak stuff up and running for our alpha release.

I made two visualizations in my spare time, this slideshow zoom (PDF) out from M51 to the whole celestial sphere to demonstrate the enormity of the SDSS dataset. I also made this animated GIF (below) of just the celestial sphere itself. In both visualizations, the over-plotted stars are from the Bright Star Catalog.

small stuff

I worked on many small things today: responding to a referee, making visualizations of SDSS data for scientific and public-outreach lectures, installing data on the astrometry.net machine, upgrading data analysis software, and arguing about the meaning of Quintero's latest paper on galaxy environments.

MW satellites, CVS repository destroyed

Willman gave a great group-meeting presentation about the status of the ultra-faint companions to the Milky Way discovered by herself and others as overdensities in the SDSS stellar catalogs. There is so much more to be done, and our current estimate is that there will be dozens of new satellites discovered when SDSS-like (and better) surveys cover the whole sky. Awesome!

In other news, the Princeton CVS repository (with all the Princeton-based SDSS data analysis code, including all the spectroscopic reduction code, and including several huge code bases to which I am a large contributor and upon which I rely heavily) got destroyed today in a RAID failure. Did you hear me? All the Princeton SDSS data analysis code! All! Let's hear it for data integrity: A couple of times per day I rsync the Princeton CVS repository to NYU, and the NYU copy saved the day. The only stupid thing is that I didn't charge for this service! I think I need an agent.

BTW and IMHO, off-site backup is the only way to go, and I strongly suggest to all you large-data and large-code-base people out there that you arrange mirroring trades (as in I'll mirror your dataset if you mirror mine) with your collaborators at distant institutions. A few minutes of finding disk space and setting cron tabs can save you a lot of future hell.

celestial cartography

I figured out how to plot the bright stars on the celestial sphere and coded it up for use in my visualizations of SDSS imaging data. Can you find the big dipper on the image below? It shows the SDSS imaging data as of last autumn, with the visible stars over-plotted in white, with symbol size related to visual magnitude.

indexing quads of stars, Spitkovsky

[Late post for Friday.]

Anatoly Spitkovsky (SLAC) gave a really nice talk showing that it is possible to compute steady-state and simple evolving models for simple magnetically and rotationally powered astrophysical systems, like pulsars, quasars, magnetars, etc. It was noted that he is on the verge of predicting and/or explaining a huge variety of detailed phenomenology. If this works, he (among others) has opened up a new era in high-energy astrophysics.

On the fifth day of the team meeting, the astrometry.net team talked quad indexes all day: making indices that are dense but heterogeneous on the sky and in code space, the four-dimensional parameter space of all possible scale- and rotation-free quadrangles. At the end of the day, Mierle and Lang got close to having an acceptable error rate and false-positive rate. Roweis and I also discussed larger strategy issues for running, marketing, and funding the project.

going alpha

The astrometry.net team (Blanton, Finkbeiner, Hogg, Lang, Masjedi, Mierle, Roweis) got very close to going alpha on the WWW today. Blanton, Fink, and Roweis wrote the FITS-image-to-star-x-y-list code, Hogg and Fink worked through the tweak code, Mierle started on the WWW interface, Masjedi worked on improving the USNO-B1.0 input catalogs, and Lang and Mierle worked on the blind solving indices. It would be nice to go alpha this week, if we can. And Willman will be our first alpha user. We worked hard today.

FITS files, astrometry failure rates, Zolotov

We (read: Lang and Mierle) continued to work on failure rates in the blind astrometry of SDSS fields. The index with fewer re-used stars helped in some respects and hurt in others. I started re-reading the USNO-B1.0 catalog (and, in the process, realized that previously I have been reading the USNO-A2.0 catalog, duh!) to make a deeper catalog for them.

I am also converting the entire USNO-B1.0 catalog into FITS binary table files. These are smart, self-documenting binary files in use by the astronomical community. If anyone out there wants USNO-B1.0 converted to 256x12 FITS files (one for each pixel in an nside=16 healpix grid), just let me know!

I spoke with Adi Zolotov about possible projects; she is another probable new member of the group!