SDSS sky

Blanton and I worked for a while on sky. It is interesting that such a simple problem (subtract the smooth, additive foreground) is so hard! We decided that the way you subtract sky is a strong function of the scientific question you are asking, and you make different choices in different cases. So to make a sky determination, you have to have a very specific question in mind. This may seem strange, but it is because the sky has structure on all scales, and sometimes you want to photometer a star in a dense field, and sometimes you want to detect low surface-brightness emission behind a field of Galactic stars. There is no one sky that gives good signal-to-noise for both of these cases. For measuring the properties of huge, well-resolved galaxies, we were using a hack, but now we are getting close. Here's a comparison (before is on top, after is below):

SDSS sky

Blanton got very close to solving the SDSS "sky" problem today (the problem that the SDSS images of large galaxies or large patches of sky show non-astronomical features resulting from our inadequate tracking of sky levels). Unfortunately, I don't quite yet have side-by-side comparisons to show, but it looks very promising. I still think there are issues but we are much closer.

post-starburst environments, SDSS sky

I nearly finished the response-to-referee for the post-starburst galaxy environments paper. Nearly is a relative word.

Blanton and I had some back-and-forth (and Blanton did a lot of work) on fitting the sky in SDSS imaging for very large, very low-redshift galaxies. Our current page of nearby RC3 galaxies shows many data artifacts, many of which relate to sky-level tracking. The problem is not easy to solve. This is annoying, because the SDSS is drift-scanned, which greatly improves the consistency and simplicity of the sky!

cluster influences, environments, clustering of mass and galaxies

In group meeting, Berlind talked about reactions to NYU work at the SDSS meeting last week. This led to a discussion of Quintero's current project of looking for signs of the direction to the cluster in galaxies in the infall regions, the results of which were very well received at the meeting, apparently. We figured out that because Quintero is looking at broadband light, he won't be extremely sensitive to some of the effects claimed in the literature.

Berlind mentioned that Park (Seoul) is working on some environmental relations similar to those measured by Blanton and me. But he is using some kind of Nth-nearest neighbor statistic that has variable length scale (and a large median length-scale, and we have shown that only very small scales—Mpc or so—matter).

After group meeting, Berlind gave a short talk in the brown-bag about confirming some non-trivial relationships between cluster ages, masses, and clustering (as measured by the autocorrelation function). Age is positively correlated with clustering amplitude at masses below the nonlinear scale, but negatively above! Wechsler (Chicago) predicted this from simulations and Berlind has confirmed.

Sheldon, Masjedi, and I talked about ratios of cluster-mass and cluster-galaxy or galaxy-mass and galaxy-galaxy cross-correlations to measure the mass-to-light ratio as a function of scale. The observed power-law galaxy autocorrelation functions can only be explained in conventional CDM cosmology if these cross-correlation ratios have very specific shapes with radius. I hope this discussion was the birth of a new (and fast) project on this.

python cartography

This picture of the celestial sphere (below) may look familiar, but it is brand new: It is the output of my first-ever Python program. I even use (for the first time in my life) object-oriented features. I spent quite a bit of time on planes, trains, and hotel rooms working on this. The only connection this has to my research is that I plan to start working in Python from here on, if possible.

My astute readers will notice Orion there near the meridian (the NYC horizon is shown as a thick great circle). My very astute readers will notice that this is a perspective projection of the sphere, not orthogonal (or orthographic).

correlated point sets, talk

I gave this talk (PDF) at the IAP in Paris, to which we moved my LPNHE talk because of the student strike on in Paris. Morad will (a) notice its disturbing similarity with my ISCAP talk, and (b) be upset that I did not update the correlation function figure to its most recent version.

I also wrote a stupid code that iteratively modifies a point set in N dimensions until it has a given two-point correlation function, something Michael Joyce assigned me to do as homework. Now my homework is to make a point set that has the same correlation function as the LRGs (the Masjedi et al plot in the talk PDF file above).

[ps: I will be en vacance until Monday. I know that all two of my loyal readers will be disappointed.]

two-point statistics

I spent the last two days arguing about two-point statistics with Michael Joyce (Paris LPNHE); especially on the remarkable power-law correlation function over five orders of magnitude in separation shown in Masjedi et al, 2006 and our determination of the scale of homogeneity of the Universe (well, of LRGs in the Universe) in Hogg et al, 2005. The integrated "fractal dimension" plot in the latter paper also shows a power-law over a large range of scales, and yet it is a different power than what you would expect naively from the correlation function in the former paper. So there are multiple, unexpected, large-scale power-laws to explain in the clustering in the Universe. In other news, we both lamented the fact that we have no general ways to make (ie, set up in the computer or algorithmically) distributions of points with arbitrary correlation functions.

python cartography

On the plane to Paris I worked out how programming/scripting in Python works. I can fully understand why the open-source side of the astronomical community is migrating to Python! Indeed, I was reading up on Python because we intend to make all of the astrometry.net software open-source and cross-platform, for the obvious reason that we want everyone to run it! Already Mierle is writing most of his astrometry.net code in Python.

My first Python project is going to be some simple celestial cartography; I also worked out on the plane all the equations I need for putting polygons and great and small circles on the sphere and projecting them onto the plane. First rule of astrometry and celestial cartography: Don't use RA, Dec pairs to locate objects, use unit vectors on the sphere. No coordinate singularities or discontinuities, and spherical trig turns into simple vector operations.

clustocentric morphologies

Quintero and I (read: Quintero) produced this plot today:

(Click to enlarge.) The galaxies shown are randomly selected galaxies in bins of clustocentric distance (scaled by virial radius) and color; the images have been rotated such that the (large) cluster that each galaxy is near is to the left. As I see it, one of these galaxies has a badly measured color (SDSS problem), and one is strongly asymmetric in a way that is likely related to the cluster. Most of the galaxies show no strong sign of the nearby cluster. Note that galaxies at 0 to 0.3 on the horizontal axis are likely falling in to the cluster now.

bright galaxies in SDSS

Wu showed me that most of the bright galaxies in the Virgo Cluster are not detected properly in the Sloan Digital Sky Survey! Admittedly, the SDSS software is optimized for 16–19 mag galaxies, but seriously, this is a disgrace! We have to decide whether we are going to punt on our Virgo Cluster update or else fix the SDSS catalog at the bright end (a huge job).

Quintero made some nice figures, but nothing was fully ready in time for this blog post!

visualization

Again my work was vicarious: I helped Morad with the plotting of PRIMUS spectra, and Quintero with the display of galaxy images. Also conversations with Kazin and Willman.

PRIMUS redshifts, galaxy morphologies

Vicariously I worked on PRIMUS redshifts with Morad, who is adapting Blanton's kcorrect code to fit redshifts to the ultra-low resolution spectroscopy we are taking with IMACS on Magellan. Why ultra-low? Because we can fit far more spectra into the field and take far more redshifts per hour than anyone else. We hope.

I worked also with Quintero on getting set up to look at galaxy morphologies in cluster infall regions. I am excited, because—unlike with many projects—we have no idea what we are going to find. Usually we have some clue.

I am also doing astrometry.net stuff, but today it occupied my brain at nice +19.

infall regions, GALEX

With Quintero, I worked on the possibility of finding evidence for galaxy transformations/evolution on infall to large galaxy clusters. We are looking at the morphologies of galaxies in the coordinate system defined by the nearest cluster.

I also started to re-build astrometric standards information relevant to solving GALEX (ie, ultraviolet) data blind. Apparently this problem is non-trivial, in that UV-bright stars are rare.

Wechsler

[Yesterday:] Risa Wechsler (Chicago) gave a nice talk in which she showed that a simple insertion of galaxies into subhalos (dark-matter concentrations within virialized dark-matter halos) according to the subhalo circular velocity (expected velocity of a test particle on a circular orbit in the subhalo) produced a halo occupation (number of galaxies per halo of a given mass) that nicely explains the correlation function of galaxies and the abundance of galaxy groups of a fixed multiplicity. It also explained the variations in the correlation function with scale, galaxy luminosity, and even redshift.

This may all sound extremely technical, but (a) up till now the determination of halo occupation has been very ad-hoc and a posteriori, and (b) until we have simulations of enormous boxes in which the small-scale structure actually looks like the galaxies we actually see, low-resolution, large simulations plus halo occupation is the only way to reasonably model measurements that the community expects to use for the next five years of high-precision cosmology.

I also did the usual group-interacting and a small bit of referee-responding.

environments

I discussed and worked on measurements of environments—the environments of galaxies and the environments of groups of galaxies. Risa Wechsler (Chicago) is visiting and she has a prediction about the relative clustering of old and young groups. Berlind confirms it (clustering is a strong function of apparent group age, based on galaxy colors). Blanton doesn't (looking at the densities of group environments). We need to resolve this.

Interestingly, there may be evidence that the color distribution in each of Berlind's groups is not consistent with a Poisson sampling of (ie, independent draws from) the color distribution of galaxies. To do this test correctly, you have to use not the color distribution of all galaxies, but rather the color distribution of galaxies in the same environments as those in the groups. There may be, truly, red groups and blue groups. This relates to an old problem we assigned to Masjedi, but never finished.

Also I have an old paper about the environments of post-starburst galaxies that I am re-working. Are the environmental measures affected by the 55 arcsec spectrograph constraint of SDSS? No, it turns out. How did I determine this? By re-running my analysis using only the roughly 40 percent of the sky which the spectrograph covers more than once.

astrometric standards data, visualizations

I moved SDSS and USNO-B1.0 astrometric standards information onto the disks of astrometry.net today, in order to get our tweak stuff up and running for our alpha release.

I made two visualizations in my spare time, this slideshow zoom (PDF) out from M51 to the whole celestial sphere to demonstrate the enormity of the SDSS dataset. I also made this animated GIF (below) of just the celestial sphere itself. In both visualizations, the over-plotted stars are from the Bright Star Catalog.

small stuff

I worked on many small things today: responding to a referee, making visualizations of SDSS data for scientific and public-outreach lectures, installing data on the astrometry.net machine, upgrading data analysis software, and arguing about the meaning of Quintero's latest paper on galaxy environments.

MW satellites, CVS repository destroyed

Willman gave a great group-meeting presentation about the status of the ultra-faint companions to the Milky Way discovered by herself and others as overdensities in the SDSS stellar catalogs. There is so much more to be done, and our current estimate is that there will be dozens of new satellites discovered when SDSS-like (and better) surveys cover the whole sky. Awesome!

In other news, the Princeton CVS repository (with all the Princeton-based SDSS data analysis code, including all the spectroscopic reduction code, and including several huge code bases to which I am a large contributor and upon which I rely heavily) got destroyed today in a RAID failure. Did you hear me? All the Princeton SDSS data analysis code! All! Let's hear it for data integrity: A couple of times per day I rsync the Princeton CVS repository to NYU, and the NYU copy saved the day. The only stupid thing is that I didn't charge for this service! I think I need an agent.

BTW and IMHO, off-site backup is the only way to go, and I strongly suggest to all you large-data and large-code-base people out there that you arrange mirroring trades (as in I'll mirror your dataset if you mirror mine) with your collaborators at distant institutions. A few minutes of finding disk space and setting cron tabs can save you a lot of future hell.

celestial cartography

I figured out how to plot the bright stars on the celestial sphere and coded it up for use in my visualizations of SDSS imaging data. Can you find the big dipper on the image below? It shows the SDSS imaging data as of last autumn, with the visible stars over-plotted in white, with symbol size related to visual magnitude.

indexing quads of stars, Spitkovsky

[Late post for Friday.]

Anatoly Spitkovsky (SLAC) gave a really nice talk showing that it is possible to compute steady-state and simple evolving models for simple magnetically and rotationally powered astrophysical systems, like pulsars, quasars, magnetars, etc. It was noted that he is on the verge of predicting and/or explaining a huge variety of detailed phenomenology. If this works, he (among others) has opened up a new era in high-energy astrophysics.

On the fifth day of the team meeting, the astrometry.net team talked quad indexes all day: making indices that are dense but heterogeneous on the sky and in code space, the four-dimensional parameter space of all possible scale- and rotation-free quadrangles. At the end of the day, Mierle and Lang got close to having an acceptable error rate and false-positive rate. Roweis and I also discussed larger strategy issues for running, marketing, and funding the project.

going alpha

The astrometry.net team (Blanton, Finkbeiner, Hogg, Lang, Masjedi, Mierle, Roweis) got very close to going alpha on the WWW today. Blanton, Fink, and Roweis wrote the FITS-image-to-star-x-y-list code, Hogg and Fink worked through the tweak code, Mierle started on the WWW interface, Masjedi worked on improving the USNO-B1.0 input catalogs, and Lang and Mierle worked on the blind solving indices. It would be nice to go alpha this week, if we can. And Willman will be our first alpha user. We worked hard today.

FITS files, astrometry failure rates, Zolotov

We (read: Lang and Mierle) continued to work on failure rates in the blind astrometry of SDSS fields. The index with fewer re-used stars helped in some respects and hurt in others. I started re-reading the USNO-B1.0 catalog (and, in the process, realized that previously I have been reading the USNO-A2.0 catalog, duh!) to make a deeper catalog for them.

I am also converting the entire USNO-B1.0 catalog into FITS binary table files. These are smart, self-documenting binary files in use by the astronomical community. If anyone out there wants USNO-B1.0 converted to 256x12 FITS files (one for each pixel in an nside=16 healpix grid), just let me know!

I spoke with Adi Zolotov about possible projects; she is another probable new member of the group!