matrix factorization paper

Tsalmantza and I worked on our matrix factorization method paper, plus some applications. We are trying to finish before I depart Heidelberg.


just like in grad school

Marshall and I (nearly) finished writing the lens-equation-solving part of our plug-in to the Tractor. This reminded me of graduate school, where I spent a lot of time solving the lens equation. One of my first efforts along this line became my first arXiv submission (astro-ph/9311077), which I am proud to say was submitted when the wonderful, wonderful arXiv was in its second year of infancy (and wasn't called arXiv). Today Marshall and I re-discovered that uniquely identifying—by root finding—all the multiple images of a lensed source is non-trivial, especially near cusp caustics! That also reminded me of graduate school, where I worked out and then abandoned dozens of semi-clever techniques.


watch out, Stripe 82!

Marshall and I continued working on our lens prior PDF code, and we also spent some time looking at candidate double-redshift (maybe lens?) objects found in SDSS spectra by Tsalmantza. But the big news for my day is that Foreman-Mackey's code for re-calibrating SDSS Stripe 82 data and finding variable stars scaled up very well; it looks like running it on a single (okay, 12-core) machine will take only a couple of weeks. I hope that is right! We also confirmed that all the imaging is spinning on NYU disks.


generating lenses from prior PDFs

Marshall and I wrote a piece of code that draws sets of four point-source positions from a prior over lensing configurations, without explicitly solving the lens equation. This code penalizes configurations by their non-compactness on the source plane, transformed to image-plane units. It works: When we MCMC the prior with Foreman-Mackey's awesome ensemble sampler we get realistic-looking lenses, even though we never actually did any root-finding. The goal is to plug this prior into the Tractor when it is working on images that contain lenses.


position and velocity modeling

As my reader knows, I have been working on a responsible and probabilistic use of the Jeans equations (which model velocity second moments) for inference. Today I re-built how I simultaneously model the density as a function of position and the velocity variance as a function of position to meet a few desiderata: (1) I want the model to be extremely flexible and data-driven, or non-parametric (meaning huge numbers of parameters). (2) I want the model to strictly enforce the Jeans equation given a parametric (or non-parametric) density or gravitational potential model. (3) I want the model to be parameterized so that I don't have to put harsh or ugly barriers in parameter space that will cause my optimizer to choke. (4) I want the parameterization to be relatively stable in the sense that I want small parameter changes to lead to small, smooth changes to the density and velocity moment models. I got all of this working with a very odd parameterization: I build a non-parametric model of the derivative with respect to position of the number density times the velocity second moment! This gets divided by a potential gradient to give the number density model, and it gets integrated and divided by the density model to get the velocity moment model. Crazy, but it works. I am sure there are much better solutions for my desiderata, but I found that I was much more willing to write code than go to the library!


RC3 galaxies

I have a huge collection of RC3 galaxy images on the web; these appear in many talks, some even occasionally credited to me. Keep those cards and letters rolling in! Today I started to re-make them with the (better and bigger) DR8 data set, which should make the collection larger, if not far better. Blanton and Ben Weaver (NYU) helped me with some of the crucial technical details.

Phil Marshall showed up at MPIA today for a week. He wants to work on lensing. I told him I will work on lensing provided that it involves image modeling. He was puzzled, but agreed.


phase-space modeling

I spent all weekend madly coding up my likelihood or Bayesian moment modeling and full phase-space modeling for one-dimensional dynamical systems. I can't see any easy, simple ways to make the code fast so it is super-slow. I have a method for inferring the gravitational potential using the position distribution and second moments of the velocity distribution and then I have one using the full position-and-velocity two-dimensional distribution function. Although there is a lot of computation involved, the problem is technically easy because all one-dimensional potentials are integrable. That makes it a nice toy system for learning about inference with different noise models and different selection functions.


Python, Gaia, and the disk

I spent the morning pair-coding a simple stellar phase-space distribution modeling code with Rix, and the afternoon solo-coding the likelihood functions for my one-dimensional dynamics code. In both cases, the attempt is to have the code benefit strongly from the things Python can do for us.

In the afternoon, I had a fruitful conversations with Coryn Bailer-Jones (MPIA) about the internals and outputs of the Gaia pipelines for stellar parameter estimation and source classification. For the former, we are discussing three-dimensional models of the Milky Way, and for the latter we are working out methods that make use of discrete models.

I also spoke with Glenn van de Ven (MPIA) about what's been done in one-dimensional dynamical modeling, and Bovy sent Rix and me some beautiful plots of the Milky Way structural parameters as a function of stellar chemical abundances.


calibration talk, Python classes

At MPIA Galaxy Coffee, Foreman-Mackey spoke about his very robust calibration model for SDSS Stripe 82. The model is a mixture of variable and non-variable stars, and for each, the observations are treated as a mixture of good data and bad data. These mixture models are effectively marginalizations over classifications; they don't produce or require hard classifications of stars into variable and non-variable, nor measurements into good and bad. This way, they take the information they can get from everything, but learn the most from the best data. The calibration model looks great. On his RR Lyrae finding, it occurred to us that maybe we need to model the stochastic as well as the periodic variations if we are going to be as sensitive as possible: On long time scales, RR Lyrae lightcurves evolve.

Late in the day, at the schwimmbad, I got my orbit-fitting code to use proper Python classes, and have the potential models inherit functions from a generic potential class. That made the code much nicer and easier to extend and test.


beta!, calibration, IGM

Astrometry.net went beta today. If you want to try out the new site, check out nova, our new image-sharing site. Note that we don't guarantee data integrity, so don't use us as a data backup system! But I think you will like what our Google Summer of Code interns, Kevin Chen and Carlos Lalimarmo, have done. Also, of course, huge effort from Lang. I am infinitely pleased.

Foreman-Mackey finished demonstrating to me that his Stripe 82 calibration process is working correctly and giving precise zeropoints for even the non-photometric runs (or at least as precise as possible) in the multi-epoch data set. He also has one candidate RR Lyrae star at enormous distance (it would be the most distant Milky Way RRL star ever, if it is real).

Nao Suzuki gave a nice talk in Hennawi's group meeting about things you can do with empirical (read PCA, which should make me groan, but he is doing good things with them) models of quasar spectra. In particular he has a very nice model-free way to estimate (under some strong but fairly reasonable assumptions) the attenuation of Lyman-alpha photons as a function of redshift. He does not confirm multiple reports of a feature at redshift around 3.2.


fitting stars and galaxies, IGM attenuation

Ross Fadely arrived in Heidelberg today for a short visit. We discussed how to demonstrate success in and set the scope for our paper on star–galaxy classification using hierarchical Bayesian methods with (known bad) template SEDs. We are going to start by visualizing a few of the correct and incorrect classifications and see why they went that way.

Joe Hennawi consulted with me about a problem he is working on with Suzuki and Prochaska, to estimate the mean attenuation by the Lyman-alpha-absorbing IGM in large samples of quasars. The idea, roughly, is to assume that there is reasonable or modelable variations among quasars intrinsically, and then model the cosmological IGM attenuation and the proximity effects directly with a probabilistic model. We figured out how to do it on averages (stacks) of spectra, and we have a rough idea of how to do it on non-coadded individual quasars, but the latter is harder. I learned something important: Though I am consistently against stacking your data, it has the great advantage of employing the central limit theorem to make your noise Gaussian!


calibration with a mixture model

Foreman-Mackey and I worked on his mixture model for photometric calibration of the heterogeneous runs that make up SDSS Stripe 82. The mixture model is a mixture of variable and non-variable stars, and for each of those, the observations are a mixture of good observations and bad observations; it is a mixture of mixtures, with a lot of degeneracies. But it seems to do a great job of getting the zeropoints, and he successfully recovers the known RR Lyrae stars in the Stripe. Can we find new ones? That is the question for this month.


one-dimensional velocity moment modeling

There is a standard technique for estimating gravitational potentials called Jeans modeling that uses the Jeans equations to relate the gravitational potential, the number density as a function of position, and the velocity dispersion. As my regular reader can imagine, I have many issues with it, some of them physical (what distributions are angle-mixed and integrable?) and some of them statistical (so you measure your data and then do arithmetic operations on those measurements?). But I spent my week of vacation (just ended) building a little sandbox for testing it out in one dimension and comparing it to better methods—methods that start from a likelihood function, or probability of getting the data given model parameters. I am sure the latter will win in every way, but I don't have my ducks in a row yet.

By the way, when I say likelihood function I don't mean I am going to do maximum likelihood, I mean I am going to transmit information from the data to the parameters of interest via a probability calculation! Just a reminder for those who hear maximum likelihood when all that is said is likelihood.


calibrating Euclid and Stripe 82

I worked with Holmes and Foreman-Mackey on calibration projects. With Holmes, we are finishing his thesis chapter on survey strategies optimized for photometric self-calibration. His results set the strategy for the Euclid proposed deep field strategy. With Foreman-Mackey, we are trying to convince ourselves that our super-robust multi-exposure calibration model is doing a good job at setting the photometric zero-points for Stripe 82. In the late afternoon, Nicholas Martin (MPIA) gave a nice talk about work on M31 satellites by Crystal Brasseur (MPIA, deceased).


galaxy structure, detecting stars

In the morning's Galaxy Coffee at MPIA, Rix talked about (among other things) Bovy's results on the structure of the Milky Way as a function of metallicity (from SEGUE data). Rix has the view that the results qualitatively confirm the view that vertical structure is set by radial mixing, where vertical action is sort-of conserved. Rix was followed by David Fisher (UMD) who spoke about bulges and pseudo-bulges. I challenged Fisher on whether his subjects of study were in fact bulgy, since he finds them by fitting two-dimensional images of near-face-on galaxies. He admitted that many of them may not be; his interest is in the action at the centers of disks, whatever the three-dimensional morphology.

In the afternoon, Lang and I worked on testing our optimal detection ideas in Stripe 82 data. It looks like we can beat what astronomers think of as optimal coadds (signal-to-noise-squared-weighted co-add images) by nearly ten percent in signal-to-noise-squared. That is like a ten-percent discount on observing time or total cost, so I am pretty pleased. We crush normal methods like exposure-time-weighted, unweighted, and (gasp) stacks where all contributing images have been smoothed to the same PSF.


velocity space

Elena D'Onghia (CfA) sent Rix a disk simulation for testing out inference methods on kinematic samples of stars. I spent a good part of the day making sure I could read and understand it.


all talk

Today was Lang's last day in Heidelberg, and Foreman-Mackey's first. I spent a long time talking with Bovy and Rix about Bovy's results on the structure of the Milky Way as a function of chemical abundances, based on SEGUE data. I looked over Lang's shoulder as he worked on optimal image stacking and optimal object detection (without stacking). I made a few-day plan with Foreman-Mackey for his first few days.