long-term writing projects

I have various fantasies about book-length writing projects. I worked a bit on one of them today; no other research, because some toboggans and a snowy slope beckoned.

model complexity for HMF

I got up early and got my ya-yas out on model complexity in Tsalmantza and my paper on HMF. I have been thinking about finishing a note on model complexity for my/our Data Analysis Recipes series, and since (a) the issue is ringing around in my otherwise empty head, and (b) the issue comes up in the HMF paper, the HMF paper just got an overly-long polemic on why you shouldn't use AIC, BIC, or (naive) Bayesian evidence integrals. I am sure (I hope, perhaps?) that we will end up removing it before we submit!

station keeping

The only research I did today, in between getting relatives to various travel destinations, was to answer email from some of my hard-working students, two of whom are close to being able to criticize objectively the LSST filter choice.

more HMF

I spent most of my research time today and over the last few days working on the HMF paper. Iain Murray, in the comments to my last post, pointed out that there might be a connection to factor analysis, and I am working that out. Certainly they are related, but factor analysis and ICA and the like are for situations where the noise properties are a function only of row or of column of the (large, rectangular) data matrix. HMF permits the noise variance matrix to be arbitrarily heterogeneous.

HMF

More matrix factorization today. PCA is bad, HMF is good. Why is PCA bad? Because it does a good job at describing the variance of your data, which can have substantial contributions from—or be dominated by—noise. Why is HMF good? Because it models the noise-deconvolved underlying distribution.

heteroscedastic matrix factorization

I worked on Tsalmantza and my paper on matrix factorization, which makes a data-driven model for spectra.

exoplanet sampling

I spent my research time working on Hou's paper on exoplanet sampling with an ensemble sampler. He gets great performance!

tiny black holes

In a day of all talking all the time, I had a nice conversation with Mike Kesden about detecting tiny black holes that might zip through the Sun. There is a mass range where they are not ruled out as the dark-matter candidate but would produce observable signatures.

Magellanic streams

Today was catch-up day so not much work got done, but I did manage to learn a lot about the Magellanic Clouds and stream from one and a half seminars by Gurtina Besla (Harvard). Her view, which is pretty convincing, is that the clouds fell in late, and the stream is caused by a tidal interaction between the clouds, not between each cloud and the Milky Way.

Harvard

I spent the day at Harvard CfA, giving a seminar and discussing various things with the locals. Among other things I learned: Doug Finkbeiner and Mario Juric have a re-calibration for PanSTARRS that looks very promising. They also built fast technology to implement it (and other scientific investigations with the large data set). Kaisey Mandel (with Bob Kirshner) has a hierarchical model for supernovae light-curves that uses all bands simultaneously, including the important H band, and fits simultaneously for dust extinction and light-curve properties. He appears to be able to improve distance indication by tens of percent, which is a big deal in this area. Alyssa Goodman, Gus Muench, and others have been working on crazy new stuff for coordinated literature and sky searching, including unpacking the NASA ADS data and doing things like making higher-end text search but also running the in-line images into Astrometry.net and building a sky index. Dinner (and drinks after) were great too.

The only bad thing—or I should say sad thing—is that one of the great pleasures for me of visiting the CfA has always been hanging out with John Huchra. What a loss for all of us.

Hui and Murray

After a discussion of the (wrong) Gurzadyan & Penrose result (in the HET seminar time slot), I introduced Iain Murray to Lam Hui, the author of one of the papers inspiring our work on marginalizing out the density field. We discussed the regimes in which making a single-point estimate of the correlation function might be worsening our results (on cosmological parameters) down-stream (in the inference chain). The rest of the day was definitely not research.

inverting large matrices

Today was an all-talking day, with Murray, Bovy, and I discussing all our projects that involve—or might involve—Gaussian processes. Murray started to describe some methods for making approximations to large variance tensors that make them computationally possible to invert. These might be of great value; when the matrix gets larger than a few thousand by a few thousand, it becomes hard to invert in general. Inversion is n-cubed.

marginalize out the density field!

In studies of the baryon acoustic feature, we like to get all Bayesian about the cosmological parameters, but then we apply all that machinery to the measured two-point functions, which are created with non-Bayesian single-point estimators! I spent a chunk of today discussing that problem with Iain Murray, who is visiting NYU for the week. Murray may have a straightforward solution to this problem, in which we try to write down the probability of the data given a density field times the probability of the density field given a two-point function. Then we can marginalize out the density field and we are left with a probability of the data given the two-point function. That would be exactly the full likelihood function we all need! It might be necessary to either approximate or else use a lot of compute cycles, but even approximate likelihood functions ought to beat single-point estimators.

I pointed out to Murray that if we are spending tens of millions (or billions, maybe?) of dollars on hardware to measure the baryon acoustic feature, it might be worth spending a few bucks to improve the inference we use to exploit it.

the IMF

On the plane home from NIPS, Lang and I pair-coded some Python tools for working with and fitting the IMF, in the hopes of weighing in on various high-mass-star issues with Dalcanton's PHAT project. Sitting across the aisle from us was Iain Murray (Edinburgh), who explained stick-breaking and Chinese-restaurant processes, which will be relevant, I very much hope!

NIPS workshops, day 2

I learned more about sparse matrix representations today. I mainly learned that there is lots to learn, but I am pretty confident that we astronomers are not using all the technology we should be. I also got fired up about making some astronomy data sets for machine learning, in analogy to the MNIST set of handwritten digits. Many conference participants asked me for data, and I would like to be able to distract some of them towards astronomy problems.

In the afternoon, Lang and I pair-coded SDSS meta-data handling. During this meeting, in the break periods (when normals ski), we implemented the SDSS astrometry meta-data (which is not standards-compliant for historical reasons) and the SDSS photometric calibration meta-data. Soon we will be ready to do some damage on SDSS Stripe 82.

NIPS workshops, day 1

In the morning, I learned about sparse codes. I got pretty stoked. We are finding that k-means (well not exactly k-means, which you should never use, but our generalization of it to be probabilistically correct) is too sparse to get good performance (on, say, explaining galaxy spectra), and we are finding that PCA (well not exactly PCA, which you should never use, but our generalization of it to be probabilistically correct) is too dense. Sparse codes looks like it might interpolate between these cases. That is, we will be able to capture more structure than a prototype approach, but not be as restricted as a linear manifold approach. Excited!

In the afternoon, Lang and I pair-coded and tested the (annoying) SDSS asTrans astrometric transformation meta-data format in Python. Soon all your SDSS interface will belong to us.

Sam Roweis Symposium

Today was the Sam Roweis Symposium at NIPS. I spoke, along with four other of Roweis's close collaborators. I learned a lot, especially how LLE and related methods work. It was a great session. One thing it all reminded me of is that the NIPS crowd is far more statistically and inferentially sophisticated than even the most sophisticated astronomers. It really is a different world.

In the morning before the Roweis symposium, two talks of note were by Martin Banks (Berkeley) and Josh Tenenbaum (MIT). Banks talked about the perceptual basis for photographic rules and concepts. The most impressive part of it, from my point of view, was that he explained the tilt-shift effect: If you limit the depth of field in an image, the objects being photographed appear tiny. The effect is actually quantitatively similar to binocular parallax, in the sense that the governing equation is identical. In binary parallax you measure distances relative to the separation of your eyes; in depth-of-field you measure distances relative to the size of your pupil entrance!

Tenenbaum talked about very general models, in which even the rules of the model are up for inference. He has beautiful demos in which he can get computers to closely mimic human behavior (on very artificial tasks). But his main point is that the highly structured models of the mind, including language, may be learned deeply; that is, it might not just be fitting parameters of a fixed grammar. He gave good evidence that it is possible that everything is learned, and noted that if the program is to be pursued, it needs to become possible to assign probabilities (or likelihoods) to computer programs. Some work already exists in this area.

Phil Gregory

Lang and I stopped in to see Phil Gregory at UBC, who has been writing good stuff about Bayesian methods for exoplanet discovery. In the conversation I sharpened up my objections to Bayesian-evidence-based model selection as it is done in practice. It could be done well in principle but that would require properly informed priors. If the priors are uninformative, small changes in the outskirts of the prior-allowed regions can have enormous effects on the evidence integrals.

Dr Ronin Wu

Wu successfully defended her PhD today. Well, actually, she defended half of it yesterday, but for scheduling reasons we had to finish today. She presented the molecular hydrogen mass function as her principal result, and all the Spitzer data on which it is based. She has the largest uniform sample of Spitzer spectroscopy in existence. She is off to start a postdoc working with Herschel data this month.

quasar redshift determination

Tsalmantza, Hennawi, and I (well, really Tsalmantza) got running a system to simultaneously figure out quasar redshifts and build up model spectra. That is, the system infers a model for the spectra, uses that model to determine the redshifts, and then updates the model. Iterate to convergence. In some sense it is supposed to be a model for how astronomers figure out redshifts! At lunch-time Kyle Cranmer (NYU) talked about marginalizing likelihoods at the LHC (and the improvement that gives in measurements) and before that Wu started her PhD defense process, to be completed tomorrow.

brute-force modeling, exoplanet evolution

Two great seminars today. The first was by Ross Fadely (Haverford), who had done full brute-force modeling of strong gravitational lensing in a CDM-substructure context. He lays down CDM-compatible substructure in an enormous outer loop of realizations, and then does lens model selection within that loop. Brute force modeling that warms the heart. He finds, unfortunately, that the lenses don't obviously contradict CDM.

The second was by Josh Winn (MIT), who talked about very clever measurements of star-spin vs orbital angular momentum mis-alignment in exoplanet systems. The data show some beautiful regularities that are not implausibly explained by a combination of few-body effects driving inward migration (of hot Jupiters) followed by tidal damping of inclinations and eccentricities on pretty short time-scales. He argued, effectively (though not explicitly) that exoplanets measure the tidal dissipation timescales (or tidal quality factor) of convective stars much better than models can predict it. In the question period, the subject of free-floating planets came up. Mental note to self: Discover these!

star-galaxy separation, substructure

Ross Fadely and Beth Willman (Haverford) came in for two days, making my visitor list pretty long! We discussed Fadely's early-days results on star–galaxy separation. He has some strange objects that are obviously stars but are better fit by galaxies. We tasked him with giving more detail on a few of these cases to see if there is something simple wrong (or too general) with the galaxy models we are using, or too restrictive about the star models we are using.

In the afternoon, Willman gave the physics colloquium about the ultra-faint galaxies. What a rich and successful field this has been! And (in my humble view) it all started here at NYU with a pair (here and here) of papers. Now these galaxies are found to be incredibly numerous, observed to contain dark matter (Willman focused on the velocity measurements of Simon & Geha), possible sites for observable dark-matter annihilation (though Willman didn't discuss that at all), and promise to increase in observed number dramatically in the next decade.

quasar redshifts and the like

My MPIA collaborators Hennawi and Tsalmantza arrived in NYC for a week today. We are collaborating on a now exponentially growing number of projects involving data-driven models of galaxy spectra. We are trying to recover the known double-redshift objects in the SDSS, which tend to be either gravitational lenses or binary galaxies or quasars. We are trying to find new double-redshift objects of both kinds. We are trying to make more robust methods for getting precise redshifts of broad-line objects (which don't have any narrow redshift indicators). We are trying to model the quasar continuum blueward of Lyman-alpha for IGM measurements. We made a tiny bit of progress today on each of these.

clumpy streams from a clumpy halo, CVs

I went up to Columbia today to see Kathryn Johnston and David Schiminovich. I took the local train, so that I would have plenty of time to read and comment on the latest draft of Joo Yoon's paper (with Johnston and me) on tidal streams. In the paper we use scaling relations—dimensional analysis and order-of-magnitude arguments—to describe the dynamical effects of halo substructure on cold stellar streams; then we compare with numerical simulations and observations. The result (against my wishes) is that the rattiness of the Pal-5 and GD-1 streams is consistent, at least roughly, with what is expected in a CDM halo. Argh! But Yoon has done a great job.

At Columbia's Pizza Lunch, Christian Knigge (Southampton) gave a nice explanation of CVs, which are accreting WDs in close binaries with red dwarfs. Because they are Roche-lobe-filling, they have orbital evolution (mass-transfer) times equal to the thermal (Kelvin-Helmholz) times in the dwarf stars. So the non-main-sequence-ness of the dwarf star gives a semi-direct measure of the instantaneous mass-transfer rate. Nice! He also argued that at some periods, there should be many non-CV binaries; Schiminovich and I might be able to detect those in our time-domain GALEX stuff.

PRIMUS

After a few days of Thanksgiving vacation, it is back to work, with a blackboard talk by Blanton about PRIMUS, which has been a great success—one of the largest redshift surveys ever undertaken, and undertaken with a small team and a ridiculously small number of nights. Blanton noted the high efficiency of the technology and methods, but also noted some of the mistakes we made early on. I hope we transmit some knowledge to future generations. Focal-plane (as opposed to objective) prism spectroscopy could change the world.

pair code de-blend

Marshall and I spent the afternoon (among loud children) pair-coding a system to fit his putative blended quasar pair; we developed some ideas about how to deal with the ideas that (1) the quasar positions could or should be generated by a lensing pattern, and that (2) the quasars might or might not vary according to a Gaussian process, and that (3) we might be able to measure the time delay and solve for lens parameters simultaneously with the de-blending operation. We didn't get very far, but these are all good ideas. And LSST-relevant.

using variability to separate overlapping sources

Phil Marshall is in town for two days. We batted around the idea of fitting objects in the SDSS Stripe 82 data that have quasar colors but are marginally resolved. These objects could be gravitational lenses, and we could use the variability as well as our knowledge of the PSF to get them deblended and modeled. We debated whether we should deblend first and interpret second, or do the deblending with the interpretation (quasar-variability model or gravitational-lens model) built in from the start.

baryon acoustic feature

In group meeting, Eyal Kazin (NYU) led a discussion of some recent baryon acoustic feature measurements, including his own. No-one really understands in detail why the significance of the BAF seemed to go down when the amount of SDSS data used to find it went up (Kazin et al). But Kazin was concentrating on line-of-sight issues: There are some apparently conflicting results in the literature about the line-of-sight BAF signal; people who use different estimators get different significances, and I smell an illegitimate search in estimator space for best signal (though not with Kazin's work; he is doing very straightforward stuff). I have an intuition that everything would be better with a probabilistic methodology that marginalizes over the density field, but I haven't got any implementation of that, so I have no ground to stand on. I can't say much else happened research-wise today.

calibrating Euclid

Rory Holmes, Rix, and I spent a while chatting about the scope of our uber-calibration or self-calibration or calibration-marginalization analyses for potential Euclid or WFIRST survey strategies. The default strategies proposed for these kinds of missions are usually not ideal for photometric calibration, which the investigators imagine will be determined with calibration observations. However, the science data usually contain far more bits of information about calibration than the calibration data, and even more if the science data are taken with calibration information preservation or generation in mind. Fundamentally, it is about redundancy—meaning diversity—in modes, positions, orientations, time coverage, cadence, and so on.

Spitzer oversight

I spent yesterday and today on Spitzer Space Telescope oversight activities, which are not research according to The Rules (or at least I think they aren't). My only scientific activities were reading Wu's thesis, talking with Tsalmantza about binary quasars and the like, and discussing the stellar IMF with Lang.

NP hard

After a weekend and early morning finishing the NSF proposal, I had my weekly spa on sampling and exoplanets with Goodman and Hou. For the famous Gliese 581 system, there is a fence of multiple likelihood maxima in orbital frequency (period) space, with one (that we know about) much taller than the others. The multiple optima lie on lines in frequency space (apparently), and what we want to make sure of is that the most likely is also the most probable. That is, we want to integrate the likelihood under the prior at each distinct optimum. That is, we need to compute the evidence integral for each of the optima. We would also like to find all optima, and, from among them, the best-of-all optima in both senses, but this is provably hard, so I (now, in my positivist–pragmatist phase) think that the question do we have the global optimum? is outside science. I don't think Goodman agreed with me on that, in part because if you know a lot about the likelihood function (and we do), you might be able to limit the numbers substantially. I think he is right, actually—the exponentially large number we care about might not be so large we can't tackle it—but it sure isn't easy.

talk, lunch, proposal

To finish his very enjoyable visit, van Haasteren (Leiden) gave the astro group meeting seminar in the morning, with lots of banter and feedback from the crowd. He focused on stochastic signals, but got the most crowd reaction about single-source signals. On the stochastic side, Dubovsky (NYU) pointed out that there are even models in which the stochastic background might have a characteristic frequency.

At lunch and afterwords, van Haasteren and I discussed possible projects. The one we liked most is to try to improve the timing itself by going hierarchical on the pulse shapes. Commenter Steve T on my post of a few days ago was pessimistic, but in fact even a bad model of the variability among pulse shapes and pulse-to-pulse correlations in shape could have a big impact on the precision of pulsar timing. Big theme of my research: You do much better if you model the noise, but then marginalize out those noise-related parameters.

Now off to a weekend of proposal re-writing.

ensemble sampling and pulsar timing

van Haasteren and I tried to insert Hou's ensemble sampler into van Haasteren's gravitational wave detection code. We weren't completely successful, but I think we can de-bug it tomorrow.

pulsar timing Gaussian process

van Haasteren described in more detail today how they can recognize that covariances in pulsar timing residuals among multiple pulsars can be the result of a stochastic background of gravitational radiation. The method they use is a Gaussian process, which is a technology much loved by Bovy, myself, and our collaborators (and at least one of my regular readers). The nice thing is that van Haasteren's project is precisely a Gaussian process, and he knows that for good physical reasons. Furthermore, the covariance matrix he constructs is highly constrained by the physics of the problem. At lunch, he described to a subset of Camp Hogg some of the issues he faces in the real world; two of the most interesting are linear algebra—he needs to invert and take the determinant of some very large, very non-sparse matrices—and visualization—if you detect a signal at low signal-to-noise ratio, how do you convince the skeptics? Camp Hogg didn't have much useful to give him on either issue, though we can help, I think, with sampling issues.

pulsar pulse profiles

Rutger van Haasteren (Leiden) is visiting this week to talk about Bayesian inference, gravitational radiation, and pulsar timing. We spent some time talking about some of the details of pulsar timing, including the fact that every pulse looks different, every pulse is measured at low signal-to-noise, but collections of pulses average to a stable and predictable mean. We realized that there is probably a great hierarchical Bayesian description of this process.

more proposal

Still more proposal writing, in all available time on the weekend, and in any time I could find today. This was relieved by a great conversation with Goodman and Hou about stellar oscillations in radial velocity data, and a nice talk by Paul Duffell (NYU) on numerical approaches to general relativity.

galaxies like our own

My break from proposal-writing was a nice talk by Risa Wechsler (Stanford) about finding galaxy systems like that of the Milky Way plus Magellanic Clouds, both in the low-redshift SDSS sample and in huge n-body (dark-matter only) simulations. In both cases, having two massive satellites is rare, but she goes way beyond this, looking at what you can infer about the Milky Way from the presence of the Magellanic Clouds alone. Pretty neat stuff, though in the end she has a depressingly strong endorsement of the standard Lambda-CDM model.

faint variables

As part of proposal-writing, I simulated fake data on very faint variable stars in SDSS Stripe 82 and then showed, using Foreman-Mackey's implementation of Hou's super-fast MCMC, that we can detect them and measure their properties, even when at no individual epoch is there enough signal to constitute a clear individual-epoch detection.

proposal non-writing

I procrastinated and accomplished less than necessary on my proposal.

proposal writing

My only respite from proposal writing was a short conversation with Price-Whelan, Foreman-Mackey, and Paul Rozdeba (NYU) about quasar variability in repeat SDSS spectra. We came up with some first steps.

black-hole orbits, code

Mike Kesden (NYU) gave the brown-bag talk today about calculation of extreme-mass-ratio black-hole–black-hole inspiral, including gravitational radiation. This is a hard and old problem; he has found what look like inconsistencies in the existing approximate calculations. The work brings up many issues of principle, which makes it great fun for discussion. In the morning, Goodman, Hou, and I discussed—among other things—how to package Hou's code for ease of distribution, use, and re-use.

simulating telescope images

John Peterson (Purdue) is one of the principals in the LSST image simulation project. This project is insane, in that it simulates the images by laying down every single photon, from source through atmosphere, optics, and CCD. I spent a large part of today talking to him about all this, including the issue that it is hard (obviously) to see diffraction effects when you treat the light as being in the form of photons; there are some beautiful approximate methods for bridging the wave–particle duality. This is a nice problem to think about: How do you properly do simulations of a telescope, including all diffraction and refraction effects, without collapsing the wave function at the wrong time?

dark matter

I spoke about dark matter at Purdue University today. I also had some fun chatting with Maxim Lyutikov's group there over lunch.

beyond the virial radius, anthropics

There was productive conversation by email today about RR Lyrae stars in deep, multi-epoch visible data among Willman, Zolotov, and myself. Their models (by Governato et al) suggest that there should be a few RR Lyrae detectable in the 100 to 400 kpc range, even in SDSS Stripe 82, if only we can do robust detection below the individual-epoch detection limit. That kind of thing is my bread and butter. Do I feel a collaborative NSF proposal coming on?

In the afternoon, the high-energy physics seminar was by Adam Brown (Princeton) on bubble nucleation in the eternal-inflation (string-theory-inspired) metaverse. It seems that in many natural situations, the most likely bubble to nucleate in our neighborhood (future light cone) could be to a disastrously different vacuum, perhaps even one in which there would be no volume at all (it seems you would just get mashed against the expanding bubble wall). This has implications for our fate, though the standard anthropics (about which we are pretty skeptical at Camp Hogg) protect this theory from making any falsifiable predictions about our past. I used to say that astronomy was only about the past light cone, so perhaps I should be ignoring these subjects?

candidate Hou

Fengji Hou passed his oral candidacy exam today with a talk about very fast ensemble sampling and exoplanet inference and discovery. Congratulations Fengji!

fitting curves

Hou, Goodman, and I discussed spectral fitting in the context of radial-velocity measurement, relevant to our exoplanet work. We feel like we might be able to go the next level down, as it were. In the afternoon, Dmitry Malyshev (NYU) and I discussed how Malyshev is fitting the number-flux relation for gamma-ray sources in the Fermi data below the point-source detection limit. He is finding some nice surprises in there.

The Milky Way disk

At group meeting, Zolotov talked about her simulations of Milky-Way-like galaxies, and why feedback is required (in the simulations) to limit star-formation. In the afternoon, Bovy and I discussed different ways to formulate or code up the Oort problem, or the contemporary version of it, in which we try to understand disk vertical structure and (we hope) departures from axisymmetry. Also related, Foreman-Mackey and I got in touch with Larry Widrow (Queens), who suggested we all get together to start some projects of mutual interest on inferring gravitational potentials using observations of disk stars. All talk, because that is all there is time for during job season!

human classification

Schiminovich came down and we classified by hand the eclipses Lang and I put together yesterday. Most of them are not eclipses, but there is a good sample that are. We have enough for a publication, for sure, but we have to make some decisions about scope and follow-up and testing and discussion. At the end of the day, Finkbeiner (Harvard) came and gave our colloquium, on the WMAP and Fermi haze results. He now thinks that these are not likely to be created by dark-matter effects, on morphological grounds. Just as I was leaving the building, Foreman-Mackey told me that he has implemented Hou's affine-invariant ensemble sampling method in pure Python, which has some chance of making us infinitely powerful. Indeed, he says it has sped up some of his code by enormous factors (as it does for Hou too). For me, one of the things I like most about all this is that the ensemble sampler has almost no tunable parameters; that is, very few choices. I am against choice.

FUV and NUV

Lang came into town and we spent part of the day getting the far-ultraviolet data to play nice with the near-ultraviolet data. This (above) is one of the transits we discovered. It is a re-discovery, actually, and the star that is being eclipsed is a dwarf nova, which is violently variable in the ultraviolet. NUV data in grey, FUV data in magenta, broken time axis etc etc.

star-galaxy separation

Today Willman, Ross Fadely (Haverford), and I discussed star–galaxy separation. This is usually done morphologically, but at faint levels, there is usually far more information in the spectral energy distribution than in the deviation from pure-psf shape for making this determination. We batted around a few different ways to set up this problem and make a stab at it. Doing this right will be necessary (we think) if LSST is going to do reliable stellar science to its multi-epoch magnitude limit.

distinction

I started my short stint as a distinguished visitor at Haverford today, with an appearance in Beth Willman's class on astronomical ideas for non-science majors, with a seminar on the dark matter, and with an appearance in an observational astronomy laboratory course. In the latter, we took data the students had taken at KPNO last week and assembled it into color JPEGs for visualization. Of course we started at Astrometry.net.

matplotlib

I love the wonderful people who built and maintain the Python matplotlib plotting package. They are heroes of science. I spent the weekend fixing up our GALEX time-stream plots, and they are now far more useful and informative, with transparency and overlays and non-linear axes, all showing multiple kinds of information.

fast sampling and the end of time

At group meeting, Hou described our very fast MCMC algorithm; in the tests we have done so far on exoplanets (radial-velocity fitting) it beats Metropolis-Hastings MCMC by a factor of about 100 in speed. He is using affine-invariant sampling that uses an ensemble of parallel chains to build the proposal distribution. It is slick, and not complicated.

In the afternoon, Ben Freivogel (Berkeley) gave an extremely amusing talk about eternal inflation, string theory, and calculation of probabilities in the multiverse. He concludes that the only consistent way to make predictions in the theory is to put a limit on the time, and—assigning reality to his physical model—therefore finds that time has high probability of coming to an end in the next Hubble time. The argument is way out in left field, relying heavily on arguments of realism, which I reject. But I appreciate the candor: He takes the position that if you need to put a limit on time in order to consistently calculate, then time is predicted by the theory to end. I don't necessarily disagree with the latter part, but there are other reactions one can have to the former—the problem of calculating. Like: maybe eternal inflation just doesn't make predictions at all.

high purity

Schiminovich made a bunch of changes to our data extraction from the GALEX time-domain data, and now we get a very high purity sample of transits in the white dwarf population. This is very good; we are ready to clean up, run on everything, and write a paper! Clean up includes improving our likelihood ratio calculations, and including the FUV data along with the NUV data.

In the late afternoon, Bolton (Utah) gave a great talk about his gravitational lens sample (SLACS), and all the great astrophysics he and his collaborators have done with it. He is doing more now with SDSS-III BOSS.

priors on quasar spectra

Hennawi, Tsalmantza, and I had a long conversation about why our likelihood optimization does better at measuring quasar redshifts than our posterior-PDF optimization. In the latter, we use a highly informative prior PDF: That any new quasar spectrum must look exactly like some quasar we have seen before. This is the hella-informative data-driven prior. It turns out it is too strict for our problem: We end up over-weighting quasar spectra that fit the continuum well, at the expense of the narrow features that best return the right redshift. This raises a great philosophical point, one I used to discuss with Roweis extensively: You don't necessarily want to model all of the features of your data well. You want to model well the parts of your data that matter most to your questions of interest. So if we want to use ultra-informative priors, we ought to also up-weight the informative features of the data, and remove the uninformative. This is done, traditionally, by filtering—which is terribly heuristic and hard to justify technically—but which has been done more quantitatively in some domains, once notably by Panter and collaborators in MOPED.

GALEX meta-data

Schiminovich and I spent a good part of the day trying to understand brightness variations in GALEX sources, taken not from the official catalogs (which are great) but rather from our own analyses of some intermediate data products ("movies" of the photon arrival data). We are looking for time variations that are real, but we are being fooled by some meta-data problems, in which exposure times are not what we think they are. It was a frustrating day, but we did find what appears to be the problem. It is not clear that we can fix it adequately, so we might have to take the Astrometry.net-endorsed route of reconstructing the meta-data from the data through consistency or internal calibration.

spectral variability

Inspired by work this summer with Kasper Schmidt (MPIA), I had Price-Whelan and Foreman-Mackey compare spectra from BOSS and SDSS to look for variability. They find enormous variations in the quasars but also in the F-type stars used for calibration! So there is something wrong with the BOSS calibration; we will investigate next week. In the morning, Eyal Kazin (NYU) gave the group meeting, on the baryon acoustic feature.

exoplanets

I spent the day at Yale with Hou, talking with Debra Fischer and Christian Schwab about various possible exoplanet projects we might do together. In the short term we came up with a few easy projects for Hou's fast sampling methods, including looking at velocity offsets between different instruments and comparing exoplanet models for stellar velocity residuals with stellar oscillation models. For the longer term, we discussed how the velocities are measured, which is extremely non-trivial, and also the Kepler data, many of which are already public. I have a very good feeling that we have started a productive collaboration.

just chatting

Not much research gets done during job season! What a hit to the world's research it is. In my research time today, all I did was talk, with Foreman-Mackey about variance tensors, with Zolotov about cusp catastrophes, and with Jiang about merger rates in the literature.

IAS

I had a great day at the Institute for Advanced Study, my old stomping grounds. I gave a black-board talk (no computer), which shocked a few people. After my talk, I learned about ultra-high magnification microlensing events from Subo Dong (IAS) who convinced me that you can discover planets in a wide range of configurations with these systems. I also discussed with an editor the possibility of doing a Sloan Digital Atlas of Galaxies, something I have been dreaming about for years.

inference talk

I thought about my IAS talk tomorrow; I am giving a talk I have never given before, about modeling the data, and the power that gives you. After our Brown Bag seminar (by Gabadadze, on the cosmological constant problem), I chatted with my various students about the possibility that we could re-reduce the WMAP data. With what I learned at Leiden, I am certain that it is possible to make a higher signal-to-noise map than the official map, and get better parameters. But that is a huge job I am not willing to take on. I also tried to talk Kazin (NYU) and Blanton into some homogeneity tests, with little effect. A scattered day, but there are lots of good ideas floating around. One issue I might mention tomorrow: There is no proper probabilistic approach (I know) for measuring two-point functions. Readers: Do you know anything?

reading papers

On the plane home, I read and commented on Bovy's extreme-deconvolution quasar target selection paper, and also Guangtun Zhu's (NYU) paper on finding maser galaxies (for, for example, Hubble Constant measurement) using the SDSS data. I also worked on various writing projects, including Zolotov's cusp paper.

New Directions in Modern Cosmology, day 4

Highlights for me today were talks by Lewis (Sussex) about lensing in the CMB and Joyce (Paris) about the scale-invariance of gravity and numerical simulations. Lewis showed some beautiful results with WMAP data, in which they can show that the statistics of the CMB are distorted by the slightly anisotropic beam of the satellite combined with the slightly anisotropic scan pattern. This was in the context of lensing, because he finds it using the same techniques that will be used to find lensing-induced distortions to the Planck map.

Joyce showed some beautiful results on gravity, including exceedingly precise simulations of one-dimensional (yes, one-dimensional) gravitational collapse, and analytic description of the behavior of n-body simulations. He is trying to understand whether the departures from scale-invariance seen in n-body simulations (the correlation function is not a power law in the dark sector, while the observed correlation function of galaxies is very close to a power law) could be somehow related to the difference between gravity and the simulations thereof. He doesn't have conclusive results, but his models of the models are impressive.

New Directions in Modern Cosmology, day 3

Another good day in Leiden. One highlight was work by Kunz (Geneva), who showed simple models for generating CDM-like inhomogeneities in the CMB without inflation. He gets very close, but in the details at large scale, causality requires some inflation-like activity. His arguments were very general. More was said about this by Magueijo (Imperial), who uses a variable speed of light to do inflation's work. In general these causality arguments come from the problem that it is very hard to set up the initial conditions.

Lachièze-Rey (Paris) described nice detectors for the baryon acoustic feature that don't require construction of the full correlation function or power spectrum. This led nicely into an afternoon discussion of homogeneity, where my results, Sylos Labini's (Rome), and Kazin's (NYU) were batted around. I didn't fare as well as I would have liked in part because we have not closed all the loopholes remaining for some kinds of technical inhomogeneities. I certainly think that it is well established that the Universe has a mean density, but Sylos Labini and others have a good point that if you can do your data analysis without assuming the mean density then you should, if for no reason other than that it is rarely well measured (so it adds uncertainty to your results). Don't get the wrong idea from my measured tone here: The universe is not a fractal on large scales!

New Directions in Modern Cosmology, day 2

Today was my first day at the meeting in Leiden, named above. It is an eclectic group, because the idea was to bring together outsiders and insiders in the whole cosmology thing and have people hash it out: A great idea. The talks were good, but I hope I will be forgiven for saying that (for my geeky self) I learned the most from the talk given by Hao Liu (Beijing), who is re-analyzing the WMAP data starting at the time stream. He is finding many very interesting things out about the calibration, attitude model, and map-making. It appears that the quadrupole amplitude is very soft. It is all about attitude and configuration modeling, just like Hipparcos, which is not surprising. Plus there are sidelobe issues. It appears that no-one, not Hou or the WMAP team, is building the full likelihood including all calibration parameters and then marginalizing out everything but the map (or the cosmological parameters). That is, there still is not an honest Bayesian analysis (if that is even possible!).

stellar oscillations

Hou, Goodman, and I had our weekly meeting today to discuss exoplanet fitting and inference.  Goodman suggested a possible statistical model for stellar oscillations that would permit us to treat them as a kind of structured Gaussian noise.  Yet another project becomes a kind of Gaussian process!  The idea is to drive a damped oscillator with a broad-band source.  I assigned that to myself as homework for my flight to Amsterdam.

occultations

In between seminars, Schiminovich and I played around a bit with our transit search.

stacking

Price-Whelan and I pair-coded some image stacking stuff, which we are going to use to criticize the (plan for the) PanSTARRS data pipeline.

automated time-domain discovery

Lang and I spent the day writing and automated system to detect rate changes in photon streams. We applied the code—which is pretty principled—to some GALEX data (extracted by Schiminovich) and we re-discovered a bunch of known things. We also discovered a bunch of other things; it remains to be seen if they are really discoveries.

timely distraction

I went up to Columbia today to work on the IGM and radiation fields with Schiminovich and then he distracted me with all sorts of crazy stuff he is finding in the GALEX time-stream. GALEX time-tags every photon (and it has one hell of a lot of photons), so it is the best time-domain project in astronomy ever. Almost nothing has been done in that time-stream to date, with some extremely notable exceptions. We talked, wrote code, and planned our exploitation.

beta distribution, hierarchical

In response to a (good) referee comment, I worked on re-fitting the exoplanet eccentricity distribution—in our hierarchical Bayesian model—with a beta distribution. This is a very useful family of distributions with two shape parameters and a lot of freedom. On the weekend and this afternoon I also started to write down the quasar absorption model that we might be able to use to make the clustering measurement the output of another hierarchical model. If we can do that successfully, it will be a real triumph.

IGM model

At the end of the day, Foreman-Mackey, Bovy, and I discussed how we might model the Lyman-alpha absorption in quasar spectra, such that the continuum (which is unobservable) becomes a fit parameter usefully constrained by the data. The long-term goal is to do Lyman-alpha clustering. The key thing is that this would be hierarchical: The parameters we care about would be parameters of the state and clustering of the IGM.

ionizing photons

Schiminovich and I planned our project to measure the intergalactic ionizing flux in the neighborhood of the Milky Way.

all code all day

Lang and I pair-coded all day. We can now optimize the likelihood of a new SDSS Catalog against the imaging pixels in a field, permitting the model both continuous changes (parameter updates) and qualitative changes (model type changes, like star to galaxy and so on). We find that we can enormously improve the goodness of fit and reduce greatly the number of parameters, so it is win–win. It was a great day! Now we must write, write, write.

practice of science

In the morning, Lang and I pair-coded. In the afternoon, Jiang and I discussed the finer points of English grammar! This sounds like a joke, but of course writing is the most important tool in the scientist's toolkit, bar none.

likelihoods

Among other discussion topics, Bovy and I debated the usefulness of the fact that if you have N pieces of data about an object, each of which says something about, say, the question of whether that object is a quasar, you can multiply together the N independently calculated likelihoods for quasar and for star. This sounds great, but it only makes sense when the N pieces of data tell you independent things. That is, it is only true if the joint probability of all the data equals the product of the probabilities of each data item separately. This is almost never the case!

In related news, we figured out why Zolotov and I are having trouble computing the Bayes factor for our project: Bayes factors are hard to compute! And they are also very slippery, because a small change to an uninformative prior—one that makes no change whatsoever to the posterior probability distribution—can have a huge impact on the Bayes factor. Once again, I am reminded that model selection is where I part company from the Bayesians. If you don't have proper, informed priors, you can't compute marginalized relative probabilities of qualitatively different models. It is cross-validation and data-prediction only that is reliable in most real situations.

finding AGN

At group meeting today, Renbin Yan (NYU) talked about finding AGN by X-ray and optical criteria. He has an optical color criterion that can replace the NII to H-alpha part of the BPT diagram; it works surprisingly well. He also showed that some supposed classes of AGN that are X-ray bright or X-ray faint are just the undetected edges of a broad distribution.

GALEX, time domain

Schiminovich came down for most of the day, and we discussed next steps on our GALEX projects. We also discussed the time domain, where we are set up to do some crazy projects, but haven't started. With Astro2010 endorsing LSST, this could be a valuable thing for the community as well as scientifically interesting. In the late afternoon, Lang and I continued to pair-code the reoptimization of the SDSS Catalog

chaos

With the chaos of the starting semester, and job season, I didn't get much research done beyond conversations with Lang and Zolotov and Bovy.

SDSS Catalog gradient descent

Lang and I worked on setting up the synthetic-image gradient-descent we are preparing for a next-generation SDSS imaging catalog. If we can get it started, we will be optimizing a multi-billion-parameter model. I have done something close before for SDSS ubercalibration, but in that project we transformed the problem into a linear fit by making some not-terrible approximations. No such luck here.

don't stack!

I finished (in my mind, anyway) specifying the full don't stack your images project for Price-Whelan today. At first we will work on synthetic data (despite the wrongness of that), and move to real data after we have clear results.

stacking images?

Price-Whelan and I spent part of the afternoon on our project to criticize (naive methods for) image stacking. We are using fake data but quickly realized that we shouldn't be. We should be using SDSS Stripe 82!

diagnosis of code

After a morning of discussion with new student Daniel Foreman-Mackey (NYU), I spent a chunk of the afternoon helping Jagannath diagnose his likelihood function. He is doing MCMC with a marginalized likelihood function; it is a pretty non-trivial piece of code. Something goes very wrong early, and we can't figure it out. It made me realize—once again—how important the idea of diagnosis is in science. And yet it is not part of any formal curriculum or training. This point is made in a childhood context by Seymour Papert in the wonderful book Mindstorms.

talking

My only real research today was conversations with Zolotov and Wu about their current papers.

flat-fielding in my sleep

While I was on the plane flying home, Rory Holmes (MPIA) implemented some things we had talked about related to internal (self-) calibration of photometric data sets, in a toy context. It looks (as expected) like the quality with which you determine your flat-field is a strong function of how you sampled it. This project is expected to evolve into something about how you should design your survey (WFIRST or LSST or the like) to ensure precise photometric calibration.

faster iterations, but a whole lot more of them

On my last day at MPIA—what a great summer it has been—Tsalmantza and I raced an iterated gradient descent against a block-diagonal method for solving the matrix factorization problem we have been working on (I have given it many names, but I now realize that Roweis would have called it matrix factorization). We were both sure that gradient descent would crush because in our (not brilliant) R implementation, the gradient descent steps took hundreds of times less time to execute than the block-diagonal steps. But it turned out that gradient descent takes more than hundreds of times more iterations! So the block-diagonal method wins. We probably could beat both with conjugate gradient, but that will wait for another day. I have packing to do!

infinitely powerful

On the weekend I read—at Lang's suggestion—An Introduction to the Conjugate Gradient Method Without the Agonizing Pain (PDF) by Richard Shewchuk (Berkeley). It made me infintely powerful, because now I can do gradient descent and conjugate gradient effortlessly. I wrote the relevant functions for my project with Tsalmantza. Shewchuk's document contains much knowledge, beautifully expressed. You must read it!

spanning a linear space

Tsalmantza and I confronted the degeneracies you have when you use vectors to span a linear space: The components of our spectral basis can be reordered and renormalized and made into linear combinations of one another, all without changing any of the final results. This means (a) really you should never plot the basis vectors, but really just plot the fits of the basis vectors to real objects; it is not the basis vectors that are the result of your optimization, it is the subspace they span. And (b) really you should break these degeneracies as you go, so your code outputs something somewhat predictable! We broke the degeneracies the same way PCA does, except much better (of course!): We normalize them sensibly (all have the same sum of squares), orthogonalize them by diagonalizing the squared matrix of their coefficients when fit to the data, and sort them by eigenvalue, most important first.

Now the code we have outputs exactly what PCA outputs, except that it optimizes chi-squared not the unscaled sum of squares, and is therefore a much better representation of the data. It is a plug-in replacement for PCA that is better in every respect. Have I mentioned that I am excited about this? Now we must write the method paper and then finish all our myriad projects we are doing using this.

the line has been fit, code too slow

I finished my fitting-a-straight-line document; I will submit it to arXiv tomorrow. In the afternoon, Tsalmantza and I found that the non-negative updates are not only very successful, they are also very fast, much faster than the unconstrained updates. We think this is just because we are doing the unconstrained updates in a silly way (for R anyway). So it is on the to-do list to speed up the normal updates.

[after the fact: The fitting-a-line document is http://arxiv.org/abs/1008.4686. Thanks to everyone who helped out!]

uberkalibration, non-negative

Rix and I discussed a project to design observing strategies that optimize calibration performance of a survey, with Euclid, WFIRST, and LSST in mind. The idea is that, as with SDSS, the best calibration will be internal calibration, and that is optimized when there is non-degenerate information about sources and pixels. Rory Holmes (MPIA) will write some code here with direct impact on the Euclid planning, but the project is general.

I worked out for Tsalmantza the non-negative updates if we moved our data-driven spectral modeling to non-negatively constrained. I hope both my readers know that non-negative matrix factorization is a solved problem (and simple at that), even for a heteroscedastic (proper chi-squared) objective function. There is no reason to ever use PCA ever again! I have in mind that we could give good LRG redshift-finding templates to the BOSS.

submitted!

Submitted my paper with Myers and Bovy on inferring the eccentricity distribution. It will appear on arXiv tomorrow. I also had long conversations with Bovy about all the on-going projects, especially those related to quasar target selection.

Gaussian processes

Tsalmantza and I implemented a Gaussian-processes-like prior on her spectral model, to enforce smoothness at wavelengths where there are very few data. We looked at the results today and they are very nice. It works great, costs almost no time per iteration, and doesn't slow down convergence. This is great for a bunch of projects we are working on.

Late in the day I discussed image modeling with Lang; we have a (somewhat) practical proposal to replace the SDSS imaging catalog with something that is optimal (under some assumptions) in the pixel space. This should improve the catalog at the faint end and in regions where the data have issues.

highly informative prior

Tsalmantza and I pair-coded two parts of our spectral modeling software (in R it has limitations, but it is open-source): (1) We set things up to include a Gaussian-process-like smoothness prior. This ended up being trivial, but it looked bad at first because it makes a block-diagonal matrix non-block-diagonal. (2) We worked through the bugs in the highly informative prior we are trying out on our quasar-redshift project: Do we measure the redshifts of quasars better if we insist that the redshift be measured by a spectrum that looks very much like an already-seen quasar? The prior is what I would call a mixture-of-archetypes. After we finished the latter part, the answer looks like it is going to be yes. If so, this is a win for Bayes; this project directly competes a marginalize-out-nuisances (with a highly informative prior) against a optimize-everything method; the first is very Bayesian (extreme, you might say) and the second is very frequentist.

Astro2010

I led a discussion at the MPIA of the results of the Astronomy and Astrophysics Decadal Survey, which was released last week. I was much happer with the outcome than many of my colleagues here. Of course it was confusing about what it all means for Euclid in which MPIA has a stake. Does this count as research? If not, I also made the affine-parameter prior PDF proper in Jagannath and my curve-fitting project, and looked at new results of quasar redshift determination with Tsalmantza.

zeroth draft, periodograms

I wrote a polemical discussion section and thereby finished the zeroth draft of Myers and my eccentricity distribution paper. Just waiting for code to finish running on that, and then I will have to take the salty language down a notch. In the afternoon, Schmidt and I discussed his periodogram code for searching SDSS stripe 82 for periodic variables, and spectral handles on his quasar variability findings.

writing

Kept on writing.

pair-coding in chaotic environments

In the morning I finished the SDSS-III DR8 visualization. In the afternoon, Myers and I pair-coded things related to the SDSS—Pan-STARRS cross-calibration we are working on. For reasons outside of our control, we pair-coded at Jumpinn.

full-survey visualization

I worked on my full-SDSS-III imaging visualizations today. These visualizations are built from every single detected object in every survey-primary bit of imaging. So they are challenging to build. Some day, it would be fun to go visualization-crazy: Why do the theorists always have better visuals?

writing retreat

I spent the week traveling (hence no posts) and writing. I wrote text about the structure function and Gaussian processes for Bovy and Kasper Schmidt (MPIA). I wrote text about fitting a curve to a set of points in phase space for Jagannath. Next week it is back to Heidelberg and back to work.

writing

I worked on writing up the accomplishments of the week. Between hierarchical exoplanet modeling, Pan-STARRS calibration, and variability-selecting quasars, there is a lot to write about.

Gaussian process

I realized today that my eccentricity distribution code can use the methods of Gaussian processes. This pleased me greatly, as I have been wanting to learn about them. I also gave a short talk on image modeling and worked on updating my well-known but out-of-date SDSS zoom (see frame above).

Pan-STARRS and SDSS

Myers and I worked out a robust likelihood function to use for comparing synthetic Pan-STARRS fluxes to real Pan-STARRS data. The synthetic magnitudes will be constructed from overlapping SDSS imaging data, with a flexible linear model. Eventually the hope is to variability-select objects, with two epochs spanning a six-to-ten-year baseline.

extremely extreme deconvolution

I demonstrated with our fake exoplanet data that my exoplanet deconvolution methodology works. This is even more extreme than Bovy's extreme deconvolution code, because it can use data with arbitrarily complicated uncertainties—not just heteroscedastic but also non-Gaussian, bimodal, or what have you. It ain't fast.

In related news, I figured out (the hard way) that if you are sampling the posterior probability distribution function but keeping track of the likelihoods, the best likelihood you see in the sampling is not necessarily all that close to the true maximum of the likelihood function. This is obvious in retrospect, but I was confused for a while. I think it is true that if your prior is not extremely informative, you ought to eventually see the maximum-likelihood point, but you might have to wait a long time, especially if your likelihood is not very sharply peaked.

sampling fake planets

Myers and I sat side-by-side for much of the day getting and checking samplings for the posterior probability distributions for exoplanet parameters for a big chunk of fake data. We decided that the distribution inference code we are writing should run on samplings—since so many astronomers are producing samplings these days—so we need lots of samplings for testing.

cross-validation

I gave my MPIA Hauskolloquium on model selection today. I strongly advocated leave-one-out cross-validation. It is very easy, conceptually simple, robust to many kinds of mistakes (about model space and noise amplitude), and employs a "prediction" utility that matches the goals of most scientists. Despite the frequentism of all this, I am still thought-of as a Bayesian around here. I am only a Bayesian when I have to be! That turns out to be frequently.

The marginalized likelihood (marginalized over orbital phase) mentioned yesterday did work well, so Jagannath and I are ready to write our paper. We discussed the introduction for the paper and the speed of the code.

marginalization over orbital phase

Jagannath and I changed our stream-fitting code to marginalize over orbital phase. This is much better (in principle) than optimizing for it at each star. Will it perform well in practice? We wait to see. On a related note, Tsalmantza and I finished the marginalization of our quasar-redshift likelihoods over spectral parameters. Here too it remains to be seen if this improves our redshift predictions. Both of these projects are, in some sense, testing Bayesianism, since frequentists can't marginalize their likelihoods.

At lunch, Christian Schwab, Andreas Quillenbach (LSW), and I discussed the crazy stellar pulsations that can look very much like exoplanets. Our project is to model them and do hypothesis testing.

cross-validation demos

I am giving a talk on model selection on Friday. Today I made a demo that compares AIC, BIC, and leave-one-out cross-validation. All these are frequentist model selection criteria (despite the B in BIC). Of them, cross-validation is by far the best for many reasons: It is based on data prediction, not theory; it is (relatively) insensitive to bad uncertainty estimates; it has an interpretation in terms of a sensible utility. One of the oddities of model selection, not often appreciated, is that no principled frequentist or Bayesian result tells you which model to choose; data analysis just puts probabilities on models. If you want to remain principled you either never choose and just propagate all models (best practice for Bayesians), or else explicitly state your utility function and make the decision that maximizes your expected utility. Any time you choose a model without specifying or calculating utility, you have made a mistake, I think. Perhaps not a serious mistake, but a mistake nonetheless.

anisotropic noise tensors

For both quasar structure functions and Jagannath's project on tidal streams, I am playing around a lot with non-trivial variance tensors to insert into Gaussian probability distribution functions. On the quasars, I am trying to make a very pedagogical introduction to the idea that the structure function—a model of quasar variance—should be cast in terms of Gaussian processes—a set of general models for variance. This was Bovy's good idea; it makes all sorts of new kinds of data analyses possible and all sorts of existing observations more useful. On the streams, we are modeling observational uncertainty (which is simple in distance, transverse angles, radial velocity, and transverse angular velocity but complex in position–velocity space) in six-dimensional phase space, so that our cold-stream data analysis has a realistic description. The short-term goal is to establish a method for fitting streams that is justified in terms of probabilistic inference.

After lunch, Rory Holmes (MPIA) led a discussion of Euclid calibration and observing strategy, because the MPIA is part of a team proposing to operate part of that mission.

fitting a curve to stars

Tsalmantza and I continued with binary quasar inspection. Myers, Hennawi and I discussed Pan-STARRS calibration with Eric Morganson (MPIA). On a related note, at the weekly MPIA Pan-STARRS meeting, Cristina Afonso (MPIA) showed that Pan-STARRS data are very stable and can be calibrated to sub-percent precision when there are many overlapping epochs. Kasper Schmidt (MPIA), Rix, and I discussed quasar variability, and I worked out a few things we can do to model it more flexibly.

Most importantly, however, Jagannath showed up in Heidelberg for a week, and we discussed our project of fitting orbits to streams of stars at lunch with Rix. He made some suggestions—in particular to consider the effect of systematic distance errors—which slightly adjusted our to-do list; we are on track to finish a zeroth order document by the end of the week.

two new binary quasars!

Tsalmantza, Roberto Decarli (MPIA), and I discovered two new binary quasars today. One we found by our fitting, and one by looking for quasars judged to be galaxies by the spectroscopic pipeline (the reason these are sometimes binaries is because the broad lines can be shifted very far from the narrow lines, tricking the pipeline). Given that there are only four binary quasars known previously, this was a pretty good day's work.

priors on spectra

Hennawi nearly convinced me that Tsalmantza and I should be putting priors on our coefficients in spectral space and switch from producing likelihoods to producing marginalized likelihoods or posterior probabilities. I spent the evening writing up the project we are doing together and writing down how we could construct and use priors.

cross-validation

I built a substantial leave-one-out cross-validation demo for my talk on model complexity. The demo is very cool and will be useful when I write up a document about model selection. The talk is in one week.

redshifts and multiple redshifts

Today Tsalmantza and I got results on two projects: We found quasars with potential double-redshifts (that is, double quasars) with our data-driven quasar model, and we determined low-redshift quasar redshifts with great precision, with this same model. In the former project, the idea is to do a two-quasar vs one-quasar hypothesis test for every quasar. In the latter project, the question is whether we can make narrow-line-quality redshift determinations of quasars for which the narrow lines are not visible. Both projects look promising; we ended the day pretty optmistic.

non-parametric fitting

I got working a simple (though not fast) system to fit a highly parameterized (what is called, for some reason, non-parametric) curve to a set of data today, in preparation for a future talk about model complexity. My model has more parameters than data, but it optimizes, and it has a complexity that is continuously variable; the complexity is not the number of parameters.

PHAT

Rix, Julianne Dalcanton (UW), and I spent a morning in the garden discussing the high-level data-analysis plan (or maybe inference plan) for Dalcanton's PHAT project. This project is imaging a large part of Andromeda and the inference is about the ages, masses, and initial mass functions of young stellar clusters. Among other things we discussed the difference between a model that assumes a flat age distribution, and a model that insists on a flat age distribution, and the difficulty of making the latter within Bayesian inference. Indeed, this was what was hard initially about Bovy and my Solar System (or orbital roulette) project.

Dalcanton

I can't say I got much done today, except consult with Lang on his astrometric calibration work on Dalcanton's PHAT project to image M31. At Galaxy Coffee, Dalcanton quickly showed the first data from PHAT, which is absolutely beautiful, and so much better than anything that has come before.

cusp hypothesis testing

I finally wrote down in detail how we can do a hypothesis test for an ultra-faint galaxy between the cusp-in-projected-phase-space and self-gravitating-blob models. Zolotov is working on the execution.

data-analysis consulting

I gave (solicited) data-analysis advice to Rainer Klement (MPIA), Yujin Yang (MPIA), and Christy Tremonti (Wisconsin). I like my role as consultant. Hennawi, Myers, and I spent a long lunch reviewing our long to-do list for this summer. If we even get one fifth done I will be happy.

Groningen

I gave a seminar at Groningen today, where I was hosted by Scott Trager. I gave my Gaia talk, which I fear is way too pessimistic; I need to accentuate the positive.

the Oort problem

After talking to Bovy yesterday about the Oort problem (and extensions, to measure the local gravitational potential gradients in the Galaxy), Rix and I asked Volker Springel (Heidelberg) about the possibility of making mock catalogs and performing method tests in realistic simulations. He was positive, but we have no plans. What I would really like to do is to set up a blind test with a cash prize! But we don't really know how wrong our answers become as our assumptions are violated, and many of the Oort assumptions are wrong in detail. After this conversation, we learned about the Meerkat project, which is an SKA precursor but very ambitious nonetheless.

still fitting a line

I finished the third (I think) draft of my old line-fitting document, taking into account all the great comments I received from blog-readers and other experts. It is in Lang's hands, then it will go to Bovy, and then to the arXiv. I can't wait to be done; it is one of the milestones for the summer.

angle-mixed methods

In the morning with Rix and later in the day with Michael Perryman (ESA, Leiden) I discussed the likely failure of angle-mixed approaches to inferring the Milky Way dynamics from a kinematic snapshot. Rix still believes we should do the Oort problem (measuring the disk potential locally) because that is likely to be close to angle-mixed, at least for some populations. I spent the other parts of the day writing, with the exception of a break in which Roberto Decarli (MPIA) explained to Tsalmantza and me how our spectral modeling could revolutionize the study of black-hole binaries. We are going to pursue that next week.

ULIRG clustering, spectral modeling

After a few days of vacation I arrived at the MPIA for my usual summer stay. Tsalmantza and I specified the scope and content of the paper we intend to complete this summer, and Ben Weiner (Arizona) spoke about the clustering and star formation of redshift-unity galaxies. He showed that neither quasars nor luminous star-forming galaxies are clustered like the massive, red galaxies of which they are supposed to be the progenitors.

fitting a line

Inspired in part by the conversation with Juric yesterday, I worked on finishing up the old, old fitting-a-line document today. I hope to do more on the airplane.

where's the dust?

Mario Juric (Harvard) dropped by for an hour in the afternoon to discuss inference of the full three-dimensional dust distribution in the Milky Way. He is taking off from ideas of Ivezic (Washington) and Finkbeiner (Harvard) about these issues, but trying to do the inference by generating the data. This is music to my ears. We talked about how to make a tractable first step.

deconvolution in the extreme

I spent the weekend and today working on my hierarchical exoplanet papers. Today, Bovy pointed out that they are really just more general versions of our extreme deconvolution method for forward-modeling noisy (and heteroscedastic) data. (On that note, Bovy just resubmitted the paper to JOAS.) Because I care—by assumption—only about the distribution function, and not about improving the individual-exoplanet measurements, I am not doing anything properly hierarchical. I think he is right; that adjusts my introduction.

Zolotov and I discussed the issue that any cusp in the projected phase space distribution function will have finite thickness, which smooths out the infinities. We talked about methods for computing the cusp function safely. Power laws are difficult.

SciCoder, hierarchy

In the afternoon I dropped by Muna's SciCoder workshop today to give a short spiel about testing, making my pitch that scientific results produce the best end-to-end tests of scientific software. In the morning I worked out a bit more about my hierarchical Bayes methods for exoplanet studies. I am finding the math to be non-trivial, although in the end I want—and expect to have—something simple to explain and simple to execute. I think I will pitch it to people who have, to start, a sampling of exoplanet parameters for each of a set of exoplanets. That is, the pitch will be to Bayesians.

Johnston

Kathryn Johnston (Columbia) spent the day at NYU, and we talked about many things. We both agree that streams—cold structures in phase space—are the future for constraining the Milky Way potential. We didn't accomplish much, but she encouraged me to finish up our various cold-structure projects!

National Science Foundation

Does writing an annual report on a grant count as research? I hope so, because that's most of what I did today. I also figured out that the paper I am writing with Myers should really be two papers; we will see if Myers agrees when I break that to him in Heidelberg. I split the minimal text I have now.

cusps and papers

I worked briefly with Zolotov on her cusp project, debugging code to generate cusp-like density functions. After that I had a long discussion with Bovy about quasar target selection, with him showing me a very good framework for incorporating variability information (a generative model for quasar lightcurves, really), and with me arguing that we should write a quasar catalog paper. If we do that, then the SDSS-III target selection paper could just be a reference to our quasar catalog paper.

epsilon

Aside from a few conversations I did nothing today.

hierarchical inclinations, cusps

I spent a bit of time working on hierarchical (read: Bayesian) methods for inferring distributions in the face of significant nuisance parameters, with the goal of getting the exoplanet mass distribution without knowing any inclinations. What we are doing also applies to binary stars and stellar rotation. In the afternoon, Zolotov and I spent some time designing her project to look at cusps in the distribution of stars in the Milky Way halo.

Jiang, semantics

Today my student Tao Jiang (working on galaxy evolution with the SDSS and SDSS-III data) passed his oral candidacy exam, and I gave my talk (remotely) at AstroInformatics2010. The latter was a bit too aggressive (Djorgovski asked me to be controversial!) and of course I had to tone it down in the question period: Indeed, the semantic world is considering issues with trust. But the only things people could point to on the issue that meta-data are probabilistic is the idea of fuzzy data and fuzzy logic, and I seem to remember that these are not—in their current forms—defined in such a way that they could communicate (and be used for marginalization over) precise likelihood or posterior probability distributions. In the question period, one audience member even suggested (as I have been writing in my blog and elsewhere) that we think of catalogs as parameters of a model of the image pixels! So that was good.

semantics

I worked today on getting some slides together for my talk at AstroInformatics2010 tomorrrow. I am giving my talk by Skype; we will see how that goes! I plan to be pretty controversial, as I have been asked to be; my working title is Semantic astronomy is doomed to fail.

quasar target selection

I did lots of small things today. Perhaps the least small was to help a tiny bit with Bovy's proposal to change SDSS-III BOSS quasar target selection over to a method that makes principled use of extreme deconvolution (XD). The Bovy method has several advantages over the current method, including:

• It is lightweight. The model is completely specified by a (large) number of Gaussians in photometry space; it can be implemented by any code easily given those Gaussian parameters.
• It is the best performing method of all those tested by the Collaboration.
• It does not, in any sense, convolve the model with the errors twice. Most empirical distribution function descriptions are descriptions of the error-convolved distribution; when you compare with a new data point you effectively apply the uncertainty convolution to the distribution twice. Not so for XD.
• It is the easiest method to extend in various ways including the following: You can modify the priors easily, even as a function of position on the sky. You can add data from GALEX or UKIDSS or variability studies straightforwardly. You can apply a utility to make the method prefer quasars that are more useful for the measurements of interest.

In general, when you need to do classification you should—if you have measurements with uncertainties you believe—model distribution functions for the classes, and when you want to model distribution functions using uncertainty information correctly you should use XD!

target selection, trust, and funding

The NSF proposal is submitted. It is about—among other things—a trust model for the VO. I think whether or not we get funded, Lang and I should write this up as a short paper, since we figured a fair bit out about the issues. Not sure if that's an LPU though. Spent a good deal of time with Bovy, briefing him on what I got from the Gaia meeting last week, and strategizing about BOSS target selection, where it seems like maybe we should make a firm proposal for being the primary algorithm. In related news, Jessica Kirkpatrick (Berkeley) was visiting today (and will be back next week for SciCoder).

write like the wind

On the airplane home, I drafted the entirety of my ELSA Conference proceedings contribution. I guess I got pretty stoked at that meeting!

ELSA Conference on Gaia, day five

This morning was the final session of the meeting. Again, there were too many things to mention. Helmi (Groeningen) began the day with a discussion of streams and substructure. Among other things, she showed that substructure falls onto the Galaxy along preferred directions, so features like the famous bifurcation in the Sagittarius stream is not unlikely caused by a pair of galaxies infalling along the same plane! She suggests looking for streams in the 3-dimensional space of actions; this is a good idea, but a young stream also is compact (1-dimensional) even in the action space, so there is more information than just what you find in action space.

At the end of the meeting, I gave my talk on catalogs as models and the idea of propagating likelihood information about image intensity measurements rather than a hard catalog. I realized during the question period that this is my main problem with the semantic ideas in astronomy, and that should be a part of my write-up. Brown (Leiden) followed with a concluding remarks talk in which he was extremely nice to me, but also advocated releasing Gaia results early and often, and releasing the results in such a way that they could be re-analyzed at the pixel level in the future. I wholeheartedly agree. Farewell to a wonderful meeting.

ELSA Conference on Gaia, day four

Today I gave Bovy's talk on inferring dynamical properties from kinematic snapshots. The talk noted that in our April Fools' paper, the planets with velocity closest to perpendicular to position do the most work. After, Juric (Harvard) asked if there are similarly important stars for the Milky Way? If so, we could just observe the hell out of those and be done! My only response is that if we had a sample of realizations of the Milky Way that generate the data—that is, detailed dynamical models—the stars whose properties vary the most across the sampling would be the critical stars. Not a very satisfying answer, since that chain would be almost impossible to produce, maybe even by 2020. Interestingly, my attitude about this turns out to be very similar to that of Pfenniger (Geneva), who gave a very nice philosophical talk about how our vision of the simplicity or complexity of the Milky Way has evolved extremely rapidly. He argued that no smooth models are likely to give right answers, not even approximately right. I don't think everyone in the business would agree with that, though I do. Before either of our talks, Juric gave a summary of his three-dimensional Milky Way modeling using photometric data. He, of course, finds that simple models don't work, but he has some great tools to precisely model the substructure.

Ludwig (Heidelberg) argued that either there are systematic issues with chemical abundance indicators or else even single stellar populations have metallicity diversity. This would do damage to the chemical tagging plans. In conversation afterwards, Freeman (ANU) told me that he thinks that the issue is with the models not the populations and that the models will improve or are improved already. I suggested to Freeman an entirely data-driven approach, but I couldn't quite specify it. This is a project for the airplane home.

Pasquato (Brussels) made very clearly the point that if stars have strong surface features that evolve (as when they have highly convective exteriors), there will be astrometric jitter if the star has an angular size that is significant relative to Gaia's angular precision. This is obvious, of course (and I mean that in the best possible way), but it suggests that maybe even parallax needs to be defined like radial velocity was yesterday!

Antoja (Barcelona), Minchev (Strasbourg), and Babusiaux (Paris) gave talks about the influence of the bar and spiral structure on the Galaxy and galaxies in general. Minchev presented his now-famous results on radial mixing, which relate in various ways to things we have been doing with the moving groups. Antoja was more focused on understanding the specific impact of specific spiral arm and bar models on the local distribution functions. Babusiaux nearly convinced me that the bulge of the Milky Way was created entirely from the disk by the action of the (or a) bar. Bars rule! After dinner, Minchev and I found ourselves at a bar.

ELSA Conference on Gaia, day three

It was spectroscopy all morning, with the Gaia Radial Velocity Spectrograph worked-over by Cropper (UCL), Katz (Paris), and Jasniewicz (Montpellier). Two things that were of great interest to me: The first is that the satellite can only do relative measurements of velocity (really of logarithmic Doppler Shift, I think) so they need absolute velocity standards. They will use a combination of asteroids and team-calibrated (from the ground) radial velocity standards that span the sphere and spectral type. The issue is more fundamental, of course: To compare stars of different types, you need to tie together radial velocity standards that are based on different lines with different gravitational redshifts and convective corrections. But the issue is even more fundamental, and that is the second thing that was of great interest to me: Lindegren (Lund) has a paper defining radial velocity that shows that it is not really a spectroscopically measured quantity: There are many physical effects that affect stellar lines. Indeed, this relates in my mind to an old conversation with Geha (Yale) and Tremaine (IAS) about whether it is possible—even in principle—to measure a velocity dispersion smaller than Geha's smallest measurements. I think it is but it is because God has been kind to us; this was not something we could expect just by dint of hard work.

Freeman (ANU) gave a talk about HERMES, an ambitious project to take detailed enough spectra of about a million stars to do full chemical tagging and separate them into thousands of unique sub-groups by detailed chemical properties. He made an argument that the chemical space will be larger than (though not much larger than) a seven-dimensional space. I have been hearing about chemical tagging for years and seen nothing, but with this it really looks like it might happen. This project is explicitly motivated by the science that it enables in the context of Gaia. Where are the American projects of this kind? Also, when will we have the technology or money to take spectra of a billion stars?

In the afternoon, there were Solar-System and asteroid talks; these are always impressive because precisions in this field are so much higher than anywhere else in astronomy. Oszkiewicz (Helsinki) gave a nice talk about fully Bayesian orbit fitting for asteroids, showing that they could have predicted the asteroid hit in Sudan in 2008 with high confidence (did they predict that?). She also showed that they can model asteroid shapes from Gaia lightcurve data with the same kind of MCMC machinery.

ELSA Conference on Gaia, day two

There is far more to report here from today's than I easily can, so I will just blurt some highlights once again:

Lindegren (Lund University) gave a beautiful talk about fundamental issues in astrometry, especially with spinning satellites. Gaia—unlike Hipparcos will work in the limit in which the spin is more strongly constrained by the density of informative stellar transits than it could be by any reasonable dynamical model of the spinning satellite subject to torques. That is, there is only a very weak dynamical model and the data do the talking. This means that any measurement by the satellite that can be accommodated by an attitude change is not constraining on the global astrometric solution! For this reason, with each of the fast rotations (scans) of the satellite, the only constraining measurements made on the global astrometric solution are comparisons between stellar separations and the basic angle of the satellite, along the direction of the scan. It really is a one-dimensional machine, at least on large scales. The two-dimensional images off the focal plane will be useful transverse only on small scales. He followed these beautiful fundamental arguments with discussions of self-calibration of the satellite, which is really what all the talks have been about these two days, in some sense.

Bombrun (ARI, Heidelberg) and Holl (Lund) gave back-to-back talks about the optimization of the linearized system of equations and the error propagation. Optimization (after the Collaboration found conjugate-gradient method) is trivial, but exact error propagation—even for the linearized system—is impossible. That's because the sparse matrix of the linear system becomes very non-sparse when you square and invert it. Holl has some very nice analytic approximations to the inverted matrix, made by expanding around invertible parts, and by making simplifying assumptions. This is key for the first generation of error propagation. In my talk I will emphasize that if the Collaboration can expose something that looks like the likelihood function, error propagation is trivial and it becomes the burden of the user, not the Collaboration. However, there is no chance of this in the short run.

Eyer (Geneva) gave an electrifying talk about variable stars, making clear what should have been obvious: Gaia will be the finest catalog of variable stars ever made, and contain in almost every class of variability hundreds of times more stars than are currently known. This opens up the possibility for all kinds of novel discovery, and enormous improvements in our understanding of the variables we now know. His group is computing observability of various kinds of variables and the numbers are simply immense. He noted that Gaia might discover WD–WD eclipsing-binary gravitational wave sources.

At the end of the day Mahabal (Caltech) spoke of automated transient classification. They are doing beautiful things in the VO/semantic framework. Of course I am a critic of this framework, because I want meta-data to be probabilistic and computer-generated, not human-designed and combinatoric (as in this is a Type IIP Supernova; much more useful to be given relative likelihoods of all non-zero possibilities). But there is no doubt that within this framework they are doing beautiful stuff.

ELSA Conference on Gaia, day one

Today was the first day of the ELSA Conference. Talks focused on the status of Gaia hardware and software, with a lot of concentration on the expected effects of radiation damage to the CCDs and strategies for dealing with that. There were many great contributions; a few facts that stuck in my mind are the following:

Prusti (ESA) emphasized the point that the intermediate or preliminary catalog released in 2015 will only be slightly less good than the final catalog of 2020, if all goes well. He argued that Gaia should release preliminary catalogs since surveys like SDSS have prepared the astronomical community for non-permanent and non-final and buggy catalogs. I agree.

Charvet (EADS) showed the spacecraft design and assembly. It is made from silicon carbide, not metal, for rigidity and stability. Apparently this makes fabrication much more difficult. The mirrors are silvered, polished silicon carbide, attached to a silicon carbide skeleton. The machining of the parts is good to a few microns and there is an on-board interferometer that continuously measures the internal distances relevant to the basic angle (between the two lines of sight) at the picometer level. It also has an on-board atomic clock. He strongly implied that this is the most challenging thing anyone at EADS has worked on.

van Leeuwen (Cambridge) spoke about spacecraft spin and attitude. He showed that his re-reduction of the Hipparcos catalog (published in book form) came from permitting the spacecraft to be jerked or clanked by configurational changes (related to temperature changes?). He found 1600 such events in the time-stream and when he modeled them, the systematic errors in the data set went down by a factor of five. I commented after his talk that the real message from his talk was not about spacecraft attitude but about the fact that it was possible to re-analyze the raw data stream. The Gaia position seems to be that raw data will be preserved and that re-analyses will be permitted, as long as they don't cost the Consortium anything. That's fine with me.

O'Mullane (ESA) gave a romp through the computer facilities and made some nice points about databases (customizability and the associated customer service is everything). He does not consider it crucial to go with open-source, in part because there is an abstraction layer between the software and the database, so change of vendor is (relatively) cheap. He then went on to say how good his experience has been with Amazon EC2, which doesn't surprise me, although he impressed me (and the crowd) by noting that while it takes months for ESA to buy him a computer, he can try out the very same device on EC2 in five minutes. That's not insignificant. From a straight-up money point of view, he is right that EC2 beats any owned hardware, unless you really can task it 24-7-365.

proposal

I worked on the proposal—and other writing projects—on the airplane and in my room at the ELSA conference on Gaia. Still not far enough along!

too little

I wrote way too little in my NSF Software Infrastructure for Scientific Innovation (NSF's words) grant proposal; it is due on Monday!

NIPS paper submitted

Lang pulled out all the stops and got our NIPS paper submitted, three minutes before deadline. In the process, he found a large number of great examples of situations in which an image-modeling or catalog-as-model approach to the SDSS data improves the catalog substantially. The paper is pretty rough around the edges, but we will tune it up and post in on arXiv, in a few weeks, I very much hope.

trusting robots; SciCoder

I worked on an upcoming NSF proposal (with Lang) on building robots that calibrate (and verify calibration meta-data for) astronomical imaging. The idea is to produce, validate, and make trustable the meta-data available through VO-like data interfaces. Right now an astronomer who makes a VO query and gets back hundreds of images spends the next two weeks figuring out if they really are useful; what will it be like when a VO query returns tens of thousands?

In the afternoon, Demetri Muna (NYU), Blanton, and I discussed Muna's SciCoder workshop, where Muna is going to have the participants learn about databases, code, and data interaction by doing real scientific projects with public SDSS data. I am pretty excited about the idea. Right now Muna is doing some of the projects ahead of time so he is prepared for all the bumps in the road.

perturbed streams

Kathryn Johnston appeared in my office today and we spent a few hours discussing simple calculations of perturbations to tidal streams by CDM-substructure-like satellites. She has greatly simplified the confused thoughts I had about them a year or two ago.

proposing on holiday

On the Memorial Day long weekend here, I outlined Lang and my next NSF proposal.

deconvolution and hierarchical Bayes

I wrote english prose and LaTeX equations like the wind on our brand-new exoplanet project. In particular, I spent some time working out the difference between hierarchical bayesian approaches to distribution estimation and deconvolution or forward-modeling approaches. There is a lot of overlap, but the key difference is that if all you care about is the distribution itself, in the Bayesian approaches you integrate out all of the individual measurements (which, in this context, should be thought of as fits to more "raw" data). That is, if you are deconvolving (forward modeling) you are trying to explain the individual-object fit results; if you are hierarchical Bayesian, you are trying to obliterate them. As I wrote text, Myers wrote code, and Lang (who came into town) worked on image modeling in preparation for the NIPS deadline.

archetypes

Blanton, Guantun Zhu (NYU), and I discussed the possibility of writing a paper on the archetypes system we set up for PRIMUS. The idea of the paper would be to split the archetypes finding and optimization out of any PRIMUS data paper because it has much wider applicability. The idea is to model a distribution of d-dimensional data by a set of delta functions in the d-dimensional space, with the set chosen to be the minimal set that adequately represents every data point. The nice thing is you can choose whatever operation you want to decide what represents what, and it can handle any kind of crazy degeneracies, missing data, or marginalization over nuisance parameters (think calibration, or extinction). The hard thing is that the search for the minimal set of archetypes is hard (in the technical algorithmic sense of the term) but Roweis cast the problem for us as a binary programming task, which is incredibly well handled by any number of open-source and commercial packages. For PRIMUS we used the IBM CPLEX code, which was astoundingly fast.

Spitzer and exoplanets

I spent a big chunk of the day working on the Spitzer Oversight Committee, which is helping the Spitzer Science Center react to funding realities and the slow shut-down of an incredibly productive and successful but finite observatory mission. We spent some time in the meeting talking about Spitzer's capabilities for (and successes in) exoplanet science, and the possibility of encouraging that even more in the future. This post bends the rules but oddly I truly find the work I do on this committee to be of great intellectual interest.