cosmology with imperfect supernova data

Alex Malz (NYU), Fed Bianco (NYU) and I discussed a possible hierarchical model for doing supernova cosmology in the face of uncertain supernova classifications and start times, which will be generic in time-domain surveys that are sparse or not designed with supernovae in mind. We started with a pretty complicated graphical model, but we were able to pare it down and then a bit more, and finally got to something awesomely simple: If we represent all supernova types with a bag of templates, and each has some prior pdf for peak brightness, then we can simultaneously model the cosmological parameters, the type probabilities, the peak-brightness distributions, and the photometry of every supernova, no matter what the bandpasses and cadences. The cool thing is that this project can (almost) be assembled from off-the-shelf parts, in the form of sub-systems of other supernova projects. And the project should create new opportunities for projects and legacy data sets.

my practice is unethical?

Today begins a few-day vacation, but I was still working in the morning: Before we left the undisclosed location of the #dsesummit, I had a conversation with Ariel Rokem (Berkeley) and Josh Greenberg (Sloan Foundation) and Jake Vanderplas (UW) and others about developing and writing out in the open: This behavior is what I do, but it is antithetical to double-blind reviewing and other kinds of referee privilege. Since those models (and especially double-blind) are ethical models (that is, they are predicated on a set of ethical principles), could it be that my work-in-the-open practice is thereby unethical? I had this to think about on the bus back to NYC.

While not wracked with existential dread on the bus, I wrote notes and issues for our paper on Hack Weeks. There is so much to say in this document, and it has several audiences.

#dsesummit, day 2

My day started with a long breakfast conversation with Yann LeCun (NYU) about adversarial methods in deep learning. In these methods, a generator and discriminator are trained simultaneously, and against one another. It is a great method for finding or describing complex density functions in high dimensions, and people in the business have high hopes. In particular, it is crushing in image applications. We discussed the problem that is currently on my mind, which is modeling the color–magnitude diagram of stars in Gaia, using one of these adversarial systems, plus a good noise model for the parallaxes. I would love to do that, and it should be much easier than the image problems, because the data are much lower in dimensionality.

I ran a very amusing session at the Summit, in which we had participants bring figures and we crowd-sourced a reaction, critique, and to-do list for each of them. We looked at a figure from politics from Michael Gill (NYU), making a causal claim about regulations and how meeting minutes are kept, a figure from geophysics from Nicholas Swanson-Hysell (Berkeley) showing the data and a model for polar wander, and a figure from neuroscience from Bijan Pesaran (NYU) showing brain region classifications. The feedback from the group was great and useful and constructive (though not always polite; my apologies!). One theme of our discussion ended up being consistency across figure elements. I feel like this crowd-sourcing session was a model for future sessions; it would even be fun to make this a regular event in some forum in NYC.

There was a lot of non-research today, but in the remainder of my research time, I worked on outline material for our growing paper on Hack Weeks.

#dsesummit, day 1

I'm at the Moore-Sloan Data Science Environments annual summit. Much of what we have been doing doesn't exactly count as research, by my (constantly weakening) standards. However, there was an absolutely great and wide-ranging discussion of Hack Weeks and Sprints and their role in education and scientific investigation. This led to a group of us committing to start a paper on the subject (not a white paper, but a paper). The just-started draft is here, and we accept pull requests.

There were some great lightning talks at dinner time. My personal favorite was Kellie Ottoboni (Berkeley) talking about the finiteness of the state space of random number generators. She (with Stark and Rivest) is looking at the possibility that there are random number generators possible with an infinite state space, capitalizing on the ideas around cryptographic hash functions. She sowed some (deserved) fear about using a 32-bit random-number generator in combinatoric contexts. Since our own emcee makes combinatoric choices, this could conceivably be relevant to our master branch!

#GaiaSprint, day 5

Today was the final day for the Sprint, and included an incredible wrap-up. My best way to communicate the awesome is just to link out to the final wrap-up slides, which we all edited simultaneously. Each participant was permitted one slide, and we worked through the full crowd (one presentation each, and questions) in a few hours. Amazing things were accomplished this week, and I anticipate multiple papers submitted to the refereed literature. My own work was on vertical heating of the Milky Way disk, measurement of the disk mid-plane location and tilt (yes, I think we have a result), and the metallicities of co-moving star pairs. The day ended with a short talk by Jim Simons (Simons) who told us about his plans for the CCA and his other centers for computational science.

#GaiaSprint, day 4

Today was another impressive day at the Sprint. Jonathan Bird (Vanderbilt) got together a break-out session to talk about low-hanging projects in Gaia DR1 that no-one is currently doing, just to record ideas and inspire conversation. That led to this impressive list! Not everything on that list is low-hanging (and not everything in this telegraphic document is really comprehensible), but there are lots of Gaia projects that could be done right now.

Meanwhile, Adrian Price-Whelan (Princeton) noticed that thousands (yes, thousands) of the co-moving stellar pairs found by Semyeong Oh (Princeton) and us have both members observed by RAVE-on. He started making plots of their differences in velocity and abundances. It looks like there are some interlopers (more than we expect from naive contamination estimates), but a big core of pairs that have both identical velocities and identical abundances. Exciting! Now if only we can convince Keith Hawkins (Columbia) to measure detailed abundances...?

In the afternoon, Jackie Faherty (AMNH), David Rodriguez (AMNH), and Brian Abbott (AMNH) came to show us a visualization tool with the TGAS data uploaded. The most fun visualization was the one that runs the clock forwards and backwards on the proper motions! They are also looking forward to putting Gaia data on the dome of the Rose Center Planetarium!

In the evening check-in, there were some impressive results. Doug Finkbeiner (Harvard) showed us his pip-installable and software-operable tools (built with Greg Green) to access the 3-d dust map built from the PanSTARRS data. Jason Sanders (Cambridge) compared age–velocity relationships expected from toy models with that observed in the TGAS+RAVE data, where he estimates ages using isochrone fitting and photometry. He finds heating at very short ages, which is apparently not surprising. Dan Foreman-Mackey (UW) showed fits that he and Tim Morton (Princeton) have been doing to get better parameters for exoplanet host stars and the input catalog to the Kepler mission. They are literally doing the entire input catalog, because this is necessary for populations studies. One thing they find is that some conclusions about planet insolation (think: habitability) will change in this era of Gaia.

I mentioned PanSTARRS above, but I should note that Finkbeiner could not actually work on the PanSTARRS data at the Gaia Sprint, because we had rules about open-ness and data sharing, which you can read on the meeting page. I can't adequately say just how appreciative we all are of the Gaia DPAC teams for making their data public. I should also say how appreciative we all are of the other surveys and collaborations and tool builders who make their data and software public for us all to use. Of course the data and tool releasers benefit from these releases enormously, but these releases also require a certain level of bravery, honesty, and time commitment; it isn't easy.

#GaiaSprint, day 3

(As usual, these blog notes are only biased, imperfect, personal highlights. They are not minutes of the meeting in any sense!) Anthony Brown (Leiden) kicked off the day by comparing the all-sky image of the Gaia TGAS catalog with the all-sky image of the stars that Gaia uses to set its attitude. This latter catalog is close to a random sampling of stars, so it makes a beautiful all-sky image.

Yesterday's check-in meeting continued this morning with Bovy showing the Oort constants. He claimed that he needed something to do while his data files unzipped, so he decided to measure the Oort constants, including constant C, which he claims has never really been measured before! This continues the theme of the awesomeness of the Gaia data: You measure things that have never before been possible while your files are unzipping. Bovy also gave us a tiny reminder of what the Oort constants are. Years ago, Bovy and I (more-or-less) failed to measure these constants in the SDSS data.

Daniel Michalik (Lund) came in by phone to tell us about the construction of the TGAS Catalog, and Alcione Mora (ESAC) told us about the Gaia Archive and how to use it. In Michalik's talk I was reminded that there are two small circles on the sky (small as in not great) where there will be close to 200 observations per star; these are great places to concentrate observing programs: Why wait to after Gaia to do the follow-up observing on the amazing time-domain astrophysics that will be discovered in those sky regions.

I spent my sprinting time working with Price-Whelan on the mid-plane of the Milky Way disk, with Bird on the age-velocity relationship, including a generative model for the ages, and with Ness on the causal relationships between metallicity, age, and vertical kinematics. On the latter, the quesion is: Is heating “caused” by age or by metallicity? (Or maybe some more sophisticated question than that.) The answer seems to be that in some parts of abundance space it is clearly age, and in others it is clearly metallicity. I hope this holds up!

At the evening check-in session, Ruth Angus (Columbia) showed that, of Semyeong Oh's comoving pairs of stars that both have gyrochronology ages, they seem (usually) to show the same-ish age. It is early, but it looks like a possible confirmation of the effectiveness of the gyrochronology, possibly in parts of the H-R diagram where it hasn't been well tested previously.

After that, Vasily Belokurov (Cambridge) blew us all away by punking the Gaia DR1 uncertainty model to find time-variable sources in the billion-star catalog. He then found a bridge of variable stars connecting the LMC to the SMC! That made me afraid, very afraid.

#GaiaSprint, day 2

It is only Tuesday, and yet there are already incredible results flowing in from the Gaia Sprint. I won't do justice in any way to what I saw today, but here are a few very personal highlights from the day. Although everyone spent the day working—the Sprint has almost no formal program—these results are from the morning and evening check-in discussions:

In the morning check-in, Branimir Sesar (MPIA) showed us the results of a hierarchical model of the RR Lyrae stars in TGAS, where he simultaneously fit for the period–luminosity relation parameters, and also parameters of a flexible model for the noise (bias and variance) in the Gaia parallaxes. He confirms the Gaia noise model and gets absolutely beautiful parameter constraints. That was a pretty good result for one day of work!

Sven Buder (MPIA) and Johanna Coronado (MPIA) used dynamical actions computed with the help of Wilma Trick (MPIA) and Jo Bovy (Toronto) to investigate the heating mechanisms in the Milky Way disk. They can clearly show that stars that are older (at least according to spectroscopic parameters and stellar models) have larger vertical actions. This was nice, but they have really beautiful gradients in vertical action across the red clump, consistent with the expected gradient in age across the red clump. This suggests that their stellar labels (from GALAH and LAMOST, respectively) and stellar models and Gaia kinematics are all consistent. Crazy! And beautiful. Is the vertical action the new age estimator?

Semyeong Oh (Princeton) showed her results (with also Price-Whelan and Spergel and me) on co-moving pairs of stars, and their locations in the color-magnitude diagram. This led to a lot of discussion about what can be concluded and what can be predicted. In particular, we expect no old stars (like no red giants, even) for the widest-separation co-moving pairs.

Sergey Koposov (Cambridge) showed us results from an insane and massive project to measure proper motions from the comparison of the SDSS imaging to the Gaia billion-source list. This project involved a complete recalibration of SDSS astrometry! His proper motions look great, and he is using them to search for substructure and analyze Milky Way structure. A simply insane project. All the more insane, because his catalog will be superseded by Gaia at the next data release in a year! And I mean “insane” in the best possible way.

With a tiny bit of consulting from me, Jonathan Bird (Vanderbilt) converted his project to measure the age–velocity relation (the vertical velocity dispersion in the disk as a function of stellar age) into a generative model for the ages. This isn't working yet, but he showed results for the relation when he assumes that the ages are God's truth. This project is one he was working on for a long time with GCS data, but with the TGAS data he obviated all his previous work in one single day. Damn, I love good data.

#GaiaSprint, day 1

Today was the first day of the NYC Gaia Sprint, with 50 participants from around the world. I had an absolutely great research day. The meeting began with a set of pitches, one per participant, that included an introduction, a statement of expertise (what that participant brings to the meeting), and a statement of goals (what that participant hopes to take home from the meeting). Pitches were all over the place: Milky Way disk and halo, testing stellar models, exoplanet science, calibration, target selection, future missions, you name it! This session took two hours. But that pitch session was the entirety of the formal program of the 45-hour meeting! That is, everyone is just supposed to work from here on. Of course we will have break-out sessions, and informal discussions, check-in and wrap-up sessions, and lots and lots of co-working. But that was it.

I started working with Boris Leistedt (NYU) on modeling a slice of the color-magnitude diagram of stars, to build a data-driven photometric distance indicator (that will beat the parallax for most TGAS stars). I also started working with Adrian Price-Whelan (Princeton) on his discovery (this morning!) that the TGAS Catalog contains the most precise measurement of the Milky Way disk midplane ever. That displaced some of our plans for running the clock back on disrupting binaries and associations.

We had two break-outs, one on likelihood formulations for doing inference with parallaxes, and another on data quality and data issues in the Gaia DR1 data sets. This latter talk was by Anthony Brown (Leiden), who is the chair of the entire Gaia DPAC data processing effort. I learned a huge amount in both of these break-outs about the noise model for the TGAS parallaxes, which I ought to be using in my own inferences.

One thing we have done in this meeting—which is standard practice for me at scientific meetings now—is open a shared, editable web document to record notes. By mid afternoon this document was more than 20 pages long, filled with crowd-sourced notes about pitches, projects, data sets, and software tools. We will preserve and publish these notes after the meeting in an informal form. One of the big outcomes of this meeting could be some standard tools, standard data sets, and advice about how to use these to do reliable science. Thinking about that as we continue to hack on the data.

GALEX, Gaia, and MCMC

Early in the morning, I met with Dun Wang (NYU), Steven Mohammed (Columbia), and David Schiminovich (Columbia) to discuss our GALEX imaging of the Galactic Plane. We gave Wang and Mohammed tasks of writing titles and abstracts for their papers on the subject. Also, Mohammed showed us his exploration of the GALEX–TGAS match, which looks like it is filled with good stuff.

In the afternoon, Dan Foreman-Mackey (UW) and I met to discuss exoplanet results, where Foreman-Mackey has new results on multiplicity based on ABC inference. We followed this with parallel work on our Data Analysis Recipes tutorial on MCMC inference. We re-organized some of the content, reduced scope very slightly, and tried to close issues.

I also worked on posterior samplings for star distances, given parallaxes. I am using Simple Monte Carlo, with two techniques, one that works well for high signal-to-noise parallaxes, and one that works well for low signal-to-noise. The issues are very subtle; a uniform-density prior has a lot of very bad properties in parallax space. I got something working and posted a gif on the twitters.

Simple Monte Carlo

I worked in the morning to build a custom (Simple Monte Carlo) sampler that samples the posterior pdf for the true parallax given a noisy parallax measurement and a sensible but interim distance prior. The problem is very ill-behaved; for many (most, even) TGAS stars, there is almost no support for the likelihood under the prior and vice versa. In related news, in the evening, I worked on Adrian Price-Whelan and my paper on a custom (Simple Monte Carlo) sampler that samples orbital parameters for single-line binary-star systems. I raised many issues and edited the text directly.

stellar parameters; machine learning in cosmology

A research-filled day started with a discussion with Vakili about final changes to his paper on centroiding compact sources. We are responding to a constructive and useful referee report. The day ended with me sending a long email out to the #GaiaSprint participants with their homework assignments, some of which are pretty non-trivial!

At stars group meeting, we heard from Tim Morton (Princeton), who has been building a system to get the best possible stellar parameters (radii, densities, distances) for exoplanet (and binary-star) host stars, given all available data. His system is very flexible in what can be used to constrain the system: photometry, spectroscopy, asteroseismology, and astrometry. What I was even more impressed with is its handling of binary stars and more complex hierarchies of stellar systems: You can have some photometry that constrains the sum of the star brightnesses, and other photometry that constrains the difference. And you can fit the binaries fixing the ages and metallicities to agree. That makes his code very useful for unresolved and marginally resolved binaries, which are always a nuisance when you want to fit models.

At cosmology group meeting, Tjitske Starkenburg (CCA) and Lauren Anderson (CCA) reviewed these two papers by Kamdar, Turk, & Brunner about using machine learning to model the outputs of cosmological simulations. The matters of greatest interest to the group were relegated to a short appendix of the first paper! These papers don't directly solve anyone's current problems, but they represent a start for using machine learning in cosmology. We closed the meeting with a discussion about where we might most productively point traditional machine-learning techniques towards unsolved problems in cosmology. Our ideas were about training on simulations but applying to real data: Maybe we could infer the (unobserved, latent) dark-matter properties given the observed galaxy properties. Or maybe we could use ideas from ML to find better statistics (that is, summary statistics from a galaxy survey) for constraining cosmological parameters.

binary stars; deconvolving the color–magnitude diagram

The day started with a long call with Adrian Price-Whelan (Princeton) regarding binary-star systems, their detection and analysis. We discussed the oddities we are finding in Gaia TGAS. This led to a call later in the day with Semyeong Oh (Princeton), who showed us really nice results. We definitely see the Gaia DR1 exposure map in the sky positions of our detected binaries, but this is as expected: We are more sensitive where we have better data! I am starting to think that we have a reliable catalog. Also on that call we found this highly relevant paper from the Hipparcos era.

Later in the day, Dustin Lang (Toronto) and I discussed my project (which is constantly shrinking in scope) to build a data-driven, noise-deconvolved model of the color–magnitude distribution of stars in TGAS. He endorsed the very simple project and had some good ideas. I now think I might actually have a do-able project for the #GaiaSprint. It also fits in to things that Lang and I have been saying for years, about shrinkage and hierarchical thinking.

supernovae in Kepler

Peter Garnavich (Notre Dame) gave our astrophysics seminar about supernovae and other oddities in Kepler and K2 time-domain data. His lightcurves of supernovae start very early and show beautiful features and regularities. He can confidently classify supernovae on the basis of the light curve alone, it looks like to me! He showed some odd things which might be (the elusive) SN 0.1a. A great re-use of #OtherPeoplesData by an admirable #ResearchParasite!

#GaiaSprint brain-storming

I had great Gaia DR1 and Gaia Sprint brain-storming sessions today with Adrian Price-Whelan and Dan Foreman-Mackey. Price-Whelan proposed that we “run back the clock” on the TGAS stars to see if any pairs emerge from disruption events in the past. I commented that if we go to short times in the past, you need neither distances nor radial velocities: The angular motions suffice. It must be that as the time scale gets longer, distances and radial velocities matter more and more; it would be nice to see this come in continuously. We left that as a conceptual to-do, but we worked out a plan that ought to work in general. This project has a lot in common with the Kinematic Consensus projects that Hans-Walter Rix (MPIA) and I have worked on in the past.

I described to Foreman-Mackey some of my ideas about a data-driven model of the color-magnitude diagram. I have this idea of transforming the space to one in which the noise model is Gaussian, but that kind of thing always makes me feel dirty. We discussed the possibility that I could make my model only one-dimensional (in the parallax direction) by cutting the data into tiny color boxes. That's crazy, but the Gaia Sprint is a hack week, where experiments reign and we want to make things work. So maybe I will write that down this weekend.

carbon stars, data-driven models, simulating faint galaxies

In the morning I hosted our weekly stars meeting at the CCA. Jill Knapp (Princeton) came in! She talked to us about carbon stars, why they are interesting, where they are found, and what we might learn about them from Gaia DR1. We proposed that we could get her results pretty fast! Kathryn Johnston (Columbia) talked to us about the use of stellar models to get very precise distances to all kinds of stars, which triggered many conversations. One is whether and how we could make sure we have such technology up and running at the Gaia Sprint in 1.5 weeks. Another is whether we could get percent-level distances to stars without using stellar models. That is part of my evil plans.

In the afternoon I hosted our weekly cosmology meeting. Lauren Anderson (CCA) talked about her matched dark-matter-only and baryonic (SPH) simulations, and how she is unsatisfied with how they understand the faint end of the galaxy population. How can we understand the completeness of the simulations, or get the most out of the galaxies that are at the low-mass end? We discussed this and the conversation edged into halo-occupation territory. I then started saying crazy stuff about machine learning and everyone quietly left the room, claiming other commitments and, oh my, look at the time!

JHU, chemical tagging; binary star relative velocities

I had a great visit to JHU today, with extensive conversations with JHU locals Ali-Haimoud, Wyse, Szalay, Kuntz, Menard, Tollerud, van Velzen, Nataf, and visitor Dalya Baron (TAU). With Wyse I caught up after many years (a decade?); we discussed chemical tagging, radial migration, and stellar spectroscopy. I gave a late-afternoon talk on The Cannon and its use in measuring stellar parameters in an interactive forum. I also had a great chance to talk flat-fielding with the HST calibration team, who are among the world's best calibrators.

Before all that, I did more pair coding with Price-Whelan (Princeton) on the likelihood ratios, and we had a realization: If we are looking for binary stars, we do expect finite, detectable velocity differences as the separation between the pairs of stars gets small. He implemented this before the day was out and the results look very good.

I've got 99 problems, and every one of them is a units conversion

Today I spent some research time pair-coding (yet more) with Price-Whelan the fully marginalized likelihood ratio test we are using for our comoving-stars project. There are subtleties! Every time we look at the code we find bugs and think-os. I am reminded of the point that we generally find bugs continuously and with no sign of slowing until we are happy that the code seems to pass our unit and functional tests, at which point we stop looking. That doesn't mean the code is correct! A corollary: Almost no-one has ever wasted time testing code. It turns out that one of the most troublesome parts of this project is units conversion. Not surprising, really, when we have AU, pc, km/s, mas/yr, the inverse squared of these (in inverse variance matrices) and many, many more.