how significant is your anomaly?

So imagine that you have a unique data set Y, and in that data set Y you measure a bunch of parameters θ by a bunch of different methods. Then you find, in your favorite analysis, your estimate of one particular parameter is way out of line: All of physics must be wrong! How do you figure out the significance of your result?

If you only ever have data Y, you can't answer this question very satisfactorily: You searched Y for an anomaly, and now you want to test the significance. That's why so many a posteriori anomaly results end up going away: That search probably tested way more hypotheses than you think it did, so any significances should be reduced accordingly.

The best approach is to use only part of your data (somehow) to search, and then use a found anomaly to propose a hypothesis test, and then test that test in the held-out or new data. But that often isn't possible, or it is already too late. But if you can do this, then there is usually a likelihood ratio that is decisive about the significance of the anomaly!

I discussed all these issues today with Kate Storey-Fisher (Stanford) and Abby Williams (Chicago) today, as we are trying to finish a paper on the anomalous amplitude of the kinematic dipole in quasar samples.

divide by your selection function, or multiply by it?

With Kate Storey-Fisher (San Sebastián), Abby Williams (Caltech) is working on a paper about large-angular-scale power, or anisotropy, in the distribution of quasars. It is a great subject; we need to estimate this power in the context of a very non-trivial all-sky selection function. The tradition in cosmology is to divide the data by this selection function. But of course you shouldn't manipulate your data. Instead, you could multiply your model by the selection function. You can guess which one I prefer! In fact you can do either, as long as you weight the data in the right way in the fit. I promised to write up a few words and equations about this for Williams.

kinematic dipole and dust

I had a long conversation with Kate Storey-Fisher (NYU) and Abby Williams (Caltech) about the dipole in the Quaia catalog caused by the kinematic motion of the Solar System barycenter with respect to the cosmic rest frame. Williams has found that the amplitude of the dipole we get depends very strongly on how we account for dust in our sample. There is currently a controversy about the amplitude of the dipole seen in WISE quasars. We now think that it is possible that the measured amplitude is a strong function of how dust is corrected for in the sample? We designed new tests for next week.

elusive quasar dipole

There should be an imprint of the kinematic dipole observed in the cosmic microwave background in any cosmological tracer: The dipole is set by the Solar System barycentric velocity relative to the local Hubble flow, and that same velocity should imprint a dipole on anything cosmological. I have been working on this in part because it is a good measurement to make with Quaia, and in part because the dipole in the quasars is controversial. I have many thoughts, but I will save them for later.

Anyways, Abby Williams (NYU) has been working on making this measurement, and her dipole amplitude and direction depend on what we hold fixed and what we vary (in particular selection-function components), and they also depend on what sky region we use. None of this is surprising; the selection function has a strong dipole in it, and it is not known precisely. But then I don't understand how the studies published previously have such good error bars. Maybe they didn't consider the various different fitting regimes?

wobble, star spots, quasar dipole

[Time to try to re-start this forum.]

I spent this morning on three different small activities. One was giving feedback to Matt Daunt (NYU) who is trying to re-build the wobble concept for stellar radial-velocity measurement in jax. He has annoying optimization issues, which are very hard to diagnose! Optimization is always nasty, in my experience.

Another activity was working on the abstract for Lily Zhao's (Flatiron) upcoming paper on stellar variability in the spectral domain, generated by rotating, spotty stars. She is concerned that the paper is too conceptual. I love conceptual papers! I think science moves forward through concepts and implentations, and no individual paper has to do it all.

My third activity this morning was working through the mathematics on a project of Abby Williams (NYU, Caltech) to measure the kinematic dipole in the all-sky Quaia quasar catalog. There are so many different ways to measure it. I think I have a justifiable likelihood function approach, and one in which we could marginalize out—or profile out—the uncertainties in the selection function we have estimated. It's a controversial subject, so I would like to do things correctly.

Dr Kate Storey-Fisher

Kate Storey-Fisher (NYU) defended her PhD here at NYU today. She killed it! She talked about emulating cosmological simulations (at the level of statistics, not maps), making invariant scalars that encode the shapes and dynamics of dark-matter halos, and her awesome 1.2 million all-sky quasar catalog from ESA Gaia and NASA WISE. It was all things my loyal reader knows lots about but I loved it. It has been an honor and a privilege to work with KSF these years, and I will miss her very very much.

purity and completeness

Today Kate Storey-Fisher (NYU) and I discussed how to estimate the stellar contamination of her Gaia and WISE quasar catalog. Because there are few large, complete samples of anything, it’s hard to do this by comparison with any kind of Ground Truth™. What we realized on the call is that it’s easier to estimate how the contamination *changed* as we went from the Gaia quasar candidate table to our final sample. We discussed how to use what external data we have to estimate this.

Gothamfest

Once a year (and differently every year), we get together as much of the astronomical community in New York City as we can and have them give fast talks. Today was great! I learned a huge amount, and no highlight reel would do. But here are some examples: Amanda Quirk (Columbia) has great data on M33 stars that maybe we could use to build images of the orbital toruses using technology that Price-Whelan and I developed over the last few years? Marc Huertas-Company (Paris) said (confidently?) that many of the star-forming galaxies found by JWST at very high redshift are likely prolate. Michael Higgins (CUNY) and Keaton Bell (CUNY) have a beautiful system to separate sources of variability out in NASA TESS data using structure in frequency space. Kate Storey-Fisher (NYU) showed results from Giulio Fabbian cross-correlating her ESA Gaia quasar sample with the ESA Planck lensing map, with better error bars than any previous survey! Ben Cassese (Columbia) showed a moving-object pipeline with NASA TESS imaging that detects outer Solar System objects, much like old work by Dustin Lang and myself.

the symmetries of the observed universe are different from the symmetries of the latent universe

Kate Storey-Fisher and I spent a long time today talking about how to build a project that is about cosmological observables, built from the concepts in her projects on applying coordinate-free geometric forms to theoretical objects in cosmology. The idea could be: Find geometric scalars that exist in the theoretical (or latent) universe, find geometric scalars that exist in any observational survey of the observable universe, and learn the relationships between these; construct cosmological tests and tests of the dark-matter model. The big issue (from my perspective) is that the symmetries that apply to the 6-dimensional phase space of the Universe are different from the symmetries that apply to the observed 3-dimensional redshift-and-angle-space of the (galaxy or quasar, say) observations. Some might say that there are no symmetries in this observational space, since there are window functions and selection functions, but this is not correct: Coordinate symmetries still exist there, it is just that these other functions must also be tracked, in the same space. Anyway, it's a nice research program to figure all this out.

spherical-harmonic transforms of point sets

On the weekend I computed the spherical-harmonic transform of Kate Storey-Fisher's quasar sample made from ESA Gaia data. I also computed the spherical-harmonic transform of the random catalog we use to map the selection function. The two transforms are extremely similar in their complex amplitudes! Since the random catalog is made assuming perfect homogeneity and isotropy, this similarity directly translates into a measurement of the isotropy of the Universe.

jackknife practice and intuitions

Last week I gave a colloquium at MPIA in which I advocated the use of jackknife and bootstrap resampling to obtain empirical uncertainty estimates in a complex data analysis. Today I actually implemented jackknife in my project on cosmic homogeneity (and isotropy). I jackknifed by sky position: I split the sky into 12 nearly-equal regions for 12-fold leave-one-out. I have intuitions about when it is a good idea to jackknife on a quantity (like sky position) and when it is a good idea to jackknife on a quantity that is completely random, but I don't know exactly where my intuition comes from. In general it must be the case that jackknifing on different things answers different questions about your noise.

do the stars make up a coordinate system?

Long, long ago, when I worked with Sam Roweis (deceased) and Dustin Lang (Perimeter) on locating images on the sky, we used to discuss coordinate systems: You don't actually need a long-lat or theta-phi coordinate system to describe the locations of things on the sky, right? You can just use angular relationships among sources to locate everything precisely and unambiguously! And with that approach, you don't need to make as many choices and standards and lines of code about reference frames. But, alas, this point of view is not in the ascendent.

Not being deterred, I put the bright stars on my maps (from this weekend) of Kate Storey-Fisher's ESA Gaia quasar sample. Can you find the big dipper and Orion? And Sirius?

Lambert's projection

At the Heidelberg Tiergarten Schwimmbad I worked out the mathematics for an equal-area projection of the sphere, centered on the poles. It turns out that I reinvented Lambert's projection from the 1770s. Here's a plot of the Gaia DR3 quasar sample (censored by some dust cuts) in my new projection:

start a paper on homogeneity

After making (yesterday) all the plots that demonstrate the uniformity and large-scale homogeneity of Kate Storey-Fisher's Gaia quasar catalog (which she is writing up now), I decided (tentatively) to write a paper on cosmic homogeneity with these data: When a catalog shows beautiful homogeneity, that is both a statement about the catalog and a statement about the Universe. I wrote a title and abstract and some figure captions today.

the fractal dimension of the Universe is 3

Back around 2004 I promised myself I would never compute a fractal dimension ever again! But I did today, using Kate Storey-Fisher's (NYU) new quasar catalog from the ESA Gaia data. And it turns out that it is 3. Good! Actual measurement with uncertainty coming soon.

new paper scope

We had a great meeting today with Kate Storey-Fisher (NYU), Hans-Walter Rix (MPIA), Christina Eilers (MIT), and me to discuss KSF's progress on the ESA Gaia quasar sample. We looked at her large-scale structure results and her jackknifes and discussed paper scope. Options range from a quasar-catalog paper to a selection-function paper to a full cosmological parameter-estimation paper. Of course we decided to do all three! But importantly we decided that this week we would focus on writing a quasar-catalog paper. That's good, and achievable.

jackknife the sky?

On the weekend, Kate Storey-Fisher (NYU) and I implemented jackknife uncertainty estimation for KSF's cosmology-with-Gaia-quasars project. We jackknifed by cutting the sky into RA slices. This is standard practice but I don't love it! It assumes that you know that your main source of error is calibration or sample consistency over the sky. It might be something way more insidious. In principle I guess you should jackknife over many things, and also randomly.

do we have the baryon acoustic feature?

Today I posted this tweet (below), which I think explains what happened today! I also gave a talk at MPIA Galaxy Coffee, with Adrian Price-Whelan (Flatiron), about the appearance of stellar parameters in the ESA Gaia XP spectral coefficients.

Seen on @katestoreyfish’s monitor rn. Is that a BARYON ACOUSTIC FEATURE in the @ESAGaia DR3 quasar sample?? #GaiaFete pic.twitter.com/0W0PPJd8H8

— David W Hogg (@davidwhogg) July 28, 2022

quasars and stellar density

Kate Storey-Fisher (NYU) has made really nice random catalogs that look very very similar, in sky coordinates, to the quasars we have. However, there is obviously more exclusion of quasars from the Galactic plane region than we can explain with any reasonable model of how dust is affecting things. It's the stellar density of course: ESA Gaia selection is very sensitive to stellar density, especially (as now) when you are using the XP spectra. Today she included the stellar density in the random-catalog regression and boom: Excellent random! Our model is not mechanistic, it is effective and data-driven.

non-separable and generative random catalogs

Standard practice in large-scale structure is to make large-scale structure and cross-correlation measurements using a catalog of tracers (quasars in our case now) with random catalogs taking the role of tracking the selection function. In most cases this random catalog is made by sampling from a model for the angular selection function and, separately, for each object, sampling from a model for the radial selection function (redshift distribution in our case now). But of course the redshift distribution depends, in detail, on the angular selection function (because, for example, some of the angular selection is set by dust extinction). Kate Storey-Fisher (NYU) and I discussed now to capture these issues in the random we are building for the ESA Gaia quasar sample we are using. One idea is to give the randoms quasar-like luminosities and building the random catalog using our causal ideas about how things make it into the catalog