dimensionless and coordinate-free?

A lot of talk about Buckingham pi in my world. This is a theorem that says that any dimensional equation in physics with k dimensional inputs can be re-written as a dimensionless equation with fewer than k dimensionless inputs. But this is useless when we think about geometric equations—and many equations in physics are geometric.

Consider, for example, the coordinate-free equation F=ma. This equation has two dimensional vector terms. If we apply Buckingham pi, we get three coupled equations with non-scalar, non-coordinate-free dimensionless ratios. That's terrible, and useless! Can we replace Buckingham pi with something that makes equations that are both dimensionless and coordinate-free?

non-convolutional neural networks

Here's a quotation from an email I sent to Schölkopf (MPI-IS) and Villar (JHU) today:

First, I believe (am I wrong?) that a CNN works by repeating the precisely identical weights for every pixel. So if, in a CNN layer, there are k channels of 3x3 filters, there are only 9k weights that set all the k responses of every pixel in the layer to the 3x3 pixels centered on that pixel in the layer below. The sparsity comes not just from the fact that each pixel in one layer connects to only the 9 pixels below it, but also from the fact that the 9k weights are the same for every pixel (except maybe at edges). That enforces a kind of translation symmetry.

Okay, now, we could make a non-convolutional neural net (NCNN) layer as follows: Each pixel is connected, like in the CNN, to just the 3x3 pixels in the layer below, centered on that pixel. And again, there will be k channels and only 9k weights for the whole layer. The only difference is that at each pixel, a rotation (of 0, 90, 180, or 270 degrees) gets applied and a flip (by the identity or across the x direction) gets applied to the weight maps. That is, every pixel has the same k filters applied but at each pixel, there has been one of the 8 rotation-reflection transformations assigned to the 9k-element 3x3 weight map. This NCNN layer would, like the CNN layer, have 9k weights in the layer, and it would be just as local and sparse as the matching CNN layer.

My conjecture is that the NCNN will perform far worse on image-recognition tasks than the CNN. It is also (fairly) easy (I believe) to build a NCNN from a light modification of a CNN code. Comparison is clean and straightforward. I am ready to bet substantial cash on this one.

First Science Results from JWST, day two

Today was day two of the First Science Results from JWST meeting at STScI. Once again, it was a blast of results from all different fields. Some things I'll think about more going forward include: Something like 3 percent of white dwarf stars show an infrared excess that is consistent with them having a Saturn-like ring system? How did I not know this previously? It makes me want to find a WD with a transiting exoplanet to map the rings and maybe even ring gaps! There is a huge class of red luminous outbursts that appear to be the result of mergers of binary stars (maybe often when one of the binary pair starts to go off the main sequence and engulf its partner). Some of these, for energetic and other reasons, look like they are created not by binary-star systems but instead by star–planet systems. I wonder if the populations can be connected to the population of stars with weird lithium and refractory abundances?

First Science Results from JWST, day one

Today was day one of the First Science Results from JWST meeting at STScI. Today (like all days, I expect) was a barrage of information on different topics, filled with exciting results and systematic errors! I love meetings like this, because it is fun to see people struggling with data they don't quite understand yet. And I can see lots of opportunities for my interests in spectrographs and imagers to be useful in this community. My favorite talks today (unfairly!) were the talks on the instruments and their status. There are some beautiful lens-flare-like artifacts in the NIRISS instrument; that would be a fun problem (for example!). There are insane “snowball” cosmic-ray hits in the NIRSpec data, the likes of which I've never seen before. One nice thing about contemporary NASA: The plan is to make all the calibration pipelines completely open and user-operable, so it is easy to intervene on these data.

towards flexible dynamical models

There are many conversations going on right now in the Flatiron dynamics community about making flexible models for galaxy dynamics. For example, there is work on replacing parametric models with non-parametric basis expansions in various bases. For another, Price-Whelan and I have been trying to think about how one might just image the orbital tori directly with the stellar element-abundance maps. We brought two of these conversations together today, in which Ben Cassese (Columbia) and Danny Horta-Darrington (Flatiron) showed that they are using near-identical forms for data-driven orbit forms in the vertical dynamics of the disk. We also spent a lot of time talking about what constitutes a sensible likelihood function for torus imaging and distribution-function-fitting projects.

can you see the orientation of a star?

Stars don't have uniform surfaces, and they rotate. Can you see the orientation of the star in a single spectrum? Of course the answer is no: You don't have a coordinate system! But if you have some previous spectra of the star, can you establish a rotation period and define an angular coordinate system, and then follow that by taking a new spectrum and saying where the star is in its rotational phase? It looks like the answer to this question might be yes, based on experiments that Lily Zhao (Flatiron) is doing. Of course we don't really care about the stellar orientation. What we care about is capturing or correcting the artificial radial-velocity signals introduces to the data from the rotating, non-uniform surface.

phase and frequency variations

If a star has a (relatively) coherent oscillation mode, and you can monitor it over a long period of time, then orbital motion of the star can be seen as either phase or frequency variations of that mode. I've been working on this in different collaborations, with Dan Hey, with Simon J Murphy, with Abby Shaum (CUNY), and recently with Nora Eisner (Flatiron). Right now, Shaum, Eisner, and I are looking at signal-processing approaches that look like demodulators. What I'm interested in—at least in terms of me learning about signal processing—is how can we make a demodulator that demodulates both phase and frequency simultaneously. There must be hybrid and combined approaches. I'm also interested in what we can measure from incoherent oscillators.

new LIGO events

Today Mathias Zaldarriaga (IAS) gave the NYU Astro Seminar. He told us about work he has been doing to increase the sensitivity of the LIGO data to inspiral events, and how that is impacting beliefs about populations of black holes.

the stability of the vacuum

The research highlight of my day today was our weekly lunchtime blackboard talk, as it often is on Mondays. TOday it was Isabel Garcia Garcia (NYU), talking about the stability of the vacuum. She was specifically talking about the stability of a false vacuum, and specifically when there are large extra dimensions. The weird thing is, in all string-like models for the fundamental particle physics model there are both large extra dimensions and an exceedingly low probability that we live in the true vacuum state. That means a decay to a different state is possible (inevitable?). Why has this vacuum lived so long?

the discussion section of a paper

I spent the afternoon writing the discussion section of my nascent paper with Andy Casey (Monash), about spectrum combinations. My philosophy of the discussion section is: Return to each of the most important assumptions you made (and, hopefully, stated explicitly in some early section), and say what you would do, what you would get, and what you would pay, if you wanted to relax that assumption. I spent a lot of time speaking about spectral variability, which can come not just from the source itself, but from the hardware, from backgrounds, or from data processing issues.

simulating data for phase and frequency modulation

Abby Shaum (CUNY) and I have been working on phase demodulation for binary detection and characterization, using coherent oscillation modes in stellar light curves. We are taking a pure signal-processing approach, which is lightweight and fast, such that we could automatically apply it to everything in Kepler or TESS. We also want to do frequency modulation for incoherent modes (which somehow our people think won't work; won't it?).

Today we discussed how to build fake data to fully test our systems. In the coherent case, this is easy! In the incoherent case this is harder. We discussed simulating it by drawing from a Gaussian Process. And we discussed simulating it by forward modeling a stochastically forced, damped harmonic oscillator.

geometric convolutional networks

Today I had a great meeting with Wilson Gregory (JHU), Drummond Fielding (Flatiron), and Soledad Villar (JHU) about a project to learn partial differential equations from simulation data. This is a toy problem from our perspective, but it is a baby step towards big, real computational problems in physics. Fielding produces training data, Gregory produces geometric convolution networks, Villar proves things, and I cheer from the sidelines.

Our approach is to replace convolutional neural networks with geometric operators that are generalizations of convolutions that know more about geometric forms like vectors, tensors, pseudovectors, and so on. By building methods that use geometric objects responsibly, we automatically enforce coordinate freedom and other deep symmetries.