causality

Schölkopf gave the second of his NYU Courant Lectures today, this time on inferring causality from observed data (data in hand, not intervention-based experiments). All of the ideas are built around concepts of conditional independence. In some cases these are the kinds of independences that are represented by graphical models; in some cases these are new kinds of independence ideas that compare distributions to features in functions that transform those distributions. He showed some simple cases where he can infer causality when we know the direction of causality. After his talk I proposed that we find some places in astrophysics where causality is debated and test them. I encourage suggestions from the peanut gallery.

One thing that Schölkopf has convinced me of in the last year or so is that the reason XDQSO beats SVM or simpler likelihood approaches to separating quasars from stars is that it contains a causal model of the noise process affecting the data. I kept thinking about it as being better because it is generative, but I was wrong: There are lots of ways to be generative that do not help in dealing with changing noise properties. It is really the causal prior information that has been injected into XD that makes it so good.