linear operators in a linear universe

In a very enlightening call with Soledad Villar (NYU), the two of us figured out the minimum-variance unbiased estimator in our toy linear universe. This is a universe in which we get to see noisy (with Gaussian-distributed noise) features X and noisy labels Y, all generated by an unobservable Z and some linear operators. In this toy, linear, Gaussian Universe—which I think is extremely general—what is the best predictor for some new y value given some new x value? We found the minimum-variance unbiased estimator today with a little quadratic programming. This is not fair, however, because the estimator we derived is one you could only construct if you had access to unobservable things, like how the Z space is related to the X and Y spaces. The whole point of this project is going to be that you can't know those things!

Now our goal is to relate this, and other less-cheat-y estimators to what's known as the BLUE: The best linear unbiased estimator (and the subject of the Gauss–Markov theorem). I personally think we can beat the BLUE handily in many situations (Villar is reserving judgement); we are trying to figure out how and when, if I'm right.

Funny thing about this project: We are definitely reinventing wheels here: All this must be known! But we are learning a huge amount, so we are forging on. Sensible? I don't know.