Today Jake vanderPlas (UW) and Boris Leistedt (NYU) gave morning tutorials on Bayesian inference. One of Leistedt's live exercises (done in Jupyter notebooks in real time) involved increasing model complexity in a linear fit and over-fitting. I got sucked into this example:
I made a model where y(x) is a (large) sum of sines and cosines (like a Fourier series). I used way more terms than there are data points, so over-fitting is guaranteed, in the case of maximum likelihood. I then did Bayesian inference with this model, but putting a prior on the coefficients that is more restrictive (more informative) as the wave number increases (or the wavelength decreases). This model is a well-behaved Gaussian process! It was nice to see the continuous evolution from fitting a rigid function to fitting a Gaussian process, all in just a few lines of Python.
No comments:
Post a Comment