In my lexicon, a *model* is an object that has a (possibly null) set of parameters and makes predictions, in the sense that it produces a probability distribution function for the data as a function of those parameters. It also needs a few more things, including a prior PDF for the *nuisance* parameters (it doesn't need a prior PDF for the parameters of greatest interest, in my humble opinion). Given all this, I would have thought that taking, for each data point, the *K* nearest neighbors in the data space (under some metric) is *not* a model. But it *can* be, if you can cleverly convert the properties of the *K* nearest neighbors into a PDF for the data point. For Fadely's calibration job, and at the suggestion of Fergus, I think we *can* do this, and I wrote it up on the airplane home.

Because the model has very few user-tuned knobs and because its complexity and coverage grows linearly with the obtained data, the KNN model is truly non-parametric. In Fergus's terminology, it has high *capacity*: It can model a lot but at the same time isn't obstructed by intractable optimization or sampling.