Bayesian bible

On the weekend I finished reading Jaynes, Probability Theory, which is an extremely long and detailed polemic about probability, inference, and decision theory. It was great!

I had many problems with the book, like its dismissal of alternate positions, and its unfair treatment of competing methods in each situation. It had too much bashing of other people. It also had some dumb things to say about quantum mechanics (did Jaynes never hear about Bell's inequalities?). But overall the book was extremely useful to me and clarifying on issues like noise and uncertainty, the subjectivity or objectivity of priors, and the locations where Bayesian methods involve ad hoc input and where they are mechanistic and driven by the rules of probability theory.

Jaynes takes a strong position that probabilities are always descriptions of your knowledge: There might be a definite fact of the matter or not, but what you know about it is probabilistic. He then refuses to use the word probability for anything else! This is a bit crazy, but it appeals to my extreme positions on consistency. So almost anything you or I would call a probability distribution, Jaynes would call a frequency distribution. All this reminds me of my careful use of the words uncertainty and error. As my mentor Neugebauer used to say about our error bars: They aren't errors! If they were errors, we would correct them. They are uncertainties.