Archive for October, 2010

“Uninformative” priors paradox?

2010 Oct 17 in Uncategorized | Comments (0)

Bayesians, puzzle me this:

If you are trying to learn the p parameter of a binomial distribution, the least informative prior is the uniform distribution, i.e. Beta(1,1)

If you are fitting the same data using a logistic regression, setting your prior to be a logistic distribution with s=1 (the same scale parameter as your generative model) is equivalent to assuming a uniform distribution over the p parameters of the binomial distribution.

But that prior is more restrictive than a logistic distribution with a larger scale parameter, and thus we could make it less informative by increasing the scale. Back in the world of binomial distributions, that would be like a Beta prior with the probability mass pushed up against the edges.

How can informativeness/restrictiveness of the prior be dependent on how you parameterize the hypothesis space?