We demonstrate the property of reparametrization invariance with a simple example on a Bernoulli statistical model. We start with the model \textsf{Ber}(q), which has parameter q. What is its Jeffreys prior? Express your answer as an un-normalized pdf \pi (q) in proportionality notation such that \pi (0.5)=2.

Question

\pi (q) \propto

Now, suppose that we write q=p^{10} and thus wish to calculate the Jeffreys prior on the statistical model parametrized by p instead, i.e. \textsf{Ber}(p^{10}). What is Jeffreys prior? Express your answer as an un-normalized pdf \tilde{\pi }(p) in proportionality notation such that \tilde{\pi }(2^{\frac{-1}{10}})=2^{\frac{1}{10}}.

\tilde{\pi }(p) \propto

Answer 1

To find the Jeffreys prior for the Bernoulli statistical model \textsf{Ber}(q), we start with the definition of the Jeffreys prior as the square root of the Fisher information:

\pi(q) \propto \sqrt{I(q)}

For the Bernoulli model, the Fisher information is given by:

I(q) = \frac{1}{q(1-q)}

Therefore, the Jeffreys prior, denoted as \pi(q), is proportional to:

\pi(q) \propto \sqrt{\frac{1}{q(1-q)}}

Now, to find the Jeffreys prior for the statistical model parametrized by p, i.e. \textsf{Ber}(p^{10}), we need to perform the change of variables. We have q = p^{10}, so p = q^{\frac{1}{10}}.

Using the chain rule of differentiation, we have:

\frac{d\pi(q)}{dq} = \frac{d\tilde{\pi}(p)}{dp} \cdot \frac{dp}{dq}

Since \pi(q) and \tilde{\pi}(p) only differ by a constant of proportionality, we can write:

\pi(q) = \frac{d\tilde{\pi}(p)}{dp} \cdot \frac{dp}{dq}

To find \frac{dp}{dq}, we differentiate both sides of the equation q = p^{10} with respect to q:

1 = 10p^9 \cdot \frac{dp}{dq}

Simplifying for \frac{dp}{dq}, we have:

\frac{dp}{dq} = \frac{1}{10p^9}

Substituting this back into the equation for \pi(q), we get:

\pi(q) = \frac{d\tilde{\pi}(p)}{dp} \cdot \frac{1}{10p^9}

Since \pi(q) = \sqrt{\frac{1}{q(1-q)}}, we have:

\sqrt{\frac{1}{q(1-q)}} = \frac{d\tilde{\pi}(p)}{dp} \cdot \frac{1}{10p^9}

Simplifying, we have:

\frac{d\tilde{\pi}(p)}{dp} = \frac{\sqrt{q(1-q)}}{10p^{10}} = \frac{\sqrt{p^{10}(1-p^{10})}}{10p^{10}} = \frac{1}{10}\sqrt{\frac{1-p^{10}}{p^{10}}}

Therefore, the Jeffreys prior for the statistical model parametrized by p, denoted as \tilde{\pi}(p), is proportional to:

\tilde{\pi}(p) \propto \sqrt{\frac{1-p^{10}}{p^{10}}}

Given that \tilde{\pi}(2^{\frac{-1}{10}}) = 2^{\frac{1}{10}}, we can find the constant of proportionality:

\tilde{\pi}(2^{\frac{-1}{10}}) \propto \sqrt{\frac{1-(2^{\frac{-1}{10}})^{10}}{(2^{\frac{-1}{10}})^{10}}} = \sqrt{\frac{1-2^{-1}}{2^{-1}}} = \sqrt{2}

Therefore, the un-normalized pdf \tilde{\pi}(p) is:

\tilde{\pi}(p) \propto \sqrt{\frac{1-p^{10}}{p^{10}}}