The uniform distribution on a set of size [mathjaxinline]K[/mathjaxinline] corresponds to a categorical distribution with [mathjaxinline]K[/mathjaxinline] categories, where each category has equal probability. Therefore, the parameter vector [mathjaxinline]\mathbf{p}[/mathjaxinline] for the uniform distribution is given by [mathjaxinline]\mathbf{p} = \left(\frac{1}{K}, \frac{1}{K}, \ldots, \frac{1}{K}\right)[/mathjaxinline].
Substituting this into the expression for [mathjaxinline]\sum _{i = 1}^ K p_ i^2[/mathjaxinline]:
[mathjaxinline]\sum _{i = 1}^ K p_ i^2 = \sum _{i = 1}^ K \left(\frac{1}{K}\right)^2 = K \cdot \frac{1}{K^2} = \frac{1}{K}[/mathjaxinline].
Therefore, [mathjaxinline]\sum _{i = 1}^ K p_ i^2 = \frac{1}{K}[/mathjaxinline].
Consider the distribution [mathjaxinline]\text {Ber}(0.25)[/mathjaxinline]. Consider the categorical statistical model [mathjaxinline]\left( \{ a_1, \ldots , a_ K \} , \{ \mathbf{P}_{\mathbf{p}}\} \right)[/mathjaxinline] for this Bernoulli distribution.
If we let [mathjaxinline]a_1 = 1[/mathjaxinline] and [mathjaxinline]a_2 = 0[/mathjaxinline], then this corresponds to a categorical distribution [mathjaxinline]\mathbf{P}_{\mathbf{p}}[/mathjaxinline] with parameter vector [mathjaxinline]\mathbf{p}[/mathjaxinline] given by...
[mathjaxinline]0.25[/mathjaxinline]
[mathjaxinline]0.75[/mathjaxinline]
[mathjaxinline][0.25~ ~ 0.75]^ T[/mathjaxinline]
[mathjaxinline][0.75~ ~ 0.25]^ T[/mathjaxinline]
correct
Let [mathjaxinline]a_ i = i[/mathjaxinline] for [mathjaxinline]i = 1, \ldots , K[/mathjaxinline]. The uniform distribution on [mathjaxinline]E = \{ 1, 2, \ldots , K\}[/mathjaxinline] can be expressed as a categorical distribution [mathjaxinline]\mathbf{P}_{\mathbf{p}}[/mathjaxinline] for some choice of parameter [mathjaxinline]\mathbf{p}[/mathjaxinline].
What is [mathjaxinline]\sum _{i = 1}^ K p_ i^2[/mathjaxinline]?
1 answer