Tukukkk

Suppose we have a point mass prior,

$$theta sim begin{cases} I(theta=1) ,& prob=frac{1}{2} \
Gamma(c,c), & prob=frac{1}{2} end{cases}$$

Then if we are asked

$lim_{c to infty} P(theta=1)$

Now here is the issue, since Gamma is a continuous distribution, it seems that in the case of gamma,we will never have $theta=1$.

To me it thus seems that regardless of the value of c, the $p(theta=1)=frac{1}{2}$

However, we also have that since expected value of a $gamma(a,b)=frac{a}{b}$ so that the expected value of the gamma is 1 when we have $a=b=c$

But, by Markov, for $X sim Gamma(c,c)$

$lim_{c to infty} Pr(|X-mu| lt epsilon) to 1$ for any $epsilon gt 0$

So is $lim_{c to infty}P(theta=1) =1$ , or is $lim_{c to infty}P(theta=1)=frac{1}{2}$

As even though the markov inequality holds, it is a continous distirbution, so we will never have it exactly equal to 1.

Thanks all

Your specified prior distribution is a mixture of a continuous and discrete part. Let $I sim text{Bern}(1/2)$ be the indicator that the parameter is taken from the gamma distribution. Then using the law of total probability you have:

$$begin{equation} begin{aligned}
mathbb{P}(theta = 1|c)
&= mathbb{P}(theta = 1|c, I=0) cdot mathbb{P}(I=0) + mathbb{P}(theta = 1|c, I=1) cdot mathbb{P}(I=1) \[6pt]
&= frac{1}{2} cdot mathbb{P}(theta = 1|c, I=0) + frac{1}{2} cdot mathbb{P}(theta = 1|c, I=1) \[6pt]
&= frac{1}{2} + frac{1}{2} cdot mathbb{P}(theta = 1| theta sim text{Ga}(c,c)) \[6pt]
&= frac{1}{2}. \[6pt]
end{aligned} end{equation}$$

(Note that the last step comes from recognising that the gamma is a continuous distribution, so the probability of a specific point is zero under this distribution.) This result holds for all values of $c$, so you are correct that it also holds in the limit:

$$lim_{c rightarrow infty} mathbb{P}(theta = 1|c) = lim_{c rightarrow infty} frac{1}{2} = frac{1}{2}.$$

Your later use of Chebychev's inequality shows that as $c rightarrow infty$ you get $theta rightarrow 1$ (convergence in probability), but this does not change the fact that $mathbb{P}(theta = 1|c) = 1/2$ for all $c > 0$. (To understand the reason for this more fully, have a read about the distinction between convergence in probability and almost-sure convergence.)