2
$\begingroup$

Let $(X_n)_{n \geq 1}$ be a recurrent irreducible Markov chain on a countable space. Let $a$ be a fixed point and $\tau$ be a stopping time such that almost surely $X_{\tau} = a$. For any $x,y$ let $G(x,y)$ be the expected number of visits to $y$, starting from $x$, for $n < \tau$:

$$ G(x,y) = \mathbf{E}_x \left[ \sum_{0 \leq n < \tau} \mathbf{1}_{X_n=y} \right] $$

We assume that $G(a,a) < \infty$ and want to show that then $G(a,x) < \infty$ for any $x$.

I have no idea how to do this, even if $\tau$ is the stopping time "first visit at $a$".

$\endgroup$
1
  • $\begingroup$ We always have the bound $\sum_{0 \leq n < \tau} 1_{X_n = y} \leq \tau$ so in the case where $\tau$ is the first stopping time at $a$, $G(a, x) \leq \mathbb E[\tau] < \infty$ since $a$ is a recurrent state. However I don't yet have an answer for the general case. $\endgroup$ Commented Oct 18, 2018 at 15:05

1 Answer 1

Reset to default
1
$\begingroup$

Fact: $$ \sum_{n \geq 0} \mathbf{P}_a (X_{n+1}=a, \tau > n) = \sum_{n \geq 0} \mathbf{P}_a (X_{n}=a, \tau > n) $$ This is a little bit counter-intuive because the summands are not equal. This identity follows from the fact that $X_\tau = a$ a.s. starting from $a$. The good thing is that the event $\tau >n $ does not depend on $X_{n+1}=a$.

Now the right-hand side is $G(a,a) < \infty$. Let us expand the left-hand side using Markov property: $$ \mathbf{P}_a (X_{n+1}=a, \tau > n) = \sum_b p_{ba} \mathbf{P}_a (X_n=b, \tau > n) $$ and summing over $n$ we get $$ G(a,a) = \sum_b p_{ba} G(a,b) $$ so for all $b$ such that $p_{ba} > 0$ we get $G(a,b) < \infty$.

More generally, for any $x$ there holds $$ G(a,x) = \sum_b p_{bx} G(a,b) $$ and we conclude, using irreducibility, that $G(a,x) <\infty$ for any $x$.

It seems to me that we didn't use the recurrence but please correct me if I am wrong.

This argument comes from https://www.stat.berkeley.edu/~aldous/RWG/book.pdf Proposition 2.3

$\endgroup$
1
  • 1
    $\begingroup$ Nice first question with self-answer ! $\endgroup$ Commented Oct 31, 2018 at 14:02

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.