Questions tagged [markov-process]
A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space processes.
2,669 questions
4 votes
1 answer
74 views
Optimal strategy for combinatorial marble-drawing game
You have $a$ amber, $b$ bronze, and $c$ crimson colored marbles in your hand, with $a\geq b\geq c$. An exact copy of this set of marbles is in a bag. Every turn, you select a marble from your hand to ...
- 2,324
0 votes
1 answer
65 views
Understanding the effect of idempotence on mixing in a Markov chain
I'm trying to understand how to get mixing time results when transitions in a chain are idempotent i.e. $P^2=P$ for transition $P$. Following is a simplified example to illustrate my confusion. Toy ...
- 13
2 votes
1 answer
154 views
A process with independent increments is a Markov process
Our professor gave us the following proof (of the statement in the title): $\mathbb{E}[\varphi(X_T) | \mathcal{F}_t] = \mathbb{E}[\varphi((X_T - X_t) + X_t) | \mathcal{F}_t] = F(X_t)$ where $F(x) = \...
- 409
2 votes
2 answers
91 views
Are inter visit time stopping times?
We work with canonical Markov chain taking value in $E$. I.e. For any transition $Q$, and initial distribution $u$, there exists a $P_u$ such that the coordinate process is $(Q ,u)$ Markov chain ...
- 1,303
-1 votes
1 answer
39 views
A Seeming contradiction in definition of a transient state in Markov chain
My book says that by definition of a transient state, i, in Markov chain, Let (the first return time) $T_i=\inf\{n \ge 1: X_n=i \mid X_0=i \},\space\space then $ $\space P(T_i < \infty \mid X_0=i) &...
- 205
3 votes
1 answer
85 views
Markov property for a Markov process
Let $(E, \mathcal{E})$ be a polish space, and let $\mu$ be a probability measure on $E$. We say that $p=(p_t)_{t\in\mathbb{R}_+}$ is a transition function if $p_0(x,A)=\delta_x(A)$, each $p_t$ is a ...
- 590
1 vote
0 answers
35 views
How to determine perturbation that maximizes the spectral gap and distance of a Markov chain
Suppose we are modeling a real world process with a Markov chain. Our best estimate of the system is $M$, but accept any matrix in some neighborhood of $M$. Is there an efficient algorithm that ...
- 327
0 votes
1 answer
81 views
Markov Chain and proof for criterion for recurrence
I am studying the proof that a state $i$ for a discret homogeneous Markov-chain $(X_n)_{n\in \mathbb{N}}$ is recurrent if and only if the series $\sum_{n\geq1} P^{(n)}_{ii}$ is divergent where $P^{(n)...
- 75