Writing algorithm formulas using MathJax

Don't use MathJax unless you need to. Here are some hacks to do formatting like this:

• Blockquotes (>). Useful for marking out a block of text as different, without doing much additional formatting to them.

• Inline MathJax. You can enclose MathJax in $ to make it render in line (ie. not on a seperate, centered line. eg. $\alpha$ gives $\alpha$

• Indenting. Number of ways to achieve this. $\quad$ (and similar) might be most familiar to you, but piles of unbreakable spaces (     ) also works as a hack-y way to do this.

• Linebreaks. You can do linebreaks by having multiple spaces at the end of a line, or using the html tag <br/>

To see it in action with your specific example:

Algorithm parameters: step size $\alpha \in (0 , 1] , \epsilon > 0$
Initialize $Q ( s, a ), \ \forall s \in S^+ , a \in A ( s ),$ arbitrarily except that $Q ( terminal , \cdot ) = 0$

Loop for each episode:
$\quad$Initialize $S$
$\quad$Loop for each step of episode:
$\qquad$Choose $A$ from $S$ using some policy derived from $Q$ (eg $\epsilon$-greedy)
$\qquad$Take action $A$, observe $R, S'$
$\qquad Q(S,A) \leftarrow Q(S,A) + \alpha[R+\gamma \max_a(S', a) - Q(S, A)]$
$\qquad S \leftarrow S'$
$\quad$ until $S$ is terminal

Which is produced from:

> Algorithm parameters: step size $\alpha \in (0 , 1] , \epsilon > 0$ Initialize $Q ( s, a ), \ \forall s \in S^+ , a \in A ( s ),$ arbitrarily except that $Q ( terminal , \cdot ) = 0$ > > Loop for each episode: $\quad$Initialize $S$ $\quad$Loop for each step of episode: $\qquad$Choose $A$ from $S$ using some policy derived from $Q$ (eg $\epsilon$-greedy) $\qquad$Take action $A$, observe $R, S'$ $\qquad Q(S,A) \leftarrow Q(S,A) + \alpha[R+\gamma \max_a(S', a) - Q(S, A)]$ $\qquad S \leftarrow S'$ $\quad$ until $S$ is terminal