Search Results

It is a finite MDP with states represented as 6 dimensional vectors of integers. The number of discrete values in each index of the state vector varies from 24 to 90. The action space varies from sta …

The wikipedia formulation does indeed show you a better view of how the update rule for action values is constructed: $$Q(s_t, a_t) \leftarrow (1-\alpha)\cdot Q(s_t, a_t) + \alpha\left[ r_t + \gamma …

As you are building policies in simulation, and can avoid the need to use approximate methods (the state space is small enough to fit in a table in memory), then your goal is to converge on the optima …

1) With an on-policy algorithm we use the current policy (a regression model with weights W, and ε-greedy selection) to generate the next state's Q. Yes. To avoid confusion, it may be better to u …

Since the convergence of QLearning is so slow I am wondering if it is possible with QLearning to interpolate the QValue of unexplored states since QLearning does not use a model? When Q learning …

The book you are reading is being somewhat lax with terms. It uses the terms "actor" and "critic", but there is another algorithm called actor-critic which is very popular recently and is quite differ …

It is just a type of namespacing, because $a$ is already assigned the chosen action. There are two contexts of action being considered in the equation, so there needs to be a symbol for each context. …

what is $R_a(s,s')$ ? In this case, it appears to represent the expected immediate reward received when taking action $a$ and transitioning from state $s$ to state $s'$. It is written this way so …

You have a couple of mistakes around assigning reward, and the update mechanism. You intend to grant 0 reward for a loss, 0.5 reward for a tie and 1 reward for a win. And you place those rewards as f …

if the length and height of the rectangle are random, as well as the starting position and the location of the Treasure, how can the bot apply the knowledge acquired to the new problem? You have …

How should I interpret this? If a lower loss means more accurate predictions of value, naively I would have expected the agent to take more high-reward actions. A lower loss means more accurate p …

Any exploration function that ensures the behaviour policy covers all possible actions will work in theory with Q learning. By covers I mean that there is a non-zero probability of selecting each acti …

The key part of the quoted text is: To perform experience replay we store the agent's experiences $e_t = (s_t,a_t,r_t,s_{t+1})$ This means instead of running Q-learning on state/action pairs as …

You are effectively implementing $\epsilon$-greedy action selection. The usual way to represent this in RL, at least that I am familiar with, is not as a "threshold" for probability of choosing the …

This update scheme: Q(s,a) += reward * gamma^(inverse position in game state) has a couple of problems: You are - apparently - incrementing Q values rather than training them to a reference targe …