1
$\begingroup$

I'm new to reinforcement learning but (I believe) I have certain experience with machine learning. I know if you work on, say, a classification problem, your workflow will be (in a very high level) something like that

  1. clean and prepare data using pandas
  2. do train/test and validation split
  3. import models from sklearn or (keras/torch for neural nets)
  4. train models, tune hyperparameters using say GridSearchCV from sklearn
  5. evaluate models on the test set/ or CV

As you can see, many things are already implemented in ready-to-use libraries. But I am wondering if it also applies to RL problems. I'm slightly aware of the gym library but do I still need to implement each model from scratch or there are libraries that have similar .fit(), .predict() architecture?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

In a typical RL setting, which is model-free RL, you won't have step 1 and 2.

  • Step 1 would be the definition of your environment (e.g. by means of gym) and reward function, which together define the task you want to solve.
  • In RL you usually train and test (and even validate) on the same environment, although you want to change the random seed to experience different episodes. So both step 2 and 5 are a bit ill defined. What you can do during training is to periodically evaluate on your test env, and so determine the best model. Then, after the training is complete, you evaluate again on the same test env to get the final performance. Also it is a good practice to perform $N$ evaluation episodes (not just 1), and then average the metrics.

For doing so there are off-the-shelf RL libraries, like sb3, tensorforce, keras-rl, torchRL, tonic, clearnRL, and so on (there are a lot.)

Moreover, if you want to do model-based or Offline/batch RL you would also work on some dataset (either previously collected, or refreshed from time to time.) If so, within the RL loop you'll have a supervised phase which looks like the pipeline you have just described, but overall is still different.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.