2
$\begingroup$

In computer vision, if we don't have a large training set, a common method is to start with a pre-trained model for some related task (e.g., ImageNet) and fine-tune that model to solve our problem.

Can something similar be done with natural language processing problems? I have a boolean classification problem on sentences and don't have a large enough training set to train a RNN from scratch. In particular, is there a good way to fine-tune a LSTM or 1D CNN or otherwise do transfer learning? And, if we want to do classification on sentences, is there a reasonable pre-trained model to start from?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

This paper might be useful....

https://arxiv.org/abs/1801.06146

Improve this answer
$\endgroup$
1
1
$\begingroup$

Consider few-shot learning approaches, recently found guys that practice such approaches for their information extraction models, claiming you will need somewhere between 8 training examples. But it depends on the cases you are looking to train and the expected satisfied precision. https://huggingface.co/knowledgator

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.