0
$\begingroup$

I want to train (fine-tune) a seq2seq model to perform the task of rephrasing input following these rules :

1- always follow the pattern "Entity Verb Entity"

2- only use simple sentences : never combine sentences

3- Don't replace existing words

4- Don't lose the overall meaning of the text or any information in it.

For example:

text = "Project Risk Management includes the processes of conducting risk management planning, identification, analysis, response planning, response implementation, and monitoring risk on a project"

Standardized Text = "Project Risk Management conducts risk management planning. Project Risk Management conducts risk identification. Project Risk Management conducts risk analysis. Project Risk Management plans responses. Project Risk Management implements responses. Project Risk Management monitors risk on a project."

Using ChatGPT the results were very good, but I want to know if I can fine tune a model (BERT, T5, any LM) locally, what should be the data format for training such a model, evaluation metrics ?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Data

As it seems like ChatGPT already works well, you could use ChatGPT to generate training data.

Models

I would look at finetuning an existing text paraphrasing model or abstractive summarization model as your task seems extremely similar.

You can find these models on HuggingFace (e.g., this paraphrasing model).

Training

In regard to training, you can take a look at how people train models on other sequence to sequence tasks. e.g., these HuggingFace tutorials on translation and summarization.

Evaluation

For metrics, you should look at how text-summarization papers/paraphrasing papers evaluate their models. For example, this paper looks at embedding similarity, ROUGE, and BLEU.

For your setting specifically, you should also be designing specialized metrics. Look into tooks like Spacy and NLTK to extract, e.g., named entities, parts of speech, and design specialized heuristics to test for your requirements (this SO post for active/passive voice detection might be useful inspiration).

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you so much for this, I'm gonna try these paraphrasing models. I think the challenge is with the rules that should be followed when paraphrasing. Otherwise I will try to train the model on a lot of documents following the rules. if this works, I will accept this answer. Thank you again. $\endgroup$ Commented Oct 1, 2023 at 7:53

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.