What are the main differences between a language model and a machine translation model?
$\begingroup$ $\endgroup$
1 
- $\begingroup$ cf. Deep Learning "12.4 Natural Language Processing", "12.4.2 Neural Language Models", and passim. $\endgroup$Geremia– Geremia2021-06-22 23:00:06 +00:00Commented Jun 22, 2021 at 23:00
Add a comment |
1 Answer
$\begingroup$ $\endgroup$
The simple language model will give you the probability of a sequence of tokens(sentence) for that language. So lets say if you have trained a model for English language your model can give you the probability for any random english sentence.
Consider some sentence
$X$ $=$ "the quick brown fox jumps over the lazy dog" $=$ $x_1 \ x_2 \ x_3 \ ... \ x_n$
model will give you $P(X)$
Moreover if the model has been trained properly, in the following scenario, where
$X$ $=$ "the quick brown fox jumps over the lazy dog" $=$ $x_1 \ x_2 \ x_3 \ ... \ x_n$
$Y$ $=$ "dog brown quick fox over the jumps lazy the" $=$ $y_1 \ y_2 \ y_3 \ ... \ y_m$
model will always give $P(X) > P(Y)$, as it has learnt the structure of language.
On the other hand Machine translation model gives you the conditional probability of the next token given your source sentence and partial-target sentence. So if
$X = $ "I am a student" and $Y = $ "je suis" $=y_1, \ y_2$
model will give you $P(y_3 | X,y1_,y_2)$ .
where $X$ is a source sentence and $Y= y_1, y_2$ is a partial target sentence. The probability of word/token "étudiant" would be maximum among all words of vocabulary.
