Word Embeddings

[securetorobert]

This is the first version of our Keras embeddings tutorial.

https://colab.sandbox.google.com/github/securetorobert/docs/blob/master/site/en/tutorials/keras/intro_word_embeddings.ipynb

[googlebot]

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored or co-authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this state. It's up to you to confirm consent of all the commit author(s), set the cla label to yes (if enabled on your project), and then merge this pull request when appropriate.

[MarkDaoust]

Please remove the .ipynb_checkpoints/ directory from the PR.

I'd prefer to avoid the use of Flatten because it ties your model to a specific sequence length. The model has to learn the meaning of the embedding at each location.

I would use a GlobalAveragePooling, an RNN, CNN, or an attention module to squeeze the sequence to a fixes size and enforce the idea that the absolute position of a word in a sequence doesn't matter (much).

There is a lot of overlap between this and basic_text_classification.ipynb. The new content seems to be mainly be the setup to use the tensorboard embedding projector. Is there a way we can put more emphasis on that, and reduce the duplication? should this maybe a sub-section of basic_text_classification.ipynb?
Could we add an inline plot for people who are only running in colab?

[random-forests]

Thanks for the feedback! Good comments. Thinking about this more, how does the following sound?

• We can include a section on using pre-trained embeddings.

• We can use a more sophisticated CNN-based model, perhaps from the right side of this flow chart: https://developers.google.com/machine-learning/guides/text-classification/step-2-5 (others are working on an RNN tutorial at the moment, so I don't think we need to include that part here).

From there, we can include a few diagrams helping to communicate the basic idea of embeddings. WDYT?

[securetorobert]

I will review the comments above and implement in a little bit. Thanks guys.

[securetorobert]

@random-forests do you need to provide consent for the CLA above?

[securetorobert]

@random-forests kindly take a look at the recent updates. Thanks

[securetorobert]

@MarkDaoust kindly review recent changes, thanks.

[random-forests]

Hi Robert, thanks again for working on this! I will circle back in a week or so (been OOO for a bit, please stay tuned).

[random-forests]

I'm only a few months late here (wowzers, my bad!) I'm going to close this PR out, and work on a new version for a bit. Will definitely credit you when we get it out! Thanks again for your help, and sorry again for the delay.