Course Material for Natural Language Processing @ Computer Science Dept, Sapienza Master in Computer Science
- Introduction to NLP, Regular Expressions, Finite State Automata and REs
- Words, Corpora and Text Normalization
- Spelling Correction and Minimum Edit Distance
- Language models, Part-of-speech-tagging
- Hidden Markov Model, Viterbi Algorithm, Logistic Regression
- Syntax, Semantics, Vector semantics (sparse), NLP tasks
- Latent Semantic Analysis and word2vec [hierarchical softmax & neg. sampling]
- Scaling word2vec, Sentiment Analysis, Language Model w/ Neural Nets
- Sequence modeling w/ Deep Learning: LM /w RNN, POS, Image Captioning
- from LSTM to Transformers
- Neural Machine Translation, Encoder/Decoder, Beam Search
- Contextual Embedding: BERT, GPT, Transfer Learning
- Multimodal NLP: Diffusion models (images), NLP as supervision for Vision (CLIP)
- text2image application (Dall-E 2): based on diffusion and CLIP
It is in the form of Jupyter Notebook slides with LaTeX math, code, drawings, plots and explanations
- Slides and material will be uploaded before every lecture on Google Classroom and here.
- Good starting point but but may be not enough.
- Textbooks are required.
| Date | Topic | NBviewer | Github | Colab |
|---|---|---|---|---|
| Word embeddings | ||||
| April 16, 18 | LSA, intro word2vec |  |
 |
|
| April 23, 30 | scaling word2vec, Sentiment Analysis, Language Model w/ NN | |
||
| April 30, May 2, 7 | Deep learning for Seq. Processing | |
|
|
| May 9, 14 | From RNN to Neural Machine Translation | |
||
| May 14, 21 | Transformers, BERT, Pre-training | |
|
|
| May 28 | ExamPrep, CLIP | |
||