====Machine learning for NLP 2 (Apprentissage pour le TAL 2)====

==Enseignant 2020-2021==
Marie Candito

== Objectives and content ==

The course provides the fundamental concepts of supervised classification via deep learning methods, with typical examples in NLP.

1. General concepts for supervised classification
  * methodology
  * evaluation metrics

2. A first classifier : k-NN


3. Linear and log-linear models
  * linear separability
  * prediction with a (log-)linear classifier
  * perceptron learning algorithm
  * kernel methods
  * logistic regression

4. Extension to Multi-layer perceptrons
  * non linearity
  * fully connected feed-forward neural network
  * universal approximation theorem


5. Learning as loss minimization
  * usual loss functions
  * stochastic gradient descent
  * backpropagation algorithm

6. vectorial representations
  * word embeddings as dense features
  * word embeddings learning

Lab sessions will illustrate the course, introducing in particular:
  * tensor manipulation in numpy / pytorch
  * sklearn and pytorch libraries

==Bibliography==
  * Hal Daumé III : An introduction to Machine Learning, http://ciml.info/
  * "Neural Network Methods in Natural Language Processing", 2016, Morgan & Claypool
    * preliminary version available here :  A primer on neural network models for natural language processing (http://u.cs.biu.ac.il/~yogo/nnlp.pdf)
  *  Goodfellow, Bengio & Courville "Deep Learning", MIT Press, 2016 http://www.deeplearningbook.org/