Artificial Neural Networks and Deep Learning
The following are last minute news you should be aware of ;-)
16/10/2019: New deck of slides uploaded 15/10/2019: Thursday lectures moved to room 2.0.1 indeed! 10/10/2019: Today's lecture moved to Aula Rogers!!! 04/10/2019: Uploded slides about feed forward neural networks 04/10/2019: Today's lecture moved to 4.0.1!!! 26/09/2019: Uploaded slides about deep learning 24/09/2019: Thursday 26/09/2019 lecture will be in room B21 !!! 19/09/2019: No lecture on the 20/09/2019 ... check the detailed schedule. 19/09/2019: The course starts today!
Contents
Course Aim & Organization
Neural networks are mature, flexible, and powerful non-linear data-driven models that have successfully been applied to solve complex tasks in science and engineering. The advent of the deep learning paradigm, i.e., the use of (neural) network to simultaneously learn an optimal data representation and the corresponding model, has further boosted neural networks and the data-driven paradigm.
Nowadays, deep neural network can outperform traditional hand-crafted algorithms, achieving human performance in solving many complex tasks, such as natural language processing, text modeling, gene expression modeling, and image recognition. The course provides a broad introduction to neural networks (NN), starting from the traditional feedforward (FFNN) and recurrent (RNN) neural networks, till the most successful deep-learning models such as convolutional neural networks (CNN) and long short-term memories (LSTM).
The course major goal is to provide students with the theoretical background and the practical skills to understand and use NN, and at the same time become familiar and with Deep Learning for solving complex engineering problems.
Teachers
The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant.
- Matteo Matteucci: the course teacher
- Giacomo Boracchi: the course teacher
- Francesco Lattari: the course teaching assistant
Course Program
This goal is pursued in the course by:
- Presenting major theoretical results underpinning NN (e.g., universal approx, vanishing/exploding gradient, etc.)
- Describing the most important algorithms for NN training (e.g., backpropagation, adaptive gradient algorithms, etc.)
- Illustrating the best practices on how to successfully train and use these models (e.g., dropout, data augmentation, etc.)
- Providing an overview of the most successful Deep Learning architectures (e.g., CNNs, sparse and dense autoencoder, LSTMs for sequence to sequence learning, etc.)
- Providing an overview of the most successful applications with particular emphasis on models for solving visual recognition tasks.
Detailed course schedule
A detailed schedule of the course can be found here; topics are just indicative while days and teachers are correct up to some last minute change (I will notify you by email). Please note that not all days we have lectures!!
Note: Lecture timetable interpretation * On Thursday, in L26.12, starts at 16:15, ends at 18:15 * On Friday, in 6.0.1, starts at 14:15, ends at 17:15
| Date | Day | Time | Room | Teacher | Type | Topic |
| 19/09/2019 | Thursday | 16:15 - 17:15 | L26.12 | Lecture | Giacomo Boracchi | Course Introduction |
| 20/09/2019 | Friday | 14:15 - 17:15 | 6.0.1 | -- | -- | -- No Lecture Today -- |
| 26/09/2019 | Thursday | 16:15 - 18:15 | B.2.1 | Lecture | Matteo Matteucci | Introduction to Machine Learning |
| 27/09/2019 | Friday | 14:15 - 17:15 | 6.0.1 | Lecture | Matteo Matteucci | Perceptron and Hebian Learning |
| 03/10/2019 | Thursday | 16:15 - 18:15 | L26.12 | -- | -- | -- No Lecture Today -- |
| 04/10/2019 | Friday | 14:15 - 17:15 | 4.0.1 | Lecture | Matteo Matteucci | Feedforward neural networks and Backpropagation |
| 10/10/2019 | Thursday | 16:15 - 18:15 | Aula Rogers | Lecture | Matteo Matteucci | Error Functions |
| 11/10/2019 | Friday | 14:15 - 16:15 | 6.0.1 | Lecture | Giacomo Boracchi | Introduction to Image Classification |
| 17/10/2019 | Thursday | 16:15 - 18:15 | 2.0.1 | Lecture | Matteo Matteucci | Neural Networks and Template Matching |
| 18/10/2019 | Friday | 14:15 - 17:15 | 6.0.1 | Lecture | Matteo Matteucci | Facing Overfitting |
| 24/10/2019 | Thursday | 16:15 - 18:15 | 2.0.1 | Lecture | Matteo Matteucci | Training tricks and Vanishing gradient |
| 25/10/2019 | Friday | 14:15 - 17:15 | 6.0.1 | Practicals | Francesco Lattari | Keras examples ... |
| 31/10/2019 | Thursday | 16:15 - 18:15 | 2.0.1 | Lecture | Giacomo Boracchi | Introduction to Convolutional Neural Networks |
| 01/11/2019 | Friday | 14:15 - 17:15 | 6.0.1 | -- | -- | -- No Lecture Today -- |
| 07/11/2019 | Thursday | 16:15 - 18:15 | 2.0.1 | Lecture | Giacomo Boracchi | CNN architectures |
| 08/11/2019 | Friday | 14:15 - 16:15 | 6.0.1 | Lecture | Giacomo Boracchi | Training with data scarcity |
| 14/11/2019 | Thursday | 16:15 - 18:15 | 2.0.1 | Lecture | Giacomo Boracchi | CNN for image segmentation |
| 15/11/2019 | Friday | 14:15 - 17:15 | 6.0.1 | Practicals | Francesco Lattari | Keras examples ... |
| 21/11/2019 | Thursday | 16:15 - 18:15 | 2.0.1 | Lecture | Giacomo Boracchi | CNN for localization and detection |
| 22/11/2019 | Friday | 14:15 - 16:15 | 6.0.1 | Lecture | Giacomo Boracchi | GAN |
| 28/11/2019 | Thursday | 16:15 - 18:15 | 2.0.1 | Lecture | Matteo Matteucci | Recurrent Neural Networks |
| 29/11/2019 | Friday | 14:15 - 17:15 | 6.0.1 | Lecture | Matteo Matteucci | Vanishing gradient and LSTM |
| 05/12/2019 | Thursday | 16:15 - 18:15 | 2.0.1 | Lecture | Matteo Matteucci | Word embedding |
| 06/12/2019 | Friday | 14:15 - 17:15 | 6.0.1 | Lecture | Matteo Matteucci | Variational Autoencoders |
| 12/12/2019 | Thursday | 16:15 - 18:15 | -- | -- | -- | -- No Lecture Today -- |
| 13/12/2019 | Friday | 14:15 - 17:15 | 6.0.1 | Practicals | Francesco Lattari | Keras examples ... |
| 19/12/2019 | Thursday | 16:15 - 18:15 | 2.0.1 | Lecture | Matteo Matteucci | ... TBD ... |
| 20/12/2019 | Friday | 14:15 - 17:15 | 6.0.1 | Lecture | Matteo Matteucci | ... TBD ... |
Course Evaluation
Course evaluation is composed of two parts:
- A written examination covering the whole program graded up to 27/32
- A home project in the form of a Kaggle style competition practicing the topics of the course graded up to 5/32
The final score will sum the grade of the written exam and the grade of the home project.
Teaching Material (the textbook)
Lectures will be based on material from different sources, teachers will provide their slides to students as soon they are available. As a general reference you can check the following text, but keep in mind that teachers will not follow it strictly
- Deep Learning. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 2016.
Course Slides
Slides from the lectures by Matteo Matteucci
- [2019/2020] Course Introduction: introductory slides of the course with useful information about the course syllabus, grading, and the course logistics.
- [2019/2020] Machine Learning vs Deep Learning: introduction to machine learning paradigms and definition of deep learning with examples
- [2019/2020] From Perceptrons to Feed Forward Neural Networks: the original Perceptron model, Hebbian learning, feed forward architecture, backpropagation and gradient descent, error functions and maximum likelihood estimation
- [2019/2020] Neural Networks Training: dealing with overfitting (weight decay, early stopping, dropout), vanishing gradient (ReLU and frinds), batch normalization
Slides from the lectures by Giacomo Boracchi are avaible in his webpage, for you
- Image Classification: Image classification and related issues, template matching, image classification via nearest neighbors methods, image classification via linear classifiers, image classification via hand-crafted features.
Additional material from the teachers
- ... coming soon ...
