Artificial Neural Networks and Deep Learning
The following are last-minute news you should be aware of ;-)
14/09/2021: Website under maintenance ... come back later
Course Aim & Organization
Neural networks are mature, flexible, and powerful non-linear data-driven models that have successfully been applied to solve complex tasks in science and engineering. The advent of the deep learning paradigm, i.e., the use of (neural) network to simultaneously learn an optimal data representation and the corresponding model, has further boosted neural networks and the data-driven paradigm.
Nowadays, deep neural network can outperform traditional hand-crafted algorithms, achieving human performance in solving many complex tasks, such as natural language processing, text modeling, gene expression modeling, and image recognition. The course provides a broad introduction to neural networks (NN), starting from the traditional feedforward (FFNN) and recurrent (RNN) neural networks, till the most successful deep-learning models such as convolutional neural networks (CNN) and long short-term memories (LSTM).
The course major goal is to provide students with the theoretical background and the practical skills to understand and use NN, and at the same time become familiar and with Deep Learning for solving complex engineering problems.
The course is composed of a blending of lectures and exercises by the course teachers and a teaching assistant.
- Matteo Matteucci: the course teacher and this is his webex room
- Giacomo Boracchi: the course co-teacher and this is his webex room
- Francesco Lattari: the course teaching assistant and this is his webex room
- Eugenio Lomurno: the course teaching assistant and this is his webex room
Course Program and Syllabus
This goal is pursued in the course by:
- Presenting major theoretical results underpinning NN (e.g., universal approx, vanishing/exploding gradient, etc.)
- Describing the most important algorithms for NN training (e.g., backpropagation, adaptive gradient algorithms, etc.)
- Illustrating the best practices on how to successfully train and use these models (e.g., dropout, data augmentation, etc.)
- Providing an overview of the most successful Deep Learning architectures (e.g., CNNs, sparse and dense autoencoder, LSTMs for sequence to sequence learning, etc.)
- Providing an overview of the most successful applications with particular emphasis on models for solving visual recognition tasks.
We have compiled a detailed syllabus of the course student can use to double check their preparation against before the exam.
- [2020/2021] Course Syllabus: a detailed list of topics covered by the course and which students are expected to know when approaching the exam
Detailed course schedule
A detailed schedule of the course can be found here; topics are just indicative while days and teachers are correct up to some last minute change (I will notify you by email). Please note that not all days we have lectures!!
Note: Lecture timetable interpretation * On Wednesday, in T.2.1, Team 1, starts at 15:15, ends at 17:00 * On Wednesday, in T.2.1, Team 2, starts at 17:30, ends at 19:15 * On Thursday, in teacher webex room, starts at 16:30, ends at 19:15
Note: Teams division is based on your Codice Persona (and should minimize overlap) * Team 1: odd Codice Persona * Team 2: even Codice Persona
|15/09/2021||Wednesday||15:15 - 17:00||T.2.1 (Team1 )||Matteo Matteucci||Course Introduction + Deep Learning Intro|
|15/09/2021||Wednesday||17:30 - 19:15||T.2.1 (Team 2)|
|16/09/2021||Thursday||16:30 - 19:15||Virtual Room||Matteo Matteucci||Introduction to Deep Learning + Introduction to Feed Forward Neural Networks|
|22/09/2021||Wednesday||15:15 - 17:00||T.2.1 (Team 1)||xxx xxx||Python Intro + Numpy|
|22/09/2021||Wednesday||17:30 - 19:15||T.2.1 (Team 2)|
|23/09/2021||Thursday||16:30 - 19:15||Virtual Room||Matteo Matteucci||Feed forward neural networks and Backpropagation|
|29/09/2021||Wednesday||15:15 - 17:00||T.2.1 (Team 1)||xxx xxx||KERAS: Tensorflow and FNN|
|29/09/2021||Wednesday||17:30 - 19:15||T.2.1 (Team 2)|
|30/09/2021||Thursday||16:30 - 19:15||Virtual Room||Matteo Matteucci||Error Functions Design (and weight decay)|
|06/10/2021||Wednesday||15:15 - 17:00||T.2.1 (Team 1)||Matteo Matteucci||Overffitting, cross-validation, and Early Stopping (flipped ?)|
|06/10/2021||Wednesday||17:30 - 19:15||T.2.1 (Team 2)|
|07/10/2021||Thursday||16:30 - 19:15||---||---||No Lectures (Graduation)|
|13/10/2021||Wednesday||15:15 - 17:00||T.2.1 (Team 1)||xxx xxx||KERAS: FFNN and Overfitting|
|13/10/2021||Wednesday||17:30 - 19:15||T.2.1 (Team 2)|
|14/10/2021||Thursday||16:30 - 19:15||Virtual Room||Matteo Matteucci||Training tricks: activation functions, network initialization, and other stuff...|
|20/10/2021||Wednesday||15:15 - 17:00||T.2.1 (Team 1)||Giacomo Boracchi||The Image Classification Problem|
|20/10/2021||Wednesday||17:30 - 19:15||T.2.1 (Team 2)|
|21/10/2021||Thursday||16:30 - 19:15||Virtual Room||Giacomo Boracchi||Convolutional Neural Networks|
|27/10/2021||Wednesday||15:15 - 17:00||T.2.1 (Team 1)||xxx xxx||KERAS: Convolutional Neural Networks|
|27/10/2021||Wednesday||17:30 - 19:15||T.2.1 (Team 2)|
|28/10/2021||Thursday||16:30 - 19:15||Virtual Room||Giacomo Boracchi||Training with data scarsity|
|03/11/2021||Wednesday||15:15 - 17:00||T.2.1 (Team 1)||xxx xxx||KERAS: Convolutional Neural Networks|
|03/11/2021||Wednesday||17:30 - 19:15||T.2.1 (Team 2)|
|04/11/2021||Thursday||16:30 - 19:15||Virtual Room||Giacomo Boracchi||Famous CNN architectures|
|10/11/2021||Wednesday||---||---||---||-- No Lecture (Prove in Itinere) --|
|11/11/2021||Thursday||16:30 - 19:15||Virtual Room||Giacomo Boracchi||Fully Convolutional CNN, CNN for image segmentation|
|17/11/2021||Wednesday||15:15 - 17:00||T.2.1 (Team 1)||Giacomo Boracchi||CNN for localization and detction|
|17/11/2021||Wednesday||17:30 - 19:15||T.2.1 (Team 2)|
|18/11/2021||Thursday||16:30 - 19:15||Virtual Room||Giacomo Boracchi||GANs|
|24/11/2021||Wednesday||15:15 - 17:00||T.2.1 (Team 1)||xxx xxx||KERAS: Autoencoder, classification, segmentation|
|24/11/2021||Wednesday||17:30 - 19:15||T.2.1 (Team 2)|
|25/11/2021||Thursday||16:30 - 19:15||Virtual Room||Matteo Matteucci||Recurrent neural networks + LSTM|
|01/12/2021||Wednesday||15:15 - 17:00||T.2.1 (Team 1)||xxx xxx||KERAS: learning with text|
|01/12/2021||Wednesday||17:30 - 19:15||T.2.1 (Team 2)|
|02/12/2021||Thursday||16:30 - 19:15||Virtual Room||Matteo Matteucci||Sequence to sequence learning and Word Embedding|
|08/12/2021||Wednesday||---||---||---||-- No Lecture (Holiday) --|
|09/12/2021||Thursday||16:30 - 19:15||Virtual Room||Matteo Matteucci||Attention Mechanism and Transformer|
|15/12/2021||Wednesday||16:15 - 18:15||Virtual Room||---||-- Spare Lecture --|
|16/12/2021||Thursday||16:30 - 19:15||Virtual Room||---||-- Spare Lecture --|
Course evaluation is composed of two parts:
- A written examination covering the whole program graded up to 22/30
- 2 home projects in the form of a "Kaggle style" challenge practicing the topics of the course graded up to 4/30 each
The final score will sum the grade of the written exam and the grade of the home projects. Home projects are not compulsory and they are issued only once a year.
Teaching Material (the textbook)
Lectures will be based on material from different sources, teachers will provide their slides to students as soon they are available. As a general reference you can check the following text, but keep in mind that teachers will not follow it strictly
- Deep Learning. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 2016.
The remaining material about the course is available through WeBeep.