Deep Learning from Scratch - Week 3

Course starts soon..


We will start now with a quiz based on the first week material

You have 7 minutes to answer the quiz.

The quiz link:
Quiz Link
It will be copied in Mattermost and in the Zoom chat.

Matrices in python

A Notebook to play with matrices

Start thinking about your Project

  • What theme would you like to address?
  • What kind of data you want to use?
  • Remember to start simple and focus on feed forward neural networks.
  • For this course, better a small project that works and can be extended than a big project abandoned halfway through.


ImageNet Classification with Deep Convolutional Neural Networks
A. Krizhevsky, I. Sutskever, G. E. Hinton
Proceedings of the 25th International Conference on Neural Information Processing Systems (Lake Tahoe, NV, Dec. 2012), 1097–1105.

QUIZ (15 mins)

1. Why is the cost function important and why we need such a function for logistic regression?
1b. Are you familiar with the concept of convex functions and why this helps gradient find the minimum?
2. Is the concept of gradient descent clear? How do learning rate correlates to number of iterations?
3. Why is vectorization faster than a loop in python?
3b. Is it always the case? Can you think of a case when vectorization is not faster, or even slower?


Cost Function

Image from this article.

Non Convex Functions

Gradient Descent

Image from this article.


A nice explanation about what vectorization is
An interesting discussion on Stackoverflow

Exercise (15-20 mins)

We go through the programming assignment that were planned for this week.

Open Source Project of the Week

Fashion MNIST Datasets

Github Repository with Examples for MNIST Datasets



It is a good playground for your first project
A reward to motivate you further

For the next week