Deep Learning from Scratch - Week 3

Course starts soon..

QUIZ

We will start now with a quiz based on the first week material

You have 7 minutes to answer the quiz.

The quiz link:

Quiz Link

It will be copied in Mattermost and in the Zoom chat.

Matrices in python

A Notebook to play with matricesStart thinking about your Project

- What theme would you like to address?
- What kind of data you want to use?
- Remember to start simple and focus on feed forward neural networks.
- For this course, better a small project that works and can be extended than a big project abandoned halfway through.

PAPER OF THE WEEK

A. Krizhevsky, I. Sutskever, G. E. Hinton

QUIZ (15 mins)

1. Why is the cost function important and why we need such a function for logistic regression?

1b. Are you familiar with the concept of convex functions and why this helps gradient find the minimum?

2. Is the concept of gradient descent clear? How do learning rate correlates to number of iterations?

3. Why is vectorization faster than a loop in python?

3b. Is it always the case? Can you think of a case when vectorization is not faster, or even slower?

DISCUSSION AND ANSWERS

Cost Function

Image from this article.

Non Convex Functions

Gradient Descent

Image from this article.

Vectorization

A nice explanation about what vectorization isand

An interesting discussion on Stackoverflow

Exercise (15-20 mins)

We go through the programming assignment that were planned for this week.Open Source Project of the Week

Fashion MNIST Datasets

Github Repository with Examples for MNIST DatasetsWhy?

Conclusions

It is a good playground for your first project

A reward to motivate you further

For the next week

- Finish the third week of the course
- Do the Programming Assignment on Planar Data Classification