Deep Learning
from Scratch

Optimization Algorithms

Course starts soon..


Quiz


We will start now with a quiz based on the first week material

You have 6 minutes to answer the quiz.

The quiz link:
quiz Link
It will be copied in Mattermost and in the Zoom chat.

Group Overview

Everybody should have a group

Project Template Folder

Please use the project template

Open Questions (10 minutes)

  1. Are momentum, RMSProp and Adam alternatives to the L2 Regularizaion or Dropout we have seen last week?
  2. About exponentially weighted average: can you think of an application in which you want to use larger beta and one where you want to use small value for beta?
  3. What would happen if we increase (instead of decreasing as in learning rate decay) the learning rate during training?
  4. Can you describe what a saddle point, a plateau, a local minimum and a global minimum are? Do these concepts refers to the weights of the network, to the cost function, to the backpropagation, to the activation functions, or to all of them?

Exercise (15-20 mins)

We go through the programming assignment that were planned for this week.
Here M. Peixeiro actually wrote an article that explains exactly the solution of the assignment for this week.

For the next week

  • Finish the third week of the course! Second course is also done!
  • Do the Programming Assignment on Tensorflow