CNNVR: Optimization

机器学习

3 keys to training neural networks:

1. Score function
2. Loss function
3. Optimization

Bad ideas

Very bad idea: Random search.
A better but still very bad idea: Random local search.

Following the gradient

Using numerical gradient:

• Approximate
• Very slow

Use Calculus gradient instead. SGD is good idea indeed.