AlexNet

论文笔记

---

ImageNet Classification with Deep Convolutional Neural Networks，比较经典的 paper。

其方法包括：

1. ReLU
2. Multiple GPU trainining
3. Local response normalization
4. Overlapping pooling
5. Data augmentation (VERY HELPFUL)
   
   • image translations
   • horizontal reflections
   • altering intensity
6. Dropout
   
   • Applied to first 2 fully-connected layers of AlexNet

The authors performed PCA on the set of RGB values through the ImageNet training set, and add multiples of the found principal components to each training image:

$$\left[ p_1, p_2, p_3 \right] \left[ \alpha_1 \lambda_1, \alpha_2 \lambda_2, \alpha_3 \lambda_3, \right]$$
where $p_i$ and $\lambda_i$ are eigenvetor and eigenvalue, $a_i$ is a random variable drawn from a Gaussian with mean $0$ and standard deviation $0.1$.