Deep Learning normalization methods
Introduction to and comparison of Batch Norm, Weight Norm, Layer Norm, Instance Norm, and Group Norm. Continue reading Deep Learning normalization methods
Tag: Batch Normalization
Batch Normalization and why it works
Batch Normalization (BatchNorm) is a very frequently used technique in Deep Learning, however, the reason why it works is often interpreted ambiguously. Continue reading Batch Normalization and why it works
Sigmoid, tanh activations and their loss of popularity
The sigmoid and tanh activation functions were very frequently used in the past but have been losing popularity in the era of Deep learning. Continue reading Sigmoid, tanh activations and their loss of popularity