Skip to content

Tung M Phung's Blog

  • Home
  • About me

Category: Deep Learning

Data Mining, Deep LearningLeave a comment

The Transformer neural network architecture

January 11, 2021April 18, 2021 Tung.M.Phung

The Transformer neural networks, explained in details. Continue reading The Transformer neural network architecture

Data Mining, Deep LearningLeave a comment

Deep Learning normalization methods

January 6, 2021January 6, 2021 Tung.M.Phung

Introduction to and comparison of Batch Norm, Weight Norm, Layer Norm, Instance Norm, and Group Norm. Continue reading Deep Learning normalization methods

Data Mining, Deep LearningLeave a comment

Attention in Deep Learning, your starting point (with code)

December 30, 2020December 31, 2020 Tung.M.Phung

Get your first intuition with Attention right with a minimal code-example. Continue reading Attention in Deep Learning, your starting point (with code)

Data Mining, Deep LearningLeave a comment

LSTM: where to start?

December 27, 2020April 3, 2021 Tung.M.Phung

Where and how you should start with LSTM? Continue reading LSTM: where to start?

Data Mining, Deep LearningLeave a comment

ELU activation: A comprehensive analysis

February 16, 2020July 31, 2020 Tung.M.Phung

Through various experiments, ELU is accepted by many researchers as a good successor of the original version (ReLU). Continue reading ELU activation: A comprehensive analysis

Data Mining, Deep LearningLeave a comment

Batch Normalization and why it works

January 6, 2020January 4, 2021 Tung.M.Phung

Batch Normalization (BatchNorm) is a very frequently used technique in Deep Learning, however, the reason why it works is often interpreted ambiguously. Continue reading Batch Normalization and why it works

Data Mining, Deep LearningLeave a comment

Rectifier Linear Unit (ReLU)

December 28, 2019July 31, 2020 Tung.M.Phung

Despise its simplicity, ReLU previously achieved the top performance over various tasks of modern Machine Learning. Continue reading Rectifier Linear Unit (ReLU)

Data Mining, Deep LearningLeave a comment

Sigmoid, tanh activations and their loss of popularity

December 27, 2019July 31, 2020 Tung.M.Phung

The sigmoid and tanh activation functions were very frequently used in the past but have been losing popularity in the era of Deep learning. Continue reading Sigmoid, tanh activations and their loss of popularity