Skip to content

Tung M Phung's Blog

Home
About me

Search for:

Tag: Entropy

Machine Learning - Data Mining, MiscellaneousLeave a comment

Information Theory concepts: Entropy, Mutual Information, KL-Divergence, and more

May 20, 2021June 25, 2021 Tung.M.Phung

An intuitive introduction to Information Content, Encoding, Entropy, Joint Entropy, Cross Entropy, KL Divergence, Conditional Entropy, and Mutual Information… Continue reading Information Theory concepts: Entropy, Mutual Information, KL-Divergence, and more

Classification Models, Machine Learning - Data Mining6 Comments

Information Gain, Gain Ratio and Gini Index

January 4, 2020July 31, 2020 Tung.M.Phung

Information Gain, Gain Ratio and Gini Index are the three fundamental criteria to measure the quality of a split in Decision Tree. Continue reading Information Gain, Gain Ratio and Gini Index