Category: Text Mining

Deep Learning, Machine Learning - Data Mining, Text MiningLeave a comment

A review of pre-trained language models: from BERT, RoBERTa, to ELECTRA, DeBERTa, BigBird, and more

Tung.M.Phung

Let us review a list of pretrained language models, including BERT, Transformer-XL, XLNet, RoBERTa, DistilBERT, ALBERT, BART, ELECTRA, ConvBERT, DeBERTa, and BigBird. Continue reading A review of pre-trained language models: from BERT, RoBERTa, to ELECTRA, DeBERTa, BigBird, and more