ELECTRA: NLP预训练模型

ELECTRA的全称是Efficiently Learning an Encoder that Classifies Token Replacements Accurately. NLP式的Generator-Discriminator ELECTRA最主要的贡献是提出了新的预训练任务和框架，把生成式的Masked language model(MLM)预训练任务改成了判别式的Replaced t

>>阅读原文<<