Self-training Improves Pre-training for Natural Language Understanding 笔记
self-training，是用训练好的模型的预测结果 作为 伪target，混入原训练集一起训练， 该文的创新在于提出一种文本相似度算法从海量文本中找出和 训练集中的文本 相似的文本， 然后用训练好的模型对这批 找出的相似的文本 进行预测出 伪target，然后混一起，然后是BERT fine-tune， 提升了fine-tune的效果，似乎跟文章题目说的pre-train没关系。 原文也提到了
相关文章
- 1. Watson Natural Language Understanding
- 2. Multi-Task Deep Neural Networks for Natural Language Understanding阅读笔记
- 3. 文献阅读笔记：XLNet: Generalized Autoregressive Pretraining for Language Understanding
- 4. UniLM: Unified Language Model Pre-training for Natural Language Understanding and Generation
- 5. 论文阅读 Multi-Task Deep Neural Networks for Natural Language Understanding
- 6. 论文阅读：《Pretraining-Based Natural Language Generation for Text Summarization》
- 7. CS224d: Deep Learning for Natural Language Process
- 8. 自然语言理解（Natural Language Understanding）
- 9. 论文阅读：Dual Supervised Learning for Natural Language Understanding and Generation
- 10. Multi-Task Deep Neural Networks for Natural Language Understanding【MT-DNN模型】
- 更多相关文章...
- • RSS
元素 - RSS 教程 - • ASP.NET Razor - 标记 - ASP.NET 教程
- • Tomcat学习笔记（史上最全tomcat学习笔记）
- • YAML 入门教程
相关标签/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
欢迎关注本站公众号,获取更多信息
相关文章
- 1. Watson Natural Language Understanding
- 2. Multi-Task Deep Neural Networks for Natural Language Understanding阅读笔记
- 3. 文献阅读笔记：XLNet: Generalized Autoregressive Pretraining for Language Understanding
- 4. UniLM: Unified Language Model Pre-training for Natural Language Understanding and Generation
- 5. 论文阅读 Multi-Task Deep Neural Networks for Natural Language Understanding
- 6. 论文阅读：《Pretraining-Based Natural Language Generation for Text Summarization》
- 7. CS224d: Deep Learning for Natural Language Process
- 8. 自然语言理解（Natural Language Understanding）
- 9. 论文阅读：Dual Supervised Learning for Natural Language Understanding and Generation
- 10. Multi-Task Deep Neural Networks for Natural Language Understanding【MT-DNN模型】