Language Model Pre-Training with Sparse Latent Typing

Keyphrases

• Downstream Task 100%
• External Knowledge 50%
• Few-shot Setting 50%
• Information Extraction 50%
• Interpretable Representation 50%
• Language Model 100%
• Pre-trained Language Model 100%
• Publicly Available 50%
• Sentence-level 50%
• Text Reconstruction 50%
• Training Objectives 100%

Computer Science

• Deep Understanding 50%
• Experimental Result 50%
• External Knowledge 50%
• Information Extraction 50%
• Language Modeling 100%
• Pre-Trained Language Models 100%
• Training Objective 100%

Social Sciences

• Language Modeling 100%