XTC: Extreme Compression for Pre-trained Transformers Made Simple and Efficient

Keyphrases

• Knowledge Distillation 100%
• Generative Transformer 100%
• Extreme Compression 100%
• Ultra-low 66%
• Low Bit Precision 66%
• State-of-the-art Techniques 33%
• Compression System 33%
• Training Strategy 33%
• Resource-constrained Devices 33%
• Model Size 33%
• Edge Method 33%
• Hyperparameters 33%
• Transformer Model 33%
• Hyperparameter Tuning 33%
• NLP Model 33%
• Ternary Quantization 33%
• Layer Reduction 33%
• Small Transformer 33%

Engineering

• Bit Precision 100%
• Cutting Edge 50%
• Multistage 50%
• State-of-the-Art Method 50%
• Compression Scheme 50%

Computer Science

• Knowledge Distillation 100%
• Leaning Parameter 66%
• Compression Scheme 33%
• New-State 33%
• Constrained Device 33%
• Bidirectional Encoder Representations From Transformers 33%