Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards

Keyphrases

• Direction Selectivity 100%
• User Preference 100%
• Diverse Users 100%
• Large Language Models 100%
• Reinforcement Learning from Human Feedback 66%
• Scalar Rewards 50%
• Verbosity 33%
• Reward Model 33%
• Real-world Application 16%
• User Needs 16%
• Model Generation 16%
• Training Model 16%
• Modeller 16%
• Model Code 16%
• Competitive Performance 16%
• Feedback Method 16%
• Unit Vector 16%
• Performance Tradeoff 16%
• Rejection Sampling 16%
• Alignment Model 16%
• User-dependent 16%
• Intuitive Control 16%
• Fine-grained Control 16%
• Preference Profile 16%
• Llama 2 16%
• Mistral 16%
• Preference Optimization 16%

Computer Science

• Multiobjective 100%
• User Preference 100%
• Large Language Model 100%
• Reinforcement Learning 80%
• World Application 20%
• Model Generation 20%
• Performance Trade 20%
• Grained Control 20%