Accepted at ICML 2024
We defined a novel Uncertainty-aware Reward Model (URM) for the preference training of LLMs based on Bayesian approximation to quantify the uncertainty of paired responses. We experimentally demonstrated that using URM in LLMs training boosts their instruction following capability and their policy optimization objectives. The URM based finetuning surpasses existing methods by a large margin on benchmarks such as Vicuna and MT-bench.
The code & data for the paper are shared at P-B-U Git Repo
Research on parameter-efficient fine-tuning methods including LoRA and QLoRA for domain-specific applications, with focus on scientific and technical text generation.
Development of RAG architectures for knowledge-intensive tasks, incorporating vector databases and advanced retrieval mechanisms for technical document processing.
Implementation of collaborative AI agent frameworks using LangChain and AutoGen for complex problem-solving tasks requiring specialized reasoning capabilities.
Systematic approaches to prompt design and optimization for improved model performance across various tasks, including automated prompt generation techniques.
Advanced code generation systems for automated programming tasks, including documentation generation, code review, and algorithm implementation assistance.