活动报名｜通过自我博弈微调 (SPIN) 可以让大模型从弱到强

Harnessing the power of human-annotated data through Supervised Fine-Tuning (SFT) is pivotal for advancing Large Language Models (LLMs). In this talk, I will introduce our newest fine-tuning method, Self-Play Fine-Tuning (SPIN), which improves LLMs without the need for additional human-annotated data. SPIN utilizes a self-play mechanism, where the LLM enhances its capabilities by generating its own training data through interactions with instances of itself. Specifically, the LLM generates its own training data from its previous iterations, refining its policy by discerning these self-generated responses from those obtained from human-annotated data. As a result, SPIN unlocks the full potential of human-annotated data for SFT. Our empirical results show that SPIN can improve the LLM’s performance across a variety of benchmarks and even outperform models trained through direct preference optimization (DPO) supplemented with extra GPT-4 preference data. Additionally, I will outline the theoretical guarantees of our method. For more details and access to our codes, visit our GitHub repository (https://github.com/uclaml/SPIN).

报告嘉宾：
陈子翔，加州大学洛杉矶分校（UCLA）计算机科学系的计算机科学博士生，导师是顾全全教授。他从清华大学获得了数学学士学位。他广泛地对深度学习、优化和控制的理论及其应用感兴趣，特别是在生成模型、特征学习和多智能体强化学习方面。最近他在研究利用生成模型在公共健康领域的科学应用。他曾是simons 计算理论研究所强化学习项目的访问研究生。扫描下方二维码或点击「阅读原文」报名