Researchers from MIT, the Improbable AI Lab, and ETH Zurich have introduced a method called Self-Distillation Fine-Tuning (SDFT) that enables large language models (LLMs) to acquire new skills without erasing previous knowledge. Traditional fine-tuning often leads to forgetting past capabilities, requiring separate models for different tasks. SDFT allows LLMs to assimilate fresh skills while retaining existing competencies, outperforming supervised fine-tuning and overcoming reinforcement learning limitations.
By leveraging in-context learning, SDFT empowers a single model to accumulate multiple skills over time, offering a promising approach for developing adaptive AI agents that can swiftly adapt to evolving business landscapes without costly retraining or compromising general reasoning abilities. This advancement addresses the industry’s need for continual learning, crucial for AI systems to evolve like human learning throughout their careers.
SDFT’s unique approach combines the benefits of on-policy learning and distillation, enabling a model to learn from its own interactions and outputs rather than relying solely on static datasets or explicit reward functions. This process enhances a model’s ability to correct its reasoning trajectories autonomously, even when faced with entirely new information.
The effectiveness of SDFT is demonstrated by its superior performance in enterprise-grade skills testing, showcasing improved task learning efficiency and reduced catastrophic forgetting compared to conventional methods. The method’s availability on GitHub, along with ongoing efforts to integrate it into popular AI libraries, signals a significant step towards democratizing advanced AI training techniques for organizations seeking versatile, efficient model solutions.
Source: VentureBeat