Guide Labs, a San Francisco-based startup, has announced the release of Steerling-8B, a new interpretable language model (LLM) featuring 8 billion parameters. This LLM, developed with a novel architecture, aims to address the challenge of understanding complex deep learning models. By making the model’s actions easily interpretable, every output token can be traced back to its training data source, enabling a deeper understanding of the model’s decision-making process.
CEO Julius Adebayo and Chief Science Officer Aya Abdelsalam Ismail have led this initiative. The Steerling-8B model offers a breakthrough in interpretability, allowing users to track references, comprehend humor, and analyze gender encoding within the model. Adebayo emphasized the significance of reliably identifying and manipulating encoded information, highlighting the fragility of current models in achieving this task.
This project originated from Adebayo’s research at MIT, where he co-authored a pivotal paper in 2020 that exposed the limitations of existing deep learning model interpretation methods. The innovative approach involves incorporating a concept layer in the model to categorize data for traceability, necessitating upfront data annotation. By leveraging additional AI models, Guide Labs successfully trained Steerling-8B, marking a substantial advancement in model interpretability.
Guide Labs’ approach to model engineering focuses on designing interpretability into the model architecture, eliminating the need for post hoc neuroscientific model analysis. This strategy streamlines the interpretability process and enhances model transparency from inception.
Source: TechCrunch