Laplace's Scissors – A tool for directly editing LLM weights
Researchers have developed Laplace's Scissors, a new tool that enables precise editing of large language models while preserving correct outputs and analyzing model scaling behaviors.
The tool, developed by Miyamoto-Renya and published on GitHub, implements a technique called 'protected readout rotation' that allows developers to modify AI models while maintaining their correct responses. This approach addresses a significant challenge in AI model editing: how to fix incorrect outputs without breaking existing functionality.
Laplace's Scissors works by rotating an LLM's output matrix to compute edits while preserving protected answers. The tool uses Rust for model loading and execution, with Triton serving as a compilation bridge for CUDA or HIP GPU acceleration. It consists of three main functions: probe, diagnose, and scaling analysis.
Numerical Probing
The probe function constructs numerical inputs from model weights and traces directions with weak output discrimination. It identifies areas in the model where small changes produce minimal impact on output quality. By injecting vectors with the same norm as a reference vector at the decoder input, the tool measures their divergence using Jeffreys divergence.
This process reveals 'weak directions' in the model where interventions have minimal impact on output quality. The tool then identifies weight coordinates and attention heads that contribute most significantly to these weak directions, allowing for precise targeting of model modifications.
Answer Repair
The diagnose function uses test cases with expected answers to find rotation planes that can repair failed outputs while protecting successful ones. For each case, the tool checks whether the model's original output matches the expected answer and identifies the minimal rotation needed to correct incorrect responses.
The tool implements strict constraints to ensure that only beneficial edits are applied. Each candidate rotation must satisfy a minimum logit gap between target tokens and competitors, preserve originally successful generation paths, and stay within predefined bounds for weight changes and output modifications.
Scaling Analysis
The third component of Laplace's Scissors enables scaling analysis by applying the same edit policy across different model checkpoints. This allows researchers to measure how deviations from reference curves develop as models scale and determine how much performance can be recovered through targeted interventions.
The tool requires specific environment setup including a Rust toolchain, Python with Triton 3.8, and CUDA or HIP drivers. It supports loading models from Hugging Face or local directories, with all hub inputs pinned to specific commit SHAs for reproducibility.
Related
