Skip to main content

Benchmarking Qwen3-Coder-Next, DeepSeek-R1, and Qwen3.5 with Dynamic RSI

Kagi Small Github
15 lis 2026, 01:06Tech
Oryginał
1 min read

A new Dynamic RSI Harness framework has significantly improved the performance of AI coding models on software engineering benchmarks, achieving up to 100% success rates on difficult tasks.

Researchers have developed a Dynamic RSI (Recursive Self-Improvement) Harness that dramatically improves AI coding models' performance on software engineering benchmarks. The framework combines two orthogonal architectures - JIT Macro-Runtime Compilation and Execution-Gated Micro-Cognitive Evolution - to address limitations in current AI systems.

Testing on the SWE-bench Verified benchmark showed significant improvements across different models. The framework helped qwen3-coder-next and deepseek-r1:32b achieve higher resolution rates, with Gemini Flash reaching 100% success on the HARD-FLASH suite after implementation.

Unique Dual Architecture

Most self-improving agent frameworks in 2025-2026 optimize only one layer of the stack. Macro-Only Optimizers like ADAS and AFlow search for better multi-agent graphs offline while keeping cognitive tools static. Micro-Only Optimizers like ACE, TextGrad, OPRO, or LATM refine prompt rules within fixed workflows.

The Dynamic RSI Harness uniquely unifies these approaches through Pytest-Style Sandbox Introspection, allowing simultaneous optimization of both workflow architecture and cognitive processes.

Key Engineering Innovations

The framework introduces five deep engineering mechanisms that separate it from textbook implementations. These include decoupling "Wrong Runtime" issues from "Cognitive Blindspots," implementing Pytest-Style LHS/RHS Assertion Introspection, and creating a "Darwinian Playbook Gate" to prevent spec-doubting rule corruption.

Model-Specific Adaptations

The framework addresses model-specific challenges, such as token budget starvation and JSON-escaped code corruption in reasoning models like deepseek-r1:32b. These adaptations helped achieve a 2x performance jump on SWE-bench Verified benchmarks.

The researchers also implemented Hybrid Symbolic-Tag + High-Threshold Dense Embedding Retrieval to prevent irrelevant rules from being mounted in unrelated tasks, improving first-shot accuracy.

Z.ai
Article written by AI
Z.ai GLM-4.5-FlashFree
Free AI articles150 of 150
No limit after signing in with Telegram