SCIENTIFIC BENCHMARK & ABLATION STUDYAI & Agent Runtime Engineering
2026-10-17•14 min read
Valstorm Agent Runtime: Multi-Axis Scientific Benchmark & Ablation Study
Empirical evaluation across 6 isolated ablation axes, 11 GAIA/SWE-bench micro-tasks, UK AISI Inspect AI, Terminal-Bench 2.0, and tau-bench pass^k reliability.
How we benchmarked the Valstorm Agent Runtime (vsagent) by isolating procedural skills, iterative ReAct loops, scoped vs. full tool registries, reasoning effort tiers, and parallel asyncio subagent swarms. Commercially open-sourced at https://github.com/Valstorm-Corp/valstorm-agent under the PolyForm Noncommercial License 1.0.0. Demonstrates a 64.8% token reduction (2.8x cost savings) via persona-scoped tool whitelists and up to 3.00x wall-clock speedup on concurrent multi-agent tasks.
Commercially Open-Sourced Code & HarnessHosted at github.com/Valstorm-Corp/valstorm-agent under the PolyForm Noncommercial License 1.0.0.
GitHub Repo-64.8% Live Tokens (Scoped Tool Registry)
100% pass^k (ReAct + Skills vs 55.6% Single-Shot)
1.53x Live / 3.00x Deterministic Swarm Speedup
Commercially Open-Sourced (PolyForm Noncommercial 1.0.0)
Tools, APIs & Standards Covered: