arxiv-2605-17172 SRC-0087 · paperOpenJarvis: Personal AI, On Personal Devices
Jon Saad-Falcon, Avanika Narayan, Robby Manihani, Tanvir Bhathal, Herumb Shandilya, Hakki Orhun Akengin, Gabriel Bo, Andrew Park
Created: 2026-05-16 · Ingested: 2026-09-14
https://arxiv.org/abs/2605.17172(opens in a new tab)Personal AI stacks, like OpenClaw and Hermes Agent, are becoming central to daily work, yet they route nearly every query (often over sensitive local data) to cloud-hosted frontier models. Replacing frontier models with local models inside existing stacks does not work: swapping Claude Opus 4.6 for Qwen3.5-9B drops accuracy by 25-39 pp across personal AI tasks like PinchBench and GAIA. Existing stacks bundle agentic prompts, tool descriptions, memory configuration, and runtime settings around a specific cloud model. Only the prompts can be tuned, and state-of-the-art prompt optimizers close just 5 pp of the local-cloud gap on their own. This motivates a decomposed personal AI stack: one that exposes individual primitives which can be optimized individually or jointly to close the local-cloud gap. We present OpenJarvis, an architecture that represents a personal AI system as a typed spec …
In brief
Dropping a local open-weight model into an agent stack built around a cloud model breaks the stack, not just the reasoning: most of the loss comes from prompts, tool descriptions, memory and runtime settings that were co-designed for the cloud model, and can be recovered by re-optimizing those fields at fixed model.
OpenJarvis represents a personal AI system as a typed spec over 5 primitives (Intelligence, Engine, Agents, Tools & Memory, Learning) and searches it with a frontier model as proposer: the teacher diagnoses failure clusters, proposes edits across 4 editable primitives, and a held-out gate accepts only non-regressing edits. Evaluation covers 8 benchmarks (508 tasks) including PinchBench, GAIA, LiveCodeBench and LiveResearchBench, 11 local models from 4 families, 3 cloud baselines, 7 hardware platforms, 5 runs each, GPT-5-mini as judge.
Swapping Claude Opus 4.6 for Qwen3.5-9B in OpenClaw and Hermes Agent costs 24.8–38.8 pp; retargeting the spec at fixed Intelligence recovers 77% of the PinchBench drop and 56–57% on GAIA. The best local spec (Qwen3.5-122B) hits 80.3% average vs 83.5% for Claude Opus 4.6, at ~800x lower marginal API cost and ~4x lower latency. Spec search adds 13.1–31.5 pp per student, 1.1–8.8 pp over LoRA at 7.1–10.9x lower optimization cost.
The design isolates causes reasonably: the fixed-Intelligence condition separates configuration from capability, and ablations separate proposer from move space (LLM proposer adds 10.0 pp on average over evolutionary search at the same 4-primitive space; widening from 1 to 4 primitives adds 5.5–16.5 pp). Weight edits are only 16–44% of accepted edits.
Limits: 5 runs, a single judge model, single-machine evaluation, and search still requires frontier cloud calls plus trace transmission; GAIA-style deep reasoning stays out of reach for a 9B model. If you are benchmarking local models as drop-in replacements, your numbers likely understate them.
Written from the abstract by claude-opus-5 on 2026-09-15, with every figure checked against it. Not a substitute for the paper.
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.
No claims cite this yet.