arxiv-2510-03215 SRC-0081 · paper

Cache-to-Cache: Direct Semantic Communication Between Large Language Models

Tianyu Fu, Zihan Min, Hanling Zhang, Jichao Yan, Guohao Dai, Wanli Ouyang, Yu Wang

Created: 2025-10-03 · Ingested: 2026-09-17

Multi-LLM systems harness the complementary strengths of diverse Large Language Models, achieving performance and efficiency gains that are not attainable by a single model. In existing designs, LLMs communicate through text, forcing internal representations to be transformed into output token sequences. This process both loses rich semantic information and incurs token-by-token generation latency. Motivated by these limitations, we ask: Can LLMs communicate beyond text? Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication. Thus, we propose Cache-to-Cache (C2C), a new paradigm for direct semantic communication between LLMs. C2C uses a neural network to project and fuse the source model's KV-cache with that of the target model to enable direct s…

In brief

Two language models can pass semantic content to each other by projecting and fusing KV-caches directly, beating the usual text hand-off on both accuracy and latency. Cache-to-Cache (C2C) trains a small fuser network that concatenates a Sharer model's KV-cache with a Receiver's, reweights it, and injects it through a per-layer learnable gate, with both LLMs frozen and token/layer alignment handled explicitly across tokenizers and depths.

Tested with Qwen3-0.6B as Receiver against Qwen2.5-0.5B, Llama3.2-1B and Qwen3-4B-Base Sharers on OpenBookQA, MMLU-Redux, ARC-C and C-Eval, C2C raised accuracy over the Receiver alone by 11.00%, 9.64% and 11.88%, and over text-to-text by 5.36%, 4.15% and 3.06%, with T2T speedups of 3.46x, 1.51x and 14.41x. On MMLU-Redux the Sharer's 80 text output tokens cost 1312ms of decoding versus 90ms of cache fusion.

The design isolates causes reasonably. An ablation separates added trainable capacity (fine-tuning the Receiver alone, 596M params) from self-fusion (Identical, same model both roles) from heterogeneous C2C, which wins on all 4 benchmarks; component ablation attributes +24.18% to residual fusion and +3.07% to gating. Baselines include query-level routing and matched Receivers.

Limits: mostly 0.5B-1.5B pairs, multiple-choice benchmarks with 64-token generations, pairwise only, and a weak Sharer can still mislead a stronger Receiver.

If you build multi-model pipelines, the text interface is not the only option.

Written from the abstract by claude-opus-5 on 2026-09-18, with every figure checked against it. Not a substitute for the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

No claims cite this yet.