STQA: A Benchmark for Stock-Focused Tabular Question Answering over Historical and Forecasted Data

Language models handle retrieval-style questions over stock tables reasonably well, but reasoning over their own forecasts stays near chance unless a forecasting tool is wired in, and even with perfect intermediate inputs it tops out well short of solved.

STQA is built on FNSPID, sampling 4,417 stock tables (open, close, high, low, volume over 700 aligned trading days) and instantiating 72 expert templates into 31,400 QA pairs, split 10,500 query reasoning, 10,400 numerical forecasting, 10,500 forecast-based reasoning, with intent and slot labels plus gold SQL. The baseline agent SQFRS routes to SQL, Time-MoE forecasting, and reasoning tools, tested with Qwen3-30B, GPT-oss-20B/120B, and Gemma 4-31B.

SQFRS beats a no-workflow Vanilla agent by 8.8425% accuracy on query reasoning and 12.2575% on forecast-based reasoning. Best FR accuracy is 62.43% (Gemma 4-31B). Without any forecaster, direct reasoning gives 50.88%-56.67%, against 69.17% for human stock investors.

Causes are isolated by an ablation chain injecting gold labels step by step: gold SLU +1.14% FR, gold routing +0.17%, gold SQL +2.55%, gold forecast data +19.85%, and FR still only 86.14% with everything gold. The forecaster is held fixed across backbones, so differences reflect the LLM.

Boundaries: templates dominate, and on 100 human-written questions QR fell 17.06% and FR 10.51%; 4 open models, equities only, 100-120 item human sets.

Use this if you are judging whether tool-augmented agents can reason under forecast uncertainty; forecast quality, not routing, is the binding constraint.