ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference

ATEST is a resource-aware test-time scaling framework for LLM reasoning that adapts sampling to answer stability and live GPU conditions. ATEST-Online needs no offline calibration: it generates samples incrementally, estimates plurality-answer stability, monitors GPU load, and learns per-step energy cost online to decide when to stop. The authors report up to a 4.1-fold energy increase for the same query under rising GPU load. On GSM8K at high load, ATEST-Online reports 60.0% accuracy with no deferrals, versus 44.0% and 42.4% deferrals for ATEST-Calibrated. Evaluated on Phi-3-mini on an A100, with transfer tests on an L40S and Qwen2.5-3B.

Why it matters

Test-time compute is often allocated with fixed budgets that ignore real energy and contention costs. Adapting sampling to live hardware conditions could matter for cost-sensitive inference deployments.

Read with care

Source claims not independently checked; results are reported on a controlled load setup with a limited set of models and hardware, and online control shows higher tail latency and lower throughput on short tasks.