Show HN: Strot – Compile an LLM's intent once, run decisions locally

STROT is a project that compiles a decision task (a query plus a set of permitted actions) into a frozen standalone Python pipeline using an LLM, once at compile time. A small local model then makes decisions on CPU without runtime API calls, producing bounded typed-JSON output. The example shows support-ticket triage into actions such as billing_dispute and technical_defect, with per-action confidence scores. The source is a GitHub README page for the STROT-Compiler repository, referencing arXiv:2505.01636 and a PyPI package.

Why it matters

It proposes moving LLM-driven classification and routing work to compile time, which could lower runtime cost and variability for small local models if the approach holds up.

Read with care

Source claims not independently checked; the example confidence scores and accuracy claims come from the README and no benchmark results are reproduced here.