Quorum — interactive demo

A 90-second walkthrough

LangSmith tells you what happened.
Quorum tells you what experiment to run next.

Drop your AI traces in. Get back a ranked, evidence-backed optimization plan — then run the experiment and record the decision, all in the browser. Every number you're about to see is deterministic arithmetic on a real trace file. Nothing is invented, nothing is uploaded.

click the glowing element — or use → arrow keys

Analyze your traces LangSmith · OpenTelemetry · generic JSON — parsed entirely client-side

Drop a trace file here or click to browse

▮ parsing… normalizing 46 calls → 10 traces · pricing per call · checking outcomes

Report sample-trace.json · langsmith

Cost coverage: 100% Outcome coverage: 100% Architecture: Complete Synthetic demonstration dataset

0traces

0LLM calls

0tokens

0priced cost

0success rate

0cost / success

Executive summary decisions before analytics

Best first action Markdown Formatter up to $0.0927 on this sample · Upper-bound estimate

Highest projected impact $0.1196 Answer Generator · Simulated (upper bound)

Priced cost / successful task $0.0408 Measured · per-call model pricing

Best first action

LLM formatting step "Markdown Formatter" consumes 25.3% of priced sample cost

Why this first: low risk, high confidence, affects 70% of traces, represents 25.3% of priced sample cost, and targets a specific auxiliary component that can be validated independently.

claude-sonnetcurrent model

7calls

22,025 / 1,778tokens in / out

$0.0927sample cost

$0.0927upper-bound impact

confidence · high risk · low effort · medium upper-bound estimate

Assumption, stated: a deterministic formatter has ~zero LLM cost. An estimated maximum, never claimed as observed. The experiment below tests it.

Highest projected impact kept separate — never conflated with the safest first move

Reduce answer-generation context

up to $0.1196 input-cost reduction (32.6%)

Upper-bound simulation. Retriever metadata in the same traces reports 20 of 175 retrieved documents actually used (11%). The scenario assumes context scales with retrieved documents — validate before relying on it.

Cost by step per-call pricing from each call's own model · unknown models stay unpriced, never guessed

Answer Generator

$0.216759.1%

Markdown Formatter ◆

$0.092725.3%

Generator (retry)

$0.03278.9%

Response Validator

$0.01233.4%

Direct Answer

$0.00892.4%

Query Router

$0.00361.0%

Document Retriever

—unpriced

Experiment prefilled from the opportunity — editable, saved locally, never uploaded

Hypothesis

A deterministic or template-based formatter can produce equivalent output for "Markdown Formatter" at negligible LLM cost.

primary metric: cost per task ↓ · guardrail: success rate ±2%

Baseline · captured from sample-trace.json

10 traces · 32 LLM calls · coverage 100%

cost/task $0.0367 · success 90%

Baseline vs variant variant.json · 10 traces · deterministic formatter in place

Warning: baseline is synthetic and the variant is a user upload — flagged automatically, results may not be comparable.

Primary metric — cost per task

$0.0367 → $0.0015 (−95.9%)

Passes — the variant meets the configured arithmetic success condition on this pair of snapshots. Two samples compared honestly; no statistical claims.

guardrail	baseline	variant	change	allowed	result
Success rate	90.0%	90.0%	0.0%	±2%	Pass (unchanged)
Markdown validity	—	—	—	—	Manual check

✓ Decision recorded: adopted · "validated on holdout traces" · experiment archived locally

That's the loop

Upload→Evidence→ Best first action→Experiment→ Comparison→Decision

Observability tools stop at "here's what you spent." Quorum turns traces into the next experiment worth running — with every figure labeled measured, calculated, or simulated, and every recommendation shipped with its own validation plan. Fully client-side. Four deterministic detectors. Zero LLM calls to produce a recommendation.