Judge Policies

A policy starts from a template, gets scoped (for whom / which group / which workspace) and can be tried on sample text before it's ever saved.

#Template catalogue

Each template ships a default instruction, surface set and (where relevant) a suggested trigger — all editable before saving. "Custom" starts empty — the admin either writes the instruction by hand or generates a draft from a description.

#Scope

model_group

A policy tied to one user's specific model group.

user

Applies to all of one user's traffic.

workspace

Tied to a workspace; it is added on top of the user scope (never overrides it) and only fires when the request carries a workspace.

Resolution order is model_group → user → workspace; within each scope, policies are ordered by `order_index`.

#Generate & test

Generating a policy draft from a description never saves anything — it just returns a filled-in form. Testing a policy against sample text before saving doesn't save either — a test always runs isolated (never clustered) and writes no real verdict row. If a tiered policy's trigger doesn't match the sample text, the policy shows as "gated out" — distinct from the judge saying "allow": the model was never called at all.

#Limits

timeout_sec

Left empty, it inherits the global `judge.default_timeout_sec` setting — if that global value later changes, every un-overridden policy follows automatically.

max_input_chars

Same inheritance rule: empty means it comes from the global `judge.max_input_chars`. When several policies share a cluster, the truncation limit used is the SMALLEST of the members' limits, and the timeout is the LARGEST.

#Streaming (response surface)

buffer

The reply is fully collected before it reaches the client — the judge can still block/sanitize, at the cost of a delayed first byte.

log

The reply has already reached the client; the judge only records — it can no longer block anything.

skip

Never runs on the streaming path at all.

When a request matches several response-surface policies, the strongest mode wins (buffer > log > skip).

#A block in the agent loop is not a 422

#Cost & Activity

Every verdict row carries a `cost_usd`: when the judge model's price is known, it's computed from prompt/completion tokens; when it isn't, `cost_usd` is left blank — it is NEVER estimated. Because of clustering, that field (along with token usage) is written only to the cluster's first policy; the Judge Activity page (the verdicts list) shows `cluster_id`, `cached` (whether the decision came from the verdict cache) and the score/category/reason on every row. This `cost_usd` is its own ledger — it never enters the totals on Observability → Cost Tracking, which aggregates gateway requests; see Judge Activity here for judge spend instead.

One caveat for totals spanning a deploy boundary: pricing verdicts by the model the gateway actually served is a later addition than `cost_usd` itself, so a verdict row written before it, for a group-configured `judge_model`, has no cost at all, while a row written after does. A window covering both will show the total step up without any extra spend having occurred — the two periods aren't directly comparable.

A row's `judge_model` shows whatever name resolved for that call — the policy's own `judge_model` field when it has one set, otherwise the global `judge.default_model` default. Like `policy_name`, it's a snapshot: it keeps whatever resolved at the time even if the policy later changes or is deleted. That name can be a model GROUP; when it is, the gateway resolves the request to an actual model, and `cost_usd` is priced against that resolved model, not the name shown on this row — BUT only when the request is served through the generic upstream path: on antigravity, bedrock, and cursor nodes the request returns before that step, so `cost_usd` falls back to the group name and stays blank there too (of the three, only bedrock is usage-billed). The result: the `judge_model` shown on a row and the model its cost actually came from can differ, and the row itself doesn't call that out separately.

That pricing relies on `_resolve_judge_model_owner` to find which node serves the model (attribution only happens if exactly one node serves it, otherwise it's left blank), and the result is cached in Redis for 120 seconds (`JUDGE_MODEL_OWNER`) — this cache is not invalidated when the node topology changes, only the TTL clears it. If a model's node changes within that window, `cost_usd` can be priced against the wrong provider for up to two minutes; that is a wrong price, not an unknown one.