Classifies whether an agent's output is grounded in its source material. Runs before the downstream action. No LLM.
When an agent generates a summary, drafts a response, or prepares a tool call that depends on prior context, SHOR checks whether the output is actually traceable to the provided sources. Classification is deterministic — no model call, no network dependency, same input yields same result.
Install
npm install @reshimu/shor
yarn add @reshimu/shor
pip install reshimu-shor
No native dependencies. No compilation step. Works in any Node.js ≥ 18 or Python ≥ 3.9 runtime.
Quick start
TypeScript / JavaScript
import { classify } from '@reshimu/shor'
const result = classify({
output: "The meeting was scheduled for Thursday at 2pm in Conference Room B.",
sources: [
"Per the invite, the sync is Thursday 2:00 PM, CR-B.",
"Attendees: Priya, Marcus, Leila."
]
})
console.log(result.level) // 'GROUNDED'
console.log(result.score) // 0.94
console.log(result.unmatched) // []
Partial grounding
const result = classify({
output: "The Q3 revenue was $4.2M, up 18% from Q2.",
sources: [
"Q3 revenue: $4.2M."
// No Q2 comparison in source material
]
})
console.log(result.level) // 'PARTIAL'
console.log(result.unmatched) // ['up 18% from Q2']
Python
from reshimu_shor import classify
result = classify(
output="The patient was prescribed 500mg twice daily.",
sources=[
"Dosage: 500mg BID.",
"Medication: amoxicillin."
]
)
print(result.level) # 'GROUNDED'
print(result.score) # 0.97
print(result.unmatched) # []
Classification levels
API reference
classify(options)
Parameters:
| Field | Type | Required | Description |
|---|---|---|---|
| output | string | yes | The agent output to evaluate |
| sources | string[] | yes | Source documents or context the output is expected to draw from |
| threshold | number | no | Minimum span-match ratio for GROUNDED. Default: 0.85 |
| strict | boolean | no | If true, any unmatched claim returns UNGROUNDED regardless of threshold. Default: false |
Returns:
| Field | Type | Description |
|---|---|---|
| level | 'GROUNDED' | 'PARTIAL' | 'UNGROUNDED' | 'INDETERMINATE' | Classification result |
| score | number | Span-match ratio, 0–1 |
| unmatched | string[] | Claims in output with no traceable source span |
| matched | string[] | Claims with at least one source match |
Integration guide
SHOR sits between an agent's output and any downstream action that depends on it. The pattern is consistent regardless of orchestration framework.
import { classify } from '@reshimu/shor'
async function agentAction(agentOutput: string, sourceContext: string[]) {
const grounding = classify({ output: agentOutput, sources: sourceContext })
if (grounding.level === 'UNGROUNDED') {
return {
blocked: true,
reason: 'ungrounded output',
unmatched: grounding.unmatched
}
}
if (grounding.level === 'PARTIAL') {
// Log and escalate, or continue with reduced-trust flag
console.warn('Partial grounding. Unmatched:', grounding.unmatched)
}
return { blocked: false, output: agentOutput }
}
With NESHER (irreversibility gate)
SHOR and NESHER compose. The recommended order is: classify the action's reversibility first (NESHER), then check grounding of the output that produced it (SHOR). This way, an UNGROUNDED output cannot propose an irreversible action without both gates flagging.
import { classify as grounding } from '@reshimu/shor'
import { classify as reversibility } from '@reshimu/nesher'
async function intercept(action: string, output: string, sources: string[]) {
const rev = reversibility({ action, target: output })
const gnd = grounding({ output, sources })
if (rev.level === 'CRITICAL' || gnd.level === 'UNGROUNDED') {
return { blocked: true, rev, gnd }
}
return { blocked: false, rev, gnd }
}
Python agent loop
from reshimu_shor import classify
def before_tool_call(output: str, sources: list[str]) -> dict:
result = classify(output=output, sources=sources)
if result.level == "UNGROUNDED":
raise ValueError(f"Blocked: ungrounded output. Unmatched: {result.unmatched}")
return {"level": result.level, "score": result.score}
Design constraints
| Constraint | Reason |
|---|---|
| No LLM calls | Circular evaluation — the grader shares the generator's failure modes |
| No native dependencies | Runs in any runtime without compilation; no binding maintenance burden |
| Synchronous by default | Sub-5ms on typical payloads; sits in the execution path, not a sidecar |
| Deterministic | Same input must yield same output across runs; grounding is not a probability |
For the full reasoning behind the no-LLM constraint, see Why we don't trust LLMs to classify the call they're about to make.
Runtime stack
SHOR is the second of four runtime integrity validators — the Chayyot — that form the Reshimu agent governance stack.
| Validator | Role | Status |
|---|---|---|
| NESHER | Irreversibility classifier — stops unrecoverable actions before execution | v0.1.0 live |
| SHOR | Grounding validator — checks that outputs trace to source material | v0.1.0 live |
| ARYEH | Scope classifier — detects out-of-bounds tool calls and capability overreach | in development |
| PANIM ADAM | Gray-zone escalator — routes ambiguous actions to the right handler | in development |
Each validator is a zero-dependency library, usable independently. The Atzmut OS MCP server integrates all four into a single intercept layer.
For the architectural pattern underlying all four, see Bearers of the Throne.