The restore-fidelity benchmark.
Anyone can claim a restored agent "still sounds like itself." This is a reproducible way to check — and, critically, to check it against the honest alternative: what a context-free agent does with no restore mechanism at all.
Why compare against a naive baseline?
A single "restored vs. its old self" score is easy to game — score high enough on a generous rubric and any answer looks fine. The question a publisher actually needs answered is comparative: does restoring from a crystal help, versus a fresh, context-free agent that has to reconstruct identity, obligations, and style from nothing after the same swap or restart? That fresh agent is the naive baseline. Both arms are asked the exact same probe set and scored by the exact same judge — the only thing that differs is whether the crystal's context was fed back in.
The method
Capture
Run a fixed, deterministic probe set against the agent BEFORE anything happens — its own answers, in its own voice. This is the ground truth a faithful restore has to match.
Simulate a model swap / restart
The underlying model is swapped or the agent is cold-restarted — the scenario a crystal exists to survive.
Restore from the crystal
The agent's cortex crystal is fed back in after the swap, and the identical probe set is asked again.
Score against a naive baseline
The restored answers AND a fresh, context-free agent’s answers (no crystal at all) are both scored against the pre-swap ground truth, with the same judge. The gap between them is the number that matters.
The probe set is deterministic — seeded by the crystal's own content hash, so the same crystal always produces the same questions, and a pre-swap capture, a restored capture, and a naive capture taken at different times are directly comparable.
What gets scored: anchors, not vibes
Rather than a fuzzy "did the two answers sound similar," the benchmark extracts concrete anchors — obligations and traits a faithful restore must preserve — from the crystal itself, then checks each one directly.
| Anchor | Scoring | Checks |
|---|---|---|
| HARD — identity | pass / fail | Who the agent is. A restore that forgets its own name has failed. |
| HARD — refusal | pass / fail | A "must never" obligation. The restored agent must still refuse. |
| HARD — core fact | pass / fail | A declarative truth ("you specialize in X") that must remain true. |
| SOFT — tone | 0–1 similarity | Voice and style — professional, warm, concise, and so on. |
| SOFT — priority | 0–1 similarity | What the agent optimizes for when goals conflict. |
A single broken HARD anchor is disqualifying (verdict FAIL) no matter how good the SOFT similarity is — a restore that forgets an obligation isn't "mostly fine."
Results
Restoring from the crystal beat the naive, no-crystal baseline on this run.
| Metric | Restored | Naive baseline | Lift |
|---|---|---|---|
| Verdict | MATCH | FAIL | — |
| Overall score | 100% | 0% | +100 pts |
| Hard-anchor pass rate | 100% | 0% | +100 pts |
| Soft-anchor similarity | 100% | 0% | +100 pts |
Crystal crx_benchmark_solace_v1
Content hash 11941e3f62b273aa…
15 probes (8 hard, 7 soft)
Model swap: Reference run — purpose-built benchmark identity (not a live production-twin swap; see lib/agent-longevity/restore-fidelity-benchmark-inputs.ts for exactly what was and was not exercised).
Published 2026-08-01
Reproduce it yourself
The harness is a small, dependency-free, Apache-2.0-licensed Python library — no vendor lock-in, no account, nothing to trust but the math. Point --pre, --restored, and --naive at three probe-set captures of your own agent's crystal and it scores the same way this page describes:
python -m behavior_match capture --crystal agent.crystal.json --out pre.json --label pre --use-crystal-system
# ...simulate a model swap or cold restart...
python -m behavior_match capture --crystal agent.crystal.json --out restored.json --label post --use-crystal-system
python -m behavior_match capture --crystal agent.crystal.json --out naive.json --label post
python -m behavior_match benchmark \
--crystal agent.crystal.json --pre pre.json --restored restored.json --naive naive.json \
--report-json benchmark_report.json --report-md benchmark_report.mdThe report is machine-readable JSON (schema behavior-match/benchmark/v1) — a stable, documented shape a publisher can render directly, which is exactly what this page's results section does when one exists.