Audit, telemetry, and recovery¶
Goal¶
Keep verifiable operational evidence and respond safely when canonical state or an external effect is uncertain.
Prerequisites¶
- Access to the deployment configuration and its key provider.
- The separately protected secure anchor.
- An approved evidence destination if audit export is enabled.
Steps¶
-
Verify journal and anchor integrity:
-
Inspect bounded evidence and metadata-only telemetry:
These commands derive historical analytics from the authoritative journal. Optional live OpenTelemetry export is a separate fail-open plane and does not replay history; configure it under Live observability.
-
If configured, inspect and drain the durable exporter queue:
Draining writes to the configured directory or WORM destination. Under the development profile, the noninteractive CLI therefore needs an explicit approval mode.
- If an effect is marked
outcome_unknown, reconcile it with the external system before any retry. Use only the operation-specific explicit recovery route.
Directory export writes ciphertext-free evidence to an existing permitted directory. HTTPS WORM export uses create-only object writes and deterministic names, but the remote service must independently enforce retention or object lock.
Expected result¶
Verification confirms the hash chain, signed checkpoint, and secure anchor. Telemetry reports counts and timing without prompts, hidden reasoning, or raw tool output. Export status identifies a durable position or a bounded actionable failure.
audit verify is always a complete journal audit. Normal startup may report
incremental after a version-two anchor has established the older prefix; inspect
storage.startup_verification in state doctor for the configured mode, actual path,
verified sequence range, inspected event count, and anchor format.
Verification¶
Retain the config hash, journal head, anchor status, relevant run/effect ID, decision revision, and export position with the operating record. Verify that exported evidence does not contain payload ciphertext, plaintext, nonces, or credential values.
Failure path¶
A chain, checkpoint, anchor, decryption, or projection-position failure activates read-only recovery and blocks new effects. Preserve the journal, key identity, secure anchor, and diagnostic output. Do not delete or rewrite events to make verification pass.
An exporter outcome_unknown blocks implicit delivery replay. Investigate the
destination and use the operator-authorized reset only after establishing whether the
object exists.
Next step¶
Use Troubleshooting to collect a safe diagnostic bundle, or read State and recovery for the underlying invariants.