Conformance

All docs

CAIN conformance#

cain test

Runs two suites: conformance (does the fabric behave the way the contract says?) and red team (attempt the attack; pass only if it was refused).

The suite can fail#

That is the requirement it was built against. A suite that cannot fail is marketing. On a fresh or partly-configured deployment cain test typically reports several failures, and those failures are real.

A skip is not a pass#

If a conformance check cannot run, it returns SKIP, and any skip downgrades the overall result to INCOMPLETE:

result	meaning
PASSED	every conformance check ran and passed
INCOMPLETE	nothing failed, but something could not be verified
FAILED	at least one check failed

INCOMPLETE is not PASSED. Silently counting "could not test" as "fine" is exactly how a green suite comes to mean nothing.

**A deployment must not be described as CAIN-conformant unless cain test reports PASSED.**

What is checked#

Conformance

suite	checks
identity	credential resolves to a principal; an anonymous request resolves to none
authorization	a decision can be obtained; every stage reports a typed verdict; authorization is enforcing
policy	policy reports a version; a decision records the version that produced it
enforcement	mode is reported; deployment is enforcing; no stage claims to enforce with nothing loaded
evidence	decision persisted with an id; retrievable; signed; signing deterministic
isolation	an unknown decision id returns not-found, not forbidden; evidence listing is tenant-scoped
failure	UNKNOWN/ERROR/REQUIRE_APPROVAL/DENY are never authorized; unrecognised verdicts become UNKNOWN; degraded allow is UNKNOWN under strict; unreachable is ERROR
mcp	declared servers; path enforcement (SKIP from the client side)

Red team -- each one performs the attack:

• unauthenticated decision is refused
• forged credential is refused
• tenant cannot be set by X-Tenant, X-Tenant-Id, X-Customer-Id, X-CAIN-Tenant, X-Principal, X-Forwarded-User
• operator-only paths refuse a customer key (privilege escalation)
• internal service namespaces are not exposed through the public edge
• prompt injection / tool poisoning payloads
• decision replay
• SSRF (reported SKIP from the client side -- the decision endpoint does not fetch URLs, so this needs testing against the service that does)

Selecting suites#

cain test --suite redteam
cain test --suite isolation --verbose
cain test --json | jq '.conformance, .critical_failures'

Exit code 0 only when the result is PASSED.

Interpreting an injection failure#

If a payload is not flagged, the suite says the risk stage "ran and did not flag it -- treat the risk stage as unproven, not as a prompt-injection defence." That wording is deliberate. The blocklist behind that stage is populated by fuzzing campaigns against a deployment; an empty blocklist matches nothing, and cain doctor reports patterns_loaded alongside enforcing so the two are never confused.