The Reports API gives AcmeTest one stable, tenant-safe view of an attempt and closes the improvement loop: mastery evidence, gate status, cohort context, form equivalence, per-source alignment, item/form analytics, and revision orders. Platform3 remains the learning-fact and aggregate system of record.
Important technical decisions
How to read each record. The stable itd-NNN fragment is its citation key. The metadata line pins lifecycle provenance; Chosen is the normative decision; every alternative bullet couples a reasonable option to the concrete drawback that rejected it; and the tradeoff is either stated in its own paragraph or directly in those option–consequence pairs. A future change adds a new ITD and records symmetric supersession rather than mutating history silently.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. AlphaTest stores no attempts, responses, result records, mastery state, candidates, or aggregates. Each response composes live Platform3 data with versioned AlphaTest blueprint/scale working state and may cache only HTTP bytes privately for at most 30 seconds, keyed by tenant and authorization fingerprint.
Alternatives rejected
- Reporting warehouse in AlphaTest: duplicates Results/Analytics and creates split truth.
- Client-side joins: blocks the AcmeTest maintainer’s job and leaks authorization complexity.
Tradeoff
Availability follows upstream health; the response declares component freshness and never substitutes stale private learning facts.
ACTIVE · DEFERRED CAPABILITY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Materialization is not released. It reopens only when Gates B, C, and D are satisfied together by authenticated wire evidence: Results and Caliper support the retry-safe protocol in ITD-032, and Results exposes the settled record through the exact attempt-keyed composition contract in ITD-028. After reopening, a scored attempt is reportable only after AlphaTest idempotently writes its settled outcome and components to Results, emits the corresponding Caliper event, and reads the exact Results record back. Partial completion returns a typed dependency problem with a retry-safe operation id.
Alternatives rejected
- Return success before write-back: violates the system-of-record promise.
- Distributed transaction across products: unavailable and couples independent upstreams.
- Ship a fail-closed endpoint as a completed capability: gives AcmeTest no working job and masks a missing upstream contract.
Tradeoff
The customer cannot materialize reports until Platform3 publishes the required seam; this is preferable to storing a shadow correlation or claiming a permanently failing route.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Paths and representations use QTI attempt/session/test ids, Results result_record ids, Content test_spec ids, CASE GUIDs, and OneRoster sourcedIds. AlphaTest adds only a report operation id for retry tracing; it never introduces aliases for upstream entities.
Alternatives rejected
- AlphaTest surrogate ids: translation glue and orphan risk.
- Display names as keys: mutable and ambiguous.
Tradeoff
Clients must retain opaque native ids and cannot rely on friendly aliases; that burden is accepted to eliminate translation and orphaned-identity risk.
ACTIVE · DEFERRED CAPABILITY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Populated AttemptReport reads are deferred until ITD-028 reopens them. The intended contract then requires every canonical report to identify candidate, administration, QTI attempt/test/session, Content blueprint, test kind, score, status, provenance, and upstream freshness. view selects bounded summary, standards, or evidence projections; omitted sections are named in omitted, never silently absent.
Alternatives rejected
- A different endpoint/schema per test kind: separate products and client branching.
- GraphQL: duplicate query primitive without a persona-blocking need.
- Populate partial reports from
producer_result_id or canonical_response: producer-specific guessing presented as a durable contract.
Tradeoff
The representation is stable enough for downstream design, but no example may claim to complete live until every required reference is dereferenceable on the wire.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Every standard and KC row is mastered, not_yet, gap, or insufficient_evidence and includes the policy/version, threshold, estimate, uncertainty, CASE GUID, and contributing QTI item/outcome references. A state without evidence is invalid.
Alternatives rejected
- Percent correct only: not actionable mastery.
- Binary default for thin evidence: overclaims certainty.
Tradeoff
Evidence-bearing rows are larger and unavailable when their evidence cannot be dereferenced; that cost is accepted so no mastery label outruns its proof.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Reports evaluates no new gate rule. It presents the pass/fail outcome written to Results under the immutable Content test_spec criteria version, including achieved/required values and the complete gap list on failure. Missing criteria or coverage evidence makes status indeterminate, never pass.
Alternatives rejected
- Local threshold: conflicts with blueprint authority.
- Default 80%: unverifiable and unsafe.
Tradeoff
Reports may return an indeterminate decision when blueprint criteria are incomplete, sacrificing a convenient answer to preserve blueprint authority.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Adaptive diagnostic reports require the mastery-engine scale score, scale/calibration version, standard error or interval, stopping reason, administered-item count, and the mastered/not-yet/gap map. Any MAP comparison is labeled a crosswalk, identifies the NWEA norms set, and is not presented as an NWEA-issued score.
Alternatives rejected
- RIT-like number alone: repeats the status quo’s non-actionable result.
- Percentile ranking as mastery: violates product intent.
Tradeoff
The diagnostic representation is larger and depends on the mastery-engine calibration receipt; that dependency is accepted because a lone scale number would not expose actionable gaps.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. testKind=formative uses the canonical attempt schema, returns immediate per-standard evidence, and writes through Results. It does not emit grade-level certification or external-test readiness unless the underlying blueprint explicitly carries qualifying coverage evidence.
Alternatives rejected
- Separate quiz report: unnecessary client surface.
- Infer readiness from a unit quiz: unsupported claim.
Tradeoff
Clients must tolerate kind-specific optional sections in one schema; that conditionality is accepted to avoid a second quiz-report product.
ACTIVE · DEFERRED CAPABILITY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Motivation. Analytics Alpha is live and already publishes approved XP, accuracy, completion, MAP-growth, and RIT/norms rollups. Those are valid upstream primitives, but none is the class or school-grade per-standard mastery cohort contract this decision requires; reusing one by name would misstate its grain and semantics.
Chosen. Cohort reads are deferred until Gate A publishes authenticated production class and school-grade per-standard mastery rollup contracts. The concrete reopening event is a versioned response for each grain that identifies class or school+grade, standards/KCs, numerator and denominator/population, formula or policy version, as-of time, suppression status and rule, source Results window, and provenance link. After reopening, class summaries proxy Analytics at GET /v1/classes/{classSourcedId}/cohort-report; grade summaries proxy it at GET /v1/schools/{schoolSourcedId}/grades/{grade}/cohort-report. Both require matching JWT resource constraints and OneRoster authorization. Reports never lists candidates and computes its own aggregate.
Alternatives rejected
- Adapt the existing XP, accuracy, completion, or MAP-growth rollup: wrong subject/grain and would fabricate mastery-cohort semantics.
- Aggregate Results rows locally: private math and divergent definitions.
- One overloaded class-or-grade identifier path: ambiguous resource type and authorization boundary.
- Return student rows for client aggregation: unnecessary personal-data exposure.
Tradeoff
Analytics exists and remains authoritative, but cohort reporting stays unavailable until it publishes this specific contract; that preserves one authoritative aggregate definition without denying valid upstream capabilities.
ACTIVE · DEFERRED CAPABILITY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Motivation. The live Analytics Alpha surface publishes several report-ready rollups, but no parallel-form-equivalence endpoint or evidence schema. Existing aggregate endpoints cannot establish observed form comparability.
Chosen. Equivalence reads are deferred until Gate A publishes an authenticated production Analytics form-equivalence contract. The concrete reopening event is a versioned response identifying blueprint and form ids, coverage/DOK match, sample sizes and population definition, method, statistic, confidence interval, tolerance, decision, analysis version, suppression status/rule, and downloadable evidence artifact. insufficient_evidence is first-class.
Alternatives rejected
- Relabel an existing Analytics rollup as equivalence evidence: fabricates a contract and statistical meaning.
equivalent: true: cannot be audited.- Assume same blueprint means equivalent observed difficulty: confuses design with evidence.
- Compute equivalence privately in Reports: duplicates Analytics-owned math.
Tradeoff
Retake certification cannot claim observed equivalence until Analytics publishes the named evidence object, while the existing Analytics rollups remain usable for their approved jobs.
ACTIVE · DEFERRED CAPABILITY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Readiness reads are deferred with AttemptReport until Gate C provides the exact gate decision and Content blueprint linkage. After reopening, each named-test entry gives projected_pass|not_ready|indeterminate, source test/version, crosswalk version, coverage fraction, DOK dominance result, qualifying blueprint slots, decision rule/version, and limitations. Only complete equal-or-higher-DOK coverage plus satisfied gate criteria may produce projected_pass.
Alternatives rejected
- Score correlation alone: does not prove content coverage.
- Marketing prose without receipts: fails the audit job.
- Join a plausible blueprint locally: cannot prove it governed the attempt.
Tradeoff
Incomplete crosswalk evidence yields indeterminate rather than an optimistic readiness claim; the conservative false-negative risk is accepted to keep projected pass auditable.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Every response includes sources[] with service, resource id, observedAt, version/etag, and status. Required source failure yields 503 and no apparently complete report. Optional enrichment failure yields 200 with incompleteSections and a problem object for each omitted section. Status 206 is reserved for a successful range response, as required by RFC 9110 §15.3.7.
Alternatives rejected
- Silent omission: indistinguishable from “no gaps.”
- Serve stale database copy: violates system-of-record boundary.
- 206 for a partial composition: nonconformant because the response is not satisfying a Range request.
Tradeoff
Live source composition can increase latency and surface explicit incomplete sections; that is accepted instead of serving a faster but stale or deceptively complete report.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Server-only base URLs select QTI, Results, Analytics, Content, CASE, OneRoster, Caliper, and mastery/blueprint services. Each upstream call propagates a request/trace id and authenticates with a least-privilege service credential or delegated token; no browser calls Platform3 directly. Implementation evidence must name the exact URL and captured call for each capability.
Alternatives rejected
- Hardcoded production URLs: breaks isolated verification.
- Local substitute endpoints: prohibited fabrication.
Tradeoff
Every deployment must configure and rotate several least-privilege upstream credentials and preserve traces; that operational cost is accepted to make real integrations provable.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Reports derives tenant and allowed students/classes/schools from the verified JWT, intersects those claim constraints with requested resources, and passes no broader access upstream. A missing resource and a resource outside those claim constraints both return the same 404 detail to prevent enumeration; logs retain the internal reason.
Alternatives rejected
- Authorize only at Platform3: composition can still leak cross-source existence.
- Caller-supplied tenant header: forgeable routing.
Tradeoff
The public 404 deliberately hides whether a resource exists, reducing caller diagnostics; internal trace ids preserve supportability without enabling enumeration.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. The path major version changes only for breaking semantics. Additive fields may appear within v1; enums include an explicit unknown fallback where upstream evolution can introduce values. Policy, calibration, crosswalk, and formula versions are data fields, not API versions.
Alternatives rejected
- Date-versioned API: needless client churn.
- Freeze schemas against upstream growth: brittle under Hyrum’s Law.
Tradeoff
Clients must ignore additive fields and handle unknown enum fallbacks; that forward-compatibility work is accepted to avoid needless major-version churn.
ACTIVE · DEFERRED CAPABILITY2026-07-24Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Report data is read-only. Attempt materialization stays deferred until Gates B, C, and D pass, then ships as one idempotent per-attempt :materialize command. Improvement analysis stays deferred until Gate F passes, then ships as one idempotent per-administration :analyze command that writes only revision workflow state and receipts. Bulk commands remain deferred until a registered administration-close workflow proves at least 100 scored attempts per batch and per-resource calls breach the published completion SLO.
Alternatives rejected
- Both bulk and single doubles retry semantics without a current blocking job.
- Bulk-only is a poor recovery unit.
- A local learning-fact or aggregate write violates the system-of-record boundary.
Tradeoff and impact of no action
Two bounded commands require separate replay state, but each maps to one owner transition and avoids an ambiguous all-purpose write.
ACTIVE · DEFERRED CAPABILITY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Populated candidate list, attempt detail, and readiness reads are deferred until Gate C passes; equivalence and cohort reads are deferred until Gate A passes. The intended post-gate shape is list + detail + focused sub-resources. Evidence is a bounded projection of attempt detail, not an independently mutable resource.
Alternatives rejected
- Detail only: forces remembered ids.
- One enormous report only: makes routine screens slow and overexposes evidence.
- Return empty or synthetic success while dependencies are incomplete: indistinguishable from a student with no results.
Tradeoff
Focused sub-resources can require more round trips than one giant response; bounded disclosure and independently citable evidence are worth that latency.
ACTIVE · DEFERRED CAPABILITY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Query execution is deferred with populated list reads. After Gate C passes, list reads support documented exact-match filters, modifiedSince, one stable allowlisted sort with id tie-break, limit 1–100, and an opaque tenant-bound cursor. Unknown filters or sorts fail with 400.
Alternatives rejected
- Arbitrary query language: security and performance ambiguity.
- Offset paging: unstable under arriving results.
- Tenant scans followed by AlphaTest-side joins: expensive, authorization-sensitive, and not an exact upstream attempt lookup.
Tradeoff
Integrators cannot issue arbitrary queries or offset jumps; the allowlist and cursor binding are accepted for stable sync, performance, and tenant safety.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Report resources expose source validators but have no client mutation, so Reports defines no If-Match write. The materializer honors Results/QTI conditional-write contracts internally; a source version race is retried from a fresh read, then surfaced as 409 with the stable source-version-conflict problem type and retryable=true if it cannot converge.
Alternatives rejected
- Mutable report overrides: creates a second truth.
- Ignore upstream ETags: risks overwriting a correction.
Tradeoff
The caller must retry a distinct operation after an unconverged conflict; the error cannot be confused with reuse of an idempotency key for a different request.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Idempotency-Key is required for materialization. For 24 hours Reports retains only an encrypted control record containing tenant id, method, canonical path, key hash, canonical request hash, operation id, lifecycle state, downstream idempotency keys, terminal status code, and timestamps—never a report body, score, response, or other learning fact. Same key and canonical request resumes or references that operation: completed operations skip all writes, recompose the report live from Platform3, return the recomposed status/body with the original operation id, and set Idempotent-Replayed: true. The replay is intentionally not byte-for-byte historical because upstream corrections must appear. Same key with a different canonical request returns the idempotency-conflict 409. Downstream keys derive deterministically from tenant, attempt, operation, and input hash.
Alternatives rejected
- Persist the original response: duplicates learning facts and conflicts with correction/deletion.
- Promise original response replay from the 30-second byte cache: impossible for the 24-hour key window.
- Optional keys: duplicate Results/Caliper writes under network retry.
- Attempt id alone: blocks legitimate correction/version operations.
Tradeoff
A replay can reflect newer authoritative facts and may return a different representation or read-time dependency error, but it never repeats the Results or Caliper writes. The operation id and replay header make that behavior explicit.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Require signed bearer JWTs with sub, tenantId, iat, exp, role, and the OAuth-style permission scope reports:read or reports:write. Optional studentIds, schoolSourcedIds, and classSourcedIds are resource claim constraints that narrow access. Anonymous access is 401; a valid token lacking the required permission scope is 403; a requested resource outside claim constraints is the non-enumerating 404 defined by ITD-014.
Alternatives rejected
- Role-only: too broad for student data.
- API key: lacks principal and fine-grained claims.
Tradeoff
JWT issuance and scoped-claim management are more involved than a static API key; that complexity is accepted to bind every request to tenant, principal, permission, and educational scope.
ACTIVE · DEFERRED CAPABILITY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. After Gate C opens list reads, ship polling with modifiedSince and cursor continuation. Until then, eventing is deferred with the read capability. Customer webhooks remain separately deferred until one registered audience has a production workflow whose result-visibility SLO is below the polling interval and supplies a stable HTTPS receiver; reopening requires signed delivery, replay, and dead-letter ITDs.
Alternatives rejected
- Webhooks now: new delivery product without a current blocked job.
- Polling without modifiedSince: wasteful full scans.
- Poll an inferred tenant scan: perpetuates the missing exact index.
Tradeoff
Polling introduces bounded visibility delay and recurring read traffic; it is accepted until a registered workflow proves that a webhook delivery product is necessary.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Failures use application/problem+json per RFC 9457 with stable absolute type, title, status, detail, instance, requestId, traceId, retryable, dependency, and optional fieldErrors. Stable types cover authentication, authorization, not found, validation, idempotency-conflict (409, same key/different request), source-version-conflict (409, authoritative versions did not converge), dependency unavailable, dependency incomplete, and internal error. Each symbolic name maps to one documented absolute type URI under the canonical docs origin.
Alternatives rejected
- Reuse idempotency conflict for a source race: gives the client the wrong recovery action.
- Ad hoc message strings: not safely automatable.
- Always 200 with error field: breaks HTTP tooling.
Tradeoff
The stable type registry is a compatibility surface and must remain additive within v1.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Tenant comes only from verified JWT tenantId. Tenant ids do not appear in paths, query, or client-controlled routing headers. Every cache key, idempotency record, upstream request, log access field, and database query is tenant-scoped.
Alternatives rejected
X-Timeback-Tenant as authority: spoofable; it may be emitted downstream only after verification where an upstream requires it.- Tenant in path: duplicates identity and creates mismatch cases.
Tradeoff
Callers must dereference stable problem-type documentation instead of relying on ad hoc prose; that indirection is accepted for machine handling and backward-compatible diagnostics.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Release requires: an unauthenticated 2xx fetch of the canonical and immutable docs URLs; the two production-signed, scoped principals and disjoint fixtures in ITD-031; contract tests for every documented status/envelope; anonymous and bidirectional cross-tenant probes; deterministic replay; the three-kind unified-runtime matrix in ITD-029; and an external AcmeTest-shaped consumer suite. For each claimed upstream capability, the run retains a redacted network/server trace naming upstream host, path family, request id, response status, and required response fields. The deployed regression first proves all seven report routes for all three test kinds against the immutable API target, then deploys architecture, coverage, and QC artifacts to their separate origins, reruns the same matrix, and proves both the API and public audit rerunner remain reachable. Smoke tests must fail if a worked example receives 503, lacks required fields, carries a coverage/audit build receipt, uses another test kind's vocabulary or prerequisites in either success or problem details, or cannot authorize the required reviewer principal; a fail-closed response is secure evidence, not success evidence.
Alternatives rejected
- Unit tests only: cannot prove real upstream use.
- Manual certification: violates autonomous eval requirement.
- Treat an SSO wall, 404 docs route, unscoped reviewer token, expected 503, or one-tenant demo mint as a passing smoke: certifies infrastructure rather than the customer job.
- Run QC only before publishing its report: misses the alias replacement that previously destroyed the target under audit.
Tradeoff
Release evidence is stricter and requires real tenant fixtures plus a post-audit redeployment probe, but it prevents inaccessible documentation, alias-clobbered runtimes, and permanently failing examples from being labeled complete.
ACTIVE2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Reports persists no report or learning-fact body. The sole write-path control state is the encrypted, tenant-scoped, 24-hour metadata record enumerated in ITD-020; it contains no student outcome and is deleted at expiry. Structured logs exclude names, response bodies, scores, item responses, and tokens; they retain hashed principal, tenant, native resource ids where operationally required, dependency statuses, and trace ids for 30 days. Deletion and correction remain upstream-owned and are reflected on the next live recomposition.
Alternatives rejected
- Independent Reports DELETE: cannot delete facts it does not own.
- Persisted response replay or a long-lived response cache: duplicates stale sensitive data.
Tradeoff
Operational idempotency metadata exists briefly, but its allowlist and expiry prevent it from becoming a shadow outcome store.
ACTIVE · DEFERRED CAPABILITY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. The candidate-scoped report index is deferred until Gate C publishes an authoritative candidate/attempt index or exact attempt lookup that can serve it without a tenant scan. After reopening, ship that narrow index because AcmeTest must discover attempts without remembering ids. Cohort and form resources remain direct purpose-built reads behind Gate A. A global /attempt-reports tenant dump has its own reopening trigger: a registered compliance export job requires it and its minimization/redaction policy is pinned.
Alternatives rejected
- Write-and-remember: blocks normal app navigation.
- Generic tenant-wide list: broad exposure without a named job.
- Scan all Results rows and infer attempt identity: neither exact nor privacy-minimal.
Tradeoff
Candidate browsing stays unavailable until Results publishes an authoritative index, sacrificing convenience to avoid tenant scans, fabricated joins, and wrong-kind leakage.
ACTIVE · DEFERRED CAPABILITY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Motivation. The approved Administration architecture pins settled Results read-back through ?qtiAttemptId=. An authenticated Results runtime probe at 2026-07-17T23:43:01Z used that exact camelCase parameter and returned 200 with a valid empty result set, confirming the lookup parameter exists. The latest retained collection evidence at 2026-07-17T19:17:07.998Z reported 31,075 records and a sample containing qti_attempt_id, administration_id, content_test_spec_id, and test_kind, but still omitted test_result_components, standard/KC classifications, and QTI item evidence. The live published Results reference and data dictionary document neither qtiAttemptId nor those evidence-bearing join fields. The runtime lookup is real; it is not yet the published, dereferenceable composition contract required by ITD-004 and ITD-005.
Chosen. Every populated AttemptReport read and its candidate index remain deferred. The single concrete reopening event is a published and deployed, tenant-scoped Platform3 contract that accepts an exact QTI attempt id and returns exactly one settled Results record with versioned, dereferenceable identifiers for: candidate; administration; QTI attempt, session, test, and item outcomes; Content test_spec; AlphaTest test kind; gate criteria and decision; standard/KC mastery rows; and item/component evidence. An authenticated production probe must follow each identifier on the wire and prove tenant A cannot resolve tenant B’s attempt before the gate opens.
Forbidden substitutes
- Treating runtime-only collection fields as a versioned public contract.
- Guessing that
producer_result_id is the QTI attempt id. - Parsing producer-specific
canonical_response as a cross-product join contract. - Scanning tenant result records and selecting a plausible row.
- Creating an AlphaTest attempt-to-result correlation table or copying components locally.
Alternatives rejected
- Compose several undocumented upstream heuristics: creates a Hyrum’s-Law dependency with no stable authority.
- Return a partially populated AttemptReport: violates the canonical required fields and can misstate gate/mastery evidence.
- Fail closed forever while documenting the endpoint as shipped: secure, but not a completed customer capability.
Tradeoff and impact of no action
AcmeTest cannot complete a Reports integration until Platform3 publishes the full evidence composition contract. Recognizing the live lookup while keeping the narrower evidence gate avoids both stale denial and premature reliance on undocumented record fields.
ACTIVE · SHIP2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Motivation. A production mastery-gate request was answered by the formative coverage deployment, proven by X-Alphatest-Build-Receipt: reports-formative-a1 and formative vocabulary. The later QC deployment contract also targeted that same mutable alias, so the audit could replace the API it was meant to verify. Coverage and audit artifacts had become competing deployable products on one alias.
Chosen. Exactly one immutable unified Reports runtime owns the canonical API origin and all /v1 routes. Architecture, coverage, and QC sites deploy to a separate stable documentation origin or immutable artifact targets and may never claim, rewrite, or alias the API origin. The runtime derives testKind only from the exact authoritative attempt → Results → Content composition; callers cannot select or override kind. The unified handler dispatches kind-specific projections internally while preserving one schema, and every dependency problem derives its vocabulary, requirements, and remediation from that authoritative kind. Every response emits one runtime release receipt, never a coverage or audit receipt. Deployment promotion is atomic, and a post-promotion matrix sends mastery-gate, adaptive-diagnostic, and formative fixture attempts through the same origin before and after every docs/coverage/QC deployment; each must return only its own vocabulary and provenance while the public audit rerunner remains reachable.
Alternatives rejected
- Coverage or audit deployments sharing the API alias: last writer wins and can make test kind or even API existence a deployment accident.
- Separate public products per test kind: violates the one-product contract and burdens AcmeTest with routing.
- Caller-supplied
testKind: permits a representation to contradict the authoritative blueprint and Results record. - A rerunner that names only a flip-flopping alias: cannot reproduce the immutable candidate that produced the evidence.
Tradeoff and impact of no action
The product must maintain distinct API and documentation/audit deployment targets, and unified promotion waits for every kind handler to pass one matrix. Without that separation, a valid request can receive a semantically wrong report or 404 after a successful audit.
ACTIVE · SHIP POLICY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. No route, docs example, or product claim is marked released until every dependency gate for that capability and Gate D have passed in the same immutable release candidate. Attempt detail and readiness require C+D; candidate history requires C+D plus the authoritative index; materialization requires B+C+D; cohort and equivalence require A+D. The evidence bundle must contain successful non-503 responses with required fields and upstream traces. Closed gates may be represented in architecture and gap documentation, but a fail-closed stub is not deployed or advertised as an available route. Reopening occurs only when the named gate evidence exists; an implementation returning 503 does not reopen anything.
Alternatives rejected
- Publish every path with 503 until upstreams arrive: looks complete structurally while the customer’s job remains impossible.
- Release each coverage cell independently: recreates alias clobber and inconsistent semantics.
- Call secure failure “partial availability”: confuses boundary safety with product completion.
Tradeoff and impact of no action
The public surface stays smaller until dependencies are real. This is stricter than preview stubs, but makes the release label falsifiable and prevents downstream deliverables from advancing on non-working examples.
ACTIVE · SHIP POLICY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Gate D cannot pass with the public demo mint or one tenant. The deployment pipeline provisions two short-lived production-signed reviewer JWTs, tenant A and tenant B, with the exact reports:read/reports:write scopes and disjoint candidate/class/school constraints, plus one authoritative fixture attempt per tenant. Automated probes prove anonymous requests return 401, same-tenant reads succeed, A requesting B and B requesting A both return the same non-enumerating 404, and no response, trace, cache entry, or idempotency replay exposes the other tenant. Tokens and student facts are redacted from retained evidence. If either credential or fixture is unavailable, certification is inconclusive and release remains closed.
Alternatives rejected
- Use
POST /api/dev/mint?tenantId=demo: it cannot prove separation and broadens a production mint surface. - Infer isolation from tenant-scoped code review: does not test deployed signer, routing, cache, or upstream claims.
- Accept a 401 from the reviewer token: proves misconfiguration, not authorized same-tenant behavior.
Tradeoff and impact of no action
The release pipeline must maintain two minimal fixtures and a secure mint path for automation. Without them, D2 cannot be certified and no tenant-data capability may be released.
ACTIVE · DEFERRED CAPABILITY2026-07-15Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. Gate B opens only when Platform3 publishes and production-proves: the versioned Results result_record plus test_result_components request/response schema; conditional and idempotent write behavior; the Caliper endpoint and event profile referencing the same native attempt/result ids; and an exact settled Results read-back. One AlphaTest operation derives distinct deterministic downstream idempotency keys, writes Results/components, emits Caliper, and records only the bounded control metadata in ITD-020. A retry after Results success but Caliper failure must observe the existing Results write, emit Caliper exactly once, then read back the settled record. Success is returned only after both upstream receipts and read-back agree. The wire proof includes a forced partial-failure replay and confirms no duplicate Results record, component, or Caliper event.
Alternatives rejected
- Local transaction or outbox containing learning facts: makes Reports a second system of record.
- Emit Caliper best-effort after returning success: breaks the promised audit trail.
- Recompute components in Reports on retry: duplicates mastery/scoring authority.
- Assume a component body from observed field names: fabricates an upstream contract.
Tradeoff and impact of no action
Materialization remains unavailable until both upstream write contracts support replayable evidence. The protocol accepts temporary dependency errors rather than silently losing Results or Caliper facts.
Motivation. Two full surface-QC runs correctly found the Reports routes unreleased, then routed the same known dependency state back to architecture after implementation cells had already built stubs and synthetic fixtures. That cycle produced zero successful customer jobs while repeatedly validating a design that already said the routes must not exist.
Chosen. Architecture, data dictionary, and customer website specification may advance while upstream gates are closed; implementation, surface QC, and integration may not start for a test kind until a machine-readable prerequisite manifest records immutable, authenticated wire receipts for every applicable dependency gate plus Gate D. For mastery gates and formative tests this means B, C, and D; adaptive diagnostics additionally require a production-learner Mastery Engine receipt that dereferences native QTI member/KC evidence, acknowledged Results components, and Caliper read-back. Cohort/equivalence adds A. A closed prerequisite parks the Reports implementation cell and emits a non-blocking owner escalation naming the missing contract or fixture. It does not create a route, return a permanent 503, advance on a sandbox payload, or roll a conforming Reports architecture back to simulate progress.
Alternatives rejected
- Build stubs and let surface QC rediscover the gates: consumes attempts without moving the customer job.
- Use synthetic report fixtures as implementation evidence: proves schema rendering, not declared upstream integration.
- Silently wait for a human: violates the autonomous loop; the correct state is a recorded escalation plus parked dependent cell.
- Relax the upstream contract to make the grid green: violates D3 and D5.
Tradeoff and impact of no action
The row can pause before implementation while other modules continue, but each retry has a concrete prerequisite receipt to seek and the loop no longer mistakes safe absence for a shippable product.
ACTIVE · SHIP POLICY2026-07-18Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. The canonical production Reports origin never exposes POST /api/dev/mint, /dev/mint, or an equivalent caller-accessible signing route; those paths return the same non-enumerating 404 as any absent route. Reviewer credentials are minted out of band by the deployment pipeline from the production signing secret, are short-lived and scope/tenant/resource constrained, and the signing secret never enters client code or a request-addressable function. A release receipt proves the mint paths are absent, anonymous tenant-data reads return 401, same-tenant reads return 200, and bidirectional cross-tenant reads return the same 404.
Alternatives rejected
- A demo-only public mint on production: broadens the attack surface and cannot prove production tenant separation.
- An authenticated mint endpoint: turns Reports into an identity provider it does not own.
- A static reviewer token: creates retention and revocation risk.
Tradeoff and impact of no action
Autonomous review requires pipeline access to the signer rather than a convenient HTTP mint, but the production API exposes no credential-creation primitive.
ACTIVE · DEFERRED CAPABILITY2026-07-18Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. When the candidate report index reopens under ITD-027, every returned row must satisfy every supplied allowlisted filter after authorization, including testKind, status, testId, and modifiedSince. testKind is compared to the authoritative Results-to-Content value, never a fixture label or caller override. An empty match returns 200 with an empty page and terminal cursor; a row of another kind is a contract violation. Cursor state binds the normalized filter set and tenant/authorization fingerprint so a cursor cannot change kind, scope, or principal between pages.
Alternatives rejected
- Return a generic fixture when no filtered row exists: creates exactly the mastery-gate-in-adaptive-list defect found by QC.
- Filter in AcmeTest: leaks wrong-kind data and duplicates server semantics.
- Ignore unknown filters: silently broadens the result set; unknown parameters instead use the typed unsupported-parameter problem.
Tradeoff and impact of no action
The index requires authoritative filterable fields and cursor binding before it can ship, but every page becomes safe to consume without client-side semantic repair.
ACTIVE · DEFERRED CAPABILITY2026-07-24Author: alphatest-loopSupersedes: noneSuperseded by: none
Motivation. The spec promises difficulty, discrimination, and distractor distributions after an administration, but the supplied Analytics and Caliper contracts do not expose the complete artifact at the required grain.
Chosen. Post-administration item and form analytics are a DEFERRED capability behind Gate F. Reopening requires authenticated, tenant-scoped Analytics/Caliper contracts that return versioned item difficulty, item discrimination, misconception-tagged distractor-response distributions, form difficulty/equivalence summaries, population and sample counts, suppression status and rule, analysis window, formula version, and immutable artifact references for one scored administration. Reports composes those owner-produced metrics and never derives cohort aggregates from candidate rows. If any required rollup is absent, AlphaTest files a Platform3 issue naming the missing field or grain, marks only the affected metric unavailable, and continues independent metrics whose owner receipts are complete; it never substitutes private math.
Alternatives rejected
- Compute item and form statistics in Reports from raw responses, duplicating Analytics.
- Block the entire improvement run when one independent metric is absent.
- Relabel generic accuracy or mastery-delta rollups as item-analysis evidence.
Tradeoff and impact of no action
Partial owner availability produces explicit unavailable metrics rather than a cosmetically complete report. Without this contract, AlphaTest either duplicates Analytics or cannot prove how an order was triggered.
ACTIVE · SHIP POLICY2026-07-24Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. For each named source assessment and immutable crosswalk version, Reports publishes alignment_percent = 100 × Σ(weight_s × coverage_s × rigor_s) / Σ(weight_s) across every source requirement s. weight_s is the source blueprint's declared point or item weight; coverage_s is 1 only when the requirement maps to a blueprint slot and 0 otherwise; rigor_s is 1 when target DOK is equal or higher and 0 otherwise. Missing, ambiguous, stale, or lower-DOK mappings therefore contribute zero and remain enumerated in the receipt; an empty denominator is invalid, never 100. The record pins formula_version, source assessment/version, Content test_spec/version, crosswalk hash, numerator, denominator, per-requirement contributions, and limitations. Predicted-versus-actual readiness is a separate Analytics-owned, population/suppression/versioned comparison included only when an authenticated external-score artifact exists; absence is explicit and does not alter structural alignment.
Alternatives rejected
- A coverage-only boolean hides weighting, rigor, and omissions.
- Treat missing mappings as excluded from the denominator, which inflates alignment.
- Blend predicted-versus-actual outcomes into structural alignment, making the score population-dependent.
Tradeoff and impact of no action
The strict zero treatment can lower the number while a source is incompletely onboarded. That is the point: without it, “superset” remains an unauditable slogan.
ACTIVE · DEFERRED CAPABILITY2026-07-24Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. After Gate F opens, POST /v1/administrations/{administrationId}/improvement-runs:analyze requires Idempotency-Key and emits deterministic revision orders only from versioned threshold policies and owner artifact references. Weak-item orders target Bank regeneration when a released policy trips on sufficient-sample discrimination, difficulty, dead or untagged distractors, or miskey evidence; calibration-delta orders target Blueprint when per-source predicted-versus-actual readiness breaches its released tolerance or the structural alignment receipt contains a correctable blueprint defect. Every order carries native tenant, administration, QTI item/form/test, Content test_spec/version, CASE/source-test ids, reason codes, observed metric and threshold, analysis policy/version, upstream evidence refs, requested action, target module, and deterministic order id. Gate F remains closed for a target until Bank or Blueprint publishes an authenticated, idempotent same-tenant service command and a wire receipt proves accepted, replayed, and rejected-cross-tenant behavior; Reports never edits a bank or blueprint directly.
Alternatives rejected
- Free-form recommendations cannot be consumed or replayed safely.
- Reports mutating Content/QTI bypasses Bank and Blueprint ownership.
- Threshold-free AI judgment is not deterministic or audit-grade.
- Emit orders against undocumented receiver endpoints and call the integration complete.
Tradeoff and impact of no action
A receiver can keep its lane closed while independent metrics and another receiver proceed, but no order is mislabeled delivered. Without versioned thresholds and acknowledgements, the loop cannot be replayed or audited.
ACTIVE · SHIP POLICY2026-07-24Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. AlphaTest Supabase stores only tenant-scoped improvement workflow state: improvement_run_id, deterministic revision_order_id, target module, native upstream ids and immutable evidence references, reason codes, policy/formula versions, threshold comparison, lifecycle status, idempotency hash, delivery attempts, target receipt, supersession links, and timestamps. It stores no student identity, response, score, mastery fact, distractor count, cohort aggregate, or copied Analytics artifact. Reads and writes are API-fronted, require verified service or reports JWT claims, are tenant-scoped in every query, and return the same non-enumerating 404 across absent and cross-tenant ids. Results, Caliper, Analytics, QTI, Content, CASE, Bank, and Blueprint remain authoritative; evidence is dereferenced live and stale or unreadable references fail closed.
Alternatives rejected
- Store analytics snapshots or student rows for convenient reporting, creating split truth.
- Keep orders only in process memory, losing replay and accountability.
- Expose the Supabase client directly with an anon key.
- Use mutable display names instead of native versioned references.
Tradeoff and impact of no action
Receipt rendering depends on live owner artifacts and can become temporarily unavailable. That cost preserves a single source of truth and a narrow, secure datastore boundary.
ACTIVE · DEFERRED CAPABILITY2026-07-24Author: alphatest-loopSupersedes: noneSuperseded by: none
Chosen. The evidence surface serves tenant-authorized, content-addressed alignment reports and item-improvement ledgers for every completed improvement run. GET /v1/improvement-runs/{improvementRunId}, /alignment-reports/{sourceKey}, and /revision-orders expose JSON metadata plus short-lived download links to immutable receipt bundles containing the Analytics/Caliper artifact references, per-source alignment contributions, policy and formula versions, threshold decisions, order lifecycle, Bank/Blueprint delivery receipts, omissions, and SHA-256 digest; they contain no credentials or copied learner facts. A run is complete only when each scored administration has item/form metric statuses, one alignment record for every named source test, every tripped order has a target acknowledgement or explicit blocked-upstream status, and the evidence links dereference. Release proof must exercise this acceptance chain on the production origin with anonymous rejection, same-tenant success, cross-tenant non-enumeration, and upstream traces.
Alternatives rejected
- A dashboard-only summary cannot support an audit file or machine verification.
- Mutable latest-report URLs erase historical provenance.
- Embed raw student outcomes in receipts, duplicating Results and Analytics.
- Mark a run complete while named sources or tripped orders silently disappear.
Tradeoff and impact of no action
Receipt generation must hash and retain control metadata while dereferencing owner artifacts. Without downloadable immutable receipts, a skeptical reviewer cannot verify that the loop ran or that every source test was counted.
Motivation. The prior policy said implementation “may not start,” yet the loop still launched a mastery-gate implementation and judged it against an eval requiring real tenant-backed success while the same architecture forbade the route. That is a self-inconsistent release contract, not useful evidence.
Chosen. Before the driver invokes any Reports implementation@test_kind doer, it must validate a machine-readable prerequisite manifest containing immutable authenticated receipts for that kind: mastery_gate and formative require Gates B, C, and D; adaptive_diagnostic also requires a production-learner Mastery Engine receipt; cohort/equivalence work additionally requires Gate A; improvement-loop work requires Gate F. A missing or failed receipt causes an atomic pre-execution transition to escalated/parked, appends a non-blocking owner escalation, and launches neither the implementation doer nor its customer/rubric evals. Production-success evals are instantiated only after the prerequisite check passes. Specification cells may continue, and a new owner receipt reactivates the parked cell on the next watchdog or bridge pass. No stub, sandbox route, permanent 503, or closed-gate evaluation can stand in for the customer job.
Alternatives rejected
- Launch implementation and let a production-success eval fail, which creates a predetermined loop rather than evidence.
- Deploy fail-closed or synthetic routes, which confuses safe absence with a shipped capability.
- Wait for a person, which violates autonomous non-blocking escalation.
- Relax upstream contracts to advance the grid, which violates D3 and D5.
Tradeoff and impact of no action
Closed rows visibly park instead of turning green, but independent cells and owner modules keep moving. Without this pre-execution rule, the same impossible implementation/eval pair can livelock forever.