diff --git a/module/AUDIT_METRICS.md b/module/AUDIT_METRICS.md index fd87791c6..3f70c4914 100644 --- a/module/AUDIT_METRICS.md +++ b/module/AUDIT_METRICS.md @@ -179,6 +179,98 @@ the two counts disagreeing means an incomplete operation, not a small one. A dry run, and a run the redrive left alone, record nothing. +`saas.document.read_unscoped` and `saas.document.search_unscoped` are +observational version-1 contracts for a read or a search that involved no +collection: the caller could read none, or its request resolved to none. +`saas.document.read` and +`saas.document.search` each record one collection and require its boundary, so +without these the refusal and the empty answer reached only the producer's own +log. Neither requires an entry or an emission idempotency key. Their payload is: + +| Field | Kind | Required | Meaning | +| --- | --- | --- | --- | +| `solution` | string | no | Names the solution, as on every document event | +| `actor_principal_id` | uuid | no | The acting principal | +| `owner_principal_id` | uuid | no | The owning principal | +| `initiator` | string | no | A provenance string, such as a webhook delivery id | +| `correlation_id` | string | yes | Stable logical operation identity across transport retries | +| `outcome` | enum | yes | `empty`, `denied`, or `failed` | +| `duration_ms` | int | no | Observed elapsed milliseconds | + +They are types of their own rather than `saas.document.read` with an optional +boundary, because the registry declares each field's requirement on its own and +cannot state "boundary is required unless the outcome is denied, empty or +failed". So `boundary` is not a field, and a payload naming a collection is +refused as an unregistered field. `returned` is not an outcome, because evidence +is always returned from some collection and belongs on the scoped type, and +there is no `result_count`, because with no collection there is no result to +count. Nor is there a `version`. A refusal is the outcome `denied`, never a type +of its own. No field is bounded or marked PII; unknown payload fields are +rejected, so there is no field to carry a query, a prompt or an excerpt. + +`saas.document.confirmed` records a person confirming the value a reference +read at one version of a document: an append-only fact about that reference. +It is a transactional version-1 contract recorded only when the confirmation +commits — the module appends the event in the same transaction as its +append-only confirmation row, and a refused confirmation writes nothing — so it +carries no `outcome`. The entry is entry_id and the confirmation id is the +emission idempotency key, so a retried emission collapses; the emission is +refused without either. Its payload is: + +| Field | Kind | Required | Meaning | +| --- | --- | --- | --- | +| `solution` | string | no | Names the write, with `version` | +| `version` | string | yes | The document version the reference was read at | +| `boundary` | string | no | The data boundary (scope node) the write landed in | +| `actor_principal_id` | uuid | no | The acting principal | +| `owner_principal_id` | uuid | no | The owning principal | +| `initiator` | string | no | A provenance string, such as a webhook delivery id | +| `extractor` | string | yes | With the three rows below, names the reference confirmed | +| `extractor_version` | string | yes | | +| `field` | string | yes | | +| `format` | string | yes | | +| `address` | string | yes, at most 1024 bytes | The reference's position in its format's own notation | + +The version and every reference field are required: a confirmation that cannot +say which version and which reference it confirmed is not one. The confirmed +value is deliberately not a field — the spine records that a reference was +confirmed, where, and by whom, never what it read — so a payload carrying it is +refused as an unregistered field. No field is marked PII. + +`saas.document.extractor_revised` records an extractor attached to a document +being revised: attached, corrected or retired. It is a transactional version-1 +contract. The entry is entry_id, and the emission is refused without it or +without an emission idempotency key. The native extractor itself stays off the +trail; its digest names it. Its payload is: + +| Field | Kind | Required | Meaning | +| --- | --- | --- | --- | +| `solution` | string | no | Names the write, with `version` | +| `version` | string | yes | The document version the revision was derived from | +| `boundary` | string | no | The data boundary (scope node) the write landed in | +| `actor_principal_id` | uuid | no | The acting principal | +| `owner_principal_id` | uuid | no | The owning principal | +| `initiator` | string | no | A provenance string, such as a webhook delivery id | +| `extractor_key` | string | yes, at most 128 bytes | The extractor's key | +| `revision` | int | yes | The key's revision | +| `digest` | string | yes, at most 64 bytes | The revision's digest, which names the native extractor | +| `parent_digest` | string | no, at most 64 bytes | The parent revision's digest; a first attachment has none | +| `retired` | bool | yes | Whether the revision retires the key | +| `request_type` | string | yes, at most 128 bytes | The native request type | +| `proposal_kind` | string | yes, at most 16 bytes | What proposed the revision | +| `proposal_source` | string | no, at most 512 bytes; **PII** | The proposing run, receipt or derivation | + +Everything a revision always names is required. `proposal_source` names the +proposing run, receipt or derivation, but nothing bounds it to an identifier: it +is an unvalidated string, so it is marked PII and redacted from exports and +webhooks. It is the only field of the four types so marked. + +On all four types, a required string, uuid or enum field means a nonempty, +non-blank value, not merely a present key; a required int or bool is satisfied +by `0` or `false`, which are values. The registry, +`services/accounts/code/pkg/business/audit_registry.go`, is the source of these +tables. + ## Version acknowledgement Successful aggregate responses include `scope_contract_version: 1` (Connect JSON: diff --git a/module/services/accounts/code/pkg/infra/postgres_audit_provisioning_test.go b/module/services/accounts/code/pkg/infra/postgres_audit_provisioning_test.go index a60a70b10..9f9647f97 100644 --- a/module/services/accounts/code/pkg/infra/postgres_audit_provisioning_test.go +++ b/module/services/accounts/code/pkg/infra/postgres_audit_provisioning_test.go @@ -281,13 +281,25 @@ func TestBringingAPartitionInStepLocksThePartitionNotItsName(t *testing.T) { }) provisioned <- outcome{err, time.Since(started)} }() - // Provisioning is at the target: trying for it, or queued for it. + // Provisioning is at the target: sleeping between tries for it, or queued + // for it behind the reader. Nothing else here sleeps, and the reader holds + // nothing else. Provisioning cannot pass the target while the reader holds + // it, so from its first try until the reader commits it stays there. + // + // The observer reads pg_stat_activity outside any transaction. A session + // collects the list of backends, and the statement each is running, the + // first time it reads them in a transaction, and shows that same list until + // the transaction ends; only the wait event is read live. Read from inside + // the reader's transaction, a first read taken before provisioning sent its + // statement hid provisioning for the rest of the test. + observer := retentionSession(t) + readerPID := int32(reader.PgConn().PID()) require.Eventually(t, func() bool { var waiting int - err := reader.QueryRow(testCtx, ` + err := observer.QueryRow(testCtx, ` SELECT count(*) FROM pg_stat_activity WHERE query LIKE '%audit_events_provision_partitions%' AND pid <> pg_backend_pid() - AND (wait_event = 'PgSleep' OR wait_event_type = 'Lock')`).Scan(&waiting) + AND (wait_event = 'PgSleep' OR $1::integer = ANY(pg_blocking_pids(pid)))`, readerPID).Scan(&waiting) return err == nil && waiting > 0 }, 10*time.Second, 20*time.Millisecond, "provisioning never came to the target") _, err = reader.Exec(testCtx, `COMMIT`)