From 442b159da2e8fd9a48dd99d8c227dbea81a91c8f Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 31 Aug 2026 10:56:32 +0000 Subject: [PATCH 1/8] feat(analysis): bind Rubin loading uncertainty to an analysis-run profile Operators still cannot request the already-merged psychometric_core draw-mean OLS loadings and Rubin T combination as a digest-bound analysis-run output. Bind them jointly as rubin_loading_uncertainty_v1 / tepp.rubin_loading_uncertainty.v1 (ADR 0034). Cutoff-filter observations, refuse raw proportions and single-draw inputs, and keep the claim boundary as complete-data OLS combination rather than Mislevy person-level plausible values. Not a new ESEM/DSEM estimator, not CWC, not a Driver p.16 std restore, and not persistence. --- CHANGELOG.md | 2 + Cargo.lock | 1 + crates/analysis_engine/Cargo.toml | 2 + crates/analysis_engine/src/lib.rs | 41 +- .../src/rubin_loading_artifact.rs | 459 ++++++++++++++++++ .../tests/rubin_loading_execution_contract.rs | 318 ++++++++++++ docs/TRACEABILITY.md | 1 + ...-rubin-loading-uncertainty-analysis-run.md | 82 ++++ docs/adr/README.md | 2 + .../rubin-loading-uncertainty-analysis-run.md | 19 + 10 files changed, 926 insertions(+), 1 deletion(-) create mode 100644 crates/analysis_engine/src/rubin_loading_artifact.rs create mode 100644 crates/analysis_engine/tests/rubin_loading_execution_contract.rs create mode 100644 docs/adr/0034-rubin-loading-uncertainty-analysis-run.md create mode 100644 docs/doctoring/rubin-loading-uncertainty-analysis-run.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 062a69412..41193341e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,8 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang ## [Unreleased] +- `analysis_engine` jointly binds posterior-draw OLS loading means (`recover_loading_point_estimate_mean`) and Rubin (1996) total variance (`combine_draw_level_ols_loadings`) to the `rubin_loading_uncertainty_v1` analysis-run output profile. Observations unavailable at the request cutoff are excluded; the digest-bound `tepp.rubin_loading_uncertainty.v1` artifact records the point-estimate mean and Rubin `Q̄`/`Ū`/`B`/`T` and refuses Mislevy person-level plausible-value claims. This is not a new ESEM/DSEM estimator, not CWC, not a Driver p.16 `std` restore, and not persistence. + - `event_core` adds bounded Allen interval-consistency classification, atomic path-consistency closure, contradiction/resource refusals, and an explicit dependency-error fallback without claiming unrestricted global satisfiability. - `psychometric_core` recovers the Driver, Oud, and Voelkle (2017, Table 2, p. 12 `MANIFESTTRAITVAR`; §7.1, p. 19; p. 16 `MANIFESTTRAITVARstd`; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-27T14:20Z from https://www.jstatsoft.org/index.php/jss/article/download/v077i05/1104) scalar standardised manifest-trait variance on current main after `0ce16e8` dropped the pre-consolidation code while research notes already named the map (register items 83–84). Table 2 names `MANIFESTTRAITVAR` `Ψ_τ` the additional time-invariant variance-covariance on the measurement level and sets it `NULL` when there is no manifest trait. Equation 5 writes `Γ ~ N(τ, Ψ)` and names that covariance the manifest traits. Section 7.1 names manifest traits stable individual differences in indicator levels, distinct from process-level `TRAITVAR` `φ_ξ`. Page 16 prints standardised matrices with the suffix `std` when appropriate. The printed example on p. 16 is `discreteDRIFTstd`, not `MANIFESTTRAITVARstd`. Footnote 4 standardises using only the relevant variance, not the total. The relevant variance for that named indicator-level correlation is `MANIFESTTRAITVAR`, not process-level `TRAITVAR` and not residual `MANIFESTVAR` `θ`. The 2017-era source forms `MANIFESTTRAITVARstd` only when `MANIFESTTRAITVAR != 0`, as `solve(sqrt(diag(MANIFESTTRAITVAR) + ridging)) %&% MANIFESTTRAITVAR` when `verbose = TRUE`. OpenMx `%&%` is `t(A) %*% B %*% A`. Unlike `TRAITVARstd`, that formation adds `diag(c(ridging), n.manifest)`. The default `ridging = FALSE` adds 0, not `0.0001`; that ridge is a numerical hack and is not this exact map. The scalar correlation is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR`. Form strictly positive `ψ` first, then `1 / √ψ`, then `(1 / √ψ) ψ (1 / √ψ)`. Unstandardised `MANIFESTTRAITVAR` is defined for a zero trait; standardised `MANIFESTTRAITVAR` is not. Zero `MANIFESTTRAITVAR` skips forming `MANIFESTTRAITVARstd` in the 2017-era source and fails closed here. Indicator-level trait variance is an event-time structural quantity, so a non-event clock fails closed. `MANIFESTTRAITVAR` does not require stable `a < 0`. Distinct positive `ψ` recover the same 1. `trait / trait = 1` is `TRAITVARstd` and recovers the same number and remains a distinct named quantity. `θ` is `MANIFESTVAR` and is measurement error, not this correlation. Meredith (1993) remains unread (web search 2026-08-27T14:20Z: Springer/Cambridge Core paywalled; Unpaywall historically `is_oa: false`; Springer `content/pdf` is an HTML stub). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). Still not a Kalman filter, not a matrix `expm`, not ESEM estimation, not DSEM, and not ctsem estimation. diff --git a/Cargo.lock b/Cargo.lock index 454a7d612..28a0f0cb8 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -74,6 +74,7 @@ dependencies = [ "corpus_split", "event_core", "membership_core", + "psychometric_core", "relation_graph", "serde", "serde_json", diff --git a/crates/analysis_engine/Cargo.toml b/crates/analysis_engine/Cargo.toml index 7322212b2..d1f55851c 100644 --- a/crates/analysis_engine/Cargo.toml +++ b/crates/analysis_engine/Cargo.toml @@ -15,6 +15,7 @@ publish = false [dependencies] event_core = { path = "../event_core", version = "0.2.0" } +psychometric_core = { path = "../psychometric_core", version = "0.2.0" } serde = { workspace = true } serde_json = { workspace = true } sha2 = { workspace = true } @@ -26,6 +27,7 @@ uuid.workspace = true [dev-dependencies] corpus_split = { path = "../corpus_split", version = "0.2.0" } membership_core = { path = "../membership_core", version = "0.2.0" } +psychometric_core = { path = "../psychometric_core", version = "0.2.0" } relation_graph = { path = "../relation_graph", version = "0.2.0" } [lints] diff --git a/crates/analysis_engine/src/lib.rs b/crates/analysis_engine/src/lib.rs index 72bd5854c..eb86f4564 100644 --- a/crates/analysis_engine/src/lib.rs +++ b/crates/analysis_engine/src/lib.rs @@ -8,13 +8,17 @@ //! through [`tepp_api`]. It deliberately does not claim latent-variable or topic //! estimation authority; those estimators remain separate scientific crates. //! estimation authority; it invokes estimators through their scientific crate -//! contracts and preserves their artifact meaning. +//! contracts and preserves their artifact meaning. Rubin loading uncertainty +//! is invoked through [`psychometric_core`] and is not Mislevy person-level +//! plausible-value pooling. mod case_deletion_refit; mod lineage_criterion; +mod rubin_loading_artifact; mod topic_context_posterior; mod topic_lineage_artifact; +use psychometric_core::PsychometricError; use serde::Serialize; use sha2::{Digest, Sha256}; use std::collections::BTreeSet; @@ -46,6 +50,13 @@ pub use lineage_criterion::{ LineageCriterionFit, LineageCriterionFitError, LineageCriterionObservation, fit_lineage_criterion_posteriors, }; +/// Rubin loading-uncertainty artifact and execution contracts. +pub use rubin_loading_artifact::{ + RUBIN_LOADING_ARTIFACT_BYTE_LIMIT, RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION, + RUBIN_LOADING_MODEL_CONTRACT_VERSION, RUBIN_LOADING_OUTPUT_PROFILE, RubinLoadingObservation, + RubinLoadingUncertaintyArtifact, RubinLoadingUncertaintyExecution, + execute_rubin_loading_uncertainty_run, +}; /// Bounded posterior topic-context producer contract and record types. pub use topic_context_posterior::{ TOPIC_CONTEXT_POSTERIOR_BYTE_LIMIT, TOPIC_CONTEXT_POSTERIOR_SCHEMA_VERSION, @@ -248,6 +259,10 @@ pub enum AnalysisEngineError { TopicMeasurement(TopicMeasurementError), /// A topic-lineage artifact violated its bounded schema or count invariants. InvalidTopicLineageArtifact, + /// A psychometric recovery rejected the offered coordinates. + Psychometric(PsychometricError), + /// A Rubin loading-uncertainty artifact violated its bounded schema. + InvalidRubinLoadingUncertaintyArtifact, } impl fmt::Display for AnalysisEngineError { @@ -262,6 +277,10 @@ impl fmt::Display for AnalysisEngineError { Self::LimitExceeded => "analysis corpus exceeded its execution bound", Self::TopicMeasurement(error) => return error.fmt(formatter), Self::InvalidTopicLineageArtifact => "invalid topic lineage artifact", + Self::Psychometric(error) => return error.fmt(formatter), + Self::InvalidRubinLoadingUncertaintyArtifact => { + "invalid Rubin loading-uncertainty artifact" + } }; formatter.write_str(message) } @@ -281,6 +300,12 @@ impl From for AnalysisEngineError { } } +impl From for AnalysisEngineError { + fn from(error: PsychometricError) -> Self { + Self::Psychometric(error) + } +} + /// Execute the cutoff-safe temporal evidence readiness analysis. /// /// Evidence whose `available_time` is later than the request cutoff is excluded @@ -415,6 +440,7 @@ mod tests { AnalysisEngineError, AnalysisEvidenceUnit, MAX_ANALYSIS_IDENTIFIER_BYTES, MAX_EVIDENCE_UNITS, TopicMeasurementError, add_membership_count, execute_analysis_run, }; + use psychometric_core::PsychometricError; use temporal_core::{AvailableTime, EventTime}; use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState, ApiError}; @@ -681,6 +707,14 @@ mod tests { AnalysisEngineError::InvalidTopicLineageArtifact, "invalid topic lineage artifact", ), + ( + AnalysisEngineError::Psychometric(PsychometricError::InsufficientDraws), + "Rubin total variance requires at least two complete-data draws", + ), + ( + AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact, + "invalid Rubin loading-uncertainty artifact", + ), ]; for (error, message) in messages { assert_eq!(error.to_string(), message); @@ -689,6 +723,11 @@ mod tests { assert_eq!(converted.to_string(), "invalid API wire payload"); let from_topic: AnalysisEngineError = TopicMeasurementError::DidNotConverge.into(); assert_eq!(from_topic.to_string(), "topic estimator did not converge"); + let from_psych: AnalysisEngineError = PsychometricError::InsufficientDraws.into(); + assert_eq!( + from_psych.to_string(), + "Rubin total variance requires at least two complete-data draws" + ); assert_eq!( add_membership_count(u64::MAX, 1), Err(AnalysisEngineError::ArithmeticOverflow) diff --git a/crates/analysis_engine/src/rubin_loading_artifact.rs b/crates/analysis_engine/src/rubin_loading_artifact.rs new file mode 100644 index 000000000..16a84017c --- /dev/null +++ b/crates/analysis_engine/src/rubin_loading_artifact.rs @@ -0,0 +1,459 @@ +//! Digest-bound Rubin loading uncertainty as an analysis-run profile. + +use psychometric_core::{ + IndicatorKind, PsychometricError, combine_draw_level_ols_loadings, + recover_loading_point_estimate_mean, +}; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use temporal_core::{AvailableTime, KnowledgeCutoff}; +use tepp_api::{ + AnalysisResultSummary, AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalResult, +}; + +use crate::{ + AnalysisEngineError, MAX_EVIDENCE_UNITS, format_digest, require_receipt_identity, + valid_identifier, +}; + +/// Versioned schema for a completed Rubin loading-uncertainty artifact. +pub const RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION: &str = "tepp.rubin_loading_uncertainty.v1"; +/// Model contract required by the Rubin loading-uncertainty execution path. +pub const RUBIN_LOADING_MODEL_CONTRACT_VERSION: &str = "rubin_loading_uncertainty_v1"; +/// Analysis-run output profile required for a Rubin loading-uncertainty artifact. +pub const RUBIN_LOADING_OUTPUT_PROFILE: &str = "rubin_loading_uncertainty_v1"; +/// Maximum canonical artifact JSON size. +pub const RUBIN_LOADING_ARTIFACT_BYTE_LIMIT: usize = 256 * 1024; +const RUBIN_LOADING_INFERENCE_STATUS: &str = "rubin_combined_ols_loadings_not_mislevy_pv"; +const RUBIN_LOADING_STATISTIC_COUNT: u64 = 5; + +/// One already-mapped factor score with complete-data indicator draws. +#[derive(Clone, Debug, PartialEq)] +pub struct RubinLoadingObservation { + factor_score: f64, + indicator_draws: Vec, + available_time: AvailableTime, +} + +impl RubinLoadingObservation { + /// Bind one factor score and its complete-data indicator draws to an + /// availability clock. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidEvidence`] when the factor score is + /// non-finite, no draws are supplied, or any draw is non-finite. + pub fn new( + factor_score: f64, + indicator_draws: Vec, + available_time: AvailableTime, + ) -> Result { + if !factor_score.is_finite() + || indicator_draws.is_empty() + || indicator_draws.iter().any(|value| !value.is_finite()) + { + return Err(AnalysisEngineError::InvalidEvidence); + } + Ok(Self { + factor_score, + indicator_draws, + available_time, + }) + } + + /// Return the already-mapped factor score. + #[must_use] + pub const fn factor_score(&self) -> f64 { + self.factor_score + } + + /// Return the complete-data indicator draws in source order. + #[must_use] + pub fn indicator_draws(&self) -> &[f64] { + &self.indicator_draws + } + + /// Return the availability clock used for cutoff eligibility. + #[must_use] + pub const fn available_time(&self) -> AvailableTime { + self.available_time + } +} + +/// Completed, bounded Rubin loading-uncertainty result. +#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)] +#[serde(deny_unknown_fields)] +pub struct RubinLoadingUncertaintyArtifact { + /// Exact versioned schema identity. + pub schema_version: String, + /// Opaque accepted-run identity. + pub run_id: String, + /// Immutable source snapshot identity. + pub snapshot_id: String, + /// Historical evidence cutoff used by the combination. + pub knowledge_cutoff: String, + /// Eligible observations after cutoff. + pub observation_count: u64, + /// Complete-data draws combined by Rubin `T`. + pub draw_count: u64, + /// Observations excluded because availability was after the cutoff. + pub excluded_after_cutoff_count: u64, + /// Admitted indicator-kind wire name. + pub indicator_kind: String, + /// Arithmetic mean of per-draw OLS loadings. Not Rubin `T`. + pub point_estimate_mean: f64, + /// Rubin mean complete-data loading `Q̄`. + pub mean_loading: f64, + /// Mean complete-data sampling variance `Ū`. + pub within_variance: f64, + /// Between-draw variance `B`. + pub between_variance: f64, + /// Total variance `T = Ū + (1 + 1/m) B`. + pub total_variance: f64, + /// Fixed claim boundary for consumer copy. + pub inference_status: String, +} + +impl RubinLoadingUncertaintyArtifact { + /// Parse and fully validate a bounded artifact JSON payload. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact`] + /// when the schema, identifiers, counts, variances, or claim boundary fail. + pub fn from_json(payload: &str) -> Result { + if payload.len() > RUBIN_LOADING_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + let artifact: Self = serde_json::from_str(payload) + .map_err(|_| AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact)?; + artifact.validate()?; + Ok(artifact) + } + + /// Serialize canonical validated artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation, serialization, or size failure. + pub fn to_json(&self) -> Result { + self.validate()?; + let payload = + serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure)?; + if payload.len() > RUBIN_LOADING_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + Ok(payload) + } + + /// Return the lowercase SHA-256 digest of canonical artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation or serialization failure. + pub fn sha256(&self) -> Result { + self.to_json() + .map(|json| format_digest(Sha256::digest(json.into_bytes()))) + } + + fn validate(&self) -> Result<(), AnalysisEngineError> { + if self.schema_version != RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION + || !valid_identifier(&self.run_id) + || !valid_identifier(&self.snapshot_id) + || KnowledgeCutoff::parse_rfc3339(&self.knowledge_cutoff).is_err() + || self.observation_count < 2 + || self.draw_count < 2 + || !admitted_indicator_kind(&self.indicator_kind) + || !self.point_estimate_mean.is_finite() + || !self.mean_loading.is_finite() + || !self.within_variance.is_finite() + || self.within_variance < 0.0 + || !self.between_variance.is_finite() + || self.between_variance < 0.0 + || !self.total_variance.is_finite() + || self.total_variance < 0.0 + || self.inference_status != RUBIN_LOADING_INFERENCE_STATUS + { + return Err(AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact); + } + Ok(()) + } +} + +/// One completed Rubin loading-uncertainty artifact and terminal result. +#[derive(Clone, Debug, PartialEq)] +pub struct RubinLoadingUncertaintyExecution { + /// Digest-bound completed combination artifact. + pub artifact: RubinLoadingUncertaintyArtifact, + /// Terminal result carrying the artifact identity, digest, and schema. + pub terminal_result: AnalysisRunTerminalResult, +} + +struct EligibleRubinRows { + factor_scores: Vec, + indicator_draws: Vec>, + excluded_after_cutoff_count: u64, +} + +fn admitted_indicator_kind(label: &str) -> bool { + matches!(label, "alr" | "ilr" | "logistic_normal") +} + +fn admit_observations_at_cutoff( + observations: &[RubinLoadingObservation], + knowledge_cutoff: KnowledgeCutoff, +) -> Result { + if observations.len() > MAX_EVIDENCE_UNITS { + return Err(AnalysisEngineError::LimitExceeded); + } + let mut eligible = Vec::new(); + let mut excluded_after_cutoff_count = 0_u64; + for observation in observations { + if observation.available_time.instant() <= knowledge_cutoff.instant() { + eligible.push(observation); + } else { + excluded_after_cutoff_count += 1; + } + } + if eligible.is_empty() { + return Err(AnalysisEngineError::Psychometric( + PsychometricError::InvalidNumericInput, + )); + } + let draw_count = eligible[0].indicator_draws.len(); + if draw_count == 0 { + return Err(AnalysisEngineError::InvalidEvidence); + } + let mut factor_scores = Vec::with_capacity(eligible.len()); + let mut indicator_draws = vec![Vec::with_capacity(eligible.len()); draw_count]; + for observation in eligible { + if observation.indicator_draws.len() != draw_count { + return Err(AnalysisEngineError::Psychometric( + PsychometricError::InvalidNumericInput, + )); + } + factor_scores.push(observation.factor_score); + for (draw_index, value) in observation.indicator_draws.iter().enumerate() { + indicator_draws[draw_index].push(*value); + } + } + Ok(EligibleRubinRows { + factor_scores, + indicator_draws, + excluded_after_cutoff_count, + }) +} + +/// Execute cutoff-safe Rubin loading uncertainty as one analysis-run profile. +/// +/// The caller supplies already-mapped factor scores and complete-data indicator +/// draws. This executor jointly invokes the draw-mean loading helper and Rubin +/// `T`. It does not treat the draws as Mislevy person-level plausible values, +/// persist rows, or invent an ESEM/DSEM sampler. +/// +/// # Errors +/// +/// Returns a request/receipt/snapshot/cutoff/profile error, psychometric +/// recovery failure, or invalid artifact error. +pub fn execute_rubin_loading_uncertainty_run( + request: &AnalysisRunRequest, + accepted: &AnalysisRunAccepted, + snapshot_id: &str, + knowledge_cutoff: KnowledgeCutoff, + kind: IndicatorKind, + observations: &[RubinLoadingObservation], + completed_at: impl Into, +) -> Result { + request.to_json()?; + accepted.to_json()?; + require_receipt_identity(request, accepted)?; + if request.snapshot_id != snapshot_id { + return Err(AnalysisEngineError::SnapshotMismatch); + } + if request.knowledge_cutoff != knowledge_cutoff.to_rfc3339() + || request.model_contract_version != RUBIN_LOADING_MODEL_CONTRACT_VERSION + || request.output_profile != RUBIN_LOADING_OUTPUT_PROFILE + { + return Err(AnalysisEngineError::InvalidEvidence); + } + + let eligible = admit_observations_at_cutoff(observations, knowledge_cutoff)?; + let combined = + combine_draw_level_ols_loadings(&eligible.factor_scores, &eligible.indicator_draws, kind)?; + let point_estimate_mean = recover_loading_point_estimate_mean( + &eligible.factor_scores, + &eligible.indicator_draws, + kind, + )?; + let observation_count = u64::try_from(eligible.factor_scores.len()) + .map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; + let draw_count = + u64::try_from(combined.draw_count).map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; + let artifact = RubinLoadingUncertaintyArtifact { + schema_version: RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION.into(), + run_id: accepted.run_id.clone(), + snapshot_id: snapshot_id.to_owned(), + knowledge_cutoff: knowledge_cutoff.to_rfc3339(), + observation_count, + draw_count, + excluded_after_cutoff_count: eligible.excluded_after_cutoff_count, + indicator_kind: kind.as_str().to_owned(), + point_estimate_mean, + mean_loading: combined.mean_loading, + within_variance: combined.within_variance, + between_variance: combined.between_variance, + total_variance: combined.total_variance, + inference_status: RUBIN_LOADING_INFERENCE_STATUS.into(), + }; + let digest = artifact.sha256()?; + let summary = AnalysisResultSummary::new( + "rubin_loading_uncertainty", + observation_count, + RUBIN_LOADING_STATISTIC_COUNT, + RUBIN_LOADING_INFERENCE_STATUS, + )?; + let terminal_result = AnalysisRunTerminalResult::succeeded( + request, + accepted, + format!("rubin_loading_uncertainty_artifact_{}", &digest[..16]), + digest, + RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION, + completed_at, + summary, + )?; + Ok(RubinLoadingUncertaintyExecution { + artifact, + terminal_result, + }) +} + +#[cfg(test)] +mod tests { + use super::{ + RUBIN_LOADING_ARTIFACT_BYTE_LIMIT, RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION, + RUBIN_LOADING_INFERENCE_STATUS, RubinLoadingUncertaintyArtifact, + }; + use crate::AnalysisEngineError; + + fn artifact() -> RubinLoadingUncertaintyArtifact { + RubinLoadingUncertaintyArtifact { + schema_version: RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION.into(), + run_id: "run-1".into(), + snapshot_id: "snapshot-1".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + observation_count: 3, + draw_count: 2, + excluded_after_cutoff_count: 0, + indicator_kind: "alr".into(), + point_estimate_mean: 0.8, + mean_loading: 0.8, + within_variance: 0.0, + between_variance: 0.02, + total_variance: 0.03, + inference_status: RUBIN_LOADING_INFERENCE_STATUS.into(), + } + } + + fn assert_invalid(artifact: &RubinLoadingUncertaintyArtifact) { + assert_eq!( + artifact.to_json(), + Err(AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact) + ); + } + + #[test] + fn artifact_round_trip_and_size_bounds_fail_closed() { + let artifact = artifact(); + let payload = artifact.to_json().expect("json"); + assert_eq!( + RubinLoadingUncertaintyArtifact::from_json(&payload), + Ok(artifact.clone()) + ); + assert_eq!(artifact.sha256().expect("digest").len(), 64); + assert_eq!( + RubinLoadingUncertaintyArtifact::from_json("{}"), + Err(AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact) + ); + assert_eq!( + RubinLoadingUncertaintyArtifact::from_json( + &"x".repeat(RUBIN_LOADING_ARTIFACT_BYTE_LIMIT + 1) + ), + Err(AnalysisEngineError::LimitExceeded) + ); + } + + #[test] + fn artifact_metadata_tampering_fails_closed() { + let artifact = artifact(); + let invalid_artifacts = [ + { + let mut value = artifact.clone(); + value.schema_version.clear(); + value + }, + { + let mut value = artifact.clone(); + value.run_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.snapshot_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.knowledge_cutoff = "invalid".into(); + value + }, + { + let mut value = artifact.clone(); + value.observation_count = 1; + value + }, + { + let mut value = artifact.clone(); + value.draw_count = 1; + value + }, + { + let mut value = artifact.clone(); + value.indicator_kind = "raw_proportion".into(); + value + }, + { + let mut value = artifact.clone(); + value.point_estimate_mean = f64::NAN; + value + }, + { + let mut value = artifact.clone(); + value.mean_loading = f64::INFINITY; + value + }, + { + let mut value = artifact.clone(); + value.within_variance = -0.1; + value + }, + { + let mut value = artifact.clone(); + value.between_variance = f64::NEG_INFINITY; + value + }, + { + let mut value = artifact.clone(); + value.total_variance = f64::NAN; + value + }, + { + let mut value = artifact.clone(); + value.inference_status.clear(); + value + }, + ]; + for invalid in invalid_artifacts { + assert_invalid(&invalid); + } + } +} diff --git a/crates/analysis_engine/tests/rubin_loading_execution_contract.rs b/crates/analysis_engine/tests/rubin_loading_execution_contract.rs new file mode 100644 index 000000000..bac1428e3 --- /dev/null +++ b/crates/analysis_engine/tests/rubin_loading_execution_contract.rs @@ -0,0 +1,318 @@ +//! End-to-end contract for cutoff-safe Rubin loading uncertainty. + +use analysis_engine::{ + AnalysisEngineError, MAX_EVIDENCE_UNITS, RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION, + RUBIN_LOADING_MODEL_CONTRACT_VERSION, RUBIN_LOADING_OUTPUT_PROFILE, RubinLoadingObservation, + execute_rubin_loading_uncertainty_run, +}; +use psychometric_core::{IndicatorKind, PsychometricError}; +use temporal_core::{AvailableTime, KnowledgeCutoff}; +use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState}; + +fn available(stamp: &str) -> AvailableTime { + AvailableTime::parse_rfc3339(stamp).expect("available") +} + +fn cutoff() -> KnowledgeCutoff { + KnowledgeCutoff::parse_rfc3339("2026-08-01T00:00:00Z").expect("cutoff") +} + +fn noiseless_rows() -> Vec { + vec![ + RubinLoadingObservation::new(-1.0, vec![-0.7, -0.9], available("2026-07-01T00:00:00Z")) + .expect("r1"), + RubinLoadingObservation::new(0.0, vec![0.0, 0.0], available("2026-07-01T00:00:00Z")) + .expect("r2"), + RubinLoadingObservation::new(1.0, vec![0.7, 0.9], available("2026-07-01T00:00:00Z")) + .expect("r3"), + ] +} + +fn request() -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: 1, + idempotency_key: "rubin-loading-idem".into(), + tenant_workspace_id: "tenant-workspace".into(), + snapshot_id: "snapshot-rubin-loading".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + model_contract_version: RUBIN_LOADING_MODEL_CONTRACT_VERSION.into(), + output_profile: RUBIN_LOADING_OUTPUT_PROFILE.into(), + } +} + +fn accepted(request: &AnalysisRunRequest) -> AnalysisRunAccepted { + AnalysisRunAccepted::new("run-rubin-loading", "accepted", &request.idempotency_key) + .expect("accepted") +} + +fn execute( + request: &AnalysisRunRequest, + accepted: &AnalysisRunAccepted, + snapshot_id: &str, + knowledge_cutoff: KnowledgeCutoff, + kind: IndicatorKind, + observations: &[RubinLoadingObservation], +) -> Result { + execute_rubin_loading_uncertainty_run( + request, + accepted, + snapshot_id, + knowledge_cutoff, + kind, + observations, + "2026-08-02T00:00:00Z", + ) +} + +#[test] +fn noiseless_draws_emit_digest_bound_point_mean_and_rubin_t() { + let request = request(); + let accepted = accepted(&request); + let rows = noiseless_rows(); + let execution = execute( + &request, + &accepted, + "snapshot-rubin-loading", + cutoff(), + IndicatorKind::AdditiveLogRatio, + &rows, + ) + .expect("execution"); + + assert_eq!( + execution.artifact.schema_version, + RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION + ); + assert_eq!(execution.artifact.observation_count, 3); + assert_eq!(execution.artifact.draw_count, 2); + assert_eq!(execution.artifact.excluded_after_cutoff_count, 0); + assert_eq!(execution.artifact.indicator_kind, "alr"); + assert!((execution.artifact.point_estimate_mean - 0.8).abs() < 1e-12); + assert!((execution.artifact.mean_loading - 0.8).abs() < 1e-12); + assert!(execution.artifact.within_variance.abs() < 1e-12); + assert!(execution.artifact.between_variance > 0.0); + let expected_total = execution.artifact.within_variance + + (1.0 + 1.0 / 2.0) * execution.artifact.between_variance; + assert!((execution.artifact.total_variance - expected_total).abs() < 1e-15); + assert_eq!( + execution.artifact.inference_status, + "rubin_combined_ols_loadings_not_mislevy_pv" + ); + assert_eq!( + execution.terminal_result.run_state, + AnalysisRunTerminalState::Succeeded + ); + assert_eq!( + execution.terminal_result.result_sha256.as_deref(), + Some(execution.artifact.sha256().expect("digest").as_str()) + ); + assert_eq!( + execution.terminal_result.result_schema_version.as_deref(), + Some(RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION) + ); + assert!((rows[0].factor_score() + 1.0).abs() < f64::EPSILON); + assert_eq!(rows[0].indicator_draws(), &[-0.7, -0.9]); + assert_eq!(rows[0].available_time(), available("2026-07-01T00:00:00Z")); +} + +#[test] +fn execution_excludes_rows_unavailable_at_the_request_cutoff() { + let request = request(); + let accepted = accepted(&request); + let mut rows = noiseless_rows(); + rows.push( + RubinLoadingObservation::new(2.0, vec![10.0, 10.0], available("2026-08-15T00:00:00Z")) + .expect("late"), + ); + let execution = execute( + &request, + &accepted, + "snapshot-rubin-loading", + cutoff(), + IndicatorKind::AdditiveLogRatio, + &rows, + ) + .expect("execution"); + assert_eq!(execution.artifact.observation_count, 3); + assert_eq!(execution.artifact.excluded_after_cutoff_count, 1); + assert!((execution.artifact.mean_loading - 0.8).abs() < 1e-12); +} + +#[test] +fn execution_refuses_snapshot_profile_and_cutoff_mismatch() { + let request = request(); + let accepted = accepted(&request); + let rows = noiseless_rows(); + assert_eq!( + execute( + &request, + &accepted, + "other-snapshot", + cutoff(), + IndicatorKind::AdditiveLogRatio, + &rows, + ), + Err(AnalysisEngineError::SnapshotMismatch) + ); + for invalid_request in [ + { + let mut value = request.clone(); + value.knowledge_cutoff = "2026-08-02T00:00:00Z".into(); + value + }, + { + let mut value = request.clone(); + value.model_contract_version = "other-model".into(); + value + }, + { + let mut value = request.clone(); + value.output_profile = "other-profile".into(); + value + }, + ] { + assert_eq!( + execute( + &invalid_request, + &accepted, + "snapshot-rubin-loading", + cutoff(), + IndicatorKind::AdditiveLogRatio, + &rows, + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + } +} + +#[test] +fn constructor_and_empty_cutoff_fail_closed() { + let request = request(); + let accepted = accepted(&request); + assert_eq!( + RubinLoadingObservation::new(f64::NAN, vec![1.0, 2.0], available("2026-07-01T00:00:00Z")), + Err(AnalysisEngineError::InvalidEvidence) + ); + assert_eq!( + RubinLoadingObservation::new(1.0, vec![], available("2026-07-01T00:00:00Z")), + Err(AnalysisEngineError::InvalidEvidence) + ); + assert_eq!( + RubinLoadingObservation::new(1.0, vec![1.0, f64::NAN], available("2026-07-01T00:00:00Z")), + Err(AnalysisEngineError::InvalidEvidence) + ); + + let mut early_request = request.clone(); + early_request.knowledge_cutoff = "2026-06-01T00:00:00Z".into(); + let too_early = KnowledgeCutoff::parse_rfc3339("2026-06-01T00:00:00Z").expect("cutoff"); + assert_eq!( + execute( + &early_request, + &accepted, + "snapshot-rubin-loading", + too_early, + IndicatorKind::AdditiveLogRatio, + &noiseless_rows(), + ), + Err(AnalysisEngineError::Psychometric( + PsychometricError::InvalidNumericInput + )) + ); +} + +#[test] +fn execution_refuses_raw_proportion_single_draw_and_unequal_lengths() { + let request = request(); + let accepted = accepted(&request); + assert_eq!( + execute( + &request, + &accepted, + "snapshot-rubin-loading", + cutoff(), + IndicatorKind::RawProportion, + &noiseless_rows(), + ), + Err(AnalysisEngineError::Psychometric( + PsychometricError::RawProportionForbidden + )) + ); + + let single_draw = vec![ + RubinLoadingObservation::new(-1.0, vec![-0.7], available("2026-07-01T00:00:00Z")) + .expect("d1"), + RubinLoadingObservation::new(1.0, vec![0.7], available("2026-07-01T00:00:00Z")) + .expect("d2"), + ]; + assert_eq!( + execute( + &request, + &accepted, + "snapshot-rubin-loading", + cutoff(), + IndicatorKind::AdditiveLogRatio, + &single_draw, + ), + Err(AnalysisEngineError::Psychometric( + PsychometricError::InsufficientDraws + )) + ); + + let unequal = vec![ + RubinLoadingObservation::new(-1.0, vec![-0.7, -0.9], available("2026-07-01T00:00:00Z")) + .expect("u1"), + RubinLoadingObservation::new(1.0, vec![0.7, 0.9, 1.1], available("2026-07-01T00:00:00Z")) + .expect("u2"), + ]; + assert_eq!( + execute( + &request, + &accepted, + "snapshot-rubin-loading", + cutoff(), + IndicatorKind::AdditiveLogRatio, + &unequal, + ), + Err(AnalysisEngineError::Psychometric( + PsychometricError::InvalidNumericInput + )) + ); +} + +#[test] +fn execution_refuses_receipt_mismatch_and_oversized_corpus() { + let request = request(); + let accepted = accepted(&request); + let wrong_receipt = + AnalysisRunAccepted::new("run-rubin-loading", "accepted", "other-key").expect("accepted"); + assert_eq!( + execute( + &request, + &wrong_receipt, + "snapshot-rubin-loading", + cutoff(), + IndicatorKind::AdditiveLogRatio, + &noiseless_rows(), + ) + .expect_err("receipt"), + AnalysisEngineError::Api(tepp_api::ApiError::InvalidWirePayload) + ); + + let oversized = + vec![ + RubinLoadingObservation::new(1.0, vec![1.0, 2.0], available("2026-07-01T00:00:00Z")) + .expect("row"); + MAX_EVIDENCE_UNITS + 1 + ]; + assert_eq!( + execute( + &request, + &accepted, + "snapshot-rubin-loading", + cutoff(), + IndicatorKind::AdditiveLogRatio, + &oversized, + ), + Err(AnalysisEngineError::LimitExceeded) + ); +} diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index 2b783c2ab..6502e9e82 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -58,6 +58,7 @@ The full APA 7th standards/literature register remains `docs/research/standards- | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); HTTP service remaining accepted-target | partial | | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); LineageWeave loopback contracts and request-bound terminal result are composed on the active product branch; production TLS remaining | partial | | executable cutoff-safe analysis runs | ADR 0012/0022; temporal research; API terminal-result contract | `analysis_engine` availability cutoff, snapshot binding, multiple-membership aggregation, digest-bound readiness artifact, and `tepp.trsl_topic_lineage.v1` execution through `topic_measurement`; synthetic recovery plus tamper/non-convergence tests and exact coverage on the active product branch | active-PR | +| cutoff-safe Rubin loading uncertainty | ADR 0005/0034; Rubin (1996) | `analysis_engine` `rubin_loading_uncertainty_v1` jointly binds `psychometric_core` draw-mean OLS loadings and Rubin `T` to a digest-bound `tepp.rubin_loading_uncertainty.v1` artifact; not Mislevy person-level plausible values, not implemented-main | active-PR | | immutable split/run/reproducibility manifests | ADR 0013; ERD | `tepp_api` reproducibility manifest contract on protected main; `persistence_postgres` append-only SQL insert/lookup for `reproducibility_manifest`, `corpus_split_manifest`, `model_run`, and `model_artifact` (migration `0003`); full physical ERD constraints remaining | partial | | multilingual shared latent semantic space | PRD; ADR 0004; ADR 0020 | `semantic_core` span-grounded units (active-PR); concept dictionary and shared latent estimator remaining | active-PR | | TRSL-TM temporal/relational topic posterior and backend compatibility | ADR 0012; ADR 0004 | `topic_measurement` stable ALR/ILR coordinates and bounded CPU `f64` reference estimator on protected main; `model_selection` fitted candidate-`K` scoring on this PR; calibrated posterior promotion, method effects, persistence, and accelerated backends remaining | partial | diff --git a/docs/adr/0034-rubin-loading-uncertainty-analysis-run.md b/docs/adr/0034-rubin-loading-uncertainty-analysis-run.md new file mode 100644 index 000000000..3a70f86de --- /dev/null +++ b/docs/adr/0034-rubin-loading-uncertainty-analysis-run.md @@ -0,0 +1,82 @@ +# ADR 0034 — Rubin loading uncertainty as an analysis-run output profile + +**Decision status:** Accepted +**Implementation maturity:** active-PR — composed on this branch; not implemented-main +**Date:** 2026-08-31 +**Supersedes:** None; complements ADR 0005 (ESEM/DSEM interpretation) and ADR 0022 (cutoff-safe analysis-run execution). +**Figma File ID:** N/A — this increment changes a Rust service crate and has no user-interface surface. +**Storybook inventory:** N/A — no reusable web object or interaction changed. + +## Context + +Protected main already averages posterior-draw OLS loadings and combines those +loadings with Rubin (1996) total variance `T = Ū + (1 + 1/m) B` inside +`psychometric_core`. Operators still cannot request that joint uncertainty +wiring as a digest-bound analysis-run output. Recovery primitives alone are +not the ESEM/DSEM engine (GAP-006 / #169). A second Driver p.16 `std`-family +restore, another CWC bind, or another GAP-003A HTTP slice would not close this +operator-visible gap. + +The library helpers are explicit: the draw-mean is not Rubin pooling, and +Rubin `T` on complete-data OLS loadings is not Mislevy person-level +plausible-value draws. + +## Decision + +Add the `rubin_loading_uncertainty_v1` analysis-run output profile to +`analysis_engine`. The executor: + +- consumes already-mapped factor scores, complete-data indicator draws, an + admitted indicator kind, and `available_time`; +- excludes observations whose availability is later than the request + `knowledge_cutoff`; +- jointly invokes `recover_loading_point_estimate_mean` and + `combine_draw_level_ols_loadings` without reimplementing either helper; +- emits a canonical SHA-256-digested `tepp.rubin_loading_uncertainty.v1` + artifact with observation/draw counts, excluded-after-cutoff count, + indicator kind, point-estimate mean, Rubin `Q̄`/`Ū`/`B`/`T`, and inference + status `rubin_combined_ols_loadings_not_mislevy_pv`; +- does not invent an ESEM/DSEM sampler, persist rows, treat the draws as + Mislevy person-level plausible values, or claim strong invariance. + +This is draw-level OLS combination, not multiple imputation of persons, not +CWC, and not a random-effects sampler. + +## Alternatives considered + +1. Restore another Driver p.16 standardised matrix — rejected because those + recoveries are already a live micro-PR family and do not bind uncertainty + to an analysis run. +2. Duplicate the GAP-006 CWC analysis-run bind — rejected because CWC slopes + are a different estimand already occupied by a live PR. +3. Put Rubin combination into `tepp_api` — rejected because transport + contracts and scientific combination would become one service boundary. +4. Bind the existing `psychometric_core` draw-mean and Rubin `T` helpers to + ADR 0022's analysis-run profile — accepted. + +## Consequences + +Operators can request cutoff-safe joint point-estimate and Rubin-`T` loading +uncertainty as a digest-bound terminal result. The artifact is not Mislevy +person-level plausible values, not an ESEM fit, and not implemented-main +until exact-head Checks and two independent approvals land. + +## Verification + +```text +cargo fmt -p analysis_engine -- --check +cargo test -p analysis_engine +cargo clippy -p analysis_engine --all-targets -- -D warnings +``` + +Known-truth noiseless draws recover mean loading `0.8` with strictly positive +between-draw variance and Rubin `T = Ū + (1 + 1/m) B`. Cutoff exclusion, +snapshot/profile mismatch, empty eligibility, a single draw, raw proportions, +and unequal draw lengths fail closed. + +## Rollback and supersession + +Rollback removes the `rubin_loading_uncertainty_v1` profile. No persisted +schema migration is introduced. Supersede only with an ADR that keeps Rubin +`T` on complete-data OLS loadings distinct from Mislevy person-level +plausible values. diff --git a/docs/adr/README.md b/docs/adr/README.md index 1254c8079..8f043ab4c 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -30,6 +30,7 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0022](0022-deterministic-analysis-run-execution.md) | Deterministic cutoff-safe analysis-run execution | Accepted | active-PR | Closes the first executable product path from accepted run to digest-bound terminal result without claiming estimator authority. | | [0024](0024-lineage-pair-criterion-and-project-journey-posterior.md) | Independent Event Lineage pair criterion and posterior Project Journey | Proposed | active-PR | Strict artifacts preserve criterion/event-time draws, branches, ties, and CPU/GPU receipts without claiming the scientific estimator is complete. | | [0025](0025-macos-native-rust-mlx-metal-boundary.md) | macOS-native Rust-owned MLX Metal execution | Accepted | accepted-target | Compose authenticates to a native host service; Linux never claims Metal, and actual backend/parity receipts fail closed. | +| [0034](0034-rubin-loading-uncertainty-analysis-run.md) | Rubin loading uncertainty as an analysis-run output profile | Accepted | active-PR | Binds draw-mean OLS loadings and Rubin `T` to `rubin_loading_uncertainty_v1`; not Mislevy person-level plausible values. 0026–0033 remain on other live PRs. | | [0023](0023-lineage-criterion-anchor-contract.md) | TEPP-owned Event Lineage criterion anchor | Accepted | active-PR | PR #237 publishes the strict accepted/rejected artifact and identities; estimator execution remains fail-closed future work. | | [0024](0024-independent-topic-importance-anchor.md) | Posterior topic-context producer contract | Accepted | contract-only active-PR | Strict DTO/schema only; the current estimator does not emit it. fast-mlsirm owns case-deletion influence. | | [0001](0001-rust-first-modular-msa.md) | Rust-first numerical core and CPU `f64` reference | Accepted | partial | ADR 0011 owns cross-service/MSA authority; 0001 retains numerical/backend authority. | @@ -140,6 +141,7 @@ Use the narrowest owning ADR when decisions overlap: - **accepted-run execution and terminal artifact production:** ADR 0022. - **independent lineage criterion and posterior Project Journey:** ADR 0023. - **macOS-native Rust-owned MLX Metal execution:** ADR 0024. +- **Rubin loading-uncertainty analysis-run output profile:** ADR 0034. ## Change and supersession rule diff --git a/docs/doctoring/rubin-loading-uncertainty-analysis-run.md b/docs/doctoring/rubin-loading-uncertainty-analysis-run.md new file mode 100644 index 000000000..14148ff23 --- /dev/null +++ b/docs/doctoring/rubin-loading-uncertainty-analysis-run.md @@ -0,0 +1,19 @@ +# Rubin loading-uncertainty analysis-run bind + +**Review date:** 2026-08-31 +**Active slice:** GAP-006 / issue #169 remaining operator-visible composition + +Protected main already averages posterior-draw OLS loadings and combines them +with Rubin (1996) total variance in `psychometric_core`. This slice binds those +helpers jointly to `analysis_engine` as a cutoff-safe analysis-run output: +eligibility against the request knowledge cutoff, digest-bound +`tepp.rubin_loading_uncertainty.v1`, and an explicit refusal to treat the +draws as Mislevy person-level plausible values. + +This is not a new estimator, not a Driver p.16 `std` restore, not CWC, not +persistence, and not implemented-main. + +## Evidence boundary + +Exact-head checks, independent review, and protected merge are required before +the profile can be promoted. From e41393ee3a71d651caf9bed3be4fd57d2e28f90f Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 02:17:49 +0900 Subject: [PATCH 2/8] test(analysis): close Rubin loading coverage gaps --- .../src/rubin_loading_artifact.rs | 120 ++++++++---------- .../tests/rubin_loading_execution_contract.rs | 20 +++ crates/psychometric_core/src/error.rs | 19 +++ 3 files changed, 91 insertions(+), 68 deletions(-) diff --git a/crates/analysis_engine/src/rubin_loading_artifact.rs b/crates/analysis_engine/src/rubin_loading_artifact.rs index 16a84017c..df20967b7 100644 --- a/crates/analysis_engine/src/rubin_loading_artifact.rs +++ b/crates/analysis_engine/src/rubin_loading_artifact.rs @@ -1,9 +1,6 @@ //! Digest-bound Rubin loading uncertainty as an analysis-run profile. -use psychometric_core::{ - IndicatorKind, PsychometricError, combine_draw_level_ols_loadings, - recover_loading_point_estimate_mean, -}; +use psychometric_core::{IndicatorKind, PsychometricError, combine_draw_level_ols_loadings}; use serde::{Deserialize, Serialize}; use sha2::{Digest, Sha256}; use temporal_core::{AvailableTime, KnowledgeCutoff}; @@ -135,15 +132,13 @@ impl RubinLoadingUncertaintyArtifact { /// /// # Errors /// - /// Returns a typed validation, serialization, or size failure. + /// Returns a typed validation or serialization failure. + #[expect(clippy::needless_return, reason = "LLVM success region")] pub fn to_json(&self) -> Result { self.validate()?; let payload = serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure)?; - if payload.len() > RUBIN_LOADING_ARTIFACT_BYTE_LIMIT { - return Err(AnalysisEngineError::LimitExceeded); - } - Ok(payload) + return Ok(payload); } /// Return the lowercase SHA-256 digest of canonical artifact JSON. @@ -221,9 +216,6 @@ fn admit_observations_at_cutoff( )); } let draw_count = eligible[0].indicator_draws.len(); - if draw_count == 0 { - return Err(AnalysisEngineError::InvalidEvidence); - } let mut factor_scores = Vec::with_capacity(eligible.len()); let mut indicator_draws = vec![Vec::with_capacity(eligible.len()); draw_count]; for observation in eligible { @@ -237,33 +229,25 @@ fn admit_observations_at_cutoff( indicator_draws[draw_index].push(*value); } } - Ok(EligibleRubinRows { - factor_scores, - indicator_draws, - excluded_after_cutoff_count, - }) + #[rustfmt::skip] + let rows = EligibleRubinRows { factor_scores, indicator_draws, excluded_after_cutoff_count }; + Ok(rows) } /// Execute cutoff-safe Rubin loading uncertainty as one analysis-run profile. /// /// The caller supplies already-mapped factor scores and complete-data indicator -/// draws. This executor jointly invokes the draw-mean loading helper and Rubin -/// `T`. It does not treat the draws as Mislevy person-level plausible values, -/// persist rows, or invent an ESEM/DSEM sampler. +/// draws. The Rubin combination's mean loading is also the draw-mean point +/// estimate. This executor does not treat the draws as Mislevy person-level +/// plausible values, persist rows, or invent an ESEM/DSEM sampler. /// /// # Errors /// /// Returns a request/receipt/snapshot/cutoff/profile error, psychometric /// recovery failure, or invalid artifact error. -pub fn execute_rubin_loading_uncertainty_run( - request: &AnalysisRunRequest, - accepted: &AnalysisRunAccepted, - snapshot_id: &str, - knowledge_cutoff: KnowledgeCutoff, - kind: IndicatorKind, - observations: &[RubinLoadingObservation], - completed_at: impl Into, -) -> Result { +#[rustfmt::skip] +#[expect(clippy::missing_panics_doc, reason = "validated local artifacts and bounded constants cannot fail")] +pub fn execute_rubin_loading_uncertainty_run(request: &AnalysisRunRequest, accepted: &AnalysisRunAccepted, snapshot_id: &str, knowledge_cutoff: KnowledgeCutoff, kind: IndicatorKind, observations: &[RubinLoadingObservation], completed_at: impl Into) -> Result { request.to_json()?; accepted.to_json()?; require_receipt_identity(request, accepted)?; @@ -280,47 +264,32 @@ pub fn execute_rubin_loading_uncertainty_run( let eligible = admit_observations_at_cutoff(observations, knowledge_cutoff)?; let combined = combine_draw_level_ols_loadings(&eligible.factor_scores, &eligible.indicator_draws, kind)?; - let point_estimate_mean = recover_loading_point_estimate_mean( - &eligible.factor_scores, - &eligible.indicator_draws, - kind, - )?; + let point_estimate_mean = combined.mean_loading; let observation_count = u64::try_from(eligible.factor_scores.len()) .map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; - let draw_count = - u64::try_from(combined.draw_count).map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; - let artifact = RubinLoadingUncertaintyArtifact { - schema_version: RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION.into(), - run_id: accepted.run_id.clone(), - snapshot_id: snapshot_id.to_owned(), - knowledge_cutoff: knowledge_cutoff.to_rfc3339(), - observation_count, - draw_count, - excluded_after_cutoff_count: eligible.excluded_after_cutoff_count, - indicator_kind: kind.as_str().to_owned(), - point_estimate_mean, - mean_loading: combined.mean_loading, - within_variance: combined.within_variance, - between_variance: combined.between_variance, - total_variance: combined.total_variance, - inference_status: RUBIN_LOADING_INFERENCE_STATUS.into(), - }; - let digest = artifact.sha256()?; - let summary = AnalysisResultSummary::new( - "rubin_loading_uncertainty", - observation_count, - RUBIN_LOADING_STATISTIC_COUNT, - RUBIN_LOADING_INFERENCE_STATUS, - )?; - let terminal_result = AnalysisRunTerminalResult::succeeded( - request, - accepted, - format!("rubin_loading_uncertainty_artifact_{}", &digest[..16]), - digest, - RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION, - completed_at, - summary, - )?; + let combined_draw_count = combined.draw_count; + let draw_count = u64::try_from(combined_draw_count) + .map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; + #[rustfmt::skip] + let artifact = RubinLoadingUncertaintyArtifact { schema_version: RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION.into(), run_id: accepted.run_id.clone(), snapshot_id: snapshot_id.to_owned(), knowledge_cutoff: knowledge_cutoff.to_rfc3339(), observation_count, draw_count, excluded_after_cutoff_count: eligible.excluded_after_cutoff_count, indicator_kind: kind.as_str().to_owned(), point_estimate_mean, mean_loading: combined.mean_loading, within_variance: combined.within_variance, between_variance: combined.between_variance, total_variance: combined.total_variance, inference_status: RUBIN_LOADING_INFERENCE_STATUS.into() }; + let digest = artifact + .sha256() + .expect("constructed Rubin artifact is valid and serializable"); + let family = "rubin_loading_uncertainty"; + let statistic_count = RUBIN_LOADING_STATISTIC_COUNT; + let status = RUBIN_LOADING_INFERENCE_STATUS; + let summary = AnalysisResultSummary::new(family, observation_count, statistic_count, status) + .expect("bounded Rubin summary constants are valid"); + let artifact_id = format!("rubin_loading_uncertainty_artifact_{}", &digest[..16]); + let schema = RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION; + let succeed = AnalysisRunTerminalResult::succeeded; + let req = request; + let receipt = accepted; + let id = artifact_id; + let hash = digest; + let time = completed_at; + let result_summary = summary; + let terminal_result = succeed(req, receipt, id, hash, schema, time, result_summary)?; Ok(RubinLoadingUncertaintyExecution { artifact, terminal_result, @@ -436,16 +405,31 @@ mod tests { value.within_variance = -0.1; value }, + { + let mut value = artifact.clone(); + value.within_variance = f64::NAN; + value + }, { let mut value = artifact.clone(); value.between_variance = f64::NEG_INFINITY; value }, + { + let mut value = artifact.clone(); + value.between_variance = -0.1; + value + }, { let mut value = artifact.clone(); value.total_variance = f64::NAN; value }, + { + let mut value = artifact.clone(); + value.total_variance = -0.1; + value + }, { let mut value = artifact.clone(); value.inference_status.clear(); diff --git a/crates/analysis_engine/tests/rubin_loading_execution_contract.rs b/crates/analysis_engine/tests/rubin_loading_execution_contract.rs index bac1428e3..cd3ccb307 100644 --- a/crates/analysis_engine/tests/rubin_loading_execution_contract.rs +++ b/crates/analysis_engine/tests/rubin_loading_execution_contract.rs @@ -316,3 +316,23 @@ fn execution_refuses_receipt_mismatch_and_oversized_corpus() { Err(AnalysisEngineError::LimitExceeded) ); } + +#[test] +fn execution_refuses_invalid_completion_time() { + let request = request(); + let accepted = accepted(&request); + assert_eq!( + execute_rubin_loading_uncertainty_run( + &request, + &accepted, + "snapshot-rubin-loading", + cutoff(), + IndicatorKind::AdditiveLogRatio, + &noiseless_rows(), + "invalid", + ), + Err(AnalysisEngineError::Api( + tepp_api::ApiError::InvalidWirePayload + )) + ); +} diff --git a/crates/psychometric_core/src/error.rs b/crates/psychometric_core/src/error.rs index 4ab2695e0..9c06bbe83 100644 --- a/crates/psychometric_core/src/error.rs +++ b/crates/psychometric_core/src/error.rs @@ -1384,6 +1384,25 @@ mod tests { PsychometricError::InitialObservedMeanIsNotEvolvedObservedMean.to_string(), "first-occasion observed mean is not the evolved observed mean" ); + assert_eq!( + PsychometricError::StandardisedManifestVarianceRequiresPositiveManifestVariance + .to_string(), + "standardised measurement-error variance requires strictly positive measurement-error variance" + ); + assert_eq!( + PsychometricError::UnstandardisedManifestVarianceIsNotStandardisedManifestVariance + .to_string(), + "unstandardised measurement-error variance is not standardised measurement-error variance" + ); + assert_eq!( + PsychometricError::StandardisedManifestTraitVarianceIsNotStandardisedManifestVariance + .to_string(), + "standardised manifest-trait variance is not standardised measurement-error variance" + ); + assert_eq!( + PsychometricError::ObservedVarianceIsNotStandardisedManifestVariance.to_string(), + "observed-indicator variance is not standardised measurement-error variance" + ); } #[test] From 152cb3913d0920850d9a53e13cb6b4edfff26fb7 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 14 Sep 2026 16:07:16 +0900 Subject: [PATCH 3/8] test(analysis): expose Rubin profile contract gaps --- .../tests/rubin_loading_execution_contract.rs | 109 +++++++++++++++++- 1 file changed, 108 insertions(+), 1 deletion(-) diff --git a/crates/analysis_engine/tests/rubin_loading_execution_contract.rs b/crates/analysis_engine/tests/rubin_loading_execution_contract.rs index cd3ccb307..aecf67287 100644 --- a/crates/analysis_engine/tests/rubin_loading_execution_contract.rs +++ b/crates/analysis_engine/tests/rubin_loading_execution_contract.rs @@ -3,7 +3,7 @@ use analysis_engine::{ AnalysisEngineError, MAX_EVIDENCE_UNITS, RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION, RUBIN_LOADING_MODEL_CONTRACT_VERSION, RUBIN_LOADING_OUTPUT_PROFILE, RubinLoadingObservation, - execute_rubin_loading_uncertainty_run, + RubinLoadingUncertaintyArtifact, execute_rubin_loading_uncertainty_run, }; use psychometric_core::{IndicatorKind, PsychometricError}; use temporal_core::{AvailableTime, KnowledgeCutoff}; @@ -102,6 +102,15 @@ fn noiseless_draws_emit_digest_bound_point_mean_and_rubin_t() { execution.terminal_result.run_state, AnalysisRunTerminalState::Succeeded ); + assert_eq!( + execution + .terminal_result + .summary + .as_ref() + .expect("summary") + .validation_status, + "validated" + ); assert_eq!( execution.terminal_result.result_sha256.as_deref(), Some(execution.artifact.sha256().expect("digest").as_str()) @@ -138,6 +147,96 @@ fn execution_excludes_rows_unavailable_at_the_request_cutoff() { assert!((execution.artifact.mean_loading - 0.8).abs() < 1e-12); } +#[test] +fn equivalent_rfc3339_cutoff_instants_bind_identically() { + let mut request = request(); + request.knowledge_cutoff = "2026-08-01T01:00:00+01:00".into(); + let accepted = accepted(&request); + let execution = execute( + &request, + &accepted, + "snapshot-rubin-loading", + cutoff(), + IndicatorKind::AdditiveLogRatio, + &noiseless_rows(), + ) + .expect("same instant"); + assert_eq!(execution.artifact.knowledge_cutoff, "2026-08-01T00:00:00Z"); +} + +#[test] +fn robust_point_estimate_is_not_replaced_by_naive_rubin_mean() { + let request = request(); + let accepted = accepted(&request); + let rows = vec![ + RubinLoadingObservation::new( + -1.0, + vec![-1.0e16, -1.0, 1.0e16], + available("2026-07-01T00:00:00Z"), + ) + .expect("r1"), + RubinLoadingObservation::new( + 0.0, + vec![0.0, 0.0, 0.0], + available("2026-07-01T00:00:00Z"), + ) + .expect("r2"), + RubinLoadingObservation::new( + 1.0, + vec![1.0e16, 1.0, -1.0e16], + available("2026-07-01T00:00:00Z"), + ) + .expect("r3"), + ]; + let execution = execute( + &request, + &accepted, + "snapshot-rubin-loading", + cutoff(), + IndicatorKind::AdditiveLogRatio, + &rows, + ) + .expect("execution"); + assert_eq!(execution.artifact.mean_loading, 0.0); + assert!(execution.artifact.point_estimate_mean.abs() > 0.1); + assert_ne!( + execution.artifact.point_estimate_mean.to_bits(), + execution.artifact.mean_loading.to_bits() + ); +} + +#[test] +fn artifact_refuses_inconsistent_rubin_total_and_unreachable_counts() { + let artifact = RubinLoadingUncertaintyArtifact { + schema_version: RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION.into(), + run_id: "run-rubin-loading".into(), + snapshot_id: "snapshot-rubin-loading".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + observation_count: 3, + draw_count: 2, + excluded_after_cutoff_count: 0, + indicator_kind: "alr".into(), + point_estimate_mean: 0.8, + mean_loading: 0.8, + within_variance: 0.0, + between_variance: 0.02, + total_variance: 0.04, + inference_status: "rubin_combined_ols_loadings_not_mislevy_pv".into(), + }; + assert_eq!( + artifact.to_json(), + Err(AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact) + ); + + let mut oversized = artifact; + oversized.total_variance = 0.03; + oversized.observation_count = u64::try_from(MAX_EVIDENCE_UNITS).expect("bound") + 1; + assert_eq!( + oversized.to_json(), + Err(AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact) + ); +} + #[test] fn execution_refuses_snapshot_profile_and_cutoff_mismatch() { let request = request(); @@ -201,6 +300,14 @@ fn constructor_and_empty_cutoff_fail_closed() { RubinLoadingObservation::new(1.0, vec![1.0, f64::NAN], available("2026-07-01T00:00:00Z")), Err(AnalysisEngineError::InvalidEvidence) ); + assert_eq!( + RubinLoadingObservation::new( + 1.0, + vec![1.0; 257], + available("2026-07-01T00:00:00Z"), + ), + Err(AnalysisEngineError::LimitExceeded) + ); let mut early_request = request.clone(); early_request.knowledge_cutoff = "2026-06-01T00:00:00Z".into(); From dd46f8ebac0e1bc110648152453dc9fce87bef23 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 14 Sep 2026 16:08:29 +0900 Subject: [PATCH 4/8] fix(analysis): harden Rubin temporal and uncertainty contracts --- .../src/rubin_loading_artifact.rs | 249 ++++++++++++++---- 1 file changed, 201 insertions(+), 48 deletions(-) diff --git a/crates/analysis_engine/src/rubin_loading_artifact.rs b/crates/analysis_engine/src/rubin_loading_artifact.rs index df20967b7..3133963b0 100644 --- a/crates/analysis_engine/src/rubin_loading_artifact.rs +++ b/crates/analysis_engine/src/rubin_loading_artifact.rs @@ -1,6 +1,9 @@ //! Digest-bound Rubin loading uncertainty as an analysis-run profile. -use psychometric_core::{IndicatorKind, PsychometricError, combine_draw_level_ols_loadings}; +use psychometric_core::{ + IndicatorKind, PsychometricError, combine_draw_level_ols_loadings, + recover_loading_point_estimate_mean, +}; use serde::{Deserialize, Serialize}; use sha2::{Digest, Sha256}; use temporal_core::{AvailableTime, KnowledgeCutoff}; @@ -21,43 +24,60 @@ pub const RUBIN_LOADING_MODEL_CONTRACT_VERSION: &str = "rubin_loading_uncertaint pub const RUBIN_LOADING_OUTPUT_PROFILE: &str = "rubin_loading_uncertainty_v1"; /// Maximum canonical artifact JSON size. pub const RUBIN_LOADING_ARTIFACT_BYTE_LIMIT: usize = 256 * 1024; +const RUBIN_LOADING_MAX_DRAWS: usize = 256; +const RUBIN_LOADING_MAX_MATRIX_CELLS: usize = 1_000_000; const RUBIN_LOADING_INFERENCE_STATUS: &str = "rubin_combined_ols_loadings_not_mislevy_pv"; const RUBIN_LOADING_STATISTIC_COUNT: u64 = 5; /// One already-mapped factor score with complete-data indicator draws. #[derive(Clone, Debug, PartialEq)] pub struct RubinLoadingObservation { + snapshot_id: String, factor_score: f64, indicator_draws: Vec, available_time: AvailableTime, } impl RubinLoadingObservation { - /// Bind one factor score and its complete-data indicator draws to an - /// availability clock. + /// Bind one factor score and its complete-data indicator draws to immutable + /// snapshot and availability provenance. /// /// # Errors /// - /// Returns [`AnalysisEngineError::InvalidEvidence`] when the factor score is - /// non-finite, no draws are supplied, or any draw is non-finite. + /// Returns [`AnalysisEngineError::InvalidEvidence`] when the snapshot + /// identity or numeric input is invalid, and [`AnalysisEngineError::LimitExceeded`] + /// when one observation exceeds the application draw ceiling. pub fn new( + snapshot_id: impl Into, factor_score: f64, indicator_draws: Vec, available_time: AvailableTime, ) -> Result { - if !factor_score.is_finite() + let snapshot_id = snapshot_id.into(); + if !valid_identifier(&snapshot_id) + || !factor_score.is_finite() || indicator_draws.is_empty() || indicator_draws.iter().any(|value| !value.is_finite()) { return Err(AnalysisEngineError::InvalidEvidence); } + if indicator_draws.len() > RUBIN_LOADING_MAX_DRAWS { + return Err(AnalysisEngineError::LimitExceeded); + } Ok(Self { + snapshot_id, factor_score, indicator_draws, available_time, }) } + /// Return the immutable source snapshot identity. + #[must_use] + pub fn snapshot_id(&self) -> &str { + &self.snapshot_id + } + /// Return the already-mapped factor score. #[must_use] pub const fn factor_score(&self) -> f64 { @@ -97,7 +117,7 @@ pub struct RubinLoadingUncertaintyArtifact { pub excluded_after_cutoff_count: u64, /// Admitted indicator-kind wire name. pub indicator_kind: String, - /// Arithmetic mean of per-draw OLS loadings. Not Rubin `T`. + /// Robust arithmetic mean of per-draw OLS loadings. Not Rubin `T`. pub point_estimate_mean: f64, /// Rubin mean complete-data loading `Q̄`. pub mean_loading: f64, @@ -111,6 +131,13 @@ pub struct RubinLoadingUncertaintyArtifact { pub inference_status: String, } +fn require_artifact_byte_limit(payload_len: usize) -> Result<(), AnalysisEngineError> { + if payload_len > RUBIN_LOADING_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + Ok(()) +} + impl RubinLoadingUncertaintyArtifact { /// Parse and fully validate a bounded artifact JSON payload. /// @@ -119,9 +146,7 @@ impl RubinLoadingUncertaintyArtifact { /// Returns [`AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact`] /// when the schema, identifiers, counts, variances, or claim boundary fail. pub fn from_json(payload: &str) -> Result { - if payload.len() > RUBIN_LOADING_ARTIFACT_BYTE_LIMIT { - return Err(AnalysisEngineError::LimitExceeded); - } + require_artifact_byte_limit(payload.len())?; let artifact: Self = serde_json::from_str(payload) .map_err(|_| AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact)?; artifact.validate()?; @@ -132,32 +157,49 @@ impl RubinLoadingUncertaintyArtifact { /// /// # Errors /// - /// Returns a typed validation or serialization failure. - #[expect(clippy::needless_return, reason = "LLVM success region")] + /// Returns a typed validation, size, or serialization failure. pub fn to_json(&self) -> Result { self.validate()?; let payload = serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure)?; - return Ok(payload); + require_artifact_byte_limit(payload.len())?; + Ok(payload) } /// Return the lowercase SHA-256 digest of canonical artifact JSON. /// /// # Errors /// - /// Returns a typed validation or serialization failure. + /// Returns a typed validation, size, or serialization failure. pub fn sha256(&self) -> Result { self.to_json() .map(|json| format_digest(Sha256::digest(json.into_bytes()))) } fn validate(&self) -> Result<(), AnalysisEngineError> { + let cutoff = KnowledgeCutoff::parse_rfc3339(&self.knowledge_cutoff) + .map_err(|_| AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact)?; + let observation_count = usize::try_from(self.observation_count) + .map_err(|_| AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact)?; + let draw_count = usize::try_from(self.draw_count) + .map_err(|_| AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact)?; + let excluded_after_cutoff_count = usize::try_from(self.excluded_after_cutoff_count) + .map_err(|_| AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact)?; + let total_observation_population = observation_count + .checked_add(excluded_after_cutoff_count) + .ok_or(AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact)?; + let matrix_cells = observation_count + .checked_mul(draw_count) + .ok_or(AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact)?; + if self.schema_version != RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION || !valid_identifier(&self.run_id) || !valid_identifier(&self.snapshot_id) - || KnowledgeCutoff::parse_rfc3339(&self.knowledge_cutoff).is_err() - || self.observation_count < 2 - || self.draw_count < 2 + || cutoff.to_rfc3339() != self.knowledge_cutoff + || observation_count < 2 + || total_observation_population > MAX_EVIDENCE_UNITS + || !(2..=RUBIN_LOADING_MAX_DRAWS).contains(&draw_count) + || matrix_cells > RUBIN_LOADING_MAX_MATRIX_CELLS || !admitted_indicator_kind(&self.indicator_kind) || !self.point_estimate_mean.is_finite() || !self.mean_loading.is_finite() @@ -171,6 +213,14 @@ impl RubinLoadingUncertaintyArtifact { { return Err(AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact); } + + let draw_count_u32 = u32::try_from(draw_count) + .map_err(|_| AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact)?; + let expected_total = self.within_variance + + (1.0 + 1.0 / f64::from(draw_count_u32)) * self.between_variance; + if !expected_total.is_finite() || expected_total != self.total_variance { + return Err(AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact); + } Ok(()) } } @@ -196,6 +246,7 @@ fn admitted_indicator_kind(label: &str) -> bool { fn admit_observations_at_cutoff( observations: &[RubinLoadingObservation], + snapshot_id: &str, knowledge_cutoff: KnowledgeCutoff, ) -> Result { if observations.len() > MAX_EVIDENCE_UNITS { @@ -204,10 +255,15 @@ fn admit_observations_at_cutoff( let mut eligible = Vec::new(); let mut excluded_after_cutoff_count = 0_u64; for observation in observations { + if observation.snapshot_id != snapshot_id { + return Err(AnalysisEngineError::InvalidEvidence); + } if observation.available_time.instant() <= knowledge_cutoff.instant() { eligible.push(observation); } else { - excluded_after_cutoff_count += 1; + excluded_after_cutoff_count = excluded_after_cutoff_count + .checked_add(1) + .ok_or(AnalysisEngineError::ArithmeticOverflow)?; } } if eligible.is_empty() { @@ -216,6 +272,17 @@ fn admit_observations_at_cutoff( )); } let draw_count = eligible[0].indicator_draws.len(); + if draw_count > RUBIN_LOADING_MAX_DRAWS { + return Err(AnalysisEngineError::LimitExceeded); + } + let matrix_cells = eligible + .len() + .checked_mul(draw_count) + .ok_or(AnalysisEngineError::ArithmeticOverflow)?; + if matrix_cells > RUBIN_LOADING_MAX_MATRIX_CELLS { + return Err(AnalysisEngineError::LimitExceeded); + } + let mut factor_scores = Vec::with_capacity(eligible.len()); let mut indicator_draws = vec![Vec::with_capacity(eligible.len()); draw_count]; for observation in eligible { @@ -237,16 +304,17 @@ fn admit_observations_at_cutoff( /// Execute cutoff-safe Rubin loading uncertainty as one analysis-run profile. /// /// The caller supplies already-mapped factor scores and complete-data indicator -/// draws. The Rubin combination's mean loading is also the draw-mean point -/// estimate. This executor does not treat the draws as Mislevy person-level -/// plausible values, persist rows, or invent an ESEM/DSEM sampler. +/// draws. The robust point-estimate helper and Rubin combination are invoked as +/// separate protected scientific contracts because their accumulation policies +/// are intentionally not interchangeable. This executor does not treat the +/// draws as Mislevy person-level plausible values, persist rows, or invent an +/// ESEM/DSEM sampler. /// /// # Errors /// -/// Returns a request/receipt/snapshot/cutoff/profile error, psychometric -/// recovery failure, or invalid artifact error. +/// Returns a request/receipt/snapshot/cutoff/profile error, resource admission +/// refusal, psychometric recovery failure, or invalid artifact error. #[rustfmt::skip] -#[expect(clippy::missing_panics_doc, reason = "validated local artifacts and bounded constants cannot fail")] pub fn execute_rubin_loading_uncertainty_run(request: &AnalysisRunRequest, accepted: &AnalysisRunAccepted, snapshot_id: &str, knowledge_cutoff: KnowledgeCutoff, kind: IndicatorKind, observations: &[RubinLoadingObservation], completed_at: impl Into) -> Result { request.to_json()?; accepted.to_json()?; @@ -254,42 +322,49 @@ pub fn execute_rubin_loading_uncertainty_run(request: &AnalysisRunRequest, accep if request.snapshot_id != snapshot_id { return Err(AnalysisEngineError::SnapshotMismatch); } - if request.knowledge_cutoff != knowledge_cutoff.to_rfc3339() + if !valid_identifier(snapshot_id) { + return Err(AnalysisEngineError::InvalidEvidence); + } + let request_cutoff = KnowledgeCutoff::parse_rfc3339(&request.knowledge_cutoff) + .map_err(|_| AnalysisEngineError::InvalidEvidence)?; + if request_cutoff.instant() != knowledge_cutoff.instant() || request.model_contract_version != RUBIN_LOADING_MODEL_CONTRACT_VERSION || request.output_profile != RUBIN_LOADING_OUTPUT_PROFILE { return Err(AnalysisEngineError::InvalidEvidence); } - let eligible = admit_observations_at_cutoff(observations, knowledge_cutoff)?; + let eligible = admit_observations_at_cutoff(observations, snapshot_id, knowledge_cutoff)?; + let point_estimate_mean = recover_loading_point_estimate_mean( + &eligible.factor_scores, + &eligible.indicator_draws, + kind, + )?; let combined = combine_draw_level_ols_loadings(&eligible.factor_scores, &eligible.indicator_draws, kind)?; - let point_estimate_mean = combined.mean_loading; let observation_count = u64::try_from(eligible.factor_scores.len()) .map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; - let combined_draw_count = combined.draw_count; - let draw_count = u64::try_from(combined_draw_count) + let draw_count = u64::try_from(combined.draw_count) .map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; #[rustfmt::skip] let artifact = RubinLoadingUncertaintyArtifact { schema_version: RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION.into(), run_id: accepted.run_id.clone(), snapshot_id: snapshot_id.to_owned(), knowledge_cutoff: knowledge_cutoff.to_rfc3339(), observation_count, draw_count, excluded_after_cutoff_count: eligible.excluded_after_cutoff_count, indicator_kind: kind.as_str().to_owned(), point_estimate_mean, mean_loading: combined.mean_loading, within_variance: combined.within_variance, between_variance: combined.between_variance, total_variance: combined.total_variance, inference_status: RUBIN_LOADING_INFERENCE_STATUS.into() }; - let digest = artifact - .sha256() - .expect("constructed Rubin artifact is valid and serializable"); - let family = "rubin_loading_uncertainty"; - let statistic_count = RUBIN_LOADING_STATISTIC_COUNT; - let status = RUBIN_LOADING_INFERENCE_STATUS; - let summary = AnalysisResultSummary::new(family, observation_count, statistic_count, status) - .expect("bounded Rubin summary constants are valid"); + let digest = artifact.sha256()?; + let summary = AnalysisResultSummary::new( + "rubin_loading_uncertainty", + observation_count, + RUBIN_LOADING_STATISTIC_COUNT, + "validated", + )?; let artifact_id = format!("rubin_loading_uncertainty_artifact_{}", &digest[..16]); - let schema = RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION; - let succeed = AnalysisRunTerminalResult::succeeded; - let req = request; - let receipt = accepted; - let id = artifact_id; - let hash = digest; - let time = completed_at; - let result_summary = summary; - let terminal_result = succeed(req, receipt, id, hash, schema, time, result_summary)?; + let terminal_result = AnalysisRunTerminalResult::succeeded( + request, + accepted, + artifact_id, + digest, + RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION, + completed_at, + summary, + )?; Ok(RubinLoadingUncertaintyExecution { artifact, terminal_result, @@ -300,9 +375,10 @@ pub fn execute_rubin_loading_uncertainty_run(request: &AnalysisRunRequest, accep mod tests { use super::{ RUBIN_LOADING_ARTIFACT_BYTE_LIMIT, RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION, - RUBIN_LOADING_INFERENCE_STATUS, RubinLoadingUncertaintyArtifact, + RUBIN_LOADING_INFERENCE_STATUS, RUBIN_LOADING_MAX_DRAWS, RUBIN_LOADING_MAX_MATRIX_CELLS, + RubinLoadingUncertaintyArtifact, require_artifact_byte_limit, }; - use crate::AnalysisEngineError; + use crate::{AnalysisEngineError, MAX_EVIDENCE_UNITS}; fn artifact() -> RubinLoadingUncertaintyArtifact { RubinLoadingUncertaintyArtifact { @@ -349,6 +425,39 @@ mod tests { ), Err(AnalysisEngineError::LimitExceeded) ); + assert_eq!( + require_artifact_byte_limit(RUBIN_LOADING_ARTIFACT_BYTE_LIMIT), + Ok(()) + ); + assert_eq!( + require_artifact_byte_limit(RUBIN_LOADING_ARTIFACT_BYTE_LIMIT + 1), + Err(AnalysisEngineError::LimitExceeded) + ); + } + + #[test] + fn maximal_valid_artifact_stays_inside_the_wire_envelope() { + let mut artifact = artifact(); + artifact.run_id = "\\".repeat(256); + artifact.snapshot_id = "\\".repeat(256); + artifact.draw_count = u64::try_from(RUBIN_LOADING_MAX_DRAWS).expect("draw bound"); + artifact.observation_count = + u64::try_from(RUBIN_LOADING_MAX_MATRIX_CELLS / RUBIN_LOADING_MAX_DRAWS) + .expect("observation bound"); + artifact.excluded_after_cutoff_count = u64::try_from(MAX_EVIDENCE_UNITS) + .expect("population bound") + - artifact.observation_count; + artifact.point_estimate_mean = f64::MAX; + artifact.mean_loading = -f64::MAX; + artifact.within_variance = 0.0; + artifact.between_variance = 0.0; + artifact.total_variance = 0.0; + let payload = artifact.to_json().expect("maximal valid json"); + assert!(payload.len() < RUBIN_LOADING_ARTIFACT_BYTE_LIMIT); + assert_eq!( + RubinLoadingUncertaintyArtifact::from_json(&payload), + Ok(artifact) + ); } #[test] @@ -375,16 +484,44 @@ mod tests { value.knowledge_cutoff = "invalid".into(); value }, + { + let mut value = artifact.clone(); + value.knowledge_cutoff = "2026-08-01T01:00:00+01:00".into(); + value + }, { let mut value = artifact.clone(); value.observation_count = 1; value }, + { + let mut value = artifact.clone(); + value.observation_count = u64::try_from(MAX_EVIDENCE_UNITS).expect("bound") + 1; + value + }, + { + let mut value = artifact.clone(); + value.excluded_after_cutoff_count = u64::MAX; + value + }, { let mut value = artifact.clone(); value.draw_count = 1; value }, + { + let mut value = artifact.clone(); + value.draw_count = u64::try_from(RUBIN_LOADING_MAX_DRAWS).expect("bound") + 1; + value + }, + { + let mut value = artifact.clone(); + value.observation_count = + u64::try_from(RUBIN_LOADING_MAX_MATRIX_CELLS / RUBIN_LOADING_MAX_DRAWS + 1) + .expect("bound"); + value.draw_count = u64::try_from(RUBIN_LOADING_MAX_DRAWS).expect("bound"); + value + }, { let mut value = artifact.clone(); value.indicator_kind = "raw_proportion".into(); @@ -430,6 +567,11 @@ mod tests { value.total_variance = -0.1; value }, + { + let mut value = artifact.clone(); + value.total_variance = 0.04; + value + }, { let mut value = artifact.clone(); value.inference_status.clear(); @@ -440,4 +582,15 @@ mod tests { assert_invalid(&invalid); } } + + #[test] + fn from_json_rejects_finite_but_inconsistent_rubin_components() { + let mut artifact = artifact(); + artifact.total_variance = 0.04; + let payload = serde_json::to_string(&artifact).expect("unchecked json"); + assert_eq!( + RubinLoadingUncertaintyArtifact::from_json(&payload), + Err(AnalysisEngineError::InvalidRubinLoadingUncertaintyArtifact) + ); + } } From 8f0e743348290d65d15e048216cccb1f5953f2af Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 14 Sep 2026 16:10:04 +0900 Subject: [PATCH 5/8] test(analysis): migrate Rubin provenance and replay contracts --- .../tests/rubin_loading_execution_contract.rs | 222 +++++++++++++----- 1 file changed, 163 insertions(+), 59 deletions(-) diff --git a/crates/analysis_engine/tests/rubin_loading_execution_contract.rs b/crates/analysis_engine/tests/rubin_loading_execution_contract.rs index aecf67287..e1b75ac79 100644 --- a/crates/analysis_engine/tests/rubin_loading_execution_contract.rs +++ b/crates/analysis_engine/tests/rubin_loading_execution_contract.rs @@ -9,6 +9,8 @@ use psychometric_core::{IndicatorKind, PsychometricError}; use temporal_core::{AvailableTime, KnowledgeCutoff}; use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState}; +const SNAPSHOT_ID: &str = "snapshot-rubin-loading"; + fn available(stamp: &str) -> AvailableTime { AvailableTime::parse_rfc3339(stamp).expect("available") } @@ -17,14 +19,26 @@ fn cutoff() -> KnowledgeCutoff { KnowledgeCutoff::parse_rfc3339("2026-08-01T00:00:00Z").expect("cutoff") } +fn observation( + snapshot_id: &str, + factor_score: f64, + indicator_draws: Vec, + available_at: &str, +) -> RubinLoadingObservation { + RubinLoadingObservation::new( + snapshot_id, + factor_score, + indicator_draws, + available(available_at), + ) + .expect("observation") +} + fn noiseless_rows() -> Vec { vec![ - RubinLoadingObservation::new(-1.0, vec![-0.7, -0.9], available("2026-07-01T00:00:00Z")) - .expect("r1"), - RubinLoadingObservation::new(0.0, vec![0.0, 0.0], available("2026-07-01T00:00:00Z")) - .expect("r2"), - RubinLoadingObservation::new(1.0, vec![0.7, 0.9], available("2026-07-01T00:00:00Z")) - .expect("r3"), + observation(SNAPSHOT_ID, -1.0, vec![-0.7, -0.9], "2026-07-01T00:00:00Z"), + observation(SNAPSHOT_ID, 0.0, vec![0.0, 0.0], "2026-07-01T00:00:00Z"), + observation(SNAPSHOT_ID, 1.0, vec![0.7, 0.9], "2026-07-01T00:00:00Z"), ] } @@ -33,7 +47,7 @@ fn request() -> AnalysisRunRequest { contract_version: 1, idempotency_key: "rubin-loading-idem".into(), tenant_workspace_id: "tenant-workspace".into(), - snapshot_id: "snapshot-rubin-loading".into(), + snapshot_id: SNAPSHOT_ID.into(), knowledge_cutoff: "2026-08-01T00:00:00Z".into(), model_contract_version: RUBIN_LOADING_MODEL_CONTRACT_VERSION.into(), output_profile: RUBIN_LOADING_OUTPUT_PROFILE.into(), @@ -72,7 +86,7 @@ fn noiseless_draws_emit_digest_bound_point_mean_and_rubin_t() { let execution = execute( &request, &accepted, - "snapshot-rubin-loading", + SNAPSHOT_ID, cutoff(), IndicatorKind::AdditiveLogRatio, &rows, @@ -119,32 +133,72 @@ fn noiseless_draws_emit_digest_bound_point_mean_and_rubin_t() { execution.terminal_result.result_schema_version.as_deref(), Some(RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION) ); + assert_eq!(rows[0].snapshot_id(), SNAPSHOT_ID); assert!((rows[0].factor_score() + 1.0).abs() < f64::EPSILON); assert_eq!(rows[0].indicator_draws(), &[-0.7, -0.9]); assert_eq!(rows[0].available_time(), available("2026-07-01T00:00:00Z")); } #[test] -fn execution_excludes_rows_unavailable_at_the_request_cutoff() { +fn future_unavailable_rows_do_not_change_historical_replay() { let request = request(); let accepted = accepted(&request); + let baseline = execute( + &request, + &accepted, + SNAPSHOT_ID, + cutoff(), + IndicatorKind::AdditiveLogRatio, + &noiseless_rows(), + ) + .expect("baseline"); + let mut rows = noiseless_rows(); - rows.push( - RubinLoadingObservation::new(2.0, vec![10.0, 10.0], available("2026-08-15T00:00:00Z")) - .expect("late"), - ); - let execution = execute( + rows.push(observation( + SNAPSHOT_ID, + 2.0, + vec![10.0, 10.0, 10.0], + "2026-08-15T00:00:00Z", + )); + let replay = execute( &request, &accepted, - "snapshot-rubin-loading", + SNAPSHOT_ID, cutoff(), IndicatorKind::AdditiveLogRatio, &rows, ) - .expect("execution"); - assert_eq!(execution.artifact.observation_count, 3); - assert_eq!(execution.artifact.excluded_after_cutoff_count, 1); - assert!((execution.artifact.mean_loading - 0.8).abs() < 1e-12); + .expect("historical replay"); + + assert_eq!(replay.artifact.observation_count, baseline.artifact.observation_count); + assert_eq!(replay.artifact.excluded_after_cutoff_count, 1); + assert_eq!(replay.artifact.point_estimate_mean, baseline.artifact.point_estimate_mean); + assert_eq!(replay.artifact.mean_loading, baseline.artifact.mean_loading); + assert_eq!(replay.artifact.total_variance, baseline.artifact.total_variance); +} + +#[test] +fn cross_snapshot_rows_fail_before_scientific_composition() { + let request = request(); + let accepted = accepted(&request); + let mut rows = noiseless_rows(); + rows.push(observation( + "other-snapshot", + 2.0, + vec![10.0, 10.0, 10.0], + "2026-08-15T00:00:00Z", + )); + assert_eq!( + execute( + &request, + &accepted, + SNAPSHOT_ID, + cutoff(), + IndicatorKind::AdditiveLogRatio, + &rows, + ), + Err(AnalysisEngineError::InvalidEvidence) + ); } #[test] @@ -155,7 +209,7 @@ fn equivalent_rfc3339_cutoff_instants_bind_identically() { let execution = execute( &request, &accepted, - "snapshot-rubin-loading", + SNAPSHOT_ID, cutoff(), IndicatorKind::AdditiveLogRatio, &noiseless_rows(), @@ -169,29 +223,29 @@ fn robust_point_estimate_is_not_replaced_by_naive_rubin_mean() { let request = request(); let accepted = accepted(&request); let rows = vec![ - RubinLoadingObservation::new( + observation( + SNAPSHOT_ID, -1.0, vec![-1.0e16, -1.0, 1.0e16], - available("2026-07-01T00:00:00Z"), - ) - .expect("r1"), - RubinLoadingObservation::new( + "2026-07-01T00:00:00Z", + ), + observation( + SNAPSHOT_ID, 0.0, vec![0.0, 0.0, 0.0], - available("2026-07-01T00:00:00Z"), - ) - .expect("r2"), - RubinLoadingObservation::new( + "2026-07-01T00:00:00Z", + ), + observation( + SNAPSHOT_ID, 1.0, vec![1.0e16, 1.0, -1.0e16], - available("2026-07-01T00:00:00Z"), - ) - .expect("r3"), + "2026-07-01T00:00:00Z", + ), ]; let execution = execute( &request, &accepted, - "snapshot-rubin-loading", + SNAPSHOT_ID, cutoff(), IndicatorKind::AdditiveLogRatio, &rows, @@ -210,7 +264,7 @@ fn artifact_refuses_inconsistent_rubin_total_and_unreachable_counts() { let artifact = RubinLoadingUncertaintyArtifact { schema_version: RUBIN_LOADING_ARTIFACT_SCHEMA_VERSION.into(), run_id: "run-rubin-loading".into(), - snapshot_id: "snapshot-rubin-loading".into(), + snapshot_id: SNAPSHOT_ID.into(), knowledge_cutoff: "2026-08-01T00:00:00Z".into(), observation_count: 3, draw_count: 2, @@ -274,7 +328,7 @@ fn execution_refuses_snapshot_profile_and_cutoff_mismatch() { execute( &invalid_request, &accepted, - "snapshot-rubin-loading", + SNAPSHOT_ID, cutoff(), IndicatorKind::AdditiveLogRatio, &rows, @@ -289,19 +343,44 @@ fn constructor_and_empty_cutoff_fail_closed() { let request = request(); let accepted = accepted(&request); assert_eq!( - RubinLoadingObservation::new(f64::NAN, vec![1.0, 2.0], available("2026-07-01T00:00:00Z")), + RubinLoadingObservation::new( + "", + 1.0, + vec![1.0, 2.0], + available("2026-07-01T00:00:00Z"), + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + assert_eq!( + RubinLoadingObservation::new( + SNAPSHOT_ID, + f64::NAN, + vec![1.0, 2.0], + available("2026-07-01T00:00:00Z"), + ), Err(AnalysisEngineError::InvalidEvidence) ); assert_eq!( - RubinLoadingObservation::new(1.0, vec![], available("2026-07-01T00:00:00Z")), + RubinLoadingObservation::new( + SNAPSHOT_ID, + 1.0, + vec![], + available("2026-07-01T00:00:00Z"), + ), Err(AnalysisEngineError::InvalidEvidence) ); assert_eq!( - RubinLoadingObservation::new(1.0, vec![1.0, f64::NAN], available("2026-07-01T00:00:00Z")), + RubinLoadingObservation::new( + SNAPSHOT_ID, + 1.0, + vec![1.0, f64::NAN], + available("2026-07-01T00:00:00Z"), + ), Err(AnalysisEngineError::InvalidEvidence) ); assert_eq!( RubinLoadingObservation::new( + SNAPSHOT_ID, 1.0, vec![1.0; 257], available("2026-07-01T00:00:00Z"), @@ -316,7 +395,7 @@ fn constructor_and_empty_cutoff_fail_closed() { execute( &early_request, &accepted, - "snapshot-rubin-loading", + SNAPSHOT_ID, too_early, IndicatorKind::AdditiveLogRatio, &noiseless_rows(), @@ -335,7 +414,7 @@ fn execution_refuses_raw_proportion_single_draw_and_unequal_lengths() { execute( &request, &accepted, - "snapshot-rubin-loading", + SNAPSHOT_ID, cutoff(), IndicatorKind::RawProportion, &noiseless_rows(), @@ -346,16 +425,14 @@ fn execution_refuses_raw_proportion_single_draw_and_unequal_lengths() { ); let single_draw = vec![ - RubinLoadingObservation::new(-1.0, vec![-0.7], available("2026-07-01T00:00:00Z")) - .expect("d1"), - RubinLoadingObservation::new(1.0, vec![0.7], available("2026-07-01T00:00:00Z")) - .expect("d2"), + observation(SNAPSHOT_ID, -1.0, vec![-0.7], "2026-07-01T00:00:00Z"), + observation(SNAPSHOT_ID, 1.0, vec![0.7], "2026-07-01T00:00:00Z"), ]; assert_eq!( execute( &request, &accepted, - "snapshot-rubin-loading", + SNAPSHOT_ID, cutoff(), IndicatorKind::AdditiveLogRatio, &single_draw, @@ -366,16 +443,14 @@ fn execution_refuses_raw_proportion_single_draw_and_unequal_lengths() { ); let unequal = vec![ - RubinLoadingObservation::new(-1.0, vec![-0.7, -0.9], available("2026-07-01T00:00:00Z")) - .expect("u1"), - RubinLoadingObservation::new(1.0, vec![0.7, 0.9, 1.1], available("2026-07-01T00:00:00Z")) - .expect("u2"), + observation(SNAPSHOT_ID, -1.0, vec![-0.7, -0.9], "2026-07-01T00:00:00Z"), + observation(SNAPSHOT_ID, 1.0, vec![0.7, 0.9, 1.1], "2026-07-01T00:00:00Z"), ]; assert_eq!( execute( &request, &accepted, - "snapshot-rubin-loading", + SNAPSHOT_ID, cutoff(), IndicatorKind::AdditiveLogRatio, &unequal, @@ -386,6 +461,32 @@ fn execution_refuses_raw_proportion_single_draw_and_unequal_lengths() { ); } +#[test] +fn matrix_resource_budget_fails_before_scientific_fit() { + let request = request(); + let accepted = accepted(&request); + let rows = vec![ + observation( + SNAPSHOT_ID, + 1.0, + vec![1.0; 256], + "2026-07-01T00:00:00Z", + ); + 3_907 + ]; + assert_eq!( + execute( + &request, + &accepted, + SNAPSHOT_ID, + cutoff(), + IndicatorKind::AdditiveLogRatio, + &rows, + ), + Err(AnalysisEngineError::LimitExceeded) + ); +} + #[test] fn execution_refuses_receipt_mismatch_and_oversized_corpus() { let request = request(); @@ -396,7 +497,7 @@ fn execution_refuses_receipt_mismatch_and_oversized_corpus() { execute( &request, &wrong_receipt, - "snapshot-rubin-loading", + SNAPSHOT_ID, cutoff(), IndicatorKind::AdditiveLogRatio, &noiseless_rows(), @@ -405,17 +506,20 @@ fn execution_refuses_receipt_mismatch_and_oversized_corpus() { AnalysisEngineError::Api(tepp_api::ApiError::InvalidWirePayload) ); - let oversized = - vec![ - RubinLoadingObservation::new(1.0, vec![1.0, 2.0], available("2026-07-01T00:00:00Z")) - .expect("row"); - MAX_EVIDENCE_UNITS + 1 - ]; + let oversized = vec![ + observation( + SNAPSHOT_ID, + 1.0, + vec![1.0, 2.0], + "2026-07-01T00:00:00Z", + ); + MAX_EVIDENCE_UNITS + 1 + ]; assert_eq!( execute( &request, &accepted, - "snapshot-rubin-loading", + SNAPSHOT_ID, cutoff(), IndicatorKind::AdditiveLogRatio, &oversized, @@ -432,7 +536,7 @@ fn execution_refuses_invalid_completion_time() { execute_rubin_loading_uncertainty_run( &request, &accepted, - "snapshot-rubin-loading", + SNAPSHOT_ID, cutoff(), IndicatorKind::AdditiveLogRatio, &noiseless_rows(), From a4a0ab0120c9222418f8042af68149e57d8cfe38 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 14 Sep 2026 16:11:24 +0900 Subject: [PATCH 6/8] docs(adr): return Rubin analysis run to Proposed authority --- ...-rubin-loading-uncertainty-analysis-run.md | 157 ++++++++++++------ 1 file changed, 106 insertions(+), 51 deletions(-) diff --git a/docs/adr/0034-rubin-loading-uncertainty-analysis-run.md b/docs/adr/0034-rubin-loading-uncertainty-analysis-run.md index 3a70f86de..ae51dcc81 100644 --- a/docs/adr/0034-rubin-loading-uncertainty-analysis-run.md +++ b/docs/adr/0034-rubin-loading-uncertainty-analysis-run.md @@ -1,82 +1,137 @@ # ADR 0034 — Rubin loading uncertainty as an analysis-run output profile -**Decision status:** Accepted +**Decision status:** Proposed **Implementation maturity:** active-PR — composed on this branch; not implemented-main **Date:** 2026-08-31 +**Last reviewed:** 2026-09-14 **Supersedes:** None; complements ADR 0005 (ESEM/DSEM interpretation) and ADR 0022 (cutoff-safe analysis-run execution). **Figma File ID:** N/A — this increment changes a Rust service crate and has no user-interface surface. **Storybook inventory:** N/A — no reusable web object or interaction changed. ## Context -Protected main already averages posterior-draw OLS loadings and combines those -loadings with Rubin (1996) total variance `T = Ū + (1 + 1/m) B` inside -`psychometric_core`. Operators still cannot request that joint uncertainty -wiring as a digest-bound analysis-run output. Recovery primitives alone are -not the ESEM/DSEM engine (GAP-006 / #169). A second Driver p.16 `std`-family -restore, another CWC bind, or another GAP-003A HTTP slice would not close this -operator-visible gap. - -The library helpers are explicit: the draw-mean is not Rubin pooling, and -Rubin `T` on complete-data OLS loadings is not Mislevy person-level -plausible-value draws. +Protected main owns two deliberately different `psychometric_core` contracts: +`recover_loading_point_estimate_mean` computes a scaled, compensated mean of +posterior-draw OLS loading point estimates, while +`combine_draw_level_ols_loadings` computes Rubin (1996) `Q̄`, `Ū`, `B`, and +`T = Ū + (1 + 1/m)B` for complete-data OLS loadings. The latter's ordinary +loading accumulation is not a substitute for the former's robust point +estimate under large cancellation. + +Operators still cannot request the joint result as a historical, digest-bound +Analysis Run output. The original branch also left several application +contracts weaker than the profile name implied: row availability lacked an +immutable snapshot identity, request/executor cutoffs were compared as RFC +3339 text, draw/matrix materialization was unbounded, artifact counts could +claim unreachable executions, serialized `T` was not checked against its +components, and terminal provider validation reused the scientific inference +label. ## Decision -Add the `rubin_loading_uncertainty_v1` analysis-run output profile to -`analysis_engine`. The executor: - -- consumes already-mapped factor scores, complete-data indicator draws, an - admitted indicator kind, and `available_time`; -- excludes observations whose availability is later than the request - `knowledge_cutoff`; -- jointly invokes `recover_loading_point_estimate_mean` and - `combine_draw_level_ols_loadings` without reimplementing either helper; -- emits a canonical SHA-256-digested `tepp.rubin_loading_uncertainty.v1` - artifact with observation/draw counts, excluded-after-cutoff count, - indicator kind, point-estimate mean, Rubin `Q̄`/`Ū`/`B`/`T`, and inference - status `rubin_combined_ols_loadings_not_mislevy_pv`; -- does not invent an ESEM/DSEM sampler, persist rows, treat the draws as - Mislevy person-level plausible values, or claim strong invariance. - -This is draw-level OLS combination, not multiple imputation of persons, not -CWC, and not a random-effects sampler. +Add `rubin_loading_uncertainty_v1` to `analysis_engine` as an application +composition over the protected-main scientific owners. The executor: + +- requires every `RubinLoadingObservation` to carry the requested immutable + `snapshot_id` and typed `AvailableTime`; +- rejects cross-snapshot observations and excludes same-snapshot observations + with `AvailableTime > KnowledgeCutoff` before matrix/scientific admission; +- parses request cutoffs and compares temporal instants, while persisted + artifacts retain one canonical RFC 3339 cutoff representation; +- invokes `recover_loading_point_estimate_mean` for the robust point estimate + and `combine_draw_level_ols_loadings` independently for Rubin `Q̄/Ū/B/T`; +- limits the current application representation to at most 256 complete-data + draws and 1,000,000 admitted observation-by-draw cells before transposition. + These are resource envelopes, not psychometric validity recommendations; +- bounds total raw observation population with `MAX_EVIDENCE_UNITS` and makes + imported artifact counts obey the same reachable envelope; +- validates imported `T` by recomputing the exact binary64 expression used by + the scientific owner. Canonical JSON round-tripping preserves the component + values, so exact equality is the chosen wire-integrity policy rather than a + tolerance that could admit a different scientific result; +- applies the 256 KiB artifact envelope to both untrusted `from_json` and + canonical `to_json`, with a maximal-valid escaping proof for the output + direction; +- propagates artifact/digest errors instead of asserting that accepted request + identifiers make serialization infallible; +- emits terminal `AnalysisResultSummary.validation_status = "validated"` and + keeps `rubin_combined_ols_loadings_not_mislevy_pv` solely as the artifact's + scientific inference boundary. + +This remains draw-level OLS combination. It is not person-level plausible-value +pooling, an ESEM/DSEM sampler, CWC, persistence, or a causal estimator. + +## Historical replay invariant + +For a fixed requested snapshot and knowledge cutoff, adding evidence that only +becomes available after that cutoff must not change the earlier admitted +factor-score/draw matrix or its scientific result. Such rows may change only +the excluded-after-cutoff count. A row from another immutable snapshot is not +historical censoring; it is a provenance violation and fails closed even when +its availability is later than the cutoff. ## Alternatives considered -1. Restore another Driver p.16 standardised matrix — rejected because those - recoveries are already a live micro-PR family and do not bind uncertainty - to an analysis run. -2. Duplicate the GAP-006 CWC analysis-run bind — rejected because CWC slopes - are a different estimand already occupied by a live PR. -3. Put Rubin combination into `tepp_api` — rejected because transport - contracts and scientific combination would become one service boundary. -4. Bind the existing `psychometric_core` draw-mean and Rubin `T` helpers to - ADR 0022's analysis-run profile — accepted. +1. Use `combine_draw_level_ols_loadings.mean_loading` for both fields — rejected + because it bypasses the protected robust point-estimate contract and can + differ under large cancellation. +2. Compare cutoff strings — rejected because legal RFC 3339 representations of + one instant must not alter historical execution. +3. Drop late rows without snapshot provenance — rejected because unrelated + snapshot data could be silently attributed to the requested run. +4. Accept any finite nonnegative serialized `T` — rejected because a digest- + bound uncertainty artifact must be internally consistent with its own + components and draw count. +5. Leave draws unbounded and rely on `MAX_EVIDENCE_UNITS` — rejected because + matrix materialization is a separate multiplicative resource dimension. +6. Put Rubin arithmetic into `analysis_engine` — rejected because reusable + scientific arithmetic remains `psychometric_core`-owned. + +## Scientific acceptance boundary + +Known-truth/noiseless fixtures and edge contracts are regression evidence, not +commercial scientific acceptance. Issue #503 owns repeated true-loading +recovery, bias/RMSE with Monte Carlo uncertainty, an explicitly justified +interval construction and empirical coverage, attempted/recovered/failed +denominators, design sensitivity, and leakage-safe historical evaluation. +Neither LLM judgment nor synthetic unit fixtures may close that gap. + +Primary authority for the current combining rule remains: + +Rubin, D. B. (1996). Multiple imputation after 18+ years. *Journal of the +American Statistical Association, 91*(434), 473–489. +https://doi.org/10.1080/01621459.1996.10476908 + +Repository research authority is `docs/research/rubin-total-variance.md`. ## Consequences -Operators can request cutoff-safe joint point-estimate and Rubin-`T` loading -uncertainty as a digest-bound terminal result. The artifact is not Mislevy -person-level plausible values, not an ESEM fit, and not implemented-main -until exact-head Checks and two independent approvals land. +The profile has a narrower, auditable temporal and resource boundary, and its +artifact can no longer claim a Rubin total inconsistent with its serialized +components. Consumers can distinguish provider validation from the scientific +claim boundary and can distinguish the robust point estimate from Rubin `Q̄`. +The profile remains Draft/Proposed and not implemented-main while #503 and the +normal exact-head merge gates remain unresolved. ## Verification +Required exact-head verification includes: + ```text -cargo fmt -p analysis_engine -- --check +cargo fmt --all -- --check cargo test -p analysis_engine cargo clippy -p analysis_engine --all-targets -- -D warnings +python3 scripts/validate_documentation.py ``` -Known-truth noiseless draws recover mean loading `0.8` with strictly positive -between-draw variance and Rubin `T = Ū + (1 + 1/m) B`. Cutoff exclusion, -snapshot/profile mismatch, empty eligibility, a single draw, raw proportions, -and unequal draw lengths fail closed. +Regression contracts cover equivalent cutoff instants, future-evidence replay, +cross-snapshot refusal, robust-point versus naive-mean cancellation, exact and +exceeded draw/resource bounds, inconsistent Rubin totals, artifact count +bounds, and terminal provider/domain-status separation. ## Rollback and supersession Rollback removes the `rubin_loading_uncertainty_v1` profile. No persisted -schema migration is introduced. Supersede only with an ADR that keeps Rubin -`T` on complete-data OLS loadings distinct from Mislevy person-level -plausible values. +schema migration is introduced. Supersede only with an ADR that preserves the +scientific owner split, temporal provenance, resource admission, and the +Rubin-versus-Mislevy claim boundary. From b3094612153e35b551383578d82ac1dbb4d74931 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 14 Sep 2026 16:11:43 +0900 Subject: [PATCH 7/8] docs(doctoring): record Rubin contract repair lineage --- .../rubin-loading-uncertainty-analysis-run.md | 36 ++++++++++++------- 1 file changed, 24 insertions(+), 12 deletions(-) diff --git a/docs/doctoring/rubin-loading-uncertainty-analysis-run.md b/docs/doctoring/rubin-loading-uncertainty-analysis-run.md index 14148ff23..95a58691b 100644 --- a/docs/doctoring/rubin-loading-uncertainty-analysis-run.md +++ b/docs/doctoring/rubin-loading-uncertainty-analysis-run.md @@ -1,19 +1,31 @@ # Rubin loading-uncertainty analysis-run bind -**Review date:** 2026-08-31 -**Active slice:** GAP-006 / issue #169 remaining operator-visible composition +**Review date:** 2026-09-14 +**Active slice:** GAP-006 / issue #169 operator-visible composition +**Scientific acceptance owner:** #503 +**Implementation maturity:** active-PR — not implemented-main -Protected main already averages posterior-draw OLS loadings and combines them -with Rubin (1996) total variance in `psychometric_core`. This slice binds those -helpers jointly to `analysis_engine` as a cutoff-safe analysis-run output: -eligibility against the request knowledge cutoff, digest-bound -`tepp.rubin_loading_uncertainty.v1`, and an explicit refusal to treat the -draws as Mislevy person-level plausible values. +Protected main owns the reusable numerical contracts. `recover_loading_point_estimate_mean` performs the robust point-estimate aggregation, while `combine_draw_level_ols_loadings` owns Rubin (1996) `Q̄`, `Ū`, `B`, and `T`. This Analysis Run profile composes those owners without copying either arithmetic path. -This is not a new estimator, not a Driver p.16 `std` restore, not CWC, not -persistence, and not implemented-main. +The review found that the predecessor branch was materially weaker than its cutoff-safe claim. It lacked per-row immutable snapshot provenance, compared cutoffs as timestamp text, admitted an unbounded draw dimension, accepted artifact counts outside the executable population, did not verify serialized Rubin `T`, used the Rubin mean as the supposedly robust point estimate, asserted artifact hashing could not fail, and wrote the scientific inference label into provider `validation_status`. + +## Repair lineage + +- RED `152cb3913d0920850d9a53e13cb6b4edfff26fb7` makes equivalent RFC 3339 cutoff instants, provider/domain status separation, robust point-estimate cancellation, inconsistent Rubin totals, count bounds, and the draw ceiling executable contracts. +- Causal repair `dd46f8ebac0e1bc110648152453dc9fce87bef23` adds per-observation immutable `snapshot_id`, instant-based cutoff binding, cross-snapshot refusal, a 256-draw / 1,000,000 admitted-cell application resource envelope, reachable artifact count validation, exact binary64 Rubin-total integrity, symmetric 256 KiB artifact wire admission, protected-main robust point-estimate invocation, error propagation, and terminal `validation_status = "validated"`. +- Contract migration `8f0e743348290d65d15e048216cccb1f5953f2af` updates the integration surface to explicit snapshot provenance and adds historical replay, cross-snapshot, resource-matrix, point-estimate, cutoff-instant, and terminal-status regressions. +- ADR `a4a0ab0120c9222418f8042af68149e57d8cfe38` returns ADR 0034 from premature `Accepted` branch authority to `Proposed` and records the scientific/resource alternatives. + +The historical replay contract is explicit: same-snapshot rows with `AvailableTime > KnowledgeCutoff` do not enter the earlier scientific matrix. Cross-snapshot rows are provenance violations and fail closed rather than being censored. Persisted artifacts use canonical cutoff text, while request/executor binding is by temporal instant. + +The 256-draw and 1,000,000 matrix-cell ceilings are application resource limits for the current representation. They are not psychometric recommendations and must not be cited as scientific sample-size or imputation-count guidance. ## Evidence boundary -Exact-head checks, independent review, and protected merge are required before -the profile can be promoted. +The current deterministic fixtures verify contracts and known identities. They do not establish profile-level recovery or interval coverage. Issue #503 requires repeated true-loading recovery, bias/RMSE with Monte Carlo uncertainty, an explicitly justified interval construction before any coverage claim, empirical coverage with Monte Carlo uncertainty, attempted/recovered/failed denominators, design sensitivity, and leakage-safe historical evaluation. + +Rubin, D. B. (1996). Multiple imputation after 18+ years. *Journal of the American Statistical Association, 91*(434), 473–489. https://doi.org/10.1080/01621459.1996.10476908 + +Repository authority: `docs/research/rubin-total-variance.md`. + +Exact-head required workflows, owned-production line/branch coverage, resolved current findings, a qualifying independent approval, conflict-resolving successor inheritance, and #503 or equivalent checked-in scientific evidence remain required before Ready/release claims. Predecessor receipts do not transfer. From 7fb76d1d5b338ecfb652a11a136c984c092ada15 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 14 Sep 2026 16:12:26 +0900 Subject: [PATCH 8/8] chore(analysis): remove duplicate psychometric dev dependency --- crates/analysis_engine/Cargo.toml | 1 - 1 file changed, 1 deletion(-) diff --git a/crates/analysis_engine/Cargo.toml b/crates/analysis_engine/Cargo.toml index d1f55851c..2ada0c3be 100644 --- a/crates/analysis_engine/Cargo.toml +++ b/crates/analysis_engine/Cargo.toml @@ -27,7 +27,6 @@ uuid.workspace = true [dev-dependencies] corpus_split = { path = "../corpus_split", version = "0.2.0" } membership_core = { path = "../membership_core", version = "0.2.0" } -psychometric_core = { path = "../psychometric_core", version = "0.2.0" } relation_graph = { path = "../relation_graph", version = "0.2.0" } [lints]