diff --git a/crates/openshell-core/README.md b/crates/openshell-core/README.md index 1cf1b7e70e..4e5177cd34 100644 --- a/crates/openshell-core/README.md +++ b/crates/openshell-core/README.md @@ -58,6 +58,21 @@ router agree on provider defaults. Profiles define: Do not duplicate provider-specific inference behavior in callers. Add shared behavior here, then consume it from the gateway, sandbox, and router. +## Static Credential Resolution + +Static credential bindings retain endpoint authorization. The gateway includes +static `ProviderProfileCredential` auth metadata in the existing runtime +credential map under `static-env:`. Provider state extracts those entries +before storing dynamic token grants and compiles placement locally alongside the +matching static binding. Missing metadata preserves legacy endpoint-scoped +substitution; malformed reserved metadata fails closed atomically. + +Request rewriting supplies the actual header name, query key, or path slot. +Header authorization is independent of authentication scheme, including Basic. +Explicit body/WebSocket rewrite policies and trusted signing retain their +existing permissions. Existing placeholder and stable-handle derivation remain +unchanged. Protobuf message shapes and service interfaces remain unchanged. + ## Middleware Contracts Built-in supervisor middleware identifiers, host-selector matching, and pure diff --git a/crates/openshell-core/src/provider_credentials.rs b/crates/openshell-core/src/provider_credentials.rs index d71f34b7c6..99c52b4e55 100644 --- a/crates/openshell-core/src/provider_credentials.rs +++ b/crates/openshell-core/src/provider_credentials.rs @@ -3,7 +3,7 @@ //! Runtime provider credential snapshots. -use crate::secrets::SecretResolver; +use crate::secrets::{CredentialPlacement, SecretResolver}; use crate::{ endpoint_path::EndpointPathPattern, host_pattern::HostPattern, @@ -67,6 +67,7 @@ struct StaticCredentialIdentityEpoch { #[derive(Debug, Clone)] struct CompiledStaticCredentialBinding { + placement: CredentialPlacement, endpoints: Vec, credential_identity: String, workload_credential_handle: String, @@ -97,6 +98,29 @@ impl fmt::Display for StaticCredentialBindingError { impl std::error::Error for StaticCredentialBindingError {} +/// Reserved entries in the existing credential map carry static placement only. +pub const STATIC_CREDENTIAL_METADATA_PREFIX: &str = "static-env:"; + +const CREDENTIAL_PATH_SLOT: &str = "{credential}"; + +fn split_static_credential_metadata( + credentials: HashMap, +) -> ( + HashMap, + HashMap, +) { + let mut metadata = HashMap::new(); + let mut grants = HashMap::new(); + for (key, credential) in credentials { + if let Some(env_key) = key.strip_prefix(STATIC_CREDENTIAL_METADATA_PREFIX) { + metadata.insert(env_key.to_string(), credential); + } else { + grants.insert(key, credential); + } + } + (metadata, grants) +} + impl ProviderCredentialState { pub fn from_environment( revision: u64, @@ -104,6 +128,7 @@ impl ProviderCredentialState { credential_expires_at_ms: HashMap, dynamic_credentials: HashMap, ) -> Self { + let (_, dynamic_credentials) = split_static_credential_metadata(dynamic_credentials); let (child_env, generation_resolver, current_resolver) = SecretResolver::from_provider_env_for_current_revision( env, @@ -173,11 +198,14 @@ impl ProviderCredentialState { non_secret_environment_keys: Vec, stable_placeholder_environment_keys: Vec, ) -> Result { + let (placement_metadata, dynamic_credentials) = + split_static_credential_metadata(dynamic_credentials); let static_credential_bindings = compile_static_credential_bindings( &env, static_credential_bindings, &non_secret_environment_keys, &stable_placeholder_environment_keys, + &placement_metadata, )?; let stable_handles = static_credential_stable_handles(&static_credential_bindings); let (child_env, generation_resolver, current_resolver) = @@ -304,11 +332,21 @@ impl ProviderCredentialState { } pub fn resolver(&self) -> Option> { - self.inner + let inner = self + .inner .read() - .expect("provider credential state poisoned") - .combined_resolver - .clone() + .expect("provider credential state poisoned"); + inner.combined_resolver.as_ref().map(|resolver| { + Arc::new( + resolver.as_ref().clone().with_placements( + inner + .static_credential_bindings + .iter() + .map(|(key, binding)| (key.clone(), binding.placement.clone())) + .collect(), + ), + ) + }) } /// Resolve provider placeholders only for credentials bound to this @@ -414,11 +452,23 @@ impl ProviderCredentialState { }) .map(|(key, epoch)| (key.clone(), epoch.revisions.clone())) .collect(); - Arc::new(resolver.scoped_to_env_keys( - &inner.known_static_credential_keys, - &allowed, - revision_fallback_allowed_revisions, - )) + Arc::new( + resolver + .as_ref() + .clone() + .with_placements( + inner + .static_credential_bindings + .iter() + .map(|(key, binding)| (key.clone(), binding.placement.clone())) + .collect(), + ) + .scoped_to_env_keys( + &inner.known_static_credential_keys, + &allowed, + revision_fallback_allowed_revisions, + ), + ) }); (resolver, classifier, revision) } @@ -694,6 +744,7 @@ impl ProviderCredentialState { credential_expires_at_ms: HashMap, dynamic_credentials: HashMap, ) -> usize { + let (_, dynamic_credentials) = split_static_credential_metadata(dynamic_credentials); let (mut child_env, generation_resolver, current_resolver) = SecretResolver::from_provider_env_for_current_revision( env, @@ -774,11 +825,14 @@ impl ProviderCredentialState { non_secret_environment_keys: Vec, stable_placeholder_environment_keys: Vec, ) -> Result { + let (placement_metadata, dynamic_credentials) = + split_static_credential_metadata(dynamic_credentials); let static_credential_bindings = match compile_static_credential_bindings( &env, static_credential_bindings, &non_secret_environment_keys, &stable_placeholder_environment_keys, + &placement_metadata, ) { Ok(bindings) => bindings, Err(error) => { @@ -939,6 +993,7 @@ impl ProviderCredentialState { .expect("provider credential state poisoned"); let dynamic_credentials = dynamic_credentials.unwrap_or_else(|| inner.current.dynamic_credentials.clone()); + let (_, dynamic_credentials) = split_static_credential_metadata(dynamic_credentials); inner.current = Arc::new(ProviderCredentialSnapshot { installation_id: uuid::Uuid::new_v4().to_string(), revision, @@ -959,7 +1014,22 @@ fn compile_static_credential_bindings( bindings: HashMap, non_secret_environment_keys: &[String], stable_placeholder_environment_keys: &[String], + placement_metadata: &HashMap, ) -> Result, StaticCredentialBindingError> { + for (key, metadata) in placement_metadata { + if !env.contains_key(key) + || !bindings.contains_key(key) + || metadata.env_vars.len() != 1 + || metadata.env_vars[0] != *key + || metadata.token_grant.is_some() + || metadata.refresh.is_some() + || !metadata.token_grant_owners.is_empty() + { + return Err(binding_error( + "static credential placement metadata does not match delivered credentials", + )); + } + } let non_secret_keys = non_secret_environment_keys .iter() .cloned() @@ -1056,6 +1126,10 @@ fn compile_static_credential_bindings( bindings .into_iter() .map(|(key, binding)| { + let placement = placement_metadata.get(&key).map_or( + Ok(CredentialPlacement::Legacy), + compile_credential_placement, + )?; let endpoints = binding .endpoints .into_iter() @@ -1065,6 +1139,7 @@ fn compile_static_credential_bindings( key, CompiledStaticCredentialBinding { endpoints, + placement, credential_identity: binding.credential_identity, workload_credential_handle: binding.workload_credential_handle, }, @@ -1073,6 +1148,49 @@ fn compile_static_credential_bindings( .collect() } +fn compile_credential_placement( + metadata: &crate::proto::ProviderProfileCredential, +) -> Result { + let placement = match metadata.auth_style.trim().to_ascii_lowercase().as_str() { + "basic" => CredentialPlacement::Header("authorization".into()), + "header" | "bearer" + if !metadata.header_name.trim().is_empty() + && metadata + .header_name + .trim() + .bytes() + .all(|b| b.is_ascii_alphanumeric() || b"!#$%&'*+-.^_`|~".contains(&b)) => + { + CredentialPlacement::Header(metadata.header_name.trim().to_ascii_lowercase()) + } + "query" + if !metadata.query_param.trim().is_empty() + && !metadata + .query_param + .chars() + .any(|c| c.is_control() || "&=#?".contains(c)) => + { + CredentialPlacement::QueryParam(metadata.query_param.trim().to_string()) + } + "path" + if metadata.path_template.starts_with('/') + && metadata.path_template.matches(CREDENTIAL_PATH_SLOT).count() == 1 + && !metadata + .path_template + .chars() + .any(|c| c.is_control() || "?#".contains(c)) => + { + CredentialPlacement::PathTemplate(metadata.path_template.clone()) + } + _ => { + return Err(binding_error( + "static credential placement metadata has an invalid request location", + )); + } + }; + Ok(placement) +} + fn compile_static_credential_endpoint( endpoint: StaticCredentialEndpointBinding, ) -> Result { @@ -1365,15 +1483,182 @@ mod tests { binding } + fn placement_metadata(style: &str) -> HashMap { + HashMap::from([( + format!("{STATIC_CREDENTIAL_METADATA_PREFIX}API_KEY"), + crate::proto::ProviderProfileCredential { + name: "api_key".into(), + env_vars: vec!["API_KEY".into()], + auth_style: style.into(), + header_name: "Authorization".into(), + query_param: "api_key".into(), + path_template: "/v1/{credential}/events".into(), + ..Default::default() + }, + )]) + } + + #[test] + fn static_metadata_infers_locations_without_becoming_a_token_grant() { + for style in ["bearer", "header", "basic", "query", "path"] { + let state = ProviderCredentialState::from_bound_environment( + 7, + HashMap::from([("API_KEY".into(), "synthetic".into())]), + HashMap::new(), + placement_metadata(style), + HashMap::from([("API_KEY".into(), binding("api.example.com", 443, "/**"))]), + vec![], + ) + .unwrap(); + assert!(state.snapshot().dynamic_credentials.is_empty()); + let reference = state.snapshot().child_env["API_KEY"].clone(); + let resolver = state + .resolver_for_endpoint("api.example.com", 443, "/v1/chat") + .unwrap(); + assert!( + resolver + .rewrite_header_value_for_name("X-Reflect", &reference) + .is_err() + ); + let header = resolver.rewrite_header_value_for_name("Authorization", &reference); + let query = crate::secrets::rewrite_target_for_eval( + &format!("/v1/chat?api_key={reference}"), + &resolver, + ); + let path = crate::secrets::rewrite_target_for_eval( + &format!("/v1/{reference}/events"), + &resolver, + ); + assert_eq!( + header.is_ok(), + matches!(style, "bearer" | "header" | "basic") + ); + assert_eq!(query.is_ok(), style == "query"); + assert_eq!(path.is_ok(), style == "path"); + } + } + + #[test] + fn malformed_static_metadata_revokes_static_material_and_preserves_real_grants() { + for malformed in 0..6 { + let state = ProviderCredentialState::from_bound_environment( + 7, + HashMap::from([("API_KEY".into(), "synthetic".into())]), + HashMap::new(), + HashMap::new(), + HashMap::from([("API_KEY".into(), binding("api.example.com", 443, "/**"))]), + vec![], + ) + .unwrap(); + let mut metadata = placement_metadata("header"); + let key = format!("{STATIC_CREDENTIAL_METADATA_PREFIX}API_KEY"); + let value = metadata.get_mut(&key).unwrap(); + match malformed { + 0 => value.env_vars = vec!["OTHER".into()], + 1 => { + value.token_grant = Some(crate::proto::ProviderCredentialTokenGrant::default()); + } + 2 => value.refresh = Some(crate::proto::ProviderCredentialRefresh::default()), + 3 => value.token_grant_owners = vec!["owner".into()], + 4 => value.auth_style = "unsupported".into(), + 5 => { + let value = metadata.remove(&key).unwrap(); + metadata.insert(format!("{STATIC_CREDENTIAL_METADATA_PREFIX}OTHER"), value); + } + _ => unreachable!(), + } + metadata.insert( + "real-grant".into(), + crate::proto::ProviderProfileCredential { + token_grant: Some(crate::proto::ProviderCredentialTokenGrant::default()), + ..Default::default() + }, + ); + assert!( + state + .install_bound_environment( + 8, + HashMap::from([("API_KEY".into(), "replacement".into())]), + HashMap::new(), + metadata, + HashMap::from([("API_KEY".into(), binding("api.example.com", 443, "/**"))]), + vec![], + ) + .is_err() + ); + assert!(state.snapshot().child_env.is_empty()); + assert_eq!(state.snapshot().dynamic_credentials.len(), 1); + assert!( + state + .snapshot() + .dynamic_credentials + .contains_key("real-grant") + ); + } + } + + #[test] + fn stable_references_survive_metadata_arrival_and_use_the_current_location() { + let handle = "a".repeat(64); + let bindings = HashMap::from([( + "API_KEY".into(), + stable_binding("api.example.com", 443, "/**", &handle), + )]); + let state = ProviderCredentialState::from_bound_environment_with_stable_placeholders( + 7, + HashMap::from([("API_KEY".into(), "synthetic".into())]), + HashMap::new(), + HashMap::new(), + bindings.clone(), + vec![], + vec!["API_KEY".into()], + ) + .unwrap(); + let reference = state.snapshot().child_env["API_KEY"].clone(); + for style in [None, Some("header"), Some("query"), None] { + state + .install_bound_environment_with_stable_placeholders( + 8, + HashMap::from([("API_KEY".into(), "synthetic".into())]), + HashMap::new(), + style.map_or_else(HashMap::new, placement_metadata), + bindings.clone(), + vec![], + vec!["API_KEY".into()], + ) + .unwrap(); + assert_eq!(state.snapshot().child_env["API_KEY"], reference); + let resolver = state + .resolver_for_endpoint("api.example.com", 443, "/v1/chat") + .unwrap(); + let query = crate::secrets::rewrite_target_for_eval( + &format!("/v1/chat?api_key={reference}"), + &resolver, + ); + assert_eq!(query.is_ok(), style != Some("header")); + assert_eq!( + resolver + .rewrite_header_value_for_name("Authorization", &reference) + .is_ok(), + style != Some("query") + ); + } + } + fn assert_binding_validation_error( env: HashMap, bindings: HashMap, non_secret_environment_keys: Vec, expected: &str, ) { - let error = - compile_static_credential_bindings(&env, bindings, &non_secret_environment_keys, &[]) - .expect_err("malformed provider metadata must fail validation"); + let error = compile_static_credential_bindings( + &env, + bindings, + &non_secret_environment_keys, + &[], + &HashMap::new(), + ) + .expect_err("malformed provider metadata must fail validation"); assert_eq!(error.to_string(), expected); } diff --git a/crates/openshell-core/src/secrets.rs b/crates/openshell-core/src/secrets.rs index 2e51c9de28..6241186631 100644 --- a/crates/openshell-core/src/secrets.rs +++ b/crates/openshell-core/src/secrets.rs @@ -114,9 +114,18 @@ pub struct UnresolvedPlaceholderError { enum UnresolvedPlaceholderReason { Unavailable, EndpointMismatch, + PlacementMismatch, } +pub const CREDENTIAL_PLACEMENT_MISMATCH: &str = + "credential is not authorized for the request location"; + impl UnresolvedPlaceholderError { + #[must_use] + pub fn is_placement_mismatch(&self) -> bool { + self.reason == UnresolvedPlaceholderReason::PlacementMismatch + } + #[must_use] pub fn unavailable(location: &'static str) -> Self { unresolved(location) @@ -136,6 +145,7 @@ impl fmt::Display for UnresolvedPlaceholderError { match self.reason { UnresolvedPlaceholderReason::Unavailable => "credential placeholder could not be resolved", + UnresolvedPlaceholderReason::PlacementMismatch => CREDENTIAL_PLACEMENT_MISMATCH, UnresolvedPlaceholderReason::EndpointMismatch => "credential is not authorized for the request endpoint", }, @@ -180,9 +190,31 @@ pub struct RewriteTargetResult { // SecretResolver // --------------------------------------------------------------------------- +/// Credential-specific destinations authorized by the provider profile. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum CredentialPlacement { + /// Historical substitution behavior when no placement metadata is delivered. + Legacy, + Header(String), + QueryParam(String), + PathTemplate(String), +} + +/// The actual destination of a workload-provided credential placeholder. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum RequestCredentialLocation { + Header(String), + QueryParam(String), + Path(String), + Body, + WebSocket, +} + #[derive(Clone, Default)] pub struct SecretResolver { by_placeholder: HashMap, + placements: HashMap, + request_location: Option, denied_env_keys: HashSet, identity_bound_env_keys: HashSet, revision_fallback_allowed_revisions: HashMap>>, @@ -292,6 +324,8 @@ impl SecretResolver { let resolver = |by_placeholder: HashMap| { (!by_placeholder.is_empty()).then_some(Self { by_placeholder, + placements: HashMap::new(), + request_location: None, denied_env_keys: HashSet::new(), identity_bound_env_keys: HashSet::new(), revision_fallback_allowed_revisions: HashMap::new(), @@ -347,6 +381,8 @@ impl SecretResolver { child_env, Some(Self { by_placeholder, + placements: HashMap::new(), + request_location: None, denied_env_keys: HashSet::new(), identity_bound_env_keys: HashSet::new(), revision_fallback_allowed_revisions: HashMap::new(), @@ -357,11 +393,13 @@ impl SecretResolver { pub fn merge<'a>(resolvers: impl IntoIterator) -> Option { let mut by_placeholder = HashMap::new(); + let mut placements = HashMap::new(); let mut denied_env_keys = HashSet::new(); let mut identity_bound_env_keys = HashSet::new(); let mut revision_fallback_allowed_revisions = HashMap::new(); for resolver in resolvers { by_placeholder.extend(resolver.by_placeholder.clone()); + placements.extend(resolver.placements.clone()); denied_env_keys.extend(resolver.denied_env_keys.iter().cloned()); identity_bound_env_keys.extend(resolver.identity_bound_env_keys.iter().cloned()); revision_fallback_allowed_revisions @@ -372,6 +410,8 @@ impl SecretResolver { } else { Some(Self { by_placeholder, + placements, + request_location: None, denied_env_keys, identity_bound_env_keys, revision_fallback_allowed_revisions, @@ -405,12 +445,92 @@ impl SecretResolver { .collect(); Self { by_placeholder, + placements: self.placements.clone(), + request_location: None, denied_env_keys, identity_bound_env_keys: bound_keys.clone(), revision_fallback_allowed_revisions, } } + /// Attach mandatory placement metadata to a resolver snapshot. + pub(crate) fn with_placements( + mut self, + placements: HashMap, + ) -> Self { + self.placements = placements; + self + } + + /// Select the actual structured request destination before resolving input. + #[must_use] + pub fn with_request_location(&self, location: RequestCredentialLocation) -> Self { + let mut scoped = self.clone(); + scoped.request_location = Some(location); + scoped + } + + fn placement_permits(&self, placeholder: &str) -> bool { + let Some(key) = placeholder_env_key(placeholder) else { + return true; + }; + let Some(placement) = self.placements.get(key) else { + return true; + }; + match (placement, self.request_location.as_ref()) { + (CredentialPlacement::Legacy, _) + | ( + CredentialPlacement::Header(_) + | CredentialPlacement::QueryParam(_) + | CredentialPlacement::PathTemplate(_), + Some(RequestCredentialLocation::Body | RequestCredentialLocation::WebSocket), + ) => true, + ( + CredentialPlacement::Header(expected), + Some(RequestCredentialLocation::Header(actual)), + ) => expected.eq_ignore_ascii_case(actual), + ( + CredentialPlacement::QueryParam(expected), + Some(RequestCredentialLocation::QueryParam(actual)), + ) => expected == actual, + ( + CredentialPlacement::PathTemplate(expected), + Some(RequestCredentialLocation::Path(actual)), + ) => { + // Normalize only this credential's slot; all other path text must match. + let mut normalized = String::new(); + let mut pos = 0; + let mut slots = 0; + while pos < actual.len() { + let next_canonical = actual[pos..] + .find(PLACEHOLDER_PREFIX) + .map(|offset| pos + offset); + let next_alias = actual[pos..] + .find(PROVIDER_ALIAS_MARKER) + .map(|offset| alias_start_for_marker(actual, pos + offset)); + let next = [next_canonical, next_alias].into_iter().flatten().min(); + let Some(start) = next else { + normalized.push_str(&actual[pos..]); + break; + }; + normalized.push_str(&actual[pos..start]); + let Some((end, token)) = self.credential_token_at(actual, start) else { + return false; + }; + if placeholder_env_key(token) == Some(key) { + normalized.push_str("{credential}"); + slots += 1; + } else { + normalized.push_str(token); + } + pos = end; + } + slots == 1 && normalized == *expected + } + _ => false, + } + } + fn unresolved_for( &self, location: &'static str, @@ -420,6 +540,8 @@ impl SecretResolver { .is_some_and(|key| self.denied_env_keys.contains(key)) { UnresolvedPlaceholderReason::EndpointMismatch + } else if !self.placement_permits(placeholder) { + UnresolvedPlaceholderReason::PlacementMismatch } else { UnresolvedPlaceholderReason::Unavailable }; @@ -435,6 +557,9 @@ impl SecretResolver { // handle retains the original endpoint, epoch, and expiration constraints. let canonical_alias = alias_env_key(value).map(placeholder_for_env_key); let value = canonical_alias.as_deref().unwrap_or(value); + if !self.placement_permits(value) { + return None; + } if placeholder_env_key(value).is_some_and(|key| self.identity_bound_env_keys.contains(key)) && revisioned_placeholder_parts(value).is_none() && stable_placeholder_parts(value).is_none() @@ -510,6 +635,9 @@ impl SecretResolver { reason: UnresolvedPlaceholderReason::EndpointMismatch, }); } + if !self.placement_permits(value) { + return Err(self.unresolved_for(location, value)); + } Ok(self.resolve_placeholder(value)) } @@ -519,6 +647,15 @@ impl SecretResolver { .map(|s| s.expires_at_ms) } + pub fn rewrite_header_value_for_name( + &self, + name: &str, + value: &str, + ) -> Result, UnresolvedPlaceholderError> { + self.with_request_location(RequestCredentialLocation::Header(name.to_string())) + .rewrite_header_value(value) + } + pub fn rewrite_header_value( &self, value: &str, @@ -562,6 +699,15 @@ impl SecretResolver { &self, text: &mut String, location: &'static str, + ) -> Result { + self.with_request_location(RequestCredentialLocation::Body) + .rewrite_text_placeholders_at_location(text, location) + } + + fn rewrite_text_placeholders_at_location( + &self, + text: &mut String, + location: &'static str, ) -> Result { if !contains_raw_reserved_marker(text) { return Ok(0); @@ -627,7 +773,8 @@ impl SecretResolver { &self, text: &mut String, ) -> Result { - self.rewrite_text_placeholders(text, "websocket") + self.with_request_location(RequestCredentialLocation::WebSocket) + .rewrite_text_placeholders_at_location(text, "websocket") } fn credential_token_at<'a>( @@ -677,7 +824,7 @@ impl SecretResolver { } let mut rewritten = decoded; - let replacements = self.rewrite_text_placeholders(&mut rewritten, "header")?; + let replacements = self.rewrite_text_placeholders_at_location(&mut rewritten, "header")?; if replacements == 0 { return Ok(None); @@ -1092,6 +1239,9 @@ fn rewrite_uri_path( path: &str, resolver: &SecretResolver, ) -> Result, UnresolvedPlaceholderError> { + let scoped = + resolver.with_request_location(RequestCredentialLocation::Path(percent_decode(path))); + let resolver = &scoped; let segments: Vec<&str> = path.split('/').collect(); let mut resolved_segments = Vec::with_capacity(segments.len()); let mut redacted_segments = Vec::with_capacity(segments.len()); @@ -1202,8 +1352,11 @@ fn rewrite_uri_query_params( let decoded_value = percent_decode(value); if contains_raw_reserved_marker(&decoded_value) { let mut rewritten = decoded_value.clone(); + let decoded_key = percent_decode(&key.replace('+', " ")); + let scoped = resolver + .with_request_location(RequestCredentialLocation::QueryParam(decoded_key)); let replacements = - resolver.rewrite_text_placeholders(&mut rewritten, "query_param")?; + scoped.rewrite_text_placeholders_at_location(&mut rewritten, "query_param")?; if replacements == 0 || contains_raw_reserved_marker(&rewritten) { return Err(unresolved("query_param")); } @@ -1326,10 +1479,12 @@ pub fn rewrite_header_line_checked( return Ok(line.to_string()); }; - resolver.rewrite_header_value(value.trim())?.map_or_else( - || Ok(line.to_string()), - |rewritten| Ok(format!("{name}: {rewritten}")), - ) + resolver + .rewrite_header_value_for_name(name, value.trim())? + .map_or_else( + || Ok(line.to_string()), + |rewritten| Ok(format!("{name}: {rewritten}")), + ) } /// Resolve placeholders in a request target (path + query) for OPA evaluation. @@ -1473,6 +1628,141 @@ mod tests { )); } + fn placement_resolver(placement: CredentialPlacement) -> SecretResolver { + let (_, resolver) = SecretResolver::from_provider_env_for_revision( + HashMap::from([("TOKEN".to_string(), "secret-value".to_string())]), + HashMap::new(), + 1, + ); + resolver + .unwrap() + .with_placements(HashMap::from([("TOKEN".to_string(), placement)])) + } + + #[test] + fn static_header_placement_cannot_be_overridden_by_request_location() { + let resolver = placement_resolver(CredentialPlacement::Header("Authorization".into())); + let token = "openshell:resolve:env:v1_TOKEN"; + assert_eq!( + resolver + .rewrite_header_value_for_name("aUtHoRiZaTiOn", &format!("token {token}")) + .unwrap(), + Some("token secret-value".into()) + ); + assert!( + resolver + .rewrite_header_value_for_name("X-Reflect", token) + .unwrap_err() + .is_placement_mismatch() + ); + assert!( + rewrite_target_for_eval(&format!("/search?q={token}"), &resolver) + .unwrap_err() + .is_placement_mismatch() + ); + assert!( + rewrite_target_for_eval(&format!("/bot{token}/events"), &resolver) + .unwrap_err() + .is_placement_mismatch() + ); + assert_eq!( + resolver + .rewrite_text_placeholders(&mut token.to_string(), "body") + .unwrap(), + 1 + ); + assert_eq!( + resolver + .rewrite_websocket_text_placeholders(&mut token.to_string()) + .unwrap(), + 1 + ); + assert!(resolver.resolve_placeholder(token).is_none()); + let basic = base64::engine::general_purpose::STANDARD.encode(format!("user:{token}")); + assert!( + resolver + .rewrite_header_value_for_name("Authorization", &format!("Basic {basic}")) + .unwrap() + .is_some() + ); + assert!( + resolver + .rewrite_header_value_for_name("X-Reflect", &format!("Basic {basic}")) + .unwrap_err() + .is_placement_mismatch() + ); + } + + #[test] + fn query_placement_checks_each_decoded_parameter_name() { + let resolver = placement_resolver(CredentialPlacement::QueryParam("api_key".into())); + let token = "openshell:resolve:env:v1_TOKEN"; + let valid = format!("/search?%61pi_key={token}"); + assert_eq!( + rewrite_target_for_eval(&valid, &resolver).unwrap().resolved, + "/search?%61pi_key=secret-value" + ); + for target in [ + format!("/search?other={token}"), + format!("/search?api_key={token}&other={token}"), + "/search?other=vendor-OPENSHELL-RESOLVE-ENV-v1_TOKEN".to_string(), + ] { + assert!( + rewrite_target_for_eval(&target, &resolver) + .unwrap_err() + .is_placement_mismatch() + ); + } + assert!( + rewrite_target_for_eval( + &format!("/search?api_key={token}&api_key={token}"), + &resolver + ) + .is_ok() + ); + } + + #[test] + fn path_placement_checks_exact_slot_and_literal_template() { + let resolver = placement_resolver(CredentialPlacement::PathTemplate( + "/bot{credential}/events".into(), + )); + let token = "openshell:resolve:env:v1_TOKEN"; + assert_eq!( + rewrite_target_for_eval(&format!("/bot{token}/events"), &resolver) + .unwrap() + .resolved, + "/botsecret-value/events" + ); + assert!( + rewrite_target_for_eval(&format!("/bot{token}/echo"), &resolver) + .unwrap_err() + .is_placement_mismatch() + ); + assert!( + rewrite_target_for_eval(&format!("/echo/{token}/events"), &resolver) + .unwrap_err() + .is_placement_mismatch() + ); + assert!( + rewrite_target_for_eval( + &format!("/bot{}/events", fully_percent_encoded(token)), + &resolver + ) + .is_ok() + ); + let alias_resolver = placement_resolver(CredentialPlacement::PathTemplate( + "/{credential}/events".into(), + )); + assert!( + rewrite_target_for_eval( + "/vendor-OPENSHELL-RESOLVE-ENV-v1_TOKEN/events", + &alias_resolver + ) + .is_ok() + ); + } + fn fully_percent_encoded(marker: &str) -> String { const HEX: &[u8; 16] = b"0123456789ABCDEF"; let mut encoded = String::with_capacity(marker.len() * 3); diff --git a/crates/openshell-server/README.md b/crates/openshell-server/README.md index 58f735c35c..8f61d62116 100644 --- a/crates/openshell-server/README.md +++ b/crates/openshell-server/README.md @@ -18,6 +18,21 @@ Retries reuse that generation and session, including after gateway restart. Credential renewal retains both; credentials for an older launch cannot authorize the new one. +## Static credential delivery + +Existing polling and configuration-push environments keep their original wire +shape, delivery behavior, identities, and stable-handle derivation. The gateway +adds placement-only `ProviderProfileCredential` entries to the existing runtime +credential map under `static-env:`, for actually delivered static keys. +Entries are matched to their owning provider binding and contain no token grant, +refresh configuration, or credential values. The supervisor infers enforcement +from the auth metadata; the gateway does not enforce placement. + +Older supervisors ignore entries without token grants. An older gateway omits +these entries, so newer supervisors retain legacy behavior. Metadata travels in +the same captured environment snapshot; no separate service, capability gate, +or additional lookup is needed. + ## Operator credential export `GetProviderCredentials` exports selected profile-declared runtime environment diff --git a/crates/openshell-server/src/grpc/policy.rs b/crates/openshell-server/src/grpc/policy.rs index 374a3b5759..716e63f17a 100644 --- a/crates/openshell-server/src/grpc/policy.rs +++ b/crates/openshell-server/src/grpc/policy.rs @@ -4026,6 +4026,18 @@ pub async fn build_provider_environment_snapshot_from_inputs( .clear(); } + // Add request metadata only after token-grant owner composition and all + // static value filtering, so it describes this exact atomic snapshot. + let static_request_metadata = super::provider::static_credential_request_metadata( + &sandbox_id, + provider_profile_catalog, + provider_records, + &provider_environment, + )?; + provider_environment + .dynamic_credentials + .extend(static_request_metadata); + info!( sandbox_id = %sandbox_id, provider_count = provider_names.len(), @@ -12184,13 +12196,27 @@ mod tests { environment.provider_env_revision, second.provider_env_revision ); - assert_eq!(environment.dynamic_credentials.len(), 2); - assert!(environment.dynamic_credentials.values().all(|credential| { - credential - .token_grant - .as_ref() - .is_some_and(|grant| grant.token_endpoint.contains("revision-b")) - })); + let grants = environment + .dynamic_credentials + .values() + .filter_map(|credential| credential.token_grant.as_ref()) + .collect::>(); + assert_eq!(grants.len(), 2); + assert!( + grants + .iter() + .all(|grant| grant.token_endpoint.contains("revision-b")) + ); + assert!( + environment + .dynamic_credentials + .iter() + .filter(|(key, _)| key.starts_with( + openshell_core::provider_credentials::STATIC_CREDENTIAL_METADATA_PREFIX + )) + .all(|(_, credential)| credential.token_grant.is_none() + && credential.token_grant_owners.is_empty()) + ); handle_submit_policy_analysis( &state, diff --git a/crates/openshell-server/src/grpc/provider.rs b/crates/openshell-server/src/grpc/provider.rs index b77b9ed45b..731d3d88b0 100644 --- a/crates/openshell-server/src/grpc/provider.rs +++ b/crates/openshell-server/src/grpc/provider.rs @@ -1532,6 +1532,104 @@ pub(super) async fn resolve_provider_environment_from_records_with_policy_bindin }) } +/// Copy authored request metadata for static values that survived snapshot +/// filtering. The reserved namespace keeps it separate from endpoint token grants; +/// older supervisors ignore entries without `token_grant` before parsing selectors. +pub(super) fn static_credential_request_metadata( + sandbox_id: &str, + catalog: &EffectiveProviderProfileCatalog, + records: &[ProviderEnvironmentRecord], + environment: &ProviderEnvironment, +) -> Result, Status> { + use openshell_core::provider_credentials::STATIC_CREDENTIAL_METADATA_PREFIX; + let mut metadata = HashMap::new(); + for record in records { + let provider = &record.provider; + let Some(profile) = get_provider_type_profile_for_scope( + catalog, + &provider.r#type, + &provider.profile_workspace, + ) else { + continue; + }; + let refresh_epochs = refresh_authorization_epochs_by_key(record)?; + for credential in &profile.credentials { + if !has_usable_static_request_metadata( + &credential.auth_style, + &credential.header_name, + &credential.query_param, + &credential.path_template, + ) { + continue; + } + for key in credential.accepted_stored_keys() { + if !environment.environment.contains_key(key) + || !environment.static_credential_keys.contains(key) + { + continue; + } + let Some(baseline) = environment.static_credential_bindings.get(key) else { + continue; + }; + // A later optional declaration with the same environment key + // must not override metadata belonging to the actual owner. + let expected = static_credential_binding( + Some(sandbox_id), + record, + key, + &baseline.endpoints, + refresh_epochs.get(key).map(String::as_str), + credential.stable_placeholder, + ); + if expected != *baseline { + continue; + } + metadata.insert( + format!("{STATIC_CREDENTIAL_METADATA_PREFIX}{key}"), + ProviderProfileCredential { + name: credential.name.clone(), + env_vars: vec![key.to_string()], + auth_style: credential.auth_style.clone(), + header_name: credential.header_name.clone(), + query_param: credential.query_param.clone(), + path_template: credential.path_template.clone(), + ..Default::default() + }, + ); + } + } + } + Ok(metadata) +} + +const CREDENTIAL_PATH_SLOT: &str = "{credential}"; + +fn has_usable_static_request_metadata( + auth_style: &str, + header_name: &str, + query_param: &str, + path_template: &str, +) -> bool { + match auth_style.trim().to_ascii_lowercase().as_str() { + "basic" => true, + "header" | "bearer" => header_name.trim().parse::().is_ok(), + "query" => { + !query_param.trim().is_empty() + && !query_param + .chars() + .any(|c| c.is_control() || "&=#?".contains(c)) + } + "path" => { + path_template.starts_with('/') + && path_template.matches(CREDENTIAL_PATH_SLOT).count() == 1 + && !path_template + .chars() + .any(|c| c.is_control() || "?#".contains(c)) + } + _ => false, + } +} + fn refresh_authorization_epochs_by_key( record: &ProviderEnvironmentRecord, ) -> Result, Status> { @@ -12164,6 +12262,97 @@ mod tests { ); } + #[tokio::test] + async fn static_request_metadata_is_minimal_owner_bound_and_tracks_delivered_values() { + use openshell_core::provider_credentials::STATIC_CREDENTIAL_METADATA_PREFIX; + let store = test_store().await; + let mut owned = custom_profile("metadata-owner"); + owned.credentials = vec![static_credential("token", "TOKEN", true)]; + owned.endpoints = vec![NetworkEndpoint { + host: "api.example.com".into(), + port: 443, + protocol: "rest".into(), + access: openshell_core::proto::NetworkAccessPreset::Full as i32, + ..Default::default() + }]; + let mut optional = owned.clone(); + optional.id = "metadata-optional".into(); + optional.credentials[0].required = false; + optional.credentials[0].header_name = "x-wrong-owner".into(); + let sources = ProviderProfileSources::from_test_profiles(vec![owned, optional]); + let catalog = sources.snapshot_catalog(&store, "default").await.unwrap(); + let record = |name: &str, id: &str, profile: &str| ProviderEnvironmentRecord { + name: name.into(), + object_id: id.into(), + resource_version: 1, + provider: Provider { + r#type: profile.into(), + profile_workspace: "default".into(), + ..Default::default() + }, + refresh_states: Vec::new(), + }; + let mut records = [ + record("owner", "owner-id", "metadata-owner"), + record("optional", "optional-id", "metadata-optional"), + ]; + records[0] + .provider + .credentials + .insert("TOKEN".into(), "synthetic-sensitive-token".into()); + let endpoints = vec![StaticCredentialEndpointBinding { + host: "api.example.com".into(), + port: 443, + path: "/**".into(), + }]; + let baseline = static_credential_binding( + Some("sandbox"), + &records[0], + "TOKEN", + &endpoints, + None, + false, + ); + let mut environment = ProviderEnvironment { + environment: HashMap::from([("TOKEN".into(), "synthetic-sensitive-token".into())]), + static_credential_keys: HashSet::from(["TOKEN".into()]), + static_credential_bindings: HashMap::from([("TOKEN".into(), baseline.clone())]), + ..Default::default() + }; + let metadata = + static_credential_request_metadata("sandbox", &catalog, &records, &environment) + .unwrap(); + let key = format!("{STATIC_CREDENTIAL_METADATA_PREFIX}TOKEN"); + assert_eq!(metadata.len(), 1); + assert_eq!( + metadata[&key], + ProviderProfileCredential { + name: "token".into(), + env_vars: vec!["TOKEN".into()], + auth_style: "bearer".into(), + header_name: "authorization".into(), + ..Default::default() + } + ); + assert_eq!( + environment.static_credential_bindings["TOKEN"], baseline, + "legacy handles and identities must not change" + ); + let encoded = metadata[&key].encode_to_vec(); + assert!( + !encoded + .windows(b"synthetic-sensitive-token".len()) + .any(|window| window == b"synthetic-sensitive-token") + ); + environment.environment.remove("TOKEN"); + assert!( + static_credential_request_metadata("sandbox", &catalog, &records, &environment) + .unwrap() + .is_empty(), + "expired or withheld material must not leave request metadata behind" + ); + } + #[test] fn external_stable_placeholder_binding_preserves_only_value_changes() { let mut record = ProviderEnvironmentRecord { diff --git a/crates/openshell-supervisor-network/README.md b/crates/openshell-supervisor-network/README.md index 7ce403a15e..81f59cf306 100644 --- a/crates/openshell-supervisor-network/README.md +++ b/crates/openshell-supervisor-network/README.md @@ -26,3 +26,13 @@ HTTP client; a token acquired for one sandbox is never served to another. Without a Workload API socket, token grants fail closed. A process hosting several instances serves `/v1/denials` from a per-instance `SandboxInstanceDenialLog` instead of the shared shorthand log files. + +## Static credential placement + +When placement metadata is available, substitution checks the declared request +location before writing upstream. HTTP placement violations return a safe 403 response; +WebSocket violations close the connection with policy denial. Both paths emit a +domain event and a detection finding without credential values or placeholders. +Legacy credentials preserve endpoint-scoped rewriting. Explicit body and WebSocket +rewrite flags authorize those locations for ordinary static credentials. SigV4 +signing retains the endpoint-scoped resolver's existing trusted access. diff --git a/crates/openshell-supervisor-network/src/l7/mod.rs b/crates/openshell-supervisor-network/src/l7/mod.rs index 3f26114998..37849b9c0c 100644 --- a/crates/openshell-supervisor-network/src/l7/mod.rs +++ b/crates/openshell-supervisor-network/src/l7/mod.rs @@ -66,6 +66,37 @@ pub(crate) fn build_credential_endpoint_mismatch_finding( .build() } +pub(crate) fn build_credential_placement_mismatch_finding( + policy_name: &str, + host: &str, + protocol: Option<&str>, +) -> openshell_ocsf::OcsfEvent { + use openshell_ocsf::{ + ActionId, ActivityId, DetectionFindingBuilder, DispositionId, FindingInfo, SeverityId, + }; + let mut evidence = vec![ + ("policy", policy_name), + ("host", host), + ("disposition", "denied"), + ]; + if let Some(protocol) = protocol { + evidence.push(("protocol", protocol)); + } + DetectionFindingBuilder::new(openshell_ocsf::ctx::ctx()) + .activity(ActivityId::Open) + .action(ActionId::Denied) + .disposition(DispositionId::Blocked) + .severity(SeverityId::High) + .is_alert(true) + .finding_info(FindingInfo::new( + "openshell.provider_credential.placement_mismatch", + "Provider credential used at an unauthorized request location", + )) + .evidence_pairs(&evidence) + .message("Provider credential placement mismatch; request denied") + .build() +} + /// TLS handling mode for proxy connections. #[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] pub enum TlsMode { diff --git a/crates/openshell-supervisor-network/src/l7/relay.rs b/crates/openshell-supervisor-network/src/l7/relay.rs index f3e9e34741..56ec88f0f6 100644 --- a/crates/openshell-supervisor-network/src/l7/relay.rs +++ b/crates/openshell-supervisor-network/src/l7/relay.rs @@ -593,12 +593,15 @@ where W: AsyncWrite + Unpin, { let endpoint_mismatch = error.is_endpoint_mismatch(); - let status = if endpoint_mismatch { + let placement_mismatch = error.is_placement_mismatch(); + let status = if endpoint_mismatch || placement_mismatch { "403 Forbidden" } else { "500 Internal Server Error" }; - let body = if endpoint_mismatch { + let body = if placement_mismatch { + r#"{"error":"credential_placement_mismatch","message":"Credential is not authorized for this request location"}"# + } else if endpoint_mismatch { r#"{"error":"credential_endpoint_mismatch","message":"Credential is not authorized for this request endpoint"}"# } else { r#"{"error":"credential_unavailable","message":"Credential placeholder could not be resolved"}"# @@ -613,11 +616,34 @@ where .into_diagnostic()?; client.flush().await.into_diagnostic()?; - ocsf_emit!(build_credential_resolution_event( - ctx, - method, - endpoint_mismatch - )); + if placement_mismatch { + ocsf_emit!( + HttpActivityBuilder::new(openshell_ocsf::ctx::ctx()) + .activity(ActivityId::for_http_method(method)) + .http_request(method_only_request(method)) + .http_response(openshell_ocsf::HttpResponse { code: 403 }) + .action(ActionId::Denied) + .disposition(DispositionId::Blocked) + .severity(SeverityId::High) + .status(StatusId::Failure) + .dst_endpoint(Endpoint::from_domain(&ctx.host, ctx.port)) + .firewall_rule(&ctx.policy_name, "credential-binding") + .message("Credential use denied: unauthorized request location") + .status_detail("credential_placement_mismatch") + .build() + ); + ocsf_emit!(crate::l7::build_credential_placement_mismatch_finding( + &ctx.policy_name, + &ctx.host, + None, + )); + } else { + ocsf_emit!(build_credential_resolution_event( + ctx, + method, + endpoint_mismatch + )); + } if endpoint_mismatch { ocsf_emit!(build_credential_endpoint_mismatch_finding(ctx)); @@ -1884,9 +1910,13 @@ where assembly_budget, resolver, generation_guard: options.generation_guard, - provider_credentials: options - .ctx - .and_then(|ctx| ctx.provider_credentials.as_ref()), + provider_credentials: if options.websocket.credential_rewrite { + options + .ctx + .and_then(|ctx| ctx.provider_credentials.as_ref()) + } else { + None + }, target: &options.target, inspector, compression, @@ -4113,6 +4143,9 @@ mod tests { scoped .secret_resolver .expect("endpoint resolver") + .with_request_location(secrets::RequestCredentialLocation::Header( + "Authorization".into() + )) .resolve_placeholder("openshell:resolve:env:v42_API_TOKEN"), Some("secret") ); @@ -4296,6 +4329,463 @@ mod tests { assert_eq!(finding["disposition"], "Blocked"); } + fn placement_test_state() -> ProviderCredentialState { + ProviderCredentialState::from_bound_environment( + 1, + TestHashMap::from([("API_TOKEN".into(), "synthetic-private-token".into())]), + TestHashMap::new(), + crate::test_support::header_placement_metadata(&["API_TOKEN"]), + TestHashMap::from([("API_TOKEN".into(), endpoint_binding("provider:API_TOKEN"))]), + Vec::new(), + ) + .expect("bound placement fixture") + } + + #[tokio::test] + async fn mixed_legacy_and_strict_credentials_preserve_opt_in_body_and_query_rewriting() { + let legacy_binding = endpoint_binding("provider:LEGACY_TOKEN"); + let state = ProviderCredentialState::from_bound_environment( + 1, + TestHashMap::from([ + ("STRICT_TOKEN".into(), "strict-private-token".into()), + ("LEGACY_TOKEN".into(), "legacy-private-token".into()), + ]), + TestHashMap::new(), + crate::test_support::header_placement_metadata(&["STRICT_TOKEN"]), + TestHashMap::from([ + ( + "STRICT_TOKEN".into(), + endpoint_binding("provider:STRICT_TOKEN"), + ), + ("LEGACY_TOKEN".into(), legacy_binding), + ]), + Vec::new(), + ) + .expect("mixed placement fixture"); + let snapshot = state.snapshot(); + let strict = &snapshot.child_env["STRICT_TOKEN"]; + let legacy = &snapshot.child_env["LEGACY_TOKEN"]; + let resolver = state + .resolver_for_endpoint("allowed.example.test", 443, "/allowed/search") + .unwrap(); + let body = format!("legacy={legacy}&strict={strict}"); + let target = format!("/allowed/search?probe={legacy}"); + let raw = format!( + "POST {target} HTTP/1.1\r\nHost: allowed.example.test\r\nAuthorization: token {strict}\r\nX-Legacy: {legacy}\r\nContent-Type: application/x-www-form-urlencoded\r\nContent-Length: {}\r\n\r\n{body}", + body.len() + ); + let request = crate::l7::rest::request_from_buffered_http( + "POST", + "/allowed/search", + &target, + raw.into_bytes(), + ) + .unwrap(); + let (mut _app, mut client) = tokio::io::duplex(8192); + let (mut upstream, mut service) = tokio::io::duplex(8192); + let service_task = tokio::spawn(async move { + let mut bytes = Vec::new(); + loop { + let mut byte = [0]; + service.read_exact(&mut byte).await.unwrap(); + bytes.push(byte[0]); + if bytes.ends_with(b"\r\n\r\n") { + break; + } + } + let headers = String::from_utf8(bytes).unwrap(); + let length = headers + .lines() + .find_map(|line| { + let (name, value) = line.split_once(':')?; + name.eq_ignore_ascii_case("Content-Length") + .then(|| value.trim().parse::().unwrap()) + }) + .unwrap(); + let mut body = vec![0; length]; + service.read_exact(&mut body).await.unwrap(); + service + .write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n") + .await + .unwrap(); + (headers, String::from_utf8(body).unwrap()) + }); + crate::l7::rest::relay_http_request_with_options_guarded( + &request, + &mut client, + &mut upstream, + crate::l7::rest::RelayRequestOptions { + resolver: Some(&resolver), + request_body_credential_rewrite: true, + ..Default::default() + }, + ) + .await + .unwrap(); + let (headers, body) = service_task.await.unwrap(); + assert!(headers.contains("/allowed/search?probe=legacy-private-token")); + assert!(headers.contains("Authorization: token strict-private-token")); + assert!(headers.contains("X-Legacy: legacy-private-token")); + assert_eq!( + body, + "legacy=legacy-private-token&strict=strict-private-token" + ); + assert!( + resolver + .rewrite_header_value_for_name("X-Legacy", strict) + .unwrap_err() + .is_placement_mismatch() + ); + } + + #[tokio::test] + async fn metadata_credentials_do_not_enable_disabled_body_rewriting() { + let state = placement_test_state(); + let resolver = state + .resolver_for_endpoint("allowed.example.test", 443, "/allowed/search") + .unwrap(); + let token = state.snapshot().child_env["API_TOKEN"].clone(); + let body = format!("token={token}"); + let raw = format!( + "POST /allowed/search HTTP/1.1\r\nHost: allowed.example.test\r\nContent-Type: application/x-www-form-urlencoded\r\nContent-Length: {}\r\n\r\n{body}", + body.len() + ); + let request = crate::l7::rest::request_from_buffered_http( + "POST", + "/allowed/search", + "/allowed/search", + raw.into_bytes(), + ) + .unwrap(); + let (mut _app, mut client) = tokio::io::duplex(8192); + let (mut upstream, mut service) = tokio::io::duplex(8192); + let error = crate::l7::rest::relay_http_request_with_options_guarded( + &request, + &mut client, + &mut upstream, + crate::l7::rest::RelayRequestOptions { + resolver: Some(&resolver), + request_body_credential_rewrite: false, + deny_uninspected_credentials: true, + ..Default::default() + }, + ) + .await + .expect_err("placement metadata must not enable body rewriting"); + assert!( + error + .downcast_ref::() + .is_some(), + "{error}" + ); + drop(upstream); + let mut sent = String::new(); + service.read_to_string(&mut sent).await.unwrap(); + assert!(!sent.contains("synthetic-private-token")); + } + + #[tokio::test] + async fn metadata_credentials_do_not_enable_disabled_websocket_rewriting() { + let state = placement_test_state(); + let token = state.snapshot().child_env["API_TOKEN"].clone(); + let ctx = L7EvalContext { + host: "allowed.example.test".into(), + port: 443, + provider_credentials: Some(state.clone()), + ..Default::default() + }; + let (mut app, mut client) = tokio::io::duplex(8192); + let (mut upstream, mut service) = tokio::io::duplex(8192); + let relay = tokio::spawn(async move { + handle_upgrade( + &mut client, + &mut upstream, + Vec::new(), + "allowed.example.test", + 443, + UpgradeRelayOptions { + websocket_request: true, + websocket: WebSocketUpgradeBehavior { + credential_rewrite: false, + deny_uninspected_credentials: true, + ..Default::default() + }, + secret_resolver: state.resolver(), + ctx: Some(&ctx), + target: "/allowed/socket".into(), + ..Default::default() + }, + ) + .await + }); + app.write_all(&masked_text_frame(token.as_bytes())) + .await + .unwrap(); + let error = tokio::time::timeout(std::time::Duration::from_secs(1), relay) + .await + .unwrap() + .unwrap() + .expect_err("disabled rewriting must deny credential frames"); + assert!(error.to_string().contains("rewrite is disabled"), "{error}"); + let mut sent = Vec::new(); + service.read_to_end(&mut sent).await.unwrap(); + assert!(sent.is_empty()); + } + + #[tokio::test] + async fn header_credential_query_reflection_is_denied_before_upstream_write() { + let state = placement_test_state(); + let resolver = state + .resolver_for_endpoint("allowed.example.test", 443, "/allowed/search") + .unwrap(); + let token = state.snapshot().child_env["API_TOKEN"].clone(); + // A pagination service copies extra query parameters into its Link header. + // No request bytes may reach it when the parameter contains a header credential. + for (target, extra_headers, body, rewrite_body) in [ + ( + format!("/allowed/search?q=rust&per_page=1&probe={token}"), + String::new(), + String::new(), + false, + ), + ( + "/allowed/search".into(), + format!("X-Reflect: {token}\r\n"), + String::new(), + false, + ), + ] { + let raw = format!( + "GET {target} HTTP/1.1\r\nHost: allowed.example.test\r\n{extra_headers}Content-Type: application/x-www-form-urlencoded\r\nContent-Length: {}\r\n\r\n{body}", + body.len() + ); + let request = crate::l7::rest::request_from_buffered_http( + "GET", + "/allowed/search", + &target, + raw.into_bytes(), + ) + .unwrap(); + let (mut app, mut client) = tokio::io::duplex(8192); + let (mut upstream, mut service) = tokio::io::duplex(8192); + let ctx = L7EvalContext { + host: "allowed.example.test".into(), + port: 443, + ..Default::default() + }; + let outcome = relay_http_request_with_credential_rejection( + &request, + &mut client, + &mut upstream, + crate::l7::rest::RelayRequestOptions { + resolver: Some(&resolver), + request_body_credential_rewrite: rewrite_body, + ..Default::default() + }, + &ctx, + None, + ) + .await + .unwrap(); + assert!(outcome.is_none()); + drop(client); + drop(upstream); + let mut response = String::new(); + app.read_to_string(&mut response).await.unwrap(); + let mut sent = Vec::new(); + service.read_to_end(&mut sent).await.unwrap(); + assert!( + sent.is_empty(), + "placement denial must precede all upstream writes" + ); + assert!(response.contains("403 Forbidden"), "{response}"); + assert!( + response.contains("credential_placement_mismatch"), + "{response}" + ); + assert!(!response.contains("synthetic-private-token")); + assert!(!response.contains(&token)); + } + let finding = crate::l7::build_credential_placement_mismatch_finding( + "test", + "allowed.example.test", + None, + ) + .to_json() + .unwrap(); + assert_eq!( + finding["finding_info"]["uid"], + "openshell.provider_credential.placement_mismatch" + ); + } + + #[tokio::test] + async fn permitted_auth_formats_preserve_pagination_without_disclosing_credentials() { + use base64::Engine as _; + let state = placement_test_state(); + let resolver = state + .resolver_for_endpoint("allowed.example.test", 443, "/allowed/search") + .unwrap(); + let token = state.snapshot().child_env["API_TOKEN"].clone(); + for (authorization, expected) in [ + ( + format!("Bearer {token}"), + "Bearer synthetic-private-token".to_owned(), + ), + ( + format!("token {token}"), + "token synthetic-private-token".to_owned(), + ), + ( + format!( + "Basic {}", + base64::engine::general_purpose::STANDARD.encode(format!("user:{token}")) + ), + format!( + "Basic {}", + base64::engine::general_purpose::STANDARD + .encode("user:synthetic-private-token") + ), + ), + ] { + let target = "/allowed/search?q=rust&per_page=1"; + let raw = format!( + "GET {target} HTTP/1.1\r\nHost: allowed.example.test\r\nAuthorization: {authorization}\r\n\r\n" + ); + let request = crate::l7::rest::request_from_buffered_http( + "GET", + "/allowed/search", + target, + raw.into_bytes(), + ) + .unwrap(); + let (mut app, mut client) = tokio::io::duplex(8192); + let (mut upstream, mut service) = tokio::io::duplex(8192); + let service_task = tokio::spawn(async move { + let mut bytes = Vec::new(); + loop { + let mut byte = [0]; + service.read_exact(&mut byte).await.unwrap(); + bytes.push(byte[0]); + if bytes.ends_with(b"\r\n\r\n") { + break; + } + } + let received = String::from_utf8(bytes).unwrap(); + let target = received + .lines() + .next() + .unwrap() + .split_whitespace() + .nth(1) + .unwrap(); + let response = format!( + "HTTP/1.1 200 OK\r\nLink: ; rel=\"next\"\r\nContent-Length: 0\r\n\r\n" + ); + service.write_all(response.as_bytes()).await.unwrap(); + received + }); + crate::l7::rest::relay_http_request_with_options_guarded( + &request, + &mut client, + &mut upstream, + crate::l7::rest::RelayRequestOptions { + resolver: Some(&resolver), + ..Default::default() + }, + ) + .await + .unwrap(); + drop(client); + let mut response = String::new(); + app.read_to_string(&mut response).await.unwrap(); + let received = service_task.await.unwrap(); + assert!( + received.contains(&format!("Authorization: {expected}")), + "{received}" + ); + assert!(response.contains("per_page=1&page=2"), "{response}"); + assert!(!response.contains("synthetic-private-token")); + } + } + + #[tokio::test] + async fn trusted_sigv4_signing_preserves_metadata_free_credentials() { + let values = TestHashMap::from([ + ("AWS_ACCESS_KEY_ID".into(), "synthetic-access-key".into()), + ( + "AWS_SECRET_ACCESS_KEY".into(), + "synthetic-signing-secret".into(), + ), + ("AWS_SESSION_TOKEN".into(), "synthetic-session-token".into()), + ]); + let bindings = values + .keys() + .map(|key: &String| { + let binding = endpoint_binding(&format!("aws:{key}")); + (key.clone(), binding) + }) + .collect(); + let state = ProviderCredentialState::from_bound_environment( + 1, + values, + TestHashMap::new(), + TestHashMap::new(), + bindings, + Vec::new(), + ) + .unwrap(); + let resolver = state + .resolver_for_endpoint("allowed.example.test", 443, "/allowed/api") + .unwrap(); + let request = crate::l7::rest::request_from_buffered_http( + "GET", + "/allowed/api", + "/allowed/api", + b"GET /allowed/api HTTP/1.1\r\nHost: allowed.example.test\r\n\r\n".to_vec(), + ) + .unwrap(); + let (mut _app, mut client) = tokio::io::duplex(8192); + let (mut upstream, mut service) = tokio::io::duplex(8192); + let service_task = tokio::spawn(async move { + let mut bytes = Vec::new(); + loop { + let mut byte = [0]; + service.read_exact(&mut byte).await.unwrap(); + bytes.push(byte[0]); + if bytes.ends_with(b"\r\n\r\n") { + break; + } + } + service + .write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n") + .await + .unwrap(); + String::from_utf8(bytes).unwrap() + }); + crate::l7::rest::relay_http_request_with_options_guarded( + &request, + &mut client, + &mut upstream, + crate::l7::rest::RelayRequestOptions { + resolver: Some(&resolver), + credential_signing: crate::l7::CredentialSigning::SigV4NoBody, + signing_service: "execute-api", + signing_region: "us-west-2", + host: "allowed.example.test", + port: 443, + ..Default::default() + }, + ) + .await + .unwrap(); + let sent = service_task.await.unwrap(); + assert!(sent.contains("AWS4-HMAC-SHA256"), "{sent}"); + assert!(sent.contains("synthetic-access-key")); + assert!(sent.contains("synthetic-session-token")); + assert!(!sent.contains("synthetic-signing-secret")); + } + #[tokio::test] async fn body_only_endpoint_mismatch_returns_typed_403() { let (_state, resolver) = endpoint_mismatch_resolver(TestHashMap::from([( diff --git a/crates/openshell-supervisor-network/src/l7/websocket.rs b/crates/openshell-supervisor-network/src/l7/websocket.rs index f2ea281a81..e0dad1aba6 100644 --- a/crates/openshell-supervisor-network/src/l7/websocket.rs +++ b/crates/openshell-supervisor-network/src/l7/websocket.rs @@ -39,6 +39,7 @@ const OPCODE_BINARY: u8 = 0x2; const OPCODE_CLOSE: u8 = 0x8; const OPCODE_PING: u8 = 0x9; const OPCODE_PONG: u8 = 0xA; +const CREDENTIAL_PLACEMENT_MISMATCH: &str = "websocket credential placement mismatch"; const CREDENTIAL_ENDPOINT_MISMATCH: &str = "websocket credential endpoint mismatch"; #[derive(Clone, Debug)] @@ -582,7 +583,27 @@ where } if let Err(termination) = &result { let error = termination.error.to_string(); - if error.contains(CREDENTIAL_ENDPOINT_MISMATCH) { + if error.contains(CREDENTIAL_PLACEMENT_MISMATCH) { + ocsf_emit!( + NetworkActivityBuilder::new(openshell_ocsf::ctx::ctx()) + .activity(ActivityId::Fail) + .action(ActionId::Denied) + .disposition(DispositionId::Blocked) + .severity(SeverityId::High) + .status(StatusId::Failure) + .dst_endpoint(Endpoint::from_domain(host, port)) + .firewall_rule(options.policy_name, "credential-binding") + .message("WebSocket credential use denied: unauthorized request location") + .status_detail("credential_placement_mismatch") + .build() + ); + ocsf_emit!(crate::l7::build_credential_placement_mismatch_finding( + options.policy_name, + host, + Some("websocket"), + )); + let _ = write_policy_violation_close(&mut client_write).await; + } else if error.contains(CREDENTIAL_ENDPOINT_MISMATCH) { emit_credential_endpoint_mismatch(host, port, options.policy_name); let _ = write_policy_violation_close(&mut client_write).await; } else if error.contains("credential") { @@ -609,7 +630,7 @@ where } async fn write_policy_violation_close(writer: &mut W) -> Result<()> { - let reason = b"credential endpoint mismatch"; + let reason = b"credential use denied"; let mut frame = Vec::with_capacity(reason.len() + 4); frame.push(0x80 | OPCODE_CLOSE); frame.push(u8::try_from(reason.len() + 2).expect("close reason fits one-byte length")); @@ -1288,7 +1309,12 @@ where resolver .rewrite_websocket_text_placeholders(&mut text) .map_err(|error| { - if error.is_endpoint_mismatch() { + if error.is_placement_mismatch() { + terminate( + WebSocketTerminationCause::PolicyDenial, + miette!(CREDENTIAL_PLACEMENT_MISMATCH), + ) + } else if error.is_endpoint_mismatch() { terminate( WebSocketTerminationCause::PolicyDenial, miette!(CREDENTIAL_ENDPOINT_MISMATCH), @@ -2713,7 +2739,7 @@ network_policies: 1, HashMap::from([("DISCORD_BOT_TOKEN".to_string(), "real-token".to_string())]), HashMap::new(), - HashMap::new(), + crate::test_support::header_placement_metadata(&["DISCORD_BOT_TOKEN"]), HashMap::from([( "DISCORD_BOT_TOKEN".to_string(), StaticCredentialBinding { @@ -2815,7 +2841,7 @@ network_policies: } #[tokio::test] - async fn guarded_websocket_uses_path_scoped_live_resolver() { + async fn guarded_websocket_rewrites_header_credentials_with_policy_opt_in() { let state = bound_websocket_provider_state(); let placeholder = b"openshell:resolve:env:v1_DISCORD_BOT_TOKEN"; let input = masked_frame(true, OPCODE_TEXT, placeholder); @@ -2847,7 +2873,68 @@ network_policies: .await; assert!( result.is_ok(), - "path-scoped live resolver must satisfy the credential guard: {result:?}" + "explicit WebSocket rewriting must permit header credentials: {result:?}" + ); + + drop(relay_write); + let mut output = Vec::new(); + upstream_read.read_to_end(&mut output).await.unwrap(); + assert_eq!(decode_masked_text_frame(&output), "real-token"); + } + + #[tokio::test] + async fn legacy_websocket_credential_rewrite_remains_available() { + let state = ProviderCredentialState::from_bound_environment( + 1, + HashMap::from([("DISCORD_BOT_TOKEN".to_string(), "real-token".to_string())]), + HashMap::new(), + HashMap::new(), + HashMap::from([( + "DISCORD_BOT_TOKEN".to_string(), + StaticCredentialBinding { + endpoints: vec![StaticCredentialEndpointBinding { + host: "gateway.example.test".into(), + port: 443, + path: "/socket".into(), + }], + credential_identity: "provider:DISCORD_BOT_TOKEN".into(), + workload_credential_handle: String::new(), + }, + )]), + Vec::new(), + ) + .expect("legacy websocket fixture"); + let placeholder = b"openshell:resolve:env:v1_DISCORD_BOT_TOKEN"; + let input = masked_frame(true, OPCODE_TEXT, placeholder); + let (mut client_write, mut relay_read) = tokio::io::duplex(4096); + let (mut relay_write, mut upstream_read) = tokio::io::duplex(4096); + client_write.write_all(&input).await.unwrap(); + drop(client_write); + + let mut options = RelayOptions { + policy_name: "test-policy", + assembly_budget: WebSocketAssemblyBudget::default(), + resolver: None, + generation_guard: None, + provider_credentials: Some(&state), + target: "/socket", + inspector: None, + compression: WebSocketCompression::None, + middleware_session: None, + middleware_context: None, + deny_uninspected_credentials: true, + }; + let result = relay_client_to_server( + &mut relay_read, + &mut relay_write, + "gateway.example.test", + 443, + &mut options, + ) + .await; + assert!( + result.is_ok(), + "legacy placement must preserve WebSocket rewriting: {result:?}" ); drop(relay_write); @@ -2914,7 +3001,7 @@ network_policies: result .as_ref() .is_err_and(|error| error.error.to_string().contains("generation changed")), - "revoked credential generation must fail before the frame write: {result:?}" + "revoked generation must fail before the frame write: {result:?}" ); drop(relay_write); diff --git a/crates/openshell-supervisor-network/src/proxy.rs b/crates/openshell-supervisor-network/src/proxy.rs index bd93c78c57..f73831c79a 100644 --- a/crates/openshell-supervisor-network/src/proxy.rs +++ b/crates/openshell-supervisor-network/src/proxy.rs @@ -6490,7 +6490,10 @@ async fn handle_forward_proxy( .end(openshell_core::proto::MiddlewareSessionEndReason::Cancellation) .await; } - if e.is_endpoint_mismatch() { + if e.is_placement_mismatch() { + crate::l7::relay::reject_credential_resolution(client, &l7_ctx, method, &e).await?; + client.shutdown().await.into_diagnostic()?; + } else if e.is_endpoint_mismatch() { if let Some(observer) = endpoint_observer.as_ref() { observer.observe_credential_failure(true); } @@ -13005,6 +13008,9 @@ network_policies: credentials .resolver .expect("endpoint resolver") + .with_request_location(secrets::RequestCredentialLocation::Header( + "Authorization".into() + )) .resolve_placeholder("openshell:resolve:env:v42_API_TOKEN"), Some("secret") ); diff --git a/crates/openshell-supervisor-network/src/test_support.rs b/crates/openshell-supervisor-network/src/test_support.rs index d1b23b6f31..5c302b91de 100644 --- a/crates/openshell-supervisor-network/src/test_support.rs +++ b/crates/openshell-supervisor-network/src/test_support.rs @@ -1,7 +1,28 @@ // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -//! Executable identity and socket-owner subprocess fixtures. +//! Credential, executable identity, and socket-owner subprocess fixtures. + +pub fn header_placement_metadata( + keys: &[&str], +) -> std::collections::HashMap { + keys.iter() + .map(|key| { + ( + format!( + "{}{key}", + openshell_core::provider_credentials::STATIC_CREDENTIAL_METADATA_PREFIX + ), + openshell_core::proto::ProviderProfileCredential { + env_vars: vec![(*key).to_string()], + auth_style: "bearer".to_string(), + header_name: "Authorization".to_string(), + ..Default::default() + }, + ) + }) + .collect() +} #[cfg(target_os = "linux")] use std::net::TcpStream; diff --git a/docs/how-it-works/policies/network-rules.mdx b/docs/how-it-works/policies/network-rules.mdx index 811d920922..c2aec12e2a 100644 --- a/docs/how-it-works/policies/network-rules.mdx +++ b/docs/how-it-works/policies/network-rules.mdx @@ -440,8 +440,9 @@ frames or messages from the server. If the endpoint carries provider credentials, OpenShell closes the connection with close code `1008` when the client sends a binary frame, because it cannot check the frame for credentials. Set `allow_uninspected_credentials: true` to relay binary frames anyway. Set -`websocket_credential_rewrite: true` only when client text messages contain -OpenShell credential placeholders that must be resolved. +`websocket_credential_rewrite: true` when client text messages contain credential +placeholders that must be resolved. This explicitly authorizes WebSocket text +rewriting for ordinary static credentials, including credentials with HTTP placement metadata. When new network rules take effect, OpenShell closes open WebSocket connections, so the client must reconnect. The client might receive close code `1012`, or the diff --git a/docs/how-it-works/policies/schema.mdx b/docs/how-it-works/policies/schema.mdx index 693efb209c..40faf25379 100644 --- a/docs/how-it-works/policies/schema.mdx +++ b/docs/how-it-works/policies/schema.mdx @@ -166,8 +166,8 @@ the gateway host. | Field | Type | Default | Description | |---|---|---|---| | `credential_binding.provider` | string | None | Binds the static credentials of an attached provider whose profile defines no endpoints. Valid only in a sandbox policy. | -| `request_body_credential_rewrite` | bool | `false` | Replaces credential placeholders in REST request bodies. | -| `websocket_credential_rewrite` | bool | `false` | Replaces credential placeholders in client WebSocket text messages. | +| `request_body_credential_rewrite` | bool | `false` | Replaces ordinary static credential placeholders in REST bodies, including those with declared HTTP placement. | +| `websocket_credential_rewrite` | bool | `false` | Replaces ordinary static credential placeholders in client WebSocket text messages, including those with declared HTTP placement. | | `allow_uninspected_credentials` | bool | `false` | Allows a provider-credentialed endpoint to omit request inspection or use `tls: skip`. | | `credential_signing` | string | None | AWS request signing: `sigv4`, `sigv4:body`, or `sigv4:no_body`. | | `signing_service` | string | None | AWS service name, such as `bedrock` or `s3`. Required with `credential_signing`. | diff --git a/docs/how-it-works/providers/overview.mdx b/docs/how-it-works/providers/overview.mdx index 73f209738e..44e0959924 100644 --- a/docs/how-it-works/providers/overview.mdx +++ b/docs/how-it-works/providers/overview.mdx @@ -436,15 +436,18 @@ startup, OpenShell replaces each credential with an opaque placeholder token in the agent's environment. When the agent sends an HTTP request containing a placeholder, the proxy resolves it immediately before forwarding the request. -Static credential resolution has two independent authorization boundaries: +Static credential resolution has three independent authorization boundaries: 1. Network policy must allow the calling binary and request destination. 2. The credential binding must include the request host, port, and path. Profile endpoints provide the binding by default. An endpointless profile can use a sandbox policy endpoint that names the attached provider instance through `credential_binding.provider`. +3. When placement metadata is supported and available, the request location + must match the declared header, query parameter, or path-template slot, or an + explicitly enabled body/WebSocket rewrite policy. -Both checks must pass. A provider profile endpoint does not grant network access +All three checks must pass. A provider profile endpoint does not grant network access unless provider policy composition or the sandbox's own policy allows the request. A plain sandbox policy endpoint does not grant credential use unless the profile already covers it or the endpoint explicitly binds an endpointless @@ -461,21 +464,27 @@ for endpoint matching and migration guidance. ### Supported injection locations -The proxy resolves credential placeholders in the following parts of an HTTP request: +The proxy automatically enforces declared placement when the gateway and +supervisor support it. Moving a header credential into a query parameter, path, +or unrelated header is denied before forwarding. Credentials without usable +placement metadata and older components retain legacy endpoint-scoped rewriting. | Location | How the agent uses it | Example | |---|---|---| -| Header value | Agent reads `$API_KEY` from env and places it in a header. | `Authorization: Bearer ` | +| Declared header value | Agent reads `$API_KEY` from env and places it in the declared header. | `Authorization: Bearer ` | | Header value (Basic auth) | Agent base64-encodes `user:` in an `Authorization: Basic` header. The proxy decodes, resolves, and re-encodes. | `Authorization: Basic ` | -| Query parameter value | Agent places the placeholder in a URL query parameter. | `GET /api?key=` | -| URL path segment | Agent builds a URL with the placeholder in the path. Supports concatenated patterns. | `POST /bot/sendMessage` | -| Supported request body | An inspected REST endpoint opts in with `request_body_credential_rewrite: true`. | `{"api_key":""}` | -| WebSocket text message | A REST or WebSocket endpoint opts in with `websocket_credential_rewrite: true`. | `{"token":""}` | +| Declared query parameter value | Agent uses the profile's `query_param`. | `GET /api?key=` | +| Declared URL path slot | Agent uses the profile's `path_template`, including concatenated patterns. | `POST /bot/sendMessage` | | AWS SigV4 signing | An endpoint configures `credential_signing`, and the proxy signs with the endpoint-bound AWS credentials. | `credential_signing: sigv4` | The proxy does not rewrite cookies, response content, unsupported request bodies, or WebSocket binary frames. +Explicit endpoint policies can enable credential rewriting in request bodies +or WebSocket text messages. These flags do not authorize arbitrary query +parameters, path slots, or unrelated headers. See [Profiles](/how-it-works/providers/profiles#understand-static-credential-endpoint-binding) +for placement configuration and upgrade requirements. + ### Clients that parse JWT claims A trusted launcher can wrap an issued placeholder in a JWT-shaped alias when a diff --git a/docs/how-it-works/providers/profiles.mdx b/docs/how-it-works/providers/profiles.mdx index 1617e00afe..3fad35a9da 100644 --- a/docs/how-it-works/providers/profiles.mdx +++ b/docs/how-it-works/providers/profiles.mdx @@ -70,6 +70,7 @@ A request can use a static credential only when all of these checks pass: | The placeholder belongs to the current attached provider state. | Sandbox provider attachment and current provider record. | | The calling binary and destination are allowed. | Effective sandbox network policy. | | The host, port, and canonical request path match the credential binding. | Provider profile endpoints, or an explicit sandbox policy binding for an endpointless profile. | +| When supported placement metadata is available, the request location matches it or an explicit body/WebSocket rewrite policy. | The credential's `auth_style` and its header, query parameter, or path template. | | The HTTP method, path, or protocol operation is allowed when L7 inspection is configured. | Effective sandbox network policy. | | The credential has not expired. | Provider credential expiry metadata. | @@ -94,6 +95,33 @@ glob matching. OpenShell removes the query string and uses a canonical, secret-redacted path for this check, so a credential embedded in a request path does not need to be revealed before authorization. +The supervisor automatically enforces usable static credential auth metadata. +No additional profile flag is required. Header credentials resolve only in the +declared header, query credentials only in the declared parameter, and path +credentials only in the `{credential}` slot of their path template. Header names +are case-insensitive. An `Authorization` binding accepts Bearer, token, and Basic +authentication. + +Existing endpoint policies that enable request-body or WebSocket text rewriting +continue to authorize those locations. Those flags do not authorize arbitrary +query parameters, URL paths, or unrelated headers. Trusted AWS signing retains +its existing endpoint authorization. + +Credentials without usable placement metadata retain legacy endpoint-scoped +substitution. This preserves custom profiles, policy-only AWS signers, and Codex +refresh and ID tokens. The Codex reference profile supplies placement for its +access token and account header. + +Gateway and supervisor upgrades can happen in either order. The gateway forwards +static auth metadata with the existing provider environment. Older supervisors +ignore it and keep their previous behavior. New supervisors connected to an older +gateway receive no static auth metadata and retain legacy behavior. Existing +credential delivery and wire identities remain unchanged. + +The hardening takes effect automatically once both components support it and +usable placement is declared. Existing credential placeholder and update +lifecycles remain unchanged. + Use an explicit sandbox policy binding when the profile intentionally defines credentials without defining service endpoints. The policy names the concrete provider instance, not the profile type: @@ -202,7 +230,6 @@ The following provider profile design items are not part of the current behavior | Roadmap item | Current behavior | |---|---| -| General profile-driven credential placement | Static `auth_style`, `header_name`, `query_param`, and `path_template` placement metadata is stored and validated, but static credential injection still depends on environment placeholders generated from provider credentials. Dynamic `token_grant` credentials support `bearer` and `header` placement for matching HTTP endpoints. | | Binary-scoped credential injection | Static placeholder resolution is endpoint-scoped. Dynamic grants in single-route REST and multi-route inspection also require admission by their owning endpoint's policy, including its binary restrictions. L4-only credential forwarding retains endpoint-selector matching. | | Credential verification on create | `openshell provider create` does not yet probe provider verification endpoints or expose `--no-verify`. | | Automatic credential scope extraction | OpenShell does not yet inspect upstream provider responses to discover credential scopes. | @@ -372,12 +399,11 @@ credentials: stable_placeholder: false # Accepted values: basic, bearer, header, query, path. - # These fields describe static credential placement. - # Static runtime injection still uses env placeholder resolution. + # These fields restrict where a static placeholder can resolve. auth_style: bearer header_name: authorization - query_param: api_key - path_template: /v1/{credential}/resources + # For auth_style: query, set query_param: api_key instead. + # For auth_style: path, set path_template: /v1/{credential}/resources instead. refresh: # Accepted values: diff --git a/e2e/python/test_sandbox_providers.py b/e2e/python/test_sandbox_providers.py index 4c39bae95f..ebfbc02d46 100644 --- a/e2e/python/test_sandbox_providers.py +++ b/e2e/python/test_sandbox_providers.py @@ -236,6 +236,8 @@ def _reply(self): self.send_response(200) self.send_header("Content-Type", "application/json") self.send_header("Content-Length", str(len(encoded))) + separator = "&" if "?" in self.path else "?" + self.send_header("Link", f"<{self.path}{separator}page=2>; rel=next") self.end_headers() self.wfile.write(encoded) @@ -725,6 +727,128 @@ def call_native_openai(host: str, port: int) -> str: assert body["model"] == "fixture-openai-model" +@pytest.mark.parametrize( + "placement_metadata", [True, False], ids=["metadata", "legacy"] +) +def test_static_header_credentials_reject_reflective_request_locations( + sandbox: Callable[..., Sandbox], + sandbox_client: SandboxClient, + placement_metadata: bool, +) -> None: + """Existing metadata selects placement checks while missing metadata stays compatible.""" + stub = sandbox_client._stub + profile_id = f"e2e-placement-{time.time_ns()}" + secret = "synthetic-placement-secret" + profile = _native_inference_profile( + profile_id=profile_id, + env_var="API_TOKEN", + port=0, + rules=[sandbox_pb2.L7Rule(allow=sandbox_pb2.L7Allow(method="*", path="/**"))], + ) + if not placement_metadata: + profile.credentials[0].auth_style = "" + profile.credentials[0].header_name = "" + profile.endpoints[0].request_body_credential_rewrite = True + + def probe_locations(host: str, port: int) -> str: + import json + import os + import urllib.error + import urllib.parse + import urllib.request + + token = os.environ["API_TOKEN"] + root = f"http://{host}:{port}" + requests = [ + urllib.request.Request( + root + + "/search?" + + urllib.parse.urlencode({"q": "rust", "per_page": "1", "probe": token}) + ), + urllib.request.Request(root + "/search", headers={"X-Api-Key": token}), + urllib.request.Request( + root + "/" + urllib.parse.quote(token, safe="") + "/search" + ), + urllib.request.Request( + root + "/search", + data=urllib.parse.urlencode({"token": token}).encode(), + headers={"Content-Type": "application/x-www-form-urlencoded"}, + ), + ] + results = [] + for request in requests: + try: + with urllib.request.urlopen(request, timeout=30) as response: + results.append( + { + "status": response.status, + "body": response.read().decode(), + "link": response.headers.get("Link"), + } + ) + except urllib.error.HTTPError as error: + results.append( + { + "status": error.code, + "body": error.read().decode(), + "link": error.headers.get("Link"), + } + ) + request = urllib.request.Request( + root + "/search?q=rust&per_page=1", + headers={"Authorization": f"token {token}"}, + ) + with urllib.request.urlopen(request, timeout=30) as response: + valid = { + "body": json.loads(response.read()), + "link": response.headers.get("Link"), + } + return json.dumps({"denied": results, "valid": valid}) + + with native_endpoint_server() as port: + profile.endpoints[0].port = port + with ( + imported_provider_profile( + stub, profile=profile, source=f"{profile_id}.yaml" + ), + provider( + stub, + name=f"{profile_id}-provider", + provider_type=profile_id, + credentials={"API_TOKEN": secret}, + profile_workspace="default", + ) as attached_provider, + ): + spec = datamodel_pb2.SandboxSpec( + policy=_default_policy(), providers=[attached_provider] + ) + with sandbox(spec=spec, delete_on_exit=True) as sb: + result = sb.exec_python( + probe_locations, + args=("host.openshell.internal", port), + timeout_seconds=180, + ) + assert result.exit_code == 0, result.stderr + payload = json.loads(result.stdout) + for result in payload["denied"][:3]: + if placement_metadata: + assert result["status"] == 403, result + assert "credential_placement_mismatch" in result["body"] + assert secret not in result["body"] + assert result["link"] is None + else: + assert result["status"] == 200, result + assert secret in result["body"], result + body_result = payload["denied"][3] + assert body_result["status"] == 200, body_result + assert secret in body_result["body"] + if not placement_metadata: + assert secret in payload["denied"][0]["link"] + assert payload["valid"]["body"]["authorization"] == f"token {secret}" + assert "per_page=1&page=2" in payload["valid"]["link"] + assert secret not in payload["valid"]["link"] + + def test_imported_anthropic_profile_allows_native_endpoint_with_attached_provider( sandbox: Callable[..., Sandbox], sandbox_client: SandboxClient, diff --git a/proto/openshell.proto b/proto/openshell.proto index 44d5ed81ee..2d1310f946 100644 --- a/proto/openshell.proto +++ b/proto/openshell.proto @@ -2712,9 +2712,10 @@ message GetSandboxProviderEnvironmentResponse { uint64 provider_env_revision = 2; // Expiration timestamps for returned environment variables. map credential_expiration_times = 103; - // Dynamic credentials that require token grants or other runtime injection. - // Maps endpoint-bound provider metadata to credential metadata. - // Supervisor uses this to inject Authorization headers for token grant credentials. + // Endpoint-bound token grants and static credential request metadata. + // Reserved static-env: entries carry authored placement fields for + // a delivered static value and have no token_grant. Older supervisors ignore + // those entries; newer supervisors separate them before grant processing. map dynamic_credentials = 4; // Endpoint allowlists for static credential environment variables. Metadata // can be incomplete when a provider profile is invalid; capable supervisors @@ -2775,6 +2776,8 @@ message ProviderEnvironmentValue { message ProviderEnvironmentSnapshot { uint64 provider_env_revision = 1; repeated ProviderEnvironmentValue values = 2; + // Token grants and reserved static-env: request metadata, atomically + // delivered with the static environment. Metadata entries have no token_grant. map dynamic_credentials = 3; string provider_attachment_epoch = 4; string policy_hash = 5; diff --git a/providers/codex.yaml b/providers/codex.yaml index bafc5bd063..fb2d76b2e5 100644 --- a/providers/codex.yaml +++ b/providers/codex.yaml @@ -33,6 +33,8 @@ credentials: description: Codex OAuth access token env_vars: [CODEX_AUTH_ACCESS_TOKEN] required: true + auth_style: bearer + header_name: authorization - name: refresh_token description: Codex OAuth refresh token env_vars: [CODEX_AUTH_REFRESH_TOKEN] @@ -41,6 +43,8 @@ credentials: description: Codex account identifier env_vars: [CODEX_AUTH_ACCOUNT_ID] required: true + auth_style: header + header_name: chatgpt-account-id - name: id_token description: Codex OAuth ID token env_vars: [CODEX_AUTH_ID_TOKEN] diff --git a/sdk/go/proto/openshellv1/openshell.pb.go b/sdk/go/proto/openshellv1/openshell.pb.go index f180617bf6..98355eb283 100644 --- a/sdk/go/proto/openshellv1/openshell.pb.go +++ b/sdk/go/proto/openshellv1/openshell.pb.go @@ -11531,9 +11531,10 @@ type GetSandboxProviderEnvironmentResponse struct { ProviderEnvRevision uint64 `protobuf:"varint,2,opt,name=provider_env_revision,json=providerEnvRevision,proto3" json:"provider_env_revision,omitempty"` // Expiration timestamps for returned environment variables. CredentialExpirationTimes map[string]*timestamppb.Timestamp `protobuf:"bytes,103,rep,name=credential_expiration_times,json=credentialExpirationTimes,proto3" json:"credential_expiration_times,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"` - // Dynamic credentials that require token grants or other runtime injection. - // Maps endpoint-bound provider metadata to credential metadata. - // Supervisor uses this to inject Authorization headers for token grant credentials. + // Endpoint-bound token grants and static credential request metadata. + // Reserved static-env: entries carry authored placement fields for + // a delivered static value and have no token_grant. Older supervisors ignore + // those entries; newer supervisors separate them before grant processing. DynamicCredentials map[string]*ProviderProfileCredential `protobuf:"bytes,4,rep,name=dynamic_credentials,json=dynamicCredentials,proto3" json:"dynamic_credentials,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"` // Endpoint allowlists for static credential environment variables. Metadata // can be incomplete when a provider profile is invalid; capable supervisors @@ -11776,9 +11777,11 @@ func (x *ProviderEnvironmentValue) GetStablePlaceholder() bool { // credentials are endpoint selectors rather than environment values and stay // in their own collection. type ProviderEnvironmentSnapshot struct { - state protoimpl.MessageState `protogen:"open.v1"` - ProviderEnvRevision uint64 `protobuf:"varint,1,opt,name=provider_env_revision,json=providerEnvRevision,proto3" json:"provider_env_revision,omitempty"` - Values []*ProviderEnvironmentValue `protobuf:"bytes,2,rep,name=values,proto3" json:"values,omitempty"` + state protoimpl.MessageState `protogen:"open.v1"` + ProviderEnvRevision uint64 `protobuf:"varint,1,opt,name=provider_env_revision,json=providerEnvRevision,proto3" json:"provider_env_revision,omitempty"` + Values []*ProviderEnvironmentValue `protobuf:"bytes,2,rep,name=values,proto3" json:"values,omitempty"` + // Token grants and reserved static-env: request metadata, atomically + // delivered with the static environment. Metadata entries have no token_grant. DynamicCredentials map[string]*ProviderProfileCredential `protobuf:"bytes,3,rep,name=dynamic_credentials,json=dynamicCredentials,proto3" json:"dynamic_credentials,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"` ProviderAttachmentEpoch string `protobuf:"bytes,4,opt,name=provider_attachment_epoch,json=providerAttachmentEpoch,proto3" json:"provider_attachment_epoch,omitempty"` PolicyHash string `protobuf:"bytes,5,opt,name=policy_hash,json=policyHash,proto3" json:"policy_hash,omitempty"` diff --git a/skills/debug-inference/SKILL.md b/skills/debug-inference/SKILL.md index 50937104a1..bf28d96d5c 100644 --- a/skills/debug-inference/SKILL.md +++ b/skills/debug-inference/SKILL.md @@ -103,8 +103,9 @@ inference request using the provider's documented API shape. | A retired managed endpoint fails DNS resolution | Client still uses the removed managed endpoint | Configure the provider's native base URL and attach an endpoint-bearing provider profile | | Direct request is denied | Missing attachment, endpoint policy, HTTP rule, or binary authorization | Inspect the attached provider profile and sandbox effective policy | | `credential_endpoint_mismatch` | Credential profile does not authorize the request recipient | Correct the host/port/path or import a narrowly scoped profile for the intended endpoint | +| `credential_placement_mismatch` | A placeholder is outside its declared header, query parameter, or path slot | Match the profile placement. Enforcement is automatic with supporting gateway/supervisor versions; explicit body/WebSocket rewrite policies remain supported. | | `request_authority_mismatch` | HTTP authority differs from the CONNECT destination | Use the same host and effective port in both authorities | -| Credential variable is absent | Provider was not attached when this process launched, or profiles collide on a key | Attach the provider and launch a new process; resolve duplicate keys explicitly | +| Credential variable is absent | Provider was not attached when this process launched, profiles collide on a key, or credential delivery failed | Check provider readiness, then launch a new process; resolve duplicate keys explicitly. Missing placement metadata or an older component alone does not withhold credentials. | | Upstream rejects the model or body | Client relied on removed model/request rewriting | Configure the real model and provider-native request format in the application | | `127.0.0.1` works on the host but not in the sandbox | Loopback refers to different runtime | Use `host.openshell.internal` or another gateway-reachable endpoint and profile | | Host-local request times out | Server bind address, gateway topology, or host firewall blocks container-to-host traffic | Verify the listener and permit only the required gateway network path and port | diff --git a/skills/openshell-cli/SKILL.md b/skills/openshell-cli/SKILL.md index 84e6d65a2c..7d0a68b84c 100644 --- a/skills/openshell-cli/SKILL.md +++ b/skills/openshell-cli/SKILL.md @@ -150,6 +150,15 @@ when a placeholder is present but requests receive `credential_endpoint_mismatch`. A profileless static provider fails closed because the gateway cannot construct a binding. +Usable credential placement is enforced automatically when both gateway and +supervisor support it. `credential_placement_mismatch` means a placeholder is +outside its declared header, query parameter, or path-template slot. Match the +profile. Explicit body/WebSocket rewrite policies retain their documented +behavior for ordinary credentials. Missing placement metadata or older components +retain legacy endpoint-scoped substitution; they do not cause withholding by +themselves. Check provider readiness after changing placement and launch a new +process if retained references were revoked. + When an inspected request receives `request_authority_mismatch`, compare its HTTP authority with the CONNECT tunnel endpoint. The host and effective port must match. For a tunnel to `api.example.com:8443`, send