Skip to content

Move SERVER_PUBKEY configuration from lib/pb into imix#2383

Open
jabbate19 wants to merge 3 commits into
mainfrom
1405-pubkey-to-imix
Open

Move SERVER_PUBKEY configuration from lib/pb into imix#2383
jabbate19 wants to merge 3 commits into
mainfrom
1405-pubkey-to-imix

Conversation

@jabbate19

@jabbate19 jabbate19 commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

What type of PR is this?

/kind bug

What this PR does / why we need it:

Previously lib/pb owned the full IMIX_ agent configuration surface — both
at compile time via build.rs and via option_env! macros in config.rs:

  • IMIX_CONFIG YAML parsing (transport list → DSN building for CALLBACK_URI,
    server_pubkey extraction) + DSN validation + legacy-env exclusivity checks
    lived in pb/build.rs.
  • IMIX_SERVER_PUBKEY lifecycle (parsing from YAML, auto-fetching from
    Tavern's /status via reqwest, emitting cargo:rustc-env, enforcing at
    compile time via env!("IMIX_SERVER_PUBKEY") gated on imix feature) lived
    in pb/build.rs + xchacha.rs.
  • Runtime transport config (IMIX_CALLBACK_URI, IMIX_CALLBACK_INTERVAL,
    IMIX_RETRY_INTERVAL, IMIX_RUN_ONCE, IMIX_TRANSPORT_EXTRA,
    IMIX_UNIQUE, IMIX_GUARDRAILS) was baked at compile time in pb/src/config.rs
    via option_env! macros (callback_uri!, callback_interval!, retry_interval!,
    run_once!, extra!, plus IMIX_UNIQUE/IMIX_GUARDRAILS JSON → selector/guardrail
    parsing).

This made standalone builds of eldritch and golem transitively depend on
lib/pb's agent-specific build logic — they had to set IMIX_ env vars or have
Tavern reachable, even though only imix (the agent binary) needs them.

Commit 1 — SERVER_PUBKEY from lib/pbimix

  • lib/pb/build.rs: remove get_pub_key(), YamlConfigResult, and all
    SERVER_PUBKEY emission. Keep transport URI/DSN + proto compilation.
    rerun-if-env-changed no longer watches IMIX_SERVER_PUBKEY.
  • lib/pb/src/xchacha.rs: replace compile-time env!("IMIX_SERVER_PUBKEY")
    (#[cfg(feature="imix")] / #[cfg(not)] branching) with runtime-configurable
    OnceLock<[u8; 32]> + set_server_pubkey(). Falls back to hardcoded dummy
    key when no real key is configured, so eldritch/golem build with no key.
  • lib/pb/Cargo.toml: drop const-decoder (compile-time base64) and reqwest
    build-dep.
  • imix/build.rs: now owns SERVER_PUBKEY — parses server_pubkey from
    IMIX_CONFIG YAML, checks IMIX_SERVER_PUBKEY env, auto-fetches from
    {callback_uri}/status. Emits IMIX_SERVER_PUBKEY for imix's own compile.
  • imix/Cargo.toml: add base64 runtime dep + build-deps for moved config
    logic (reqwest, serde, serde_yaml, serde_json, urlencoding, which).
  • imix/src/run.rs: add init_crypto() that reads IMIX_SERVER_PUBKEY
    (runtime env first, then option_env! from build.rs), base64-decodes, and
    calls pb::xchacha::set_server_pubkey() early in run_agent().

Commit 2 — All remaining IMIX_ config from lib/pbimix

Covers every env var the old pb/config.rs baked at compile time:

Env var Previously in pb Now owned by imix
IMIX_CONFIG (YAML: transports + server_pubkey) pb/build.rs:parse_yaml_config() emitting IMIX_CALLBACK_URI + SERVER_PUBKEY DSN imix/build.rs:parse_yaml_config() — same logic, now in imix
IMIX_CALLBACK_URI pb/build.rs:parse_yaml_config() / validate_dsn_config() + pb/config.rs:option_env! macro → CALLBACK_URI const imix/build.rs bakes as rustc-envimix/src/imix_config.rs::COMPILE_CALLBACK_URIRuntimeImixConfig.callback_uripb::config::init_runtime_config()
IMIX_CALLBACK_INTERVAL pb/build.rs:validate_dsn_config() rerun + pb/config.rs:option_env!CALLBACK_INTERVAL const imix/build.rs rerun + imix_config.rs::COMPILE_CALLBACK_INTERVALRuntimeImixConfig.callback_interval
IMIX_RETRY_INTERVAL pb/config.rs:option_env!RETRY_INTERVAL const imix_config.rs::COMPILE_RETRY_INTERVALRuntimeImixConfig.retry_interval
IMIX_RUN_ONCE pb/config.rs:option_env!RUN_ONCE const imix_config.rs::COMPILE_RUN_ONCE_BAKEDRuntimeImixConfig.run_once
IMIX_TRANSPORT_EXTRA + IMIX_TRANSPORT_EXTRA_* pb/build.rs exclusivity check + pb/config.rs:option_env!DEFAULT_EXTRA_CONFIG imix/build.rs exclusivity check + imix_config.rs::COMPILE_EXTRARuntimeImixConfig.transport_extra
IMIX_UNIQUE pb/config.rs:option_env!parse_host_unique_selectors()host_unique::from_imix_unique() imix_config.rs::COMPILE_UNIQUE_JSONRuntimeImixConfig.unique_jsonpb::config::parse_host_unique_selectors() now reads runtime config
IMIX_GUARDRAILS pb/config.rs:option_env!parse_guardrails()guardrails::from_imix_guardrails() imix_config.rs::COMPILE_GUARDRAILS_JSONRuntimeImixConfig.guardrails_json
IMIX_BEACON_ID pb/config.rs:std::env::var() (already runtime-only) kept as runtime read — does not block building
  • lib/pb/build.rs: stripped to proto generation only. Removed
    IMIX_CONFIG, IMIX_CALLBACK_URI, IMIX_CALLBACK_INTERVAL,
    IMIX_TRANSPORT_EXTRA_*, IMIX_UNIQUE, IMIX_GUARDRAILS,
    IMIX_SERVER_PUBKEY handling. Only keeps IMIX_DEBUG (debug infra,
    present in all crates) + PROTOC. Build-deps reduced to
    tonic-prost-build + which.
  • lib/pb/Cargo.toml: drops serde/serde_yaml/urlencoding/serde_json
    build-deps (all were for IMIX_ YAML/DSN parsing, now live in imix).
  • lib/pb/src/config.rs: rewritten to runtime model:
    • Removed all option_env! macros (callback_uri!, callback_interval!,
      retry_interval!, run_once!, extra!) and their compiled constants.
    • Introduced RuntimeImixConfig struct (callback_uri, callback_interval,
      retry_interval, run_once, transport_extra, unique_json,
      guardrails_json) with defaults, OnceLock<RUNTIME_CONFIG>,
      init_runtime_config() / runtime_config() accessor.
    • Deprecated consts kept as defaults for API compat (CALLBACK_URI_DEFAULT,
      CALLBACK_INTERVAL_DEFAULT, etc.).
    • get_transport_type, parse_transports, parse_dsn, host_unique and
      guardrails parsing all read from runtime_config() instead of consts.
  • imix/src/imix_config.rs (new): owns all compile-time baked IMIX_ via
    option_env! macros, provides build_runtime_config() with runtime-env →
    baked → default precedence, constructing RuntimeImixConfig.
  • imix/src/run.rs: early init calls init_crypto() + init_runtime_config()
    (via imix_config::build_runtime_config()pb::config::init_runtime_config())
    before Config::default_with_imix_version(), so all pb values reflect real
    agent deployment settings.
  • imix/src/lib.rs + main.rs: add mod imix_config;.

Remaining IMIX_ in lib/pb after this PR:

  • IMIX_DEBUG rerun + cfg flag (debug infrastructure, present in every crate,
    not a build requirement for eldritch/golem).
  • IMIX_BEACON_ID runtime std::env::var (read at agent startup, does not
    require build-time env).
  • host_unique/build.rs and guardrails/build.rs still have their own
    IMIX_UNIQUE / IMIX_GUARDRAILS validation/baking — out of scope,
    they are small helper crates whose build.rs validation is independent
    of pb's now-runtime init path.

Result

  • cargo check -p pb / -p eldritch-core / -p eldritch / -p golem succeeds
    with zero IMIX_ env vars — pb is now buildable as a pure protobuf
    library for direct consumption by eldritchv2/golem.
  • cargo check -p imix (with IMIX_SERVER_PUBKEY) succeeds — all agent
    tuning is now fully owned by imix and injected at runtime into pb.

Which issue(s) this PR fixes:

Fixes #1405

Previously lib/pb/build.rs owned the full IMIX_SERVER_PUBKEY lifecycle:
- parsing server_pubkey from IMIX_CONFIG YAML,
- auto-fetching from Tavern's /status endpoint via reqwest,
- emitting cargo:rustc-env=IMIX_SERVER_PUBKEY,
- and xchacha.rs enforced it at compile time via env!("IMIX_SERVER_PUBKEY")
  gated on the `imix` feature.

This made standalone builds of eldritch and golem transitively depend on
Tavern being reachable or IMIX_SERVER_PUBKEY being set, even though only
imix (the agent) needs encrypted C2.

Changes:
- lib/pb/build.rs: remove get_pub_key(), YamlConfigResult, and all
  SERVER_PUBKEY emission. Keep only transport URI/DSN parsing and proto
  compilation. Rerun-if-env-changed no longer watches IMIX_SERVER_PUBKEY.
- lib/pb/src/xchacha.rs: replace compile-time env!("IMIX_SERVER_PUBKEY")
  with runtime-configurable OnceLock. Expose set_server_pubkey([u8; 32]).
  Falls back to a hardcoded dummy key when no real key is configured, so
  eldritch and golem build without any key.
- lib/pb/Cargo.toml: drop const-decoder (used only for compile-time
  base64 decode) and reqwest build-dep (used only for Tavern fetch).
- imix/build.rs: now owns SERVER_PUBKEY — parses server_pubkey from
  IMIX_CONFIG YAML, checks IMIX_SERVER_PUBKEY env, and auto-fetches from
  {callback_uri}/status. Emits IMIX_SERVER_PUBKEY for imix's own compile.
- imix/Cargo.toml: add base64 runtime dep and build-deps for the moved
  config logic (reqwest, serde, serde_yaml, serde_json, urlencoding, etc.).
- imix/src/run.rs: add init_crypto() that reads IMIX_SERVER_PUBKEY
  (runtime env first, then option_env! from build.rs), base64-decodes,
  and calls pb::xchacha::set_server_pubkey() early in run_agent().

Result:
- cargo build -p pb / -p eldritch / -p golem succeeds with no env vars
- cargo build -p imix requires IMIX_SERVER_PUBKEY (or IMIX_CONFIG, or
  a running Tavern for auto-fetch), owned entirely by imix.
…imix

Previously lib/pb owned the full IMIX_ configuration surface via
compile-time env var baking:

- build.rs: parsed IMIX_CONFIG YAML, built DSN for IMIX_CALLBACK_URI,
  validated DSN vs legacy env vars, and emitted cargo:rustc-env for
  callback config. This ran for every consumer of pb, including
  eldritch and golem which don't need agent transport configuration.
- config.rs: defined macros that used option_env!("IMIX_CALLBACK_URI"),
  option_env!("IMIX_CALLBACK_INTERVAL"), IMIX_RETRY_INTERVAL,
  IMIX_RUN_ONCE, IMIX_TRANSPORT_EXTRA, IMIX_UNIQUE, IMIX_GUARDRAILS
  to create compile-time constants. Building eldritch/golem would
  bake defaults or fail if those env vars had unexpected values.

This meant building eldritch or golem required understanding imix agent
deployment concerns and pulled IMIX_ requirements transitively through
pb.

Changes:
- lib/pb/build.rs: stripped to proto generation only. All IMIX_CONFIG,
  CALLBACK_URI/interval/extra, UNIQUE, GUARDRAILS, and SERVER_PUBKEY
  handling removed. Only keeps IMIX_DEBUG feature flag (debug infra)
  and PROTOC detection. Build-deps reduced to tonic-prost-build + which.
- lib/pb/Cargo.toml: removed serde/serde_yaml/urlencoding/serde_json
  build-deps — all were for IMIX_ YAML/DSN parsing, now in imix.
- lib/pb/src/config.rs: rewritten to runtime-configurable model:
  * Removed all option_env!/env! macros (callback_uri!, callback_interval!,
    retry_interval!, run_once!, extra!) and their compiled constants.
  * Introduced RuntimeImixConfig struct holding all imix-tunable values
    (callback_uri, callback_interval, retry_interval, run_once,
    transport_extra, unique_json, guardrails_json) with sensible defaults.
  * OnceLock<RUNTIME_CONFIG> + init_runtime_config() / runtime_config()
    accessor. Tests use defaults; imix overrides at startup.
  * Deprecated consts kept as defaults for API compat.
  * get_transport_type, parse_transports, parse_dsn, host_unique and
    guardrails parsing all now read from runtime_config() instead of
    compile-time constants.
  * IMIX_BEACON_ID kept as runtime std::env::var (does not block builds).
- imix/build.rs: already owns SERVER_PUBKEY fetching (from first commit).
  No change needed for this step — it continues to handle IMIX_CONFIG,
  CALLBACK_URI, and SERVER_PUBKEY baking via rustc-env.
- imix/src/imix_config.rs (new): owns all compile-time baked IMIX_ values
  via option_env! macros (IMIX_CALLBACK_URI, CALLBACK_INTERVAL,
  RETRY_INTERVAL, RUN_ONCE, TRANSPORT_EXTRA, UNIQUE, GUARDRAILS).
  Provides build_runtime_config() that prefers runtime env vars then
  falls back to baked values, constructing RuntimeImixConfig for pb.
- imix/src/run.rs: early init calls init_runtime_config() after
  init_crypto() and before Config::default_with_imix_version, so all
  pb::config values reflect real agent deployment settings.
- imix/src/lib.rs + main.rs: add `mod imix_config;`.

Result:
- cargo check -p pb / -p eldritch-core / -p eldritch / -p golem
  succeeds with zero IMIX_ env vars set — truly standalone.
- cargo check -p imix (with IMIX_SERVER_PUBKEY) succeeds — agent config
  is now fully owned by imix and injected at runtime into pb.
- host_unique and guardrails still have their own build.rs validation;
  left as-is per scope decision.
@jabbate19
jabbate19 force-pushed the 1405-pubkey-to-imix branch from 5766d7b to 0244c52 Compare July 11, 2026 04:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] Golem and test builds expect a pubkey but shouldn't

1 participant