Move SERVER_PUBKEY configuration from lib/pb into imix#2383
Open
jabbate19 wants to merge 3 commits into
Open
Conversation
Previously lib/pb/build.rs owned the full IMIX_SERVER_PUBKEY lifecycle:
- parsing server_pubkey from IMIX_CONFIG YAML,
- auto-fetching from Tavern's /status endpoint via reqwest,
- emitting cargo:rustc-env=IMIX_SERVER_PUBKEY,
- and xchacha.rs enforced it at compile time via env!("IMIX_SERVER_PUBKEY")
gated on the `imix` feature.
This made standalone builds of eldritch and golem transitively depend on
Tavern being reachable or IMIX_SERVER_PUBKEY being set, even though only
imix (the agent) needs encrypted C2.
Changes:
- lib/pb/build.rs: remove get_pub_key(), YamlConfigResult, and all
SERVER_PUBKEY emission. Keep only transport URI/DSN parsing and proto
compilation. Rerun-if-env-changed no longer watches IMIX_SERVER_PUBKEY.
- lib/pb/src/xchacha.rs: replace compile-time env!("IMIX_SERVER_PUBKEY")
with runtime-configurable OnceLock. Expose set_server_pubkey([u8; 32]).
Falls back to a hardcoded dummy key when no real key is configured, so
eldritch and golem build without any key.
- lib/pb/Cargo.toml: drop const-decoder (used only for compile-time
base64 decode) and reqwest build-dep (used only for Tavern fetch).
- imix/build.rs: now owns SERVER_PUBKEY — parses server_pubkey from
IMIX_CONFIG YAML, checks IMIX_SERVER_PUBKEY env, and auto-fetches from
{callback_uri}/status. Emits IMIX_SERVER_PUBKEY for imix's own compile.
- imix/Cargo.toml: add base64 runtime dep and build-deps for the moved
config logic (reqwest, serde, serde_yaml, serde_json, urlencoding, etc.).
- imix/src/run.rs: add init_crypto() that reads IMIX_SERVER_PUBKEY
(runtime env first, then option_env! from build.rs), base64-decodes,
and calls pb::xchacha::set_server_pubkey() early in run_agent().
Result:
- cargo build -p pb / -p eldritch / -p golem succeeds with no env vars
- cargo build -p imix requires IMIX_SERVER_PUBKEY (or IMIX_CONFIG, or
a running Tavern for auto-fetch), owned entirely by imix.
9 tasks
…imix
Previously lib/pb owned the full IMIX_ configuration surface via
compile-time env var baking:
- build.rs: parsed IMIX_CONFIG YAML, built DSN for IMIX_CALLBACK_URI,
validated DSN vs legacy env vars, and emitted cargo:rustc-env for
callback config. This ran for every consumer of pb, including
eldritch and golem which don't need agent transport configuration.
- config.rs: defined macros that used option_env!("IMIX_CALLBACK_URI"),
option_env!("IMIX_CALLBACK_INTERVAL"), IMIX_RETRY_INTERVAL,
IMIX_RUN_ONCE, IMIX_TRANSPORT_EXTRA, IMIX_UNIQUE, IMIX_GUARDRAILS
to create compile-time constants. Building eldritch/golem would
bake defaults or fail if those env vars had unexpected values.
This meant building eldritch or golem required understanding imix agent
deployment concerns and pulled IMIX_ requirements transitively through
pb.
Changes:
- lib/pb/build.rs: stripped to proto generation only. All IMIX_CONFIG,
CALLBACK_URI/interval/extra, UNIQUE, GUARDRAILS, and SERVER_PUBKEY
handling removed. Only keeps IMIX_DEBUG feature flag (debug infra)
and PROTOC detection. Build-deps reduced to tonic-prost-build + which.
- lib/pb/Cargo.toml: removed serde/serde_yaml/urlencoding/serde_json
build-deps — all were for IMIX_ YAML/DSN parsing, now in imix.
- lib/pb/src/config.rs: rewritten to runtime-configurable model:
* Removed all option_env!/env! macros (callback_uri!, callback_interval!,
retry_interval!, run_once!, extra!) and their compiled constants.
* Introduced RuntimeImixConfig struct holding all imix-tunable values
(callback_uri, callback_interval, retry_interval, run_once,
transport_extra, unique_json, guardrails_json) with sensible defaults.
* OnceLock<RUNTIME_CONFIG> + init_runtime_config() / runtime_config()
accessor. Tests use defaults; imix overrides at startup.
* Deprecated consts kept as defaults for API compat.
* get_transport_type, parse_transports, parse_dsn, host_unique and
guardrails parsing all now read from runtime_config() instead of
compile-time constants.
* IMIX_BEACON_ID kept as runtime std::env::var (does not block builds).
- imix/build.rs: already owns SERVER_PUBKEY fetching (from first commit).
No change needed for this step — it continues to handle IMIX_CONFIG,
CALLBACK_URI, and SERVER_PUBKEY baking via rustc-env.
- imix/src/imix_config.rs (new): owns all compile-time baked IMIX_ values
via option_env! macros (IMIX_CALLBACK_URI, CALLBACK_INTERVAL,
RETRY_INTERVAL, RUN_ONCE, TRANSPORT_EXTRA, UNIQUE, GUARDRAILS).
Provides build_runtime_config() that prefers runtime env vars then
falls back to baked values, constructing RuntimeImixConfig for pb.
- imix/src/run.rs: early init calls init_runtime_config() after
init_crypto() and before Config::default_with_imix_version, so all
pb::config values reflect real agent deployment settings.
- imix/src/lib.rs + main.rs: add `mod imix_config;`.
Result:
- cargo check -p pb / -p eldritch-core / -p eldritch / -p golem
succeeds with zero IMIX_ env vars set — truly standalone.
- cargo check -p imix (with IMIX_SERVER_PUBKEY) succeeds — agent config
is now fully owned by imix and injected at runtime into pb.
- host_unique and guardrails still have their own build.rs validation;
left as-is per scope decision.
jabbate19
force-pushed
the
1405-pubkey-to-imix
branch
from
July 11, 2026 04:20
5766d7b to
0244c52
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind bug
What this PR does / why we need it:
Previously
lib/pbowned the fullIMIX_agent configuration surface — bothat compile time via
build.rsand viaoption_env!macros inconfig.rs:IMIX_CONFIGYAML parsing (transport list → DSN building forCALLBACK_URI,server_pubkeyextraction) + DSN validation + legacy-env exclusivity checkslived in
pb/build.rs.IMIX_SERVER_PUBKEYlifecycle (parsing from YAML, auto-fetching fromTavern's
/statusviareqwest, emittingcargo:rustc-env, enforcing atcompile time via
env!("IMIX_SERVER_PUBKEY")gated onimixfeature) livedin
pb/build.rs+xchacha.rs.IMIX_CALLBACK_URI,IMIX_CALLBACK_INTERVAL,IMIX_RETRY_INTERVAL,IMIX_RUN_ONCE,IMIX_TRANSPORT_EXTRA,IMIX_UNIQUE,IMIX_GUARDRAILS) was baked at compile time inpb/src/config.rsvia
option_env!macros (callback_uri!,callback_interval!,retry_interval!,run_once!,extra!, plusIMIX_UNIQUE/IMIX_GUARDRAILSJSON → selector/guardrailparsing).
This made standalone builds of
eldritchandgolemtransitively depend onlib/pb's agent-specific build logic — they had to set IMIX_ env vars or haveTavern reachable, even though only
imix(the agent binary) needs them.Commit 1 —
SERVER_PUBKEYfromlib/pb→imixlib/pb/build.rs: removeget_pub_key(),YamlConfigResult, and allSERVER_PUBKEYemission. Keep transport URI/DSN + proto compilation.rerun-if-env-changedno longer watchesIMIX_SERVER_PUBKEY.lib/pb/src/xchacha.rs: replace compile-timeenv!("IMIX_SERVER_PUBKEY")(
#[cfg(feature="imix")]/#[cfg(not)]branching) with runtime-configurableOnceLock<[u8; 32]>+set_server_pubkey(). Falls back to hardcoded dummykey when no real key is configured, so eldritch/golem build with no key.
lib/pb/Cargo.toml: dropconst-decoder(compile-time base64) andreqwestbuild-dep.
imix/build.rs: now ownsSERVER_PUBKEY— parsesserver_pubkeyfromIMIX_CONFIGYAML, checksIMIX_SERVER_PUBKEYenv, auto-fetches from{callback_uri}/status. EmitsIMIX_SERVER_PUBKEYfor imix's own compile.imix/Cargo.toml: addbase64runtime dep + build-deps for moved configlogic (
reqwest,serde,serde_yaml,serde_json,urlencoding,which).imix/src/run.rs: addinit_crypto()that readsIMIX_SERVER_PUBKEY(runtime env first, then
option_env!from build.rs), base64-decodes, andcalls
pb::xchacha::set_server_pubkey()early inrun_agent().Commit 2 — All remaining
IMIX_config fromlib/pb→imixCovers every env var the old
pb/config.rsbaked at compile time:pbimixIMIX_CONFIG(YAML: transports + server_pubkey)pb/build.rs:parse_yaml_config()emittingIMIX_CALLBACK_URI+SERVER_PUBKEYDSNimix/build.rs:parse_yaml_config()— same logic, now in imixIMIX_CALLBACK_URIpb/build.rs:parse_yaml_config()/validate_dsn_config()+pb/config.rs:option_env!macro →CALLBACK_URIconstimix/build.rsbakes asrustc-env→imix/src/imix_config.rs::COMPILE_CALLBACK_URI→RuntimeImixConfig.callback_uri→pb::config::init_runtime_config()IMIX_CALLBACK_INTERVALpb/build.rs:validate_dsn_config()rerun +pb/config.rs:option_env!→CALLBACK_INTERVALconstimix/build.rsrerun +imix_config.rs::COMPILE_CALLBACK_INTERVAL→RuntimeImixConfig.callback_intervalIMIX_RETRY_INTERVALpb/config.rs:option_env!→RETRY_INTERVALconstimix_config.rs::COMPILE_RETRY_INTERVAL→RuntimeImixConfig.retry_intervalIMIX_RUN_ONCEpb/config.rs:option_env!→RUN_ONCEconstimix_config.rs::COMPILE_RUN_ONCE_BAKED→RuntimeImixConfig.run_onceIMIX_TRANSPORT_EXTRA+IMIX_TRANSPORT_EXTRA_*pb/build.rsexclusivity check +pb/config.rs:option_env!→DEFAULT_EXTRA_CONFIGimix/build.rsexclusivity check +imix_config.rs::COMPILE_EXTRA→RuntimeImixConfig.transport_extraIMIX_UNIQUEpb/config.rs:option_env!→parse_host_unique_selectors()→host_unique::from_imix_unique()imix_config.rs::COMPILE_UNIQUE_JSON→RuntimeImixConfig.unique_json→pb::config::parse_host_unique_selectors()now reads runtime configIMIX_GUARDRAILSpb/config.rs:option_env!→parse_guardrails()→guardrails::from_imix_guardrails()imix_config.rs::COMPILE_GUARDRAILS_JSON→RuntimeImixConfig.guardrails_jsonIMIX_BEACON_IDpb/config.rs:std::env::var()(already runtime-only)lib/pb/build.rs: stripped to proto generation only. RemovedIMIX_CONFIG,IMIX_CALLBACK_URI,IMIX_CALLBACK_INTERVAL,IMIX_TRANSPORT_EXTRA_*,IMIX_UNIQUE,IMIX_GUARDRAILS,IMIX_SERVER_PUBKEYhandling. Only keepsIMIX_DEBUG(debug infra,present in all crates) +
PROTOC. Build-deps reduced totonic-prost-build+which.lib/pb/Cargo.toml: dropsserde/serde_yaml/urlencoding/serde_jsonbuild-deps (all were for IMIX_ YAML/DSN parsing, now live in imix).
lib/pb/src/config.rs: rewritten to runtime model:option_env!macros (callback_uri!,callback_interval!,retry_interval!,run_once!,extra!) and their compiled constants.RuntimeImixConfigstruct (callback_uri,callback_interval,retry_interval,run_once,transport_extra,unique_json,guardrails_json) with defaults,OnceLock<RUNTIME_CONFIG>,init_runtime_config()/runtime_config()accessor.CALLBACK_URI_DEFAULT,CALLBACK_INTERVAL_DEFAULT, etc.).get_transport_type,parse_transports,parse_dsn, host_unique andguardrails parsing all read from
runtime_config()instead of consts.imix/src/imix_config.rs(new): owns all compile-time bakedIMIX_viaoption_env!macros, providesbuild_runtime_config()with runtime-env →baked → default precedence, constructing
RuntimeImixConfig.imix/src/run.rs: early init callsinit_crypto()+init_runtime_config()(via
imix_config::build_runtime_config()→pb::config::init_runtime_config())before
Config::default_with_imix_version(), so all pb values reflect realagent deployment settings.
imix/src/lib.rs+main.rs: addmod imix_config;.Remaining
IMIX_inlib/pbafter this PR:IMIX_DEBUGrerun + cfg flag (debug infrastructure, present in every crate,not a build requirement for eldritch/golem).
IMIX_BEACON_IDruntimestd::env::var(read at agent startup, does notrequire build-time env).
host_unique/build.rsandguardrails/build.rsstill have their ownIMIX_UNIQUE/IMIX_GUARDRAILSvalidation/baking — out of scope,they are small helper crates whose build.rs validation is independent
of pb's now-runtime init path.
Result
cargo check -p pb / -p eldritch-core / -p eldritch / -p golemsucceedswith zero
IMIX_env vars — pb is now buildable as a pure protobuflibrary for direct consumption by eldritchv2/golem.
cargo check -p imix(withIMIX_SERVER_PUBKEY) succeeds — all agenttuning is now fully owned by imix and injected at runtime into pb.
Which issue(s) this PR fixes:
Fixes #1405