Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/_release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
# release.yml (production repos) and test-release.yml (the shared test repo),
# which supply `dist_repo` and `record`.
#
# Always: build and `make publish` to <dist_repo>. Claude and Codex releases
# also bump each plugin manifest version. Antigravity's native manifest has no
# version field, so its repository tag is the release version.
# Always: build and `make publish` to <dist_repo>. Claude, Codex, and Grok
# releases also bump each plugin manifest version. Antigravity's native
# manifest has no version field, so its repository tag is the release version.
# When record=true (production): commit the bump to main when needed, tag
# v<version>-<plugin>, and create a GitHub Release on the monorepo. After the
# distribution is deployed, its repo gets an unsuffixed v<version> tag and
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
# antigravity -> braintrustdata/braintrust-antigravity-plugin
# claude -> braintrustdata/braintrust-claude-plugin
# codex -> braintrustdata/braintrust-codex-plugin
# grok -> braintrustdata/braintrust-grok-plugin
#
# Full flow: stamp manifest versions where the agent format supports them ->
# commit to main when needed -> monorepo tag/release using
Expand All @@ -22,7 +23,7 @@ on:
description: "Plugin bundle to release"
required: true
type: choice
options: [antigravity, claude, codex]
options: [antigravity, claude, codex, grok]

permissions:
contents: write
Expand All @@ -33,6 +34,6 @@ jobs:
with:
version: ${{ inputs.version }}
plugin: ${{ inputs.plugin }}
dist_repo: ${{ inputs.plugin == 'antigravity' && 'braintrustdata/braintrust-antigravity-plugin' || inputs.plugin == 'codex' && 'braintrustdata/braintrust-codex-plugin' || 'braintrustdata/braintrust-claude-plugin' }}
dist_repo: ${{ inputs.plugin == 'antigravity' && 'braintrustdata/braintrust-antigravity-plugin' || inputs.plugin == 'codex' && 'braintrustdata/braintrust-codex-plugin' || inputs.plugin == 'grok' && 'braintrustdata/braintrust-grok-plugin' || 'braintrustdata/braintrust-claude-plugin' }}
record: true
secrets: inherit
2 changes: 1 addition & 1 deletion .github/workflows/test-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ on:
description: "Plugin bundle to release. Overwrites braintrustdata/test-coding-agent-dist"
required: true
type: choice
options: [antigravity, claude, codex]
options: [antigravity, claude, codex, grok]

permissions:
contents: write
Expand Down
3 changes: 2 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ cargo test --manifest-path bt-daemon/Cargo.toml --all-features

## Versioning and distribution

Versioning is per distribution. Claude and Codex plugins carry their version in
Versioning is per distribution. Claude, Codex, and Grok plugins carry their version in
their plugin manifests, and `scripts/set-plugin-version.py` updates those
manifests for a release. Antigravity's schema does not expose a version field,
so its distribution is versioned by repository tags and GitHub Releases.
Expand All @@ -54,6 +54,7 @@ is unversioned.
| antigravity | `braintrustdata/braintrust-antigravity-plugin` |
| claude | `braintrustdata/braintrust-claude-plugin` |
| codex | `braintrustdata/braintrust-codex-plugin` |
| grok | `braintrustdata/braintrust-grok-plugin` |

A distribution repository is a generated artifact. Each deploy clones it,
replaces the tracked tree with a fresh build, and pushes the result.
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ For further instructions, see the instructions for your desired coding agent
| Google Antigravity | [braintrustdata/braintrust-antigravity-plugin](https://github.com/braintrustdata/braintrust-antigravity-plugin) |
| Claude Code | [braintrustdata/braintrust-claude-plugin](https://github.com/braintrustdata/braintrust-claude-plugin) |
| Codex | [braintrustdata/braintrust-codex-plugin](https://github.com/braintrustdata/braintrust-codex-plugin) |
| Grok | [braintrustdata/braintrust-grok-plugin](https://github.com/braintrustdata/braintrust-grok-plugin) |
| OpenCode | npm: [`@braintrust/trace-opencode`](https://www.npmjs.com/package/@braintrust/trace-opencode) |
| Pi | npm: [`@braintrust/pi-extension`](https://www.npmjs.com/package/@braintrust/pi-extension) |

Expand Down
14 changes: 8 additions & 6 deletions bt-daemon/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ Each coding agent reads an independent non-credential `braintrust.json` file:
- OpenCode: `$XDG_CONFIG_HOME/opencode/braintrust.json`, falling back to
`~/.config/opencode/braintrust.json`
- Pi: `~/.pi/agent/braintrust.json`
- Grok: `~/.grok/braintrust.json`

`BT_DAEMON_CONFIG` can override the path for isolated tests and managed hosts.

Expand Down Expand Up @@ -140,16 +141,17 @@ different profiles, organizations, projects, experiments, or parent spans.
## Status

Phases 0–5 are implemented: protocol, daemon lifecycle, Braintrust sink,
Codex and Claude translators, `bt daemon` integration, and thin hook shims for
both shipped plugins. Every coding-agent capture request returns after the raw
Codex, Claude, and Grok translators, `bt daemon` integration, and thin hook shims for
all shipped plugins. Every coding-agent capture request returns after the raw
event is flushed to its journal; authentication, correlation, translation, and
reporting run on daemon-owned workers. Restart recovery replays the redacted
journal with deterministic span ids, so resubmitted rows merge into the same
spans instead of creating duplicates. Claude and Codex lifecycle entries
reference a daemon-owned transcript mirror, so recovery does not depend on
mutable external paths without re-recording the transcript on every turn.
Explicit turn/session-end flushes are bounded, and sessions can target project
logs or an experiment.
reference a daemon-owned transcript mirror; Grok records independent bounded
updates and events mirrors. Recovery therefore does not depend on mutable
external paths or copy a full transcript into every event. Explicit
turn/session-end flushes are bounded, and sessions can target project logs or
an experiment.

Memory is bounded end to end, while on-disk records stay complete: the daemon
never holds a transcript or a whole journal in memory, mirroring and replay
Expand Down
78 changes: 71 additions & 7 deletions bt-daemon/src/delivery_ledger.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,16 @@ use std::path::{Path, PathBuf};
struct LedgerFile {
#[serde(default, alias = "terminal_span_ids")]
completed_span_ids: HashSet<String>,
#[serde(default)]
late_merge_span_ids: HashSet<String>,
}

struct DeliveryLedger {
path: PathBuf,
known: HashSet<String>,
pending: HashSet<String>,
known_late_merges: HashSet<String>,
pending_late_merges: HashSet<String>,
}

impl DeliveryLedger {
Expand All @@ -46,15 +50,17 @@ impl DeliveryLedger {
crate::ids::session_storage_id(source, session_id),
&fingerprint_id[..32]
));
let known = match tokio::fs::read(&path).await {
Ok(bytes) => serde_json::from_slice::<LedgerFile>(&bytes)?.completed_span_ids,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => HashSet::new(),
let persisted = match tokio::fs::read(&path).await {
Ok(bytes) => serde_json::from_slice::<LedgerFile>(&bytes)?,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => LedgerFile::default(),
Err(error) => return Err(error.into()),
};
Ok(Self {
path,
known,
known: persisted.completed_span_ids,
pending: HashSet::new(),
known_late_merges: persisted.late_merge_span_ids,
pending_late_merges: HashSet::new(),
})
}

Expand All @@ -64,7 +70,12 @@ impl DeliveryLedger {
let row = match op {
SpanOp::Insert(row) | SpanOp::Merge(row) => row,
};
!self.known.contains(&row.span_id) && !self.pending.contains(&row.span_id)
if row.allow_late_merge {
!self.known_late_merges.contains(&row.span_id)
&& !self.pending_late_merges.contains(&row.span_id)
} else {
!self.known.contains(&row.span_id) && !self.pending.contains(&row.span_id)
}
})
.cloned()
.collect()
Expand All @@ -75,17 +86,21 @@ impl DeliveryLedger {
let row = match op {
SpanOp::Insert(row) | SpanOp::Merge(row) => row,
};
if row.end_ms.is_some() {
if row.allow_late_merge {
self.pending_late_merges.insert(row.span_id.clone());
} else if row.end_ms.is_some() {
self.pending.insert(row.span_id.clone());
}
}
}

async fn commit(&mut self) -> anyhow::Result<()> {
if self.pending.is_empty() {
if self.pending.is_empty() && self.pending_late_merges.is_empty() {
return Ok(());
}
self.known.extend(self.pending.drain());
self.known_late_merges
.extend(self.pending_late_merges.drain());
let parent = self.path.parent().expect("ledger path has a parent");
tokio::fs::create_dir_all(parent).await?;
let temp = self
Expand All @@ -95,6 +110,7 @@ impl DeliveryLedger {
&temp,
serde_json::to_vec(&LedgerFile {
completed_span_ids: self.known.clone(),
late_merge_span_ids: self.known_late_merges.clone(),
})?,
)
.await?;
Expand Down Expand Up @@ -237,6 +253,15 @@ mod tests {
})
}

fn late_merge(span_id: &str) -> SpanOp {
SpanOp::Merge(SpanRow {
span_id: span_id.into(),
root_span_id: "root".into(),
allow_late_merge: true,
..Default::default()
})
}

#[tokio::test]
async fn a_destination_receives_a_terminal_span_only_once_across_sink_instances() {
let temp = tempfile::tempdir().unwrap();
Expand Down Expand Up @@ -276,6 +301,45 @@ mod tests {
assert!(repeated_output.lock().unwrap().is_empty());
}

#[tokio::test]
async fn a_completed_span_receives_one_late_merge_across_sink_instances() {
let temp = tempfile::tempdir().unwrap();
let first = RecordingSink::default();
let mut first = LedgerSink::new(
Box::new(first),
temp.path(),
"grok",
"session-1",
Some(&config("project-a")),
)
.await;
assert_eq!(first.emit(&[terminal("span-1")]).await.unwrap(), 1);
first.flush().await.unwrap();

let second = RecordingSink::default();
let mut second = LedgerSink::new(
Box::new(second),
temp.path(),
"grok",
"session-1",
Some(&config("project-a")),
)
.await;
assert_eq!(second.emit(&[late_merge("span-1")]).await.unwrap(), 1);
second.flush().await.unwrap();

let third = RecordingSink::default();
let mut third = LedgerSink::new(
Box::new(third),
temp.path(),
"grok",
"session-1",
Some(&config("project-a")),
)
.await;
assert_eq!(third.emit(&[late_merge("span-1")]).await.unwrap(), 0);
}

#[tokio::test]
async fn a_different_destination_replays_the_same_terminal_span() {
let temp = tempfile::tempdir().unwrap();
Expand Down
112 changes: 107 additions & 5 deletions bt-daemon/src/dispatch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -214,12 +214,15 @@ impl Session {
}

/// Agent transcript files are external mutable state. Mirror them into
/// daemon-owned storage at lifecycle boundaries and journal only a reference,
/// so recovery/replay does not depend on a path that Claude may later rewrite
/// or delete — and so the transcript is stored once rather than re-copied into
/// every event. Fail open: without a reference the translator reads the live
/// path exactly as before.
/// daemon-owned storage at lifecycle boundaries and journal only bounded
/// references, so recovery sees exactly the bytes that live translation
/// observed instead of depending on mutable external paths or copying a full
/// transcript into every event. Capture failures remain fail-open.
pub(crate) async fn hydrate_transcript_reference(data_dir: &std::path::Path, env: &mut Envelope) {
if env.source == "grok" {
hydrate_grok_transcript_references(data_dir, env).await;
return;
}
let should_capture = match env.source.as_str() {
"codex" => true,
"claude-code" => matches!(
Expand Down Expand Up @@ -265,6 +268,57 @@ pub(crate) async fn hydrate_transcript_reference(data_dir: &std::path::Path, env
}
}

async fn hydrate_grok_transcript_references(data_dir: &std::path::Path, env: &mut Envelope) {
let Some(transcript) = env
.payload
.get("transcriptPath")
.or_else(|| env.payload.get("transcript_path"))
.and_then(serde_json::Value::as_str)
.map(std::path::PathBuf::from)
else {
return;
};
let Some(session_dir) = transcript.parent() else {
return;
};

let mut mirrors = serde_json::Map::new();
for (name, key) in [
("updates.jsonl", "updates"),
("events.jsonl", "events"),
("system_prompt.txt", "system_prompt"),
] {
let source = session_dir.join(name);
let Some(source_str) = source.to_str() else {
continue;
};
let mirror_session = crate::ids::session_namespace(&env.source, &env.session_id);
match crate::transcript_mirror::capture(data_dir, &mirror_session, source_str).await {
Ok((mirror, through)) => {
mirrors.insert(
key.to_string(),
serde_json::json!({
"path": source,
"mirror": mirror,
"through": through,
}),
);
}
Err(error) => {
tracing::debug!(session_id = %env.session_id, %error, file = name, "Grok transcript mirror skipped");
}
}
}
if !mirrors.is_empty() {
if let Some(payload) = env.payload.as_object_mut() {
payload.insert(
"_bt_grok_transcript_mirrors".to_string(),
serde_json::Value::Object(mirrors),
);
}
}
}

struct SessionActor {
session_id: String,
translator_session_id: String,
Expand Down Expand Up @@ -753,3 +807,51 @@ pub(crate) fn is_tool_lifecycle_event(event: &str) -> bool {
| "tool.execute.after"
)
}

#[cfg(test)]
mod tests {
use super::*;

#[tokio::test]
async fn grok_hydration_mirrors_transcripts_and_system_prompt_at_one_boundary() {
let tmp = tempfile::tempdir().unwrap();
let session = tmp.path().join("native-session");
tokio::fs::create_dir(&session).await.unwrap();
tokio::fs::write(session.join("chat_history.jsonl"), b"")
.await
.unwrap();
tokio::fs::write(session.join("updates.jsonl"), b"{\"update\":1}\n")
.await
.unwrap();
tokio::fs::write(session.join("events.jsonl"), b"{\"event\":1}\n")
.await
.unwrap();
tokio::fs::write(session.join("system_prompt.txt"), b"You are Grok.")
.await
.unwrap();
let mut env = Envelope {
source: "grok".into(),
source_version: None,
plugin_version: None,
session_id: "session-1".into(),
event: "stop".into(),
ts_ms: 1,
managed_run_id: None,
capture: None,
payload: serde_json::json!({
"transcriptPath": session.join("chat_history.jsonl")
}),
route: None,
config: None,
};

hydrate_transcript_reference(tmp.path(), &mut env).await;

let mirrors = &env.payload["_bt_grok_transcript_mirrors"];
for name in ["updates", "events", "system_prompt"] {
let path = mirrors[name]["mirror"].as_str().unwrap();
assert!(std::path::Path::new(path).is_file());
assert!(mirrors[name]["through"].as_u64().unwrap() > 0);
}
}
}
Loading
Loading