Skip to content

feat(testing): add Langfuse::Testing module for in-memory span capture - #100

Open
Alan-Marx wants to merge 4 commits into
simplepractice:mainfrom
Alan-Marx:feat/test-exporter-config
Open

feat(testing): add Langfuse::Testing module for in-memory span capture#100
Alan-Marx wants to merge 4 commits into
simplepractice:mainfrom
Alan-Marx:feat/test-exporter-config

Conversation

@Alan-Marx

@Alan-Marx Alan-Marx commented Jul 31, 2026

Copy link
Copy Markdown

Summary

We recently added the gem to our application and have set up an in-memory span exporter in our test suite so that we can test our observed code without needing to mock Langfuse network requests or stub Langfuse methods. We think this approach provides a pretty good DX and gets us pretty close to what is happening in production so thought we would offer it up to you guys for review.

  • Adds require "langfuse/testing" + include Langfuse::Testing as a supported public API for asserting against emitted spans in tests without real network calls or private ivar access
  • Introduces OtelSetup.test_mode flag that causes build_exporter to return a fresh InMemorySpanExporter on each provider build, surviving shutdown/reset! cycles transparently
  • Provides two instance methods: emitted_langfuse_spans (flush + return recorded spans) and reset_langfuse (flush + clear buffer between tests); see docs/TESTING.md for RSpec and Minitest integration examples

Test plan

  • bundle exec rspec spec/langfuse/testing_spec.rb — all examples pass
  • bundle exec rspec — full suite passes (1347 examples, 0 failures)
  • bundle exec rubocop — no offenses
  • In an app: require "langfuse/testing", include Langfuse::Testing, call Langfuse.observe, assert on emitted_langfuse_spans — spans captured without network calls
  • Confirm reset_langfuse clears spans between tests and emitted_langfuse_spans returns [] on a clean slate
  • [ x Confirm test_mode survives Langfuse.reset! (no OTLP exporter used after reset)

Note

Medium Risk
Changes the tracer provider initialization path and exporter selection, but production behavior stays on OTLP unless langfuse/testing is required; mis-loading that file in a non-test process would silently drop real exports.

Overview
Adds a test-only entry point (require "langfuse/testing") so apps can run the real Langfuse tracing pipeline and assert on exported spans without OTLP network calls or stubbing internal APIs.

Langfuse::Testing turns on OtelSetup.test_mode at load time and exposes emitted_langfuse_spans (flush + read finished spans) and reset_langfuse (flush + clear the in-memory buffer between examples). docs/TESTING.md documents RSpec/Minitest setup patterns.

OtelSetup now tracks the active span_exporter, wires the exporter through publish_provider (including rollback/shutdown cleanup), and build_exporter returns InMemorySpanExporter when test_mode is set—so in-memory export survives shutdown / reset! and each provider rebuild gets a fresh buffer. Specs cover publish-race behavior for span_exporter and the test-mode exporter path.

Reviewed by Cursor Bugbot for commit 880b375. Bugbot is set up for automated code reviews on this repo. Configure here.

Alan Marx and others added 2 commits July 31, 2026 13:11
Introduces `require "langfuse/testing"` + `include Langfuse::Testing` as
a supported, public way to assert against emitted spans in tests without
real network calls or reaching into private SDK internals.

- `OtelSetup.test_mode` flag causes `build_exporter` to return a fresh
  `InMemorySpanExporter` on each provider build; survives `shutdown` cycles
- `Langfuse::Testing#emitted_langfuse_spans` — flushes and returns recorded spans
- `Langfuse::Testing#reset_langfuse` — flushes and clears the span buffer
- `docs/TESTING.md` covers RSpec and Minitest integration patterns

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 8d28b91. Configure here.

Comment thread lib/langfuse/otel_setup.rb Outdated
Alan Marx and others added 2 commits July 31, 2026 17:34
@span_exporter was set in build_tracer_provider before the provider won
the setup mutex. On a lost race the losing provider is shut down (stopping
its exporter) while @span_exporter still pointed at it, causing
emitted_langfuse_spans and reset_langfuse to read a stopped exporter.

build_exporter is now called in setup before build_tracer_provider, and
@span_exporter is assigned inside publish_provider under the mutex only
when the provider wins. rollback_provider also clears it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Alan-Marx
Alan-Marx force-pushed the feat/test-exporter-config branch from c56a700 to 880b375 Compare August 1, 2026 00:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants