Skip to content

Support RubyLLM 2 while preserving RubyLLM 1 compatibility - #161

Open
crmne wants to merge 1 commit into
patvice:mainfrom
crmne:ruby-llm-2-compatibility
Open

crmne wants to merge 1 commit into
patvice:mainfrom
crmne:ruby-llm-2-compatibility

Conversation

@crmne

@crmne crmne commented Sep 22, 2026 •

Copy link
Copy Markdown

RubyLLM 2 removes RubyLLM::Content and reads tool schemas through parameters_schema, so loading and using MCP tools fails even if the dependency constraint is relaxed. This supports RubyLLM >= 1.9 and < 3 while preserving the RubyLLM 1.x content API.

  • Expose the normalized server schema through both schema APIs.
  • Convert tool results into RubyLLM 2 strings and attachment arrays. Preserve every text, image, audio, and resource block in multipart results, including text mixed with attachments.
  • Initialize attachments with decoded IO sources so RubyLLM can read their bytes, while preserving MCP's encoded data and MIME type.
  • Adapt prompt/resource messages and sampling responses to both message APIs. Keep validated structured output authoritative and tool errors intact.
  • Add a standalone compatibility spec and CI matrix for the minimum 1.9 series, current 1.x, and 2.x. The existing full suite continues to run on 1.x.

Validation on Ruby 4.0.7:

  • Compatibility spec: 12 examples, 0 failures on RubyLLM 1.9.2, 1.16.0, and 2.0.0.
  • Full existing suite including the new compatibility spec: 1,911 examples, 0 failures on RubyLLM 1.16.0.
  • RuboCop: all changed Ruby files pass.

This PR is limited to RubyLLM compatibility; it does not move MCP into RubyLLM core or change OAuth and transport policy.

@jordan-brough

jordan-brough commented Sep 22, 2026 •

Copy link
Copy Markdown

@crmne here's a PR with some proposed updates to your changes here, targeting this branch: crmne#1

@jordan-brough

jordan-brough commented Sep 22, 2026 •

Copy link
Copy Markdown

Another PR, stacked on top of the other one that I mentioned already: crmne#2

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, some things I noticed while working on the follow-ups in crmne#1.

With the fixture tool fix from that PR the regular suite now loads on RubyLLM 2, but about 375 examples still fail there. Most are in chat_spec (gemini-2.0-flash is gone from the 2.0 registry, bare claude-sonnet-4 resolves to Vertex AI, and OpenAI now calls /v1/responses so the recorded cassettes don't match), about 57 are in tool_spec asserting on MCP::Content where 2.0 tools now return a String, and the rest are smaller groups in the elicitation, sample, resource, and agents specs. Happy to share the full breakdown.

Two different ways I can see to get the suite green on both versions:

  1. pin providers explicitly, skip the registry lookup, and pin OpenAI to chat completions on 2.0 so the existing cassettes replay on both (the approach in Support RubyLLM 2.0 #159)
  2. or, record cassettes that hold both versions' requests, since VCR here matches on method and URL only. I have a set recorded against 2.0's defaults on a 2.0-only branch if useful, though they'd need a 1.x pass added.

It seems worth doing something here because the sampling bug fixed in crmne#1 only shows up in the regular suite, which CI currently never runs on 2.0.

@NielsKSchjoedt

Copy link
Copy Markdown

There is still at least one thing that needs handling in this PR: #159 (comment)

@NielsKSchjoedt

Copy link
Copy Markdown

Can I do anything to help getting this moving @crmne ?

@patvice

patvice commented Oct 8, 2026

Copy link
Copy Markdown
Owner

Hey everyone! Apologies on the slowest for this MR. I'm going to get this reviewed/merged today and get a new version out that will support RubyLLM 2.0 today!

@patvice

patvice commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Actually looks like RubyLLM is shipping with MCP implementation now in 2.1. Great looking API (wish I thought of that!) and with the direction of the new MCP spec implementation has simplified a lot.

So moving forward, I believe the plan is get our a compatibility change to support 2.0. Then after that deprecate the gem, and suggest new version of RubyLLM use the native gems support. I'll work on getting that all shipped in the next few days and slowly wind down the gem to be in a frozen state.

@NielsKSchjoedt

Copy link
Copy Markdown

Thanks for the update and for all the work on the gem! For what it's worth from a downstream user: we've just moved our app to RubyLLM 2.1's built-in RubyLLM::MCP (the two can't be loaded side by side, since both define RubyLLM::MCP), and the switch was smooth. The deprecation plan sounds right to us.

@patvice

patvice commented Oct 8, 2026

Copy link
Copy Markdown
Owner

That's great to hear, thanks @NielsKSchjoedt

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants