Skip to content

RubyLLM 2 follow-ups: generation, sampling, and prompt argument fixes - #1

Open
jordan-brough wants to merge 5 commits into
crmne:ruby-llm-2-compatibilityfrom
jordan-brough:pr161-generation-and-sampling-fixes
Open

jordan-brough wants to merge 5 commits into
crmne:ruby-llm-2-compatibilityfrom
jordan-brough:pr161-generation-and-sampling-fixes

Conversation

@jordan-brough

@jordan-brough jordan-brough commented Sep 22, 2026 •

Copy link
Copy Markdown

Five small follow-ups to patvice#161, one commit each. All of them work on RubyLLM 1.x and 2.x.

1. Prompts and sampling that end with assistant text now call the model

On RubyLLM 2, Chat#complete treats a conversation that already ends with an assistant message as finished and returns that message. An MCP prompt or sampling request that ends with a partial assistant answer therefore came back unchanged, and the provider was never called.

The fix is a small RubyLLM::MCP.generate_response(chat) helper. On RubyLLM 2 it calls Chat#generate for one new response, then Chat#complete to finish any tool calls. RubyLLM 1 has no Chat#generate, so there it just calls Chat#complete as before. Prompt#ask, Sample#chat, and the default sampling handler use it.

2. Sampling responses report the real stop reason

On RubyLLM 2, every sampling response said stopReason: "endTurn", even when the model hit its token limit. The code looked for stop_reason, but RubyLLM 2 messages carry finish_reason.

Now a finish_reason is mapped to the MCP names (endTurn, maxTokens, toolUse, contentFilter; anything else is camelized, which MCP allows). Messages without one still go through the existing stop_reason handling and default.

3. Required prompt arguments are validated correctly

The check was inverted: supplying a required argument raised, and omitting it passed. The error was also built with a positional string, but BaseError takes message:, so the failure surfaced as an unrelated ArgumentError. It now raises PromptArgumentError only when a required argument is missing, with String or Symbol keys accepted.

4. Sampling requests build messages RubyLLM 2 accepts

Sample#create_message and the sampling handler passed an MCP::Content as message content. RubyLLM 2 rejects that with ArgumentError, so every sampling request failed before reaching the model. They now build messages through Content#message_options, the same way Prompt already does.

5. The regular spec suite loads under RubyLLM 2

spec_helper.rb loads spec/support/simple_multiply_tool.rb, which used the RubyLLM 1 param syntax that RubyLLM 2 removed. So bundle exec rspec could not start on RubyLLM 2, which is why CI only runs the standalone compatibility spec there. The tool now declares its parameters with whichever method the installed RubyLLM provides.

Loading is not the same as passing. native/messages_spec.rb now passes in full on RubyLLM 2 (94 examples). Other files still fail there for reasons outside this PR: VCR cassettes recorded against RubyLLM 1 request shapes, model ids the RubyLLM 2 registry no longer resolves, and assertions that expect MCP::Content where RubyLLM 2 tools return a String.

Verification

Bundled with RUBY_LLM_VERSION="~> 1.0" and "~> 2.0" in turn. The new specs (generate_response_spec.rb, prompt_arguments_spec.rb) and the compatibility spec pass on both. The compatibility spec gains examples for changes 1, 2, 4, and 5 that assert the behavior of whichever RubyLLM is installed. On RubyLLM 1 the full suite was also run against the fixture servers with no version-related failures.

RubyLLM 2's Chat#complete returns the existing assistant message when the
conversation already ends with one, so an MCP prompt or sampling request
that ends with assistant text echoed that text back without calling the
provider. Add RubyLLM::MCP.generate_response, which calls Chat#generate
when the installed RubyLLM defines it and then Chat#complete, and use it
from Prompt#ask, Sample#chat, and SamplingActions#default_chat_completion.
RubyLLM 1 has no Chat#generate and its Chat#complete always generates, so
only Chat#complete is called there.
RubyLLM 2 messages carry a finish_reason (:stop, :max_tokens, :tool_calls,
:content_filter) instead of a stop_reason, so sampling responses always
reported endTurn even when the model was cut off by its token limit. Map
finish_reason to the MCP stop reason names when the message provides one,
camelizing unknown values as before, and keep the existing stop_reason
handling and the endTurn default for RubyLLM 1 messages.
Prompt#include raised when a required argument was supplied and let a
missing one through, and it built the error with a positional string that
BaseError#initialize rejects, so any required argument produced an
ArgumentError. Raise only when a required argument is absent, accepting
String or Symbol keys, and construct the error with its message keyword.
…on RubyLLM 2

Sample#create_message and the SamplingActions message builders passed an
MCP::Content object as the content of a RubyLLM::Message. RubyLLM 2 only
accepts a String there, so every sampling request raised an ArgumentError
before the model was called. Build these messages from
Content#message_options, as Prompt and Resource already do, which yields
the Content itself on RubyLLM 1 and separate text and attachments on
RubyLLM 2.
…nstalled

RubyLLM 1 declares tool parameters with Tool.param(desc:) and RubyLLM 2
with Tool.parameter(description:). spec_helper loads SimpleMultiplyTool,
so the regular spec suite could not start under RubyLLM 2. Pick the method
the installed RubyLLM provides so the suite loads on both.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant