Skip to content

THRIFT-6563: Decode unversioned Ruby binary message names as UTF-8 - #4198

Merged
kpumuk merged 1 commit into
apache:masterfrom
kpumuk:rb-old-format-message-name
Nov 18, 2026
Merged

kpumuk merged 1 commit into
apache:masterfrom
kpumuk:rb-old-format-message-name

Conversation

@kpumuk

@kpumuk kpumuk commented Nov 17, 2026

Copy link
Copy Markdown
Member

In the old (unversioned) binary message format, accepted when strict_read is false, Thrift::BinaryProtocol#read_message_begin and the native Thrift::BinaryProtocolAccelerated#read_message_begin returned the method name as raw ASCII-8BIT bytes. The versioned format reads the same name with read_string, which returns UTF-8. A non-ASCII name sent in the old format therefore didn't compare equal to the same name as a string literal, so name-based dispatch or comparison failed.

Both implementations now decode the unversioned name the same way their read_string does: Bytes.convert_to_string in Ruby and convert_to_string in the C extension. Wire bytes and the strict_read rejection of unversioned headers don't change. A new shared binary-protocol spec reads a UTF-8 name with and without a version header for both protocol classes.

Reproduction, run from lib/rb with the native extension built (bundle exec ruby -Ilib -Iext old-format-name.rb):

require "thrift"

protocols = [Thrift::BinaryProtocol]
protocols << Thrift::BinaryProtocolAccelerated if defined?(Thrift::BinaryProtocolAccelerated)

protocols.each do |protocol_class|
  transport = Thrift::MemoryBufferTransport.new
  protocol_class.new(transport, false, false).write_message_begin("héllo", Thrift::MessageTypes::CALL, 1)
  name, = protocol_class.new(transport, false, false).read_message_begin
  puts "#{protocol_class}: #{name.inspect} #{name.encoding} #{name == "héllo"}"
end

Before (master 50242d7ea1, Ruby 4.0.9):

Thrift::BinaryProtocol: "h\xC3\xA9llo" ASCII-8BIT false
Thrift::BinaryProtocolAccelerated: "h\xC3\xA9llo" ASCII-8BIT false

After:

Thrift::BinaryProtocol: "héllo" UTF-8 true
Thrift::BinaryProtocolAccelerated: "héllo" UTF-8 true

The pure-Ruby BinaryProtocol gives the same results without the native extension.

  • Did you create an Apache Jira ticket? (THRIFT-6563)
  • If a ticket exists: Does your pull request title follow the pattern "THRIFT-NNNN: describe my issue"?
  • Did you squash your changes to a single commit? (not required, but preferred)
  • Did you do your best to avoid breaking changes? If one was needed, did you label the Jira ticket with "Breaking-Change"?
  • CI required for this change.

Client: rb

Generated-By: amp/0.0.1794916837-g564d50 (model: anthropic/claude-opus-5-5; operator: kpumuk@apache.org)
Copilot AI balanced review requested due to automatic review settings November 17, 2026 23:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@mergeable mergeable Bot added the ruby Pull requests that update Ruby code label Nov 17, 2026
@kpumuk
kpumuk merged commit 9f1e81e into apache:master Nov 18, 2026
93 checks passed
@kpumuk
kpumuk deleted the rb-old-format-message-name branch November 18, 2026 02:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ruby Pull requests that update Ruby code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants