Skip to content

Return a plain Hash from OutputBuilder for faster serialization - #408

Open
a5-stable wants to merge 2 commits into
ruby-grape:masterfrom
a5-stable:output_builder-only-for-building
Open

a5-stable wants to merge 2 commits into
ruby-grape:masterfrom
a5-stable:output_builder-only-for-building

Conversation

@a5-stable

@a5-stable a5-stable commented Oct 25, 2026 •

Copy link
Copy Markdown

Summary

This change makes serializable_hash return the Hash or Array that OutputBuilder built, instead of the OutputBuilder itself.

Since Rails 8.1, ActiveSupport serializes a plain Hash in C, but an OutputBuilder is not a Hash, so it falls back to the slower Ruby path.
Also, because OutputBuilder inherits from SimpleDelegator, every Array reference in it goes through const_missing.

OutputBuilder was introduced in #204 only to build the Hash or Array, so I think that there is no reason to use it for serialization.

Benchmark results:

ruby 4.0.5 (2026-05-20 revision 64336ffd0e) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
               1.1.0     3.909k i/100ms
             patched     6.348k i/100ms
Calculating -------------------------------------
               1.1.0     41.693k (±10.9%) i/s   (23.98 μs/i) -    211.086k in   5.062842s
             patched     64.694k (± 4.2%) i/s   (15.46 μs/i) -    323.748k in   5.004277s

Comparison:
patched:    64694.3 i/s
  1.1.0:    41693.2 i/s - 1.55x  slower
Benchmark
require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"
  gem "grape-entity", "1.1.0"
  gem "benchmark-ips"
end

# 1.1.0's OutputBuilder cannot drop its SimpleDelegator superclass, so this is a copy of the patched one.
class PlainOutputBuilder
  def initialize(entity)
    @entity = entity
    @output_hash = {}
    @output_collection = []
  end

  def add(exposure, result)
    if result.is_a?(Array) && exposure.for_merge
      @output_collection << result
    elsif exposure.for_merge
      return unless result

      @output_hash.merge! result, &merge_strategy(exposure.for_merge)
    else
      @output_hash[exposure.key(@entity)] = result
    end
  end

  def output
    if @output_collection.empty?
      output = @output_hash
    else
      output = @output_collection
      output << @output_hash unless @output_hash.empty?
      output.flatten!
    end
    output
  end

  private

  def merge_strategy(for_merge)
    if for_merge.respond_to? :call
      for_merge
    else
      -> {}
    end
  end
end

module PlainOutput
  def map_entity_exposures(entity, options)
    return super unless $patched

    new_options = nesting_options_for(options)
    output = PlainOutputBuilder.new(entity)

    normalized_exposures(entity, new_options).each_with_object(output) do |exposure, out|
      exposure.with_attr_path(entity, new_options) do
        result = yield(exposure, new_options)
        out.add(exposure, result)
      end
    end.output
  end
end
Grape::Entity::Exposure::NestingExposure.prepend(PlainOutput)

class UserEntity < Grape::Entity
  expose :id, :name
end

User = Struct.new(:id, :name)
users = Array.new(1000) { |i| User.new(i, "user #{i}") }

Benchmark.ips do |x|
  x.report("1.1.0") { $patched = false; UserEntity.represent(users).to_json }
  x.report("patched") { $patched = true; UserEntity.represent(users).to_json }
  x.compare!
end

@numbata numbata left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution, @a5-stable!

A few things before this can merge:

  • CI is red. spec/grape_entity/entity_spec.rb:1705 still expects the result to be an OutputBuilder. It's a plain Hash now, so that line can go. The assertions after it already cover what matters.

  • Please add a regression spec for the fix above. Use one merge: true exposure returning an array plus one plain exposure, and check that serializable_hash is stable when called repeatedly. That keeps a future lazy wrapper from bringing the bug back.

  • Please add a CHANGELOG entry. serializable_hash / represent(..., serializable: true) now return a real Hash/Array. That changes Hash === result / Array === result (and case/when) for downstream code, so users should hear about it.

I also left an inline comment on output.

private

# If output_collection contains at least one element we have to represent the output as a collection
def output

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

output is public now, but it changes @output_collection every time it runs. If it's called twice when there's a merged array and a non-empty hash, @output_hash gets appended a second time.
As output = @output_collection doesn't copy the array. It makes a second name for the same array object, so everything done through output happens to @output_collection too. That's the same growing-result bug the old delegator had. One fix is to build the result without changing state:

 def output
    return @output_hash if @output_collection.empty?

    [*@output_collection, *(@output_hash unless @output_hash.empty?)].flatten
  end

The other option is to rename it to build!/finalize so it's clear it should only be called once. Renaming it would also stop the output local variable from shadowing the method.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@numbata
Thanks!
I fixed output so it no longer mutates the builder's state, by @output_collection.dup:

def output
  return @output_hash if @output_collection.empty?

  result = @output_collection.dup
  result << @output_hash unless @output_hash.empty?
  result.flatten
end

[*@output_collection, *(@output_hash unless @output_hash.empty?)].flatten would spread the hash into key/value pairs, so I could not use:

[*[%w[a b]], *{ name: 'John' }].flatten
# => ["a", "b", :name, "John"] (expected ["a", "b", { name: "John" }])

I also added the spec about this behaviour.

@a5-stable
a5-stable force-pushed the output_builder-only-for-building branch 2 times, most recently from 0049cf9 to 7b69374 Compare October 27, 2026 10:29
@a5-stable
a5-stable force-pushed the output_builder-only-for-building branch from 7b69374 to 3cfa434 Compare October 27, 2026 10:38
@github-actions

Copy link
Copy Markdown

Danger Report

Errors

  • Added changelog entries outside the unreleased section in CHANGELOG.md at lines 9.

View run

@a5-stable

Copy link
Copy Markdown
Author

@numbata
Thanks for the review 🙇‍♀️

  • Updated spec/grape_entity/entity_spec.rb:1705 (removed the OutputBuilder expectation).
  • Added a regression spec for the fix above: serializable_hash called repeatedly with a merge: true array exposure plus a plain exposure. I also added a unit spec for OutputBuilder#output in spec/grape_entity/exposure/nesting_exposure/output_builder_spec.rb.
  • Added a CHANGELOG entry.

I also fixed the CI failure caused by the latest RuboCop (the new Style/DirectiveScope cop). The changes are comment-only.

@a5-stable
a5-stable requested a review from numbata October 27, 2026 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants