Use pre-boxed objects for common attribute values

[Artfunkel]

• Use pre-boxed objects for common attribute values, to avoid wasteful heap allocation
• Reduce List and Linq allocations during parsing
• Remove obsolete Odds and Evens methods

After a short amount of scrolling around my map, without these changes I see over 200,000 boxed booleans in the heap. The allocation count continues to climb as new tiles are parsed.

With these changes, there are exactly two boxed boolean allocations: one for true and one for false. The boxed numeric values (0 and 1) are helpful too, but it's bool which provides the biggest win.

[Copilot]

Pull request overview

This PR focuses on reducing allocations during vector tile parsing by pre-sizing collections and reusing pre-boxed objects for common attribute values, while also removing the previously used odd/even tag-splitting helpers.

Changes:

• Pre-size parsed layer/feature collections to reduce List<T> growth allocations.
• Replace LINQ-based tag splitting with a single indexed loop and introduce pre-boxed objects for common attribute values (notably bool).
• Remove the obsolete GetOdds/GetEvens extension methods and their associated tests.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
tests/IEnumerableExtensionTests.cs	Removes tests for the deleted GetOdds/GetEvens extension methods.
src/VectorTileParser.cs	Preallocates layer list and feature list capacity based on proto counts.
src/ExtensionMethods/IEnumerableExtensions.cs	Removes the obsolete public GetOdds/GetEvens LINQ helpers.
src/AttributesParser.cs	Eliminates LINQ allocations, parses tags via indexed loop, and uses pre-boxed objects for common values.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[bertt]

can you add a benchmark.net test that proves this new method is better?

[Artfunkel]

It appears that I can't. The library doesn't seem to detect boxing as a memory allocation, and your current sample data doesn't contain any boolean values anyway.

What I can do is use a tile from my data source and then take screenshots of the Visual Studio heap diagnostics window.

Before:

After:

(Protobuf has its own boxed boolean too, so there are three in total.)

[bertt]

should be possible to create a vector tile with booleans (in the benchmark setup) and let the benchmark test read it, see what benchmark reports about the heap/stack allocations.

[Artfunkel]

This is what I did. It reports that 96 bytes were allocated both times, even though we can see hundreds of boxed booleans.

[bertt]

ah ok, when inspecting the heap diagnostics window, does it run in release mode (as is needed for benchmark test)? Maybe some optimalization is going on.

[Artfunkel]

You can see the exact same allocations in both release and debug builds.

[Artfunkel]

Ping!

[bertt]

I did a quick test with a benchmark test (n=1)

result:

current:

Method	Mean	Error	StdDev	Allocated
ParseBooleanAttributeTile	994.8 us	18.43 us	27.58 us	703.65 KB

before:

Method	Mean	Error	StdDev	Allocated
ParseBooleanAttributeTile	2.876 ms	0.0540 ms	0.0451 ms	1.4 MB

So allocated memory is improved from 1.4 MB to 703 KB, also mean parsing time is 3 times faster.

Can you add such a test and recheck results?

Benchmark code I've used:

namespace mapbox.vector.tile.benchmark;

/// <summary>
/// Benchmarks parsing of a tile with many boolean attributes, to measure
/// the impact of the pre-boxed Boxes.Boolean_True/False optimisation.
/// </summary>
[MemoryDiagnoser]
public class BooleanAttributesBenchmark
{
    private MemoryStream _tileStream = null!;

    [GlobalSetup]
    public void Setup()
    {
        const int featureCount = 1000;
        const uint extent = 4096;

        var attributes = new List<KeyValuePair<string, object>>
        {
            new("is_active",    (object)true),
            new("is_visible",   (object)true),
            new("is_selected",  (object)false),
            new("has_children", (object)false),
            new("is_root",      (object)true),
        };

        var features = new List<VectorTileFeature>(featureCount);
        for (var i = 0; i < featureCount; i++)
        {
            var geometry = new List<ArraySegment<Coordinate>>
            {
                new(new[] { new Coordinate { X = 0, Y = 0 } }),
            };
            features.Add(new VectorTileFeature(
                id: i.ToString(),
                geometry: geometry,
                attributes: attributes,
                geometryType: Tile.GeomType.Point,
                extent: extent));
        }

        var layer = new VectorTileLayer("bool_layer", 2, extent)
        {
            VectorTileFeatures = features,
        };

        var buffer = new MemoryStream();
        VectorTileEncoder.Encode(new List<VectorTileLayer> { layer }, buffer);
        _tileStream = new MemoryStream(buffer.ToArray());
    }

    [IterationSetup]
    public void ResetStream() => _tileStream.Position = 0;

    [Benchmark]
    public List<VectorTileLayer> ParseBooleanAttributeTile()
    {
        _tileStream.Position = 0;
        return VectorTileParser.Parse(_tileStream);
    }
}

[Artfunkel]

I added the test you posted, but I increased the number of attributes to 100,000 because the results included a warning that the test completed too quickly.

AttributeParser-ReduceAllocations

Mean	Error	StdDev	Gen0	Gen1	Gen2	Allocated
90.54 ms	1.732 ms	1.779 ms	6000.0000	5000.0000	1000.0000	67.9 MB

master

Mean	Error	StdDev	Gen0	Gen1	Gen2	Allocated
198.0 ms	3.78 ms	4.64 ms	13000.0000	10000.0000	2000.0000	139.33 MB

[bertt]

thanks it's in release 5.3.1