Skip to content

Keep is_buyer_maker aligned with its trade when TradesData sorts and de-duplicates - #9

Merged
terbed merged 1 commit into
terbed:mainfrom
mikgrasis:fix/is-buyer-maker-alignment
Oct 25, 2026
Merged

terbed merged 1 commit into
terbed:mainfrom
mikgrasis:fix/is-buyer-maker-alignment

Conversation

@mikgrasis

Copy link
Copy Markdown
Contributor

Summary

  • TradesData._sort_trades sorts price, time, amount and id by trade id (and then by timestamp), but is_buyer_maker is held as a separate array and is never re-ordered. _merge_trades then pairs the sorted columns with the unsorted side array, so for a trades file that is not already in id order every side ends up on the wrong trade. Prices, times and quantities stay correct, so the damage is easy to miss.
  • This happens with real Binance USD-M monthly trade dumps, some of which are not in id order. For BTCUSDT 2023-10 the resulting side agrees with kline taker-buy volume only at chance level (about 0.0 correlation of 15-minute signed delta), against 0.9999 for months that happen to be in order.
  • Fix: when preprocess=True, is_buyer_maker is stored as a column of the internal data frame, so sorting and de-duplicating re-order it together with its trade, and _merge_trades reads it back from the sorted frame. A length check raises a ValueError if is_buyer_maker does not match the number of trades. Inputs that are already sorted behave exactly as before.

Related Issues

  • None found.

Testing

  • NUMBA_DISABLE_JIT=1 pytest -q
  • Other: the new tests in tests/bars/test_trades_data_side_alignment.py were run directly (pytest was not available in my environment). The shuffled-input test fails on main and passes with this change; the sorted-input test passes on both. I also fed the first 3 million rows of a real, out-of-order BTCUSDT trades dump through TradesData and compared the result with the same rows pre-sorted by (time, id): timestamps, prices and sides are identical with this change.

Documentation

  • Added/updated docstrings
  • Updated README or other docs

Checklist

  • Tests added or updated
  • Documentation added or updated
  • Linting and type checks pass

@terbed
terbed self-requested a review October 25, 2026 21:52

@terbed terbed left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, thanks for the fix!

@terbed
terbed merged commit 8be0908 into terbed:main Oct 25, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants