Preserve DateTimeOffset's offset across BSON round-trip - #141
Merged
Conversation
BsonValue.FromDateTimeOffset / BsonSpanWriter.WriteDateTimeOffset stored a DateTimeOffset as a plain BSON DateTime (UTC millisecond timestamp only), so the offset was silently dropped on every write and read back as +00:00. Adds a distinct BsonType.DateTimeOffset wire tag (10 bytes: the existing 8-byte UTC timestamp plus a 2-byte offset-in-minutes trailer) so the offset now survives the round-trip, while every place that branches on BsonType.DateTime (skip-length, BsonValue accessors/equality/comparison, BLQL predicate/index-key building, projection, schema generation, the source-generated entity readers) gets a matching arm - generally reusing the same instant-based logic, so index ordering and range queries are unaffected. Backward compatible: a DateTimeOffset field written before this change is still tagged BsonType.DateTime on disk (offset already unrecoverable, not something a reader can fix retroactively) and keeps decoding exactly as it does today (Offset=0) - only newly-written values gain a correct offset. Fixes #140 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
mrdevrobot
had a problem deploying
to
Production
September 3, 2026 19:56 — with 
GitHub Actions
Failure
GitHub Actions
Failure
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
BsonValue.ReadFrom currently turns wire BsonType.DateTime values into BsonType.DateTimeOffset in-memory, which can incorrectly change persisted type/layout on rewrite and misrepresent plain DateTime fields.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR extends BLite’s BSON layer to preserve a DateTimeOffset’s original UTC offset across write/read round-trips by introducing a dedicated wire tag and updating all relevant readers, writers, query evaluation, indexing/sorting, schema generation, and source-generated entity reads to understand the new representation while keeping ordering/indexing keyed by instant.
Changes:
- Introduces
BsonType.DateTimeOffset (0x14)with a 10-byte payload (8-byte Unix ms + 2-byte offset-minutes) and adds reader/writer support plus skip-length handling. - Updates BLQL/query/projection/expression evaluation and index-key building to handle both
DateTimeandDateTimeOffsettags, comparing/sorting by instant for backward-compatible scans. - Adds/updates tests to validate offset-preserving round-trip and legacy
BsonType.DateTimeback-compat decoding.
File summaries
| File | Description |
|---|---|
| tests/BLite.Tests/BsonValueTests.cs | Updates expectations for BsonValue.FromDateTimeOffset to emit the new DateTimeOffset type. |
| tests/BLite.Tests/BsonSpanReaderWriterTests.cs | Adds round-trip tests for preserving offsets and for legacy-tag back-compat decoding. |
| src/BLite.SourceGenerators/CodeGenerator.cs | Treats ReadDateTimeOffset as needing the wire type in generated readers. |
| src/BLite.Core/Text/TextNormalizer.cs | Formats BsonType.DateTimeOffset using AsDateTimeOffset. |
| src/BLite.Core/Storage/StorageEngine.TimeSeries.cs | Accepts both date tags for TTL timestamp extraction. |
| src/BLite.Core/Storage/StorageEngine.Collections.cs | Includes DateTimeOffset in text-building conversion. |
| src/BLite.Core/Query/BsonProjectionCompiler.cs | Adds DateTimeOffset decoding paths to avoid wire desync. |
| src/BLite.Core/Query/BsonExpressionEvaluator.cs | Extends predicate evaluation/compare logic to decode DateTimeOffset safely. |
| src/BLite.Core/Query/Blql/BsonValueComparer.cs | Compares DateTime/DateTimeOffset by instant across tags; aligns type ordering. |
| src/BLite.Core/Query/Blql/BlqlFilter.cs | Uses instant-based index keys for both date tags. |
| src/BLite.Core/DynamicCollection.cs | Ensures retention/index key building supports DateTimeOffset tag. |
| src/BLite.Core/Collections/DocumentCollection.cs | Reads timestamps from either date tag when extracting ticks. |
| src/BLite.Core/Collections/BsonSchemaGenerator.cs | Maps CLR DateTimeOffset to BsonType.DateTimeOffset in schema generation. |
| src/BLite.Bson/BsonValue.cs | Adds DateTimeOffset value representation/accessors and serialization support. |
| src/BLite.Bson/BsonType.cs | Adds the new BsonType.DateTimeOffset enum value and documentation. |
| src/BLite.Bson/BsonSpanWriter.cs | Writes the new tag and 10-byte layout for DateTimeOffset (element + array). |
| src/BLite.Bson/BsonSpanReader.cs | Adds ReadDateTimeOffset(BsonType) and skip-length support for the new layout. |
Review details
- Files reviewed: 17/17 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…adFrom BsonValue.ReadFrom decoded a wire BsonType.DateTime value via FromDateTimeOffset, which now tags BsonType.DateTimeOffset. That silently flipped the in-memory type for every plain DateTime field (and every legacy DateTimeOffset value still under the old tag), so a later WriteTo would re-emit it in the new 10-byte layout without ever having recovered an offset - an unintended, silent wire-format upgrade on read-then-write. Decode that arm via FromDateTime instead, matching the wire tag actually read and reproducing the original 8-byte format on WriteTo. Found by Copilot's review on #141. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
DateTimeOffsetvalues now round-trip with their original offset instead of being silently normalised to UTC (Offset=0) on read. Root cause and repro are in DateTimeOffset loses its offset on BSON round-trip (WriteDateTimeOffset/ReadDateTimeOffset) #140.BsonType.DateTimeOffsetwire tag (0x14, unassigned by the BSON spec) with a 10-byte layout: the existing 8-byte UTC millisecond timestamp + a 2-byte signed offset-in-minutes trailer.BsonType.DateTimefor a value that could be aDateTimeOffset(skip-length,BsonValueaccessors/equality/comparison, BLQL predicate/index-key building, projection, expression evaluation, schema generation, source-generated entity readers) gets a matchingBsonType.DateTimeOffsetarm. Index/sort ordering is deliberately still keyed by instant, not offset - only how the value displays/reads back changes.Backward compatibility
A
DateTimeOffsetfield written before this change is on disk tagged plainBsonType.DateTime(offset already lost at write time - not something a reader can recover). Reading that legacy tag keeps decoding it exactly as it always has (Offset=0, no crash, no behavior change). Only values written after this fix, through the newWriteDateTimeOffset, get taggedBsonType.DateTimeOffsetand gain a correctly round-tripped offset. Old and new-tagged documents can coexist in the same collection -BsonValueComparercompares them by instant across both tags so range scans keep working during the transition.The generated entity (de)serializers pick this up automatically:
ReadDateTimeOffsetwas added to the source generator's "coerced" read set, so it already passes the wire-readBsonTypeinto the newReadDateTimeOffset(BsonType)overload - no regeneration-breaking API change, existing calls to the parameterlessReadDateTimeOffset()keep their current (legacy,Offset=0) behavior.Testing
BsonSpanReaderWriterTests.WriteAndRead_DateTimeOffset_PreservesOffset- new type tag round-trips both instant and offset.BsonSpanReaderWriterTests.ReadDateTimeOffset_LegacyDateTimeTag_StillDecodesAsOffsetZero- back-compat characterisation: reading a value written under the old plainBsonType.DateTimetag still decodes asOffset=0, unchanged.BsonValueTests.FromDateTimeOffset_RoundTrips_ThroughUnixMsupdated:BsonValue.FromDateTimeOffset(...).Typeis nowBsonType.DateTimeOffset, notBsonType.DateTime(its previous assertion only happened to pass because the test used a zero offset, which is exactly why this went unnoticed).MultiProcessWalSharedMemoryTestsfailures tracked separately in macOS: cross-process WAL writer lock (flock/fcntl) fails on FileStream-derived fd #134 are unaffected by this change - reproduced identically onmainbefore this branch.Fixes #140
🤖 Generated with Claude Code