Changes to support Windows Server 2025 32KiB databases

[takker-hero-se]

Fixes #78

Summary

Windows Server 2025 introduced optional 32KiB ESE database pages for Active Directory (NTDS.dit). This PR fixes two issues that prevented libesedb from parsing such databases.

Changes

1. Page tag count mask (libesedb_page_header.c, +9 lines)

In the 32KiB page format, the available_page_tag field (uint16) uses a new layout:

• Upper 4 bits: ctagReserved (reserved, must be masked out)
• Lower 12 bits: actual number of page tags

Without masking, libesedb reads all 16 bits as the tag count, inflating the value and causing out-of-bounds page tag reads.

Fix: page_header->available_page_tag &= 0x0fff when io_handle->page_size >= 32768.

2. Leaf page validation in B-tree walk (libesedb_page_tree.c, +45 lines)

The backward walk in libesedb_page_tree_get_get_first_leaf_page_number() and forward walk in libesedb_page_tree_get_number_of_leaf_values() did not check whether each page in the leaf chain is actually a leaf page. In 32KiB databases, some pages referenced in the chain may be zeroed or non-leaf pages, causing errors or incorrect record counts.

Fix: Check LIBESEDB_PAGE_FLAG_IS_LEAF before processing each page. Added proper libcerror_error_set() error handling for the libesedb_page_get_flags() calls.

Testing

Tested with real Active Directory NTDS.dit databases:

	8KiB pages (WS2019)	32KiB pages (WS2025)
Tables	14	14
datatable records	7,008	7,029
link_table records	13,904	485
Errors/crashes	None	None

No regression on 8KiB page databases.

References

• Microsoft: Database 32k pages for Active Directory

[takker-hero-se]

CI failure analysis

The 2 failed jobs (macOS x64 gcc python and mingw-w64-gcc-python) are unrelated to this PR's changes.

Both fail with:

import pyesedb
ImportError: No such file or directory

This is a shared library path issue in the CI environment, not an API incompatibility. This PR only modifies internal page parsing logic in libesedb_page_header.c and libesedb_page_tree.c — no public API changes.

Additionally, mingw-w64-gcc-python also fails on the current main branch with the same error, confirming it is a pre-existing CI environment issue.

The remaining 21/23 jobs (including all C compilation and test jobs) pass successfully.

[codecov]

Codecov Report

❌ Patch coverage is 20.00000% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 33.30%. Comparing base (6620ab0) to head (35f8478).
⚠️ Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
libesedb/libesedb_page_tree.c	16.66%	6 Missing and 4 partials ⚠️
libesedb/libesedb_page_header.c	33.33%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main      #79       +/-   ##
===========================================
+ Coverage   22.09%   33.30%   +11.20%     
===========================================
  Files          52       54        +2     
  Lines       12277    12399      +122     
  Branches     2836     2890       +54     
===========================================
+ Hits         2713     4129     +1416     
+ Misses       9140     7145     -1995     
- Partials      424     1125      +701     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[joachimmetz]

Thanks for the proposed changes will take a look when time permits.

[joachimmetz]

No need to address the failing tests. I'll make comparable changes in a separate change list.

In the 32KiB page format, the available_page_tag field (uint16) uses a new layout:

Note that this appears not to be limited to 32k page size.

To reproduce https://github.com/dfirlabs/esedb-specimens on Windows 11 (0x620,300)

libesedb_page_read_file_io_handle: reading page: 14 at offset: 61440 (0x0000f000)
libesedb_page_header_read_data: page header:
00000000: 83 e2 41 96 65 0f 65 0f  bb 01 00 00 00 00 00 00   ..A.e.e. ........
00000010: 0d 00 00 00 00 00 00 00  02 00 00 00 21 07 00 00   ........ ....!...
00000020: 2b 08 23 10 02 28 01 00                            +.#..(..

libesedb_page_header_read_data: XOR checksum                                    : 0x9641e283
libesedb_page_header_read_data: ECC checksum                                    : 0x0f650f65
libesedb_page_header_read_data: database modification time:
00000000: bb 01 00 00 00 00 00 00                            ........

libesedb_page_header_read_data: previous page number                            : 13
libesedb_page_header_read_data: next page number                                : 0
libesedb_page_header_read_data: father data page (FDP) object identifier        : 2
libesedb_page_header_read_data: available data size                             : 1825
libesedb_page_header_read_data: available uncommitted data size                 : 0
libesedb_page_header_read_data: available data offset                           : 2091
libesedb_page_header_read_data: available page tag                              : 35 (0x1023)
libesedb_page_header_read_data: page flags                                      : 0x00012802
        Is leaf
        0x0800 (primary?)
        Is new record format

[joachimmetz]

In 32KiB databases, some pages referenced in the chain may be zeroed or non-leaf pages, causing errors or incorrect record counts.

Interesting do you have some examples that could be shared?

Note that in your changes both libesedb_page_tree_get_get_first_leaf_page_number and libesedb_page_tree_get_number_of_leaf_values now can stop prematurely if there is non-leaf page in the chain.

[joachimmetz]

Closing in favor of 768a047