perf(xlsx): speed up simple table conversion - #2546
Open
Dex Hunter (dexhunter) wants to merge 1 commit into
Open
Dex Hunter (dexhunter) wants to merge 1 commit into
Dex Hunter (dexhunter) wants to merge 1 commit into
Conversation
Fixes microsoft#1379 Co-Authored-By: Aiden <aiden@weco.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1379.
XLSX conversion keeps pandas’ HTML rendering but handles its simple tables without building a general HTML DOM. This preserves the existing table formatting, with the original HTML converter handling formatting options, embedded images and content the shortcut cannot handle. The escape calls support both current markdownify and its older one-argument API.
On a generated workbook with two sheets of 10,000 rows and eight mixed-type columns each, the final committed-source replay reduced median
MarkItDown.convert_streamtime from 12.71s to 3.19s (about 75%). Three fresh-process samples ranged from 12.42–12.85s before and 3.13–3.24s after; initialization was outside the timer. This is a representative workload, not the reporter’s workbook. All three timed workloads and ten fidelity/configuration checks matched the original Markdown exactly.Validation on Linux/Python 3.12:
pre-commit run --all-files: passed.hatch test --python 3.12: 1,001 passed, 14 upstream skips.The benchmark used pandas 3.0.6 and markdownify 1.2.3. The Weco run records 11 evaluated candidates plus a separate baseline and two rejected attempts. Candidate 011 (step 13) is the committed choice; its search measurement was 3.21s. The final replay above is separate from that search.
Weco run and candidate history
Developed with AI assistance; the code and tests were reviewed and validated with the checks above.