Repository navigation
Fix unmatched baseline entries in time linear in the baseline size - #6977
Merged
Merged
Conversation
Fixing unmatched ignoreErrors entries and too high counts took time quadratic in the number of entries: 18.7 s for 8,000 unmatched entries of one path. - The entries are indexed by path and by the value of the first matched field, so findEntries() no longer goes through every entry of the path for each error. - Removing a NEON entry after the first one of a block list, and lowering a count, are edits of the tokens of the entry. Printing the whole ignoreErrors list again for each error with the format-preserving printer was linear in its length. The edit covers only the lines of the entry, so neighbouring entries are removed in the same round instead of conflicting. 8,000 unmatched entries now take 1.8 s, 20,000 take 4.0 s. The fixed config files are the same as before. Amp-Thread-ID: https://ampcode.com/threads/T-01a1b1c2-4005-7120-a2fa-17285a96ea5b
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixing unmatched
ignoreErrorsentries and too high counts (added in 538b35f) took time quadratic in the number of entries. On WooCommerce's 67k-line baseline this showed up after analysis; with 20,000 unmatched entries of one path it took 252 s against 3 s on 2.3.0.Two parts of the fixer were linear per fixed error:
findEntries()went through every entry with the same path for each error. The entries are now indexed by path and by the value of the first matched field (message,rawMessage,identifier) an error must match, the same matching then runs only on that bucket. This is the same idea as phpstan-fixer's change that computes the line map of an edited baseline once instead of once per fix.ignoreErrorslist again with the format-preserving printer (Printer::diff()scanned the unchanged prefix and suffix of the list). Removing a whole entry after the first one of a block list, and lowering a count, are now edits of the tokens of the entry: the lines from the one after the previous entry through the line break after the entry, which is exactly what the printer printed. Everything else, like the first entry, list items or inline lists, still goes through the printer.The removal edit covers only the lines of the entry, so neighbouring stale entries no longer conflict and are removed in the same round. The
fix-baselinee2e round count changes for that reason; its expected files do not. The first entry still goes through the printer, its hunk overlaps the second entry's, so all entries are never removed in one round (that would leaveignoreErrors:null) - there's a test for it.One trivial file with N unmatched entries of one path:
Running
--fixto the end on a mixed baseline (generated entries with a raised count, 300 stalemessageandrawMessageentries, comments above and after entries, amessageslist, an entry without a path), as LF and as CRLF, gives byte-identical files before and after, in 2 rounds instead of 8. Thefix,fix-baselineandfix-ignored-entriese2e jobs pass locally.