Skip to content

fix: review fixes 01-1.3 - #4

Merged
daggerok merged 1 commit into
masterfrom
review/2026-10-perf-fixes
Oct 12, 2026
Merged

daggerok merged 1 commit into
masterfrom
review/2026-10-perf-fixes

Conversation

@daggerok

Copy link
Copy Markdown
Owner

Review fixes for the csv repo from the 2026-10-11 review plan.

01-1.3 CSV parser splits by newline before quotes (applied)

Commit fcca090 - fix(csv): 01-1.3 parse quoted newlines

What changed in src/main.tsx:

  • New parseCSVText(text) does one RFC 4180 quote-aware pass over the whole text: quoted fields may contain commas, "" and newlines; \r outside quotes is dropped
  • extractValidTableData now uses parseCSVText(rawText).filter(row => row.some(cell => cell !== '')) instead of rawText.split(/\r?\n/) + parseCSVRow
  • parseCSVRow had no other callers (checked with grep), so it is removed per the plan; the header doc comment that named it is updated

Why: a quoted field with a newline inside (Fidelity Activity/History descriptions) was split into two broken rows, and the "most frequent row length" heuristic silently dropped them. In the plan example the old parser also stopped recording at the broken row, so every row after it was lost too

How verified

Throwaway Bun script outside the repo: old extractValidTableData taken from origin/master, new one from this branch, both run on the same inputs (bun verify.ts):

### plan example (quoted newline)
old: [["Symbol","Description","Qty"]]
new: [["Symbol","Description","Qty"],["AAPL","Apple, Inc.\nsecond line","10"],["MSFT","Microsoft","5"]]
### CRLF file + trailing CRLF
old: [["Symbol","Description","Qty"],["AAPL","Apple","10"],["MSFT","Microsoft","5"]]
new: [["Symbol","Description","Qty"],["AAPL","Apple","10"],["MSFT","Microsoft","5"]]
### empty + blank trailing lines
old: [["Symbol","Description","Qty"],["AAPL","Apple","10"],["MSFT","Microsoft","5"]]
new: [["Symbol","Description","Qty"],["AAPL","Apple","10"],["MSFT","Microsoft","5"]]
### quoted quotes
old: [["Symbol","Description","Qty"],["AAPL","say \"hi\", ok","10"],["MSFT","","5"]]
new: [["Symbol","Description","Qty"],["AAPL","say \"hi\", ok","10"],["MSFT","","5"]]
### CRLF inside quoted field
old: [["Symbol","Description","Qty"]]
new: [["Symbol","Description","Qty"],["AAPL","a\r\nb","10"],["MSFT","c","5"]]
### preamble + postamble
old: [["Symbol","Description","Qty"],["AAPL","Apple","10"],["MSFT","Microsoft","5"]]
new: [["Symbol","Description","Qty"],["AAPL","Apple","10"],["MSFT","Microsoft","5"]]

Inputs the old parser already handled give identical output; the two quoted-newline inputs are now parsed into 3 rows of 3 columns

Gate run locally:

  • bun install -E - 513 packages installed
  • bun run build - built in 1.84s, no errors
  • git diff --check - clean

Known behavior differences (by design of the plan's parser, not tested in the browser):

  • An unterminated quote now swallows the rest of the file into one field (before it only damaged one line)
  • A line of only commas (,,) is now dropped as an empty row (before it was kept as a row of empty cells)
  • A CRLF inside a quoted field is kept as \r\n in the cell value

02-2.4 validate saved state shape (NOT applied)

Stopped per the plan rules: the anchor const raw = localStorage.getItem(LOCALSTORAGE_DATA_KEY); matches twice in src/main.tsx (in NEEDS_BOOT_RESTORE and in loadPersistedDataSets). Also the plan's check (d.rows, d.headers) does not match the real type interface DataSet { fileName: string; data: string[][]; } and would reject every valid saved dataset. Left for a follow-up once the plan is corrected

Manual checks still needed

  • bun run serve, load a real Fidelity Activity/History CSV with multi-line descriptions and check that no rows are missing and the multi-line cell renders
  • Load a regular CRLF CSV and a CSV with preamble/postamble text and check the table looks the same as before
  • Remember Data on: reload the page and check persisted datasets restore as before

Split the whole text into rows in one RFC 4180 quote-aware pass instead of splitting by newline before parsing quotes, so quoted fields with embedded newlines no longer break rows. Remove the now unused parseCSVRow.
@daggerok
daggerok merged commit 4346e85 into master Oct 12, 2026
3 checks passed
@daggerok
daggerok deleted the review/2026-10-perf-fixes branch October 12, 2026 22:48

This branch was successfully deployed

2 active deployments
github-pages — fcca090c Deployed Oct 12, 2026 by daggerok via deploy #35
ci — fcca090c Deployed Oct 12, 2026 by daggerok via npm-check-updates #39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant