Skip to content

fix(cpp): preserve quoted CSV line endings in tsfile-cli - #996

Open
ColinLeeo wants to merge 3 commits into
apache:developfrom
ColinLeeo:colin/fix-cli-csv-crlf
Open

ColinLeeo wants to merge 3 commits into
apache:developfrom
ColinLeeo:colin/fix-cli-csv-crlf

Conversation

@ColinLeeo

Copy link
Copy Markdown
Contributor

CSV imports currently turn CRLF inside quoted fields into LF because read_record strips each physical line's trailing carriage return before checking quote state. For example, "line\r\nbreak" loses its \r during import and subsequent CSV round trips.

Check quote state before trimming record boundaries so quoted CRLF, LF, and CR remain field data. Open input files in binary mode and keep redirected stdin/stdout in binary mode on Windows to prevent CRT newline translation.

Regression coverage checks mixed line endings, blank continuation lines, escaped quotes, physical line counts, EOF without a final newline, STRING tags, STRING/TEXT fields, and both write → cat CSV → write and write → export CSV → write.

Validation:

  • Fresh Release build with LZ4 enabled on macOS.
  • CLI and common CSV tests: 200 passed.
  • Eight binary CLI scenarios covering LF/CRLF record boundaries, file/stdin input, and optional final newlines: passed.
  • Scoped Spotless and clang-format 17.0.6 checks: passed.

The Windows-specific stream setup has not been exercised locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant