Skip to content

feat: export tables to CSV, JSON or SQL, and import CSV in one transaction - #20

Merged
Meanski merged 4 commits into
mainfrom
feat/db-import-export
Oct 8, 2026
Merged

Meanski merged 4 commits into
mainfrom
feat/db-import-export

Conversation

@Meanski

@Meanski Meanski commented Oct 8, 2026

Copy link
Copy Markdown
Owner

Item 9 of the parity roadmap.

  • Export a table to CSV, JSON, or SQL (INSERT statements in the connection's dialect). You choose where to save in a native dialog. Exports stop at 200,000 rows and say so if the table is bigger.
  • Import CSV, chosen in a native dialog, into a table:
    • Header names match the table's columns case-insensitively. Unknown or repeated columns, or rows with the wrong number of fields, are refused before anything is written.
    • Empty fields import as NULL.
    • All rows go in one transaction, as batched multi-row INSERTs, so a failure part-way leaves the table untouched.
    • Limits: 50 MB and 200,000 rows.
  • The CSV parser handles quoted fields, embedded commas, quotes and newlines, CRLF line endings and a BOM. The writers quote and escape per format.
  • Main only touches files the user picked in a native dialog, and the renderer never supplies a path.
  • The PostgreSQL and MySQL connectors moved out of database.ts into dbPostgres.ts and dbMysql.ts (shared types in dbTypes.ts), keeping the schema fixes from feat: edit, insert and delete database rows #18 and feat: ER diagram of tables and foreign keys #19.

Tests cover the CSV parser and writers, header mapping, batching, transaction rollback, limits, and the export/import handlers for both dialects. The features page is updated.

Copilot AI balanced review requested due to automatic review settings October 8, 2026 00:12
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Deploying noxed with  Cloudflare Pages  Cloudflare Pages

Latest commit: 60686bb
Status: ✅  Deploy successful!
Preview URL: https://0dab4172.noxed.pages.dev
Branch Preview URL: https://feat-db-import-export.noxed.pages.dev

View logs

@coderabbitai

coderabbitai Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: Meanski/noxed/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 3280d1bf-0e74-4bc6-bc3c-e1e859d1aa84

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

CSV edge cases, IPC validation, table-context loss, non-round-trippable SQL values, and unbounded export memory usage remain unresolved.

4 open findings
What changed in this PR

Adds native-dialog table export to CSV, JSON, or SQL and transactional CSV import for PostgreSQL, MySQL, and MariaDB.

Changes:

  • Adds import/export UI and IPC APIs.
  • Implements CSV parsing, format writers, limits, and transactional batching.
  • Extracts database connectors into dialect-specific modules and adds tests/documentation.
File Description
ResultsTabsBar.tsx Adds table import/export actions.
ExportTableModal.tsx Adds export format selection.
DatabaseExplorer.tsx Integrates import/export workflows.
DatabaseExplorer.test.tsx Tests renderer workflows.
harness.tsx Mocks transfer APIs.
preload/​index.ts Exposes transfer IPC calls.
preload/​index.d.ts Types transfer APIs.
dbTypes.ts Defines shared connector types and batching.
dbTransfer.ts Implements CSV and export serialization.
dbPostgres.ts Extracts PostgreSQL connector and imports.
dbMysql.ts Extracts MySQL/MariaDB connector and imports.
database.ts Adds native file transfer handlers.
dbTransfer.test.ts Tests parsing and writers.
database.test.ts Tests handlers and transactions.
docs/​features.html Documents table transfers.

🧠 Review effort: Balanced


Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

Comment thread src/main/ipc/database.ts Outdated
Comment thread src/main/ipc/database.ts Outdated
Comment thread src/main/ipc/dbTransfer.ts Outdated
Comment thread src/renderer/src/components/Database/DatabaseExplorer.tsx Outdated
Copilot AI balanced review requested due to automatic review settings October 8, 2026 00:20

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Nontransactional MySQL tables and several serialization edge cases can cause partial imports or corrupted exports.

4 open findings
4 resolved since last review
Previously missed (2)

In code that hasn't changed since last review

Medium severity MySQL nontransactional tables break import atomicity

src/​main/​ipc/​dbMysql.ts:103

A MySQL transaction does not provide the promised all-or-nothing import for nontransactional tables such as MyISAM: earlier batches remain inserted when a later batch fails and rollback() is ineffective. Check the target table's storage engine and reject imports unless it supports transactions, or use an import strategy that is atomic for that engine.

Medium severity Case-insensitive column mapping collapses distinct PostgreSQL columns

src/​main/​ipc/​dbTransfer.ts:159

This map silently collapses PostgreSQL columns that differ only by case, which are legal when quoted (for example, Foo and foo). A CSV header can then be routed to whichever column appeared last and import data into the wrong column. Detect case-folding collisions and either prefer an exact-case match or reject an ambiguous header before inserting.

🧠 Review effort: Balanced


Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

Comment thread src/main/ipc/database.ts Outdated
Comment thread src/main/ipc/dbTransfer.ts
Comment thread src/main/ipc/dbTransfer.ts Outdated
Comment thread src/main/ipc/dbTransfer.ts
…d-trip in SQL exports; keep all-empty last CSV rows
Copilot AI balanced review requested due to automatic review settings October 8, 2026 00:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

SQL exports can corrupt supported values, and imports are not atomic for non-transactional MySQL/MariaDB tables.

2 open findings
4 resolved since last review
Previously missed (2)

In code that hasn't changed since last review

Medium severity Prevent non-transactional MySQL imports from claiming atomicity

src/​main/​ipc/​dbMysql.ts:103

A transaction does not guarantee rollback for non-transactional MySQL/MariaDB table engines such as MyISAM (and some Aria configurations). If a later row or batch fails, earlier inserts remain even though rollback() is called, violating the advertised all-or-nothing import behavior. Check the target table engine and refuse imports when transactions are unsupported, or use an import strategy that guarantees atomicity.

Medium severity Preserve PostgreSQL special floating-point values during export

src/​main/​ipc/​dbTransfer.ts:147

This silently exports PostgreSQL's valid NaN, Infinity, and -Infinity floating-point values as SQL NULL, changing the data when the script is replayed. Emit PostgreSQL's accepted quoted special literals instead; MySQL need not receive these values because it does not support storing them as ordinary floating-point values.

🧠 Review effort: Balanced


Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

Comment thread src/main/ipc/dbTransfer.ts
Comment thread src/main/ipc/dbTransfer.ts Outdated
Copilot AI balanced review requested due to automatic review settings October 8, 2026 00:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Nontransactional MySQL tables violate import atomicity, and SQL serialization can corrupt or reject supported values.

0 open findings

2 resolved since last review
Previously missed (4)

In code that hasn't changed since last review

Medium severity Add tests covering multi-batch imports and rollback

src/​main/​ipc/​__tests__/​database.test.ts:264

This test imports only two rows, and the MySQL test below does the same, so neither crosses the 60,000-bind-parameter batch boundary despite the PR description claiming batching coverage. A regression in second-batch placeholder numbering or rollback after an earlier batch would pass. Make the batch limit injectable for tests or construct an import that forces at least two batches and fails the latter.

Medium severity Reject nontransactional engines to preserve atomic imports

src/​main/​ipc/​dbMysql.ts:100

beginTransaction() does not make inserts atomic for nontransactional MySQL/MariaDB engines such as MyISAM or MEMORY; rollback is ignored and earlier batches remain written. This contradicts the all-or-nothing import contract for supported connections. Check the target table engine's transaction capability before inserting and reject unsupported tables (or explicitly narrow the feature contract).

Medium severity Generate SQL literals compatible with NO_BACKSLASH_ESCAPES

src/​main/​ipc/​dbTransfer.ts:146

This escaping depends on MySQL's default backslash mode. With the valid NO_BACKSLASH_ESCAPES SQL mode, an original value such as C:\temp is replayed with two backslashes, so the SQL export silently changes data. Emit mode-independent literals (for example, encoded hex text) or include an explicit compatible SQL mode in the script.

Medium severity Preserve PostgreSQL special values and typed object serialization

src/​main/​ipc/​dbTransfer.ts:177

This generic normalization loses valid PostgreSQL values. pg returns float NaN/Infinity as non-finite numbers, which become NULL, and object-valued types such as interval and point are JSON-stringified by plain() into text those columns cannot parse. The resulting SQL either corrupts data or cannot be replayed; preserve column type metadata and serialize/cast these values to PostgreSQL-compatible text.

🧠 Review effort: Balanced


Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

@Meanski
Meanski merged commit f9605f8 into main Oct 8, 2026
5 checks passed
@Meanski
Meanski deleted the feat/db-import-export branch October 8, 2026 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants