Skip to content

Run CLI E2E tests from portable command and golden files - #1010

Closed
ColinLeeo wants to merge 1 commit into
apache:developfrom
ColinLeeo:colin/cli-file-e2e
Closed

ColinLeeo wants to merge 1 commit into
apache:developfrom
ColinLeeo:colin/cli-file-e2e

Conversation

@ColinLeeo

Copy link
Copy Markdown
Contributor

CLI end-to-end tests currently call run_cli() inside Google Test and depend on the C++ source/build tree. They cannot be reused directly to validate a separately installed CLI.

Move executable-level scenarios into a standalone cpp/test/tools/e2e/ bundle containing cases.json, fixtures, goldens, and a Python standard-library runner. The runner invokes the specified executable, checks exit status independently from stdout/stderr and exported files, preserves multi-command write/read scenarios, and retains actual outputs, exit codes, diffs, and identity hashes. Existing parser and injected metadata/stream failure tests remain C++ unit tests.

The bundle has 155 cases and 1,076 command invocations, preserves the original 141 goldens, and adds read-back checks for successful writes plus a redirected-stdin round trip. CTest runs the same bundle against the build-tree CLI; CI uploads failure results. Python 3.9+ is required when tools and tests are enabled. Product implementation code is unchanged.

Validation on macOS ARM64:

  • Original 170 tests passed with their existing assertions before migration.
  • CMake Release build of TsFile_Test passed; 42 retained CLI/parser/format/statistics unit tests passed.
  • CliE2E and CliE2ERunnerTest passed: all 155 cases / 1,076 commands and 14 runner regression tests.
  • Exported the standalone bundle from the Git index and reran all cases successfully outside the checkout.
  • POSIX-step skip behavior was also exercised on macOS; this is not a Windows execution result. Linux and Windows validation remains for PR CI.
  • Apache RAT, Black 26.3.1, clang-format 17, and non-fixture whitespace checks passed.

Usage:

python3 cpp/test/tools/e2e/run_cli_e2e.py \
  --cli /path/to/installed/tsfile-cli \
  --output-dir /tmp/tsfile-cli-results

@ColinLeeo ColinLeeo closed this Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant