docs: 0.21.0 train — engine 0.21.0, extensions 0.3.0, drivers 0.3.0 - #243
Merged
Conversation
All five were filed reviewing PR #232 and are Windows-only: install.sh bootstraps no JDK, so it shares none of these paths. #233 Resolve-Java never probed <install>\jdk, so every re-run downloaded the ~180 MB Temurin zip again and unpacked it over a perfectly good JDK. It is now step 1 — the same order both generated launchers implement — and a bundled JDK below the floor is REPLACED rather than worked around with JAVA_HOME, because the launcher prefers <install>\jdk unconditionally. Reuse and bootstrap set identical state through one Use-EmbeddedJdk helper. #234 Install-EmbeddedJdk removed the existing JDK before its replacement existed, so a failed unpack (running out of disk lands squarely in that window) left a previously working install with no JVM at all. Now: stage, assert bin\java.exe in the staged tree, and only then delete and move. #235 the generated .bat computed JAVA_MAJOR and spent it only on the --add-opens decision, so a stale JAVA_HOME - which outranks the PATH here, deliberately, because it is the JVM the installer probed - silently won and the REPL died with UnsupportedClassVersionError. It now refuses a JVM below %REQUIRED_JAVA%, guarded by `not "0"` for parity with the .ps1 launcher: refuse only a version positively read as too low, never one it failed to parse. Both launchers now name JAVA_HOME as the likely cause. #236 a bad -Version was only discovered by Download-Jar, after ~300 MB of JDK had landed in $Target, and nothing in the failure output mentioned it. Test-RequestedVersion now settles the inputs before Resolve-Java; listings are memoised (successes only) so it costs no extra HTTP, and $script:LastListingError preserves the cause -Quiet swallows. For the residual paths the main flow is wrapped in try/finally - PowerShell runs finally on `exit`, preserving the exit code - which names the JDK left behind. The JDK is KEPT on purpose: with #233 the next attempt reuses it. #237 install.cmd fetched install.ps1 from refs/heads/main, unverified, into a fixed %TEMP% path it never cleaned up. It now fetches a pinned release tag (PS1_REF), verifies it against a pinned SHA-256 with certutil before powershell.exe ever sees the file, and downloads into a fresh %RANDOM% directory removed at a single exit point. Overridable with SOFTCLIENT4ES_INSTALL_REF / _SHA256. Verified without a Windows host: 37 checks driving the real function bodies out of install.ps1's AST (JDK reuse/replace/fallthrough/bootstrap ordering, stage-then-swap under a throwing and a malformed unpack, the generated .bat and .ps1 text, and the top-level try/finally executed from its own extent), 15 static checks on install.cmd including refetching the pinned tag to confirm the pinned SHA-256, and real runs against JFrog (-Help, -ListVersions, a bogus -Version, a missing Scala variant, and an end-to-end install with a JDK pre-placed at <Target>\jdk). Not covered: cmd.exe executing install.cmd or the generated .bat, certutil output parsing on real Windows, and Expand-Archive on a real Temurin zip. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Post-release documentation sweep, core half. The web half is the companion PR;
the two are swept together because they drift in different ways.
Coordinates bumped (39 replacements across 8 files):
java-client 0.20.4 -> 0.21.0
community-extensions 0.2.4 -> 0.3.0
arrow-extensions 0.2.5 -> 0.3.0
jdbc / adbc / flight 0.2.5 -> 0.3.0
REPL install examples 0.20.4 -> 0.21.0
Every version was verified PUBLISHED on JFrog before being written, not after:
jdbc, adbc and arrow-flight-sql at 0.3.0 for ES 6/7/8/9, and java-client at
0.21.0. The failure this prevents is silent -- a documented version with no -all
bundle falls back to the plain artifact, which carries no arrow extensions and
therefore no cross-index JOIN, so the docs would demonstrate the path that loses
the headline feature.
FOUR HISTORICAL STATEMENTS DELIBERATELY NOT BUMPED. A line saying when something
ARRIVED is a fact about the past; bumping it inverts its meaning:
* documentation/client/repl.md:54 "Since 0.20.4" (Windows Java bootstrap)
* documentation/sql/joins.md:114 "since arrow-extensions 0.2.5 ... Before
0.2.5 all of these were rejected"
A blind sweep would have turned the second into "aliases work since 0.3.0",
telling every 0.2.5/0.20.4 user they lack a feature they have. The sweep script
skips any line carrying "since"/"before", and reports which lines it skipped so
the decision is reviewable rather than implicit.
TWO CORRUPTIONS THE DIFF CAUGHT, both quoted-output drift:
* repl.md:217 -- the sweep RENAMED "0.20.4" to "0.21.0" inside a captured
--list-versions listing. 0.20.4 is still published, so the listing must GAIN
a line, not rename one: 0.20.4 restored, 0.21.0 appended, total 4 -> 5.
* install.ps1:547 -- a comment illustrating an ascending series
"(0.20.2, 0.20.3, 0.20.4, ...)" had its third term rewritten, making the
series skip a release. Reverted.
Reading the diff caught both; trusting the regex would not have.
No SQL-documentation change was needed for #238's behaviour change. The row
order of an un-ordered extraction now interleaves across slices, and a
quota-capped result is an arbitrary subset -- both are already documented in
documentation/client/scroll.md, which landed with the feature. The SQL docs were
checked and make no claim about row ordering, so there is no false statement to
correct.
Also verified, no change required: install.cmd pins PS1_REF=v0.21.0 and
PS1_SHA256, and the pinned digest matches the actual install.ps1 at tag v0.21.0
byte for byte -- a mismatch there makes the Windows installer refuse to run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The partitioning clause has never been spelled PARTITIONED BY, and the argument
shape was wrong too. The grammar is
keyword("PARTITION") ~ keyword("BY") ~ ident ~ opt(granularity)
so the column comes first and the granularity follows it in parentheses:
) PARTITION BY birthdate (MONTH) -- parses
) PARTITIONED BY (birthdate MONTH) -- "end of input expected"
Verified against the real parser, all four forms:
PARTITIONED BY (birthdate MONTH) FAIL (as documented until now)
PARTITION BY birthdate (MONTH) OK
PARTITION BY birthdate MONTH FAIL (parentheses are required)
PARTITION BY birthdate OK (granularity defaults to DAY)
Both worked examples and the two rows of the index-vs-template table are
corrected, and the clause's shape is now stated in prose next to the first
example so the parentheses rule is not left to be inferred from a sample.
Why this mattered more than a typo: before #214 made trailing input a hard
error, the parser SILENTLY DROPPED the unrecognised clause. Following these docs
produced a plain concrete index instead of an index template, with no error --
a wrong result rather than a failure. Since #214 it fails loudly, which is how
it surfaced.
Found by parse-probing every SQL example in the documentation through
Parser.apply rather than reading them.
The same wrong form is in the REPL help JSON, which ships inside the artefact --
filed separately rather than fixed here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Post-release documentation sweep, core half. Companion: softclient4es-web#40 — the two are swept together because they drift in different ways.
Two commits: the mechanical version sweep, and one SQL correctness fix found by parse-probing.
1. Coordinates bumped — 39 replacements, 8 files
java-client0.20.40.21.0community-extensions0.2.40.3.0arrow-extensions0.2.50.3.00.2.50.3.00.20.40.21.0Every version verified published on JFrog — including the four
softclient4es{6,7,8,9}-cli-all_2.13bundles at0.21.0, re-checked after the REPL release completed (all200). That check matters because the failure is silent: a documented version with no-allbundle falls back to the plain artifact, which carries no arrow extensions and therefore no cross-index JOIN.Historical statements deliberately NOT bumped
documentation/client/repl.md:54— "Since0.20.4" (Windows Java bootstrap)documentation/sql/joins.md:114— "since arrow-extensions 0.2.5 … Before 0.2.5 all of these were rejected"A blind sweep would turn the second into "aliases work since 0.3.0", telling every 0.2.5 user they lack a feature they have. The sweep skips
since/beforelines and reports which, so the decision is reviewable.Two corruptions the diff caught
repl.md:217— the sweep renamed0.20.4→0.21.0inside a captured--list-versionslisting. 0.20.4 is still published, so the listing must gain a line. Fixed: 0.20.4 restored, 0.21.0 appended, total 4 → 5.install.ps1:547— a comment illustrating an ascending series(0.20.2, 0.20.3, 0.20.4, …)had its third term rewritten. Reverted.That listing has since been re-captured from the real installer (
install.sh --list-versions --es-version 8) now that the bundles are live, and is byte-identical to what is committed — so it is verified against the artefact, not reasoned about.2.
PARTITION BY, notPARTITIONED BY—ddl_statements.mdThe partitioning clause has never been spelled
PARTITIONED BY, and the argument shape was wrong too. The grammar iskeyword("PARTITION") ~ keyword("BY") ~ ident ~ opt(granularity):) PARTITIONED BY (birthdate MONTH)← as documentedend of input expected) PARTITION BY birthdate (MONTH)) PARTITION BY birthdate MONTH) PARTITION BY birthdateDAYBoth worked examples and both rows of the index-vs-template table are corrected, and the shape is now stated in prose so the parentheses rule isn't left to be inferred from a sample.
This was worse than a typo. Before #214 made trailing input a hard error, the parser silently dropped the unrecognised clause — following these docs produced a plain concrete index instead of an index template, with no error. A wrong result, not a failure.
Found by parse-probing every documented SQL example through
Parser.apply(120 statements) rather than reading them.Related, filed not fixed here
CURRENT_TIMESTAMPis not a valid columnDEFAULT; the grammar takes_ingest.timestamp. Two stray lines, both repos.PARTITIONED BYerror in the REPL help JSON, which ships inside the artefact, so\h CREATE TABLEhands the user a form that cannot work.No SQL-doc change needed for #238
Row order of an un-ordered extraction now interleaves and a quota-capped result is an arbitrary subset — both already documented in
documentation/client/scroll.md, which landed with the feature. No SQL page promises row ordering, so there is no false statement to correct.Note on
install.cmdinstall.cmd'sPS1_REF/PS1_SHA256pinning arrived with #242, not this PR. With #242 merged I re-verified it againstmain: the pinned digest matchesinstall.ps1at tagv0.21.0byte for byte, so the Windows installer chain is intact for this release. Recorded here as a release check, not as work this PR did.