diff --git a/.claude/skills/release-pipeline/SKILL.md b/.claude/skills/release-pipeline/SKILL.md index a0564e7f..b36f39d2 100644 --- a/.claude/skills/release-pipeline/SKILL.md +++ b/.claude/skills/release-pipeline/SKILL.md @@ -13,9 +13,9 @@ before answering "is X supported" — they answer different questions. | | | |---|---| -| Projects | `boot`, `framework`, `security` | +| Projects | `ai`, `boot`, `framework`, `security` | | Version format | GA `major.minor.patch` only — M/RC/SNAPSHOT are rejected by `isGaVersion` | -| Buildable ranges | `boot` → `3.3.0`–`<4.0.0` and `4.0.0`–`<4.0.8` (both synthesized, different paths) and `>= 4.0.8` (archive); `framework` → `>= 6.1.0` (overlay); `security` → `>= 6.2.0` (overlay, two eras) | +| Buildable ranges | `ai` → `>= 1.0.0` (overlay); `boot` → `3.3.0`–`<4.0.0` and `4.0.0`–`<4.0.8` (both synthesized, different paths) and `>= 4.0.8` (archive); `framework` → `>= 6.1.0` (overlay); `security` → `>= 6.2.0` (overlay, two eras) | | Published | check `catalog.json`; an empty `projects` object means nothing has shipped yet | A project is a sequence of **layout eras** (`LayoutEra`, ADR-0004), not a single floor. An era @@ -30,10 +30,14 @@ without the other yields a tree that classifies but converts wrongly: | `framework` `>= 6.1.0` | `framework-docs` | `overlay` | the committed `antora.yml`, topped up with the version and the attributes the build contributes | no | | `security` `6.2.0` – `<6.5.1` | `docs` | `overlay` | the committed `antora.yml`, topped up with attributes derived from `gradle/libs.versions.toml` and `gradle.properties` | no | | `security` `>= 6.5.1` | `docs` | `overlay` | the same, plus the `modules/ROOT/examples/docs-src` symlink that era added | no | +| `ai` `>= 1.0.0` | `spring-ai-docs/src/main/antora` | `overlay` | the committed `antora.yml` unchanged — its build contributes no attribute at all | no | `overlay` is the cheapest to add and the one to reach for first on a new project: check what that project's `generateAntoraResources` actually produces. Spring Framework's is one -attribute (`spring-version`), so nothing is downloaded at all. +attribute (`spring-version`), so nothing is downloaded at all. Spring AI's is the degenerate +case — its whole generated template is `version` plus `prerelease`, neither of which is an +asciidoc attribute, so the overlay is pure passthrough and `generatedAttributesFor` returns +`{}`. Eras need not be contiguous, and `eraFor` returns `undefined` below the oldest floor: @@ -48,6 +52,11 @@ Eras need not be contiguous, and `eraFor` returns `undefined` below the oldest f - **4.1.0** is a *publication* fact, not a layout one: it is inside the archive era but its zip is unpublished, so it is probed over the network (below) instead of being encoded as an unbuildable range that would keep refusing it after upstream publishes. +- **Spring AI 0.8.x** is a publication fact too, and a permanent one, so it *is* encoded as a + floor: `spring-ai-docs/src/main/antora/antora.yml` is byte-identical at v0.8.0 and v2.0.1, but + `org/springframework/ai/spring-ai-bom` on Maven Central begins at `1.0.0-M5` — 0.8.0 and 0.8.1 + went to Spring's milestone repository only, and no consumer can pin a dependency to the + versions those 50 pages describe. The archive era's floor is not a compatibility guess either: upstream published that zip for 2.2.x–2.4.2, then not again until 4.0.8. The synthesized era's 3.3.0 floor has nothing to do @@ -150,6 +159,15 @@ exercise every AsciiDoc construct: adding Spring Framework surfaced hand-written `floating_title`, block images and role ``s — none of which Boot uses. Run `--strict` and add a rule per construct. +**`--strict` does not catch everything, so read the output too.** It gates on the converter's +own unknown-construct warnings, which fire only for a node the walker does not recognise. +Anything Asciidoctor substitutes *before* the walker runs is invisible to it: Spring AI's 16 +inline `stem:[…]` expressions arrive as plain text in MathJax delimiters (`\$…\$` for asciimath, +`\(…\)` for latexmath), and were escaped as prose into `\\$\\vec{a}\\$` across a page of vector +maths while `--strict` reported zero warnings. `escapeText` in `inline-html.ts` now carries them +through as `$…$`. Diff a page or two of a new corpus against the upstream site before believing +a clean run. + Then: - extend `tests/unit/upstream-sources.test.ts` diff --git a/NOTICE b/NOTICE index 3fb52990..ed6d02e2 100644 --- a/NOTICE +++ b/NOTICE @@ -34,6 +34,11 @@ Spring Security License: Apache License, Version 2.0 Notice: https://github.com/spring-projects/spring-security/blob/main/NOTICE.txt +Spring AI + Copyright (c) 2023-present, the Spring AI authors. + Source: https://github.com/spring-projects/spring-ai + License: Apache License, Version 2.0 + Spring Data (and its sub-projects: JPA, JDBC, MongoDB, Redis, etc.) Copyright (c) 2008-present, the Spring Data authors. Source: https://github.com/spring-projects/spring-data-* diff --git a/README.md b/README.md index dbee5237..7a021c99 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ > Pre-converted, version-pinned Spring documentation in Markdown — distributed as GitHub Release archives. -This repository hosts LLM-friendly Markdown versions of the Spring ecosystem reference docs (Framework, Boot, Security, Data, Cloud, etc.) generated from upstream AsciiDoc sources. Each `(project, version)` pair is published as a separate GitHub Release so consumers can fetch exactly the docs that match their project's dependencies. +This repository hosts LLM-friendly Markdown versions of the Spring ecosystem reference docs (Framework, Boot, Security, AI, Data, Cloud, etc.) generated from upstream AsciiDoc sources. Each `(project, version)` pair is published as a separate GitHub Release so consumers can fetch exactly the docs that match their project's dependencies. [![License](https://img.shields.io/badge/license-Apache--2.0-blue)](./LICENSE) [![Upstream](https://img.shields.io/badge/upstream-Apache--2.0-green)](./NOTICE) @@ -180,7 +180,7 @@ Consumers should prefer `catalog.json` over scraping the Releases page — it's ## Generation pipeline 1. **Detect new upstream release** — `nightly-detect.yml` runs `detect-upstream-versions.ts`, which diffs upstream's tags against `catalog.json` and files one issue per missing GA version. It builds nothing: a new upstream line can change the documentation layout, so a human decides. -2. **Fetch** — `fetch-upstream.ts` assembles one Antora content source from two halves. The authored half is always a sparse checkout of the docs subtree at the release tag. The generated half — the resolved `antora.yml` attributes, the sample sources `include-code::` reads, and the configuration-property metadata `configprop:` validates against — depends on the version's layout era ([ADR-0004](./.please/docs/decisions/0004-synthesize-3x-component.md)): Spring Boot 4.0.8+ merge the content archive published to Maven Central; Boot 3.3-4.0.7 reconstruct it from the tag plus the published `spring-boot-*` jars, because those archives are excluded from Spring's Maven Central sync until 4.0.8 ([ADR-0006](./.please/docs/decisions/0006-synthesize-4-0-x-component.md) covers 4.0.0-4.0.7, which reconstruct the same way from paths 4.0.0 moved); and Spring Framework 6.1+ and Spring Security 6.2+ need neither, because the tag already carries a complete descriptor and its own examples — the committed `antora.yml` is overlaid rather than rebuilt, with the version for Framework and, for Security, with the documentation URLs and four dependency versions its build resolves, all read out of the committed version catalog and `gradle.properties`. Either way: no submodules, no Gradle, no JVM. +2. **Fetch** — `fetch-upstream.ts` assembles one Antora content source from two halves. The authored half is always a sparse checkout of the docs subtree at the release tag. The generated half — the resolved `antora.yml` attributes, the sample sources `include-code::` reads, and the configuration-property metadata `configprop:` validates against — depends on the version's layout era ([ADR-0004](./.please/docs/decisions/0004-synthesize-3x-component.md)): Spring Boot 4.0.8+ merge the content archive published to Maven Central; Boot 3.3-4.0.7 reconstruct it from the tag plus the published `spring-boot-*` jars, because those archives are excluded from Spring's Maven Central sync until 4.0.8 ([ADR-0006](./.please/docs/decisions/0006-synthesize-4-0-x-component.md) covers 4.0.0-4.0.7, which reconstruct the same way from paths 4.0.0 moved); and Spring Framework 6.1+, Spring Security 6.2+ and Spring AI 1.0+ need neither, because the tag already carries a complete descriptor and its own examples — the committed `antora.yml` is overlaid rather than rebuilt, with the version for Framework and, for Security, with the documentation URLs and four dependency versions its build resolves, all read out of the committed version catalog and `gradle.properties`. Spring AI tops up nothing at all: its build generates only the component version, which the overlay writes anyway. Either way: no submodules, no Gradle, no JVM. 3. **Convert** — `convert.ts` drives Antora's own pipeline modules with Spring's Asciidoctor extensions registered, so `xref:`, `include::`, `include-code::`, `javadoc:` and `configprop:` are resolved by the same code that produces docs.spring.io. Our converter then emits Markdown from the resolved AST: GFM alerts for admonitions, headed code fences for tab groups, relative `.md` links for internal xrefs, absolute `docs.spring.io` URLs for references into components we do not build. An unhandled construct fails the build rather than being dropped. 4. **Package** — `package-release.ts` writes `NOTICE`, checksums every file, and builds a reproducible `tar.gz`: entries sorted, timestamps and ownership pinned, gzip's mtime field suppressed. The same converted tree always yields byte-identical bytes. 5. **Release** — a `-` tag push runs `release.yml`, which rebuilds from the tag, verifies the manifest against it, and publishes the archive, its checksum and the manifest. @@ -194,7 +194,7 @@ The pipeline is matrix-parallelized: typical full ecosystem rebuild (~10 project |---|---| | **Tag immutability** | Once a `-` tag is published, it is not deleted. Re-generation creates a new tag suffix (`framework-6.2.0+rebuild.1`) and updates `catalog.json` to point at the latest | | **Pre-release versions** | Not built. Only GA versions of upstream projects | -| **Buildable versions** | Spring Boot `3.3.0`+. 3.2 and older predate the Antora component entirely. 4.0.0-4.0.7 moved to the 4.x layout but published no content archive, so they are reconstructed from the tag like 3.3-3.x rather than downloaded; 4.1.0 is tagged with no archive published yet | +| **Buildable versions** | Spring Boot `3.3.0`+, Spring Framework `6.1.0`+, Spring Security `6.2.0`+, Spring AI `1.0.0`+. Boot 3.2 and older predate the Antora component entirely; 4.0.0-4.0.7 moved to the 4.x layout but published no content archive, so they are reconstructed from the tag like 3.3-3.x rather than downloaded, and 4.1.0 is tagged with no archive published yet. Spring AI 0.8.x shares the 1.x layout, but its artifacts never reached Maven Central, so no consumer can pin a dependency to what those docs describe | | **Reconstructed appendix** | The generated appendix — auto-configuration class listings and configuration-property tables — is a Gradle build output with no published equivalent, so every reconstructed version omits it: ~101 pages for 3.3-3.x, 92 for 4.0.0-4.0.7. The prose corpus (reference, how-to, tutorial, specification) is complete | | **Coverage window** | The newest N missing GA versions per project, N being the `limit` input of `matrix-build.yml` (default 3) | | **Backfill** | Older versions can be requested via issue and built on-demand | diff --git a/scripts/lib/inline-html.ts b/scripts/lib/inline-html.ts index 74a6a6fa..d2ef9f69 100644 --- a/scripts/lib/inline-html.ts +++ b/scripts/lib/inline-html.ts @@ -70,6 +70,43 @@ const BACKTICK_RUN = /`+/g /** Markdown characters that change how a text run parses. */ const MARKDOWN_SPECIALS = /[\\*_[\]]/g +/** + * One inline `stem:[…]` expression, as Asciidoctor has already rendered it. + * + * The stem macro is substituted away before a converter ever sees the node: it + * arrives here as ordinary text wrapped in the MathJax delimiters Asciidoctor + * emits — `\$…\$` for asciimath, which is the default and what Spring AI's + * playbook selects, and `\(…\)` for latexmath. + * + * That makes it invisible to the unknown-construct warning `--strict` gates on, + * so it has to be recognized here or not at all: escaped as prose, + * `\$\vec{a}\$` reaches the reader as `\\$\\vec{a}\\$`, and every backslash the + * expression carries doubles with it. + * + * A delimiter Asciidoctor emits carries exactly one backslash. Source that + * escapes one in order to write about it — `\\$x\\$`, or `\\(x\\)` — arrives + * with two, and is prose. The lookbehinds tell the two apart: without them the + * scan starts at the second backslash and publishes `$x\$`, live math where + * the page meant to show a delimiter. + * + * A singly escaped `\$x\$` written as prose stays indistinguishable, because + * Asciidoctor emits the same text for it as for `stem:[x]`. Nothing here can + * separate those: the substitution runs before the walker does and leaves the + * text node no provenance — not even the `stem` attribute, which is unset on + * the very pages that carry the macro. + */ +const STEM_SPAN = /(? void, +): string { + // Every stem delimiter starts with one, and ordinary prose rarely contains + // any — so this skips the scan below for almost every run in a corpus. + if (!text.includes('\\')) + return escapeProse(text, atLineStart) + + const spans = [...text.matchAll(STEM_SPAN)] + if (spans.length === 0) + return escapeProse(text, atLineStart) + + let out = '' + let remainder = '' + let cursor = 0 + let lineStart = atLineStart + + for (const span of spans) { + const before = text.slice(cursor, span.index) + remainder += before + out += escapeProse(before, lineStart) + // One alternative matched, so exactly one group is defined: asciimath's + // `\$…\$` or latexmath's `\(…\)`. + out += `$${span[1] ?? span[2] ?? ''}$` + lineStart = false + cursor = span.index + span[0].length + } + + const tail = text.slice(cursor) + remainder += tail + + const empty = spans.some(span => (span[1] ?? span[2] ?? '') === '') + if (empty || STEM_DELIMITER.test(remainder)) { + onUnpairedStem?.(text) + return escapeProse(text, atLineStart) + } + + return out + escapeProse(tail, lineStart) +} + /** * Rewrite a resolved Antora xref target for the Markdown tree. * @@ -342,6 +445,15 @@ export interface InlineOptions { readonly imageBase?: string /** Called with the `src` of an image that could not be given a base URL. */ readonly onImageWithoutBase?: (src: string) => void + /** + * Called with a text run whose `stem:[…]` delimiters do not pair up, or that + * holds an empty expression. + * + * The run is escaped as ordinary prose rather than converted, so the caller + * has to surface it: left unreported it would ship a visibly wrong page, and + * guessing a pairing instead would ship a silently wrong one. + */ + readonly onUnpairedStem?: (run: string) => void } /** @@ -371,7 +483,13 @@ const ABSOLUTE_URL = /^[a-z][\w+.-]*:\/\//i * @returns Markdown equivalent of `html`. */ export function inlineHtmlToMarkdown(html: string, options: InlineOptions = {}): string { - const { onUnknownTag, externalComponents = {}, imageBase, onImageWithoutBase } = options + const { + onUnknownTag, + externalComponents = {}, + imageBase, + onImageWithoutBase, + onUnpairedStem, + } = options let out = '' const emit = (text: string): void => { out += text @@ -380,7 +498,11 @@ export function inlineHtmlToMarkdown(html: string, options: InlineOptions = {}): const walk = (nodes: readonly HtmlNode[]): void => { for (const node of nodes) { if (node.kind === 'text') { - emit(escapeText(decodeEntities(node.value), out === '' || out.endsWith('\n'))) + emit(escapeText( + decodeEntities(node.value), + out === '' || out.endsWith('\n'), + onUnpairedStem, + )) continue } @@ -448,7 +570,11 @@ export function inlineHtmlToMarkdown(html: string, options: InlineOptions = {}): // It carries no children — the alt text is an attribute — so nothing // is walked. const src = node.attrs.get('src') ?? '' - const alt = escapeText(decodeEntities(node.attrs.get('alt') ?? ''), false) + const alt = escapeText( + decodeEntities(node.attrs.get('alt') ?? ''), + false, + onUnpairedStem, + ) if (src === '' || (imageBase === undefined && !ABSOLUTE_URL.test(src))) { // Same rule as a block image with no base: an empty destination // reads as a broken image, while the alt text alone still says what diff --git a/scripts/lib/markdown-converter.ts b/scripts/lib/markdown-converter.ts index 8e37df20..77d5e51e 100644 --- a/scripts/lib/markdown-converter.ts +++ b/scripts/lib/markdown-converter.ts @@ -115,6 +115,9 @@ const BLOCK_TITLE_LEVEL = '#### ' /** Deepest ATX heading GFM defines. */ const MAX_HEADING_LEVEL = 6 +/** How much of a text run a warning quotes before eliding the rest. */ +const EXCERPT_LIMIT = 60 + /** A `dlist` item: a list of terms and the description that follows them. */ type DefinitionItem = readonly [readonly AsciidoctorListItem[], AsciidoctorListItem | undefined] @@ -174,6 +177,16 @@ function flattenForCell(text: string): string { export function convertDocument(doc: AsciidoctorNode, options: ConvertOptions): ConvertResult { const warnings = new Set() + /** + * A one-line, bounded excerpt of a text run, for naming it in a warning. + * + * The run can be a whole paragraph, and a warning is read in a terminal. + */ + const excerpt = (run: string): string => { + const oneLine = run.replace(NEWLINE_RUN, ' ').trim() + return oneLine.length <= EXCERPT_LIMIT ? oneLine : `${oneLine.slice(0, EXCERPT_LIMIT)}…` + } + /** * Render an explicit `[[id]]` as its own inline-HTML anchor block. * @@ -197,6 +210,10 @@ export function convertDocument(doc: AsciidoctorNode, options: ConvertOptions): onImageWithoutBase: src => warnings.add(`inline image "${src}" dropped: no published image base for this project`), onUnknownTag: tag => warnings.add(`unknown inline tag <${tag}>`), + onUnpairedStem: run => + warnings.add( + `stem:[…] delimiters do not pair up, so the run was left escaped: "${excerpt(run)}"`, + ), }) /** Render a node list into joinable chunks, dropping the ones that render empty. */ diff --git a/scripts/lib/upstream-sources.ts b/scripts/lib/upstream-sources.ts index cefa6091..7a6ce213 100644 --- a/scripts/lib/upstream-sources.ts +++ b/scripts/lib/upstream-sources.ts @@ -34,6 +34,9 @@ const SPRING_FRAMEWORK_RAW = 'https://raw.githubusercontent.com/spring-projects/ /** Spring Security sources served straight from a release tag. */ const SPRING_SECURITY_RAW = 'https://raw.githubusercontent.com/spring-projects/spring-security' +/** Spring AI sources served straight from a release tag. */ +const SPRING_AI_RAW = 'https://raw.githubusercontent.com/spring-projects/spring-ai' + /** Maven Central base for released Spring artifacts. */ const MAVEN_CENTRAL = 'https://repo1.maven.org/maven2' @@ -480,6 +483,79 @@ function requiredSource(sources: Readonly>, path: string) } const PROJECTS: Readonly> = { + ai: { + repo: 'spring-projects/spring-ai', + // No Maven coordinates: the single era is an overlay. Spring AI publishes no + // Antora content archive — `org/springframework/ai/spring-ai-docs` is absent + // from Maven Central (probed 2026-09-17) — and needs none, because nothing + // its build generates reaches the corpus. + // + // No `xref:` names another component either: the only qualified references + // across the 121 pages of v2.0.1 are four `xref:ROOT:` into this component's + // own module, so there is nothing for a reference to dangle into. + externalComponentsFor: () => ({}), + tagPrefix: 'v', + eras: [ + { + // The layout itself reaches further back than this floor — `spring-ai-docs/ + // src/main/antora/antora.yml` is byte-identical at v0.8.0, v1.0.0, v1.1.0, + // v2.0.0 and v2.0.1, single `ROOT` module throughout — so 1.0.0 is a + // publication fact rather than a layout one, like Boot's 4.0.8 archive + // floor. `org/springframework/ai/spring-ai-bom` on Maven Central begins at + // `1.0.0-M5`, and its oldest GA is `1.0.0`: 0.8.0 and 0.8.1 went to Spring's + // milestone repository only, so no consumer can pin a dependency to the + // versions those docs describe. + // + // One era covers everything above it: v2.0.1 keeps the component path, the + // `mvnw process-resources` collector and the two-line generated template + // v1.0.0 has, so 2.x is not a second era. + since: '1.0.0', + componentPath: 'spring-ai-docs/src/main/antora', + assembly: { + descriptor: 'overlay', + // `resources/antora-resources/antora.yml` is the entire generated half, + // unchanged across the range: + // + // version: ${antora-component.version} + // prerelease: ${antora-component.prerelease} + // + // Neither line is an asciidoc attribute. `version` is the descriptor + // field `overlayDescriptor` already writes from the catalog version, and + // `prerelease` is false for everything this pipeline builds. So the + // build contributes no attribute at all — and the committed descriptor + // declares no `asciidoc.attributes` block to top up, which is why this + // is the one overlay era whose descriptor is pure passthrough. + // + // The corpus agrees: the only `{…}` references in it are prompt-template + // placeholders inside code samples (`{format}`, `{question}`, …), and the + // `spring-ai-version` the Hana page mentions is a Maven property written + // `${spring-ai-version}` inside an XML listing, not an attribute + // reference. + generatedAttributesFor: () => ({}), + // None to declare: no mode 120000 blob exists anywhere under + // `spring-ai-docs` at v1.0.0 or v2.0.1. Unlike Framework and Security + // there is no examples tree to reach through one — the corpus contains + // no `include-code::`. + internalSymlinks: [], + }, + }, + ], + // Retargets `javadoc:` macros. The corpus exercises none — its one `javadoc:` + // occurrence is a `./mvnw … javadoc:javadoc` command line inside a listing — + // but the playbook sets `javadoc-location` unconditionally, so a later version + // adding the macro should resolve rather than dangle. Verified against + // `…/2.0.1/api/org/springframework/ai/chat/client/ChatClient.html`. + javadocLocationFor: version => `https://docs.spring.io/spring-ai/docs/${version}/api`, + // The release tag, not the reference site, for the same reason as Framework and + // Security: `…/spring-ai/reference/2.0.1/_images/` answers a 301 to + // `…/reference/2.0/_images/` (verified 2026-09-17), so a URL built from a + // catalog version would be pinned in appearance only. Note `modules/ROOT/images` + // — Spring AI uses Antora's short image directory, not the `assets/images` the + // other two components ship. + imageBaseFor: version => + `${SPRING_AI_RAW}/v${version}/spring-ai-docs/src/main/antora/modules/ROOT/images`, + }, + boot: { repo: 'spring-projects/spring-boot', mavenGroupPath: 'org/springframework/boot', diff --git a/tests/unit/detect-upstream-versions.test.ts b/tests/unit/detect-upstream-versions.test.ts index 10bb7528..8784b0c6 100644 --- a/tests/unit/detect-upstream-versions.test.ts +++ b/tests/unit/detect-upstream-versions.test.ts @@ -5,7 +5,7 @@ describe('parseArgs', () => { test('defaults to every supported project, no limit, and text output', () => { const args = parseArgs([]) - expect(args).toEqual({ projects: ['boot', 'framework', 'security'], limit: null, json: false }) + expect(args).toEqual({ projects: ['ai', 'boot', 'framework', 'security'], limit: null, json: false }) }) test('accepts --project value and --project=value forms', () => { @@ -16,7 +16,7 @@ describe('parseArgs', () => { test('accepts --limit and --json', () => { const args = parseArgs(['--limit', '5', '--json']) - expect(args).toEqual({ projects: ['boot', 'framework', 'security'], limit: 5, json: true }) + expect(args).toEqual({ projects: ['ai', 'boot', 'framework', 'security'], limit: 5, json: true }) }) test('rejects an unknown project', () => { diff --git a/tests/unit/inline-html.test.ts b/tests/unit/inline-html.test.ts index ccf98b2d..1d1e3786 100644 --- a/tests/unit/inline-html.test.ts +++ b/tests/unit/inline-html.test.ts @@ -1,6 +1,20 @@ import { describe, expect, test } from 'bun:test' import { inlineHtmlToMarkdown } from '../../scripts/lib/inline-html.ts' +/** + * Converts `html`, collecting the runs the stem rule refused to read as math. + * + * Every stem case asserts on both halves — what reached the page, and what the + * build was told about what did not — so they share one call rather than each + * rebuilding the collector. + */ +function convertStem(html: string): { result: string, runs: string[] } { + const runs: string[] = [] + const result = inlineHtmlToMarkdown(html, { onUnpairedStem: run => runs.push(run) }) + + return { result, runs } +} + describe('inlineHtmlToMarkdown', () => { test('converts code, strong and em', () => { expect(inlineHtmlToMarkdown('spring')).toBe('`spring`') @@ -103,6 +117,83 @@ describe('inlineHtmlToMarkdown', () => { expect(result).toBe('kept') }) + test('carries a stem expression through unescaped, as inline math', () => { + // Asciidoctor substitutes `stem:[…]` before the converter sees it, so the + // expression arrives as text in MathJax delimiters — `\$…\$` for asciimath, + // Spring AI's notation. Escaped as prose it would read `\\$\\vec{a}\\$`. + expect(inlineHtmlToMarkdown('the vector \\$\\vec{a}\\$ is at \\$(a_1, a_2)\\$.')) + .toBe('the vector $\\vec{a}$ is at $(a_1, a_2)$.') + }) + + test('carries a latexmath stem expression through the same way', () => { + expect(inlineHtmlToMarkdown('angle \\(\\theta\\) between them')) + .toBe('angle $\\theta$ between them') + }) + + test('still escapes the prose around a stem expression', () => { + expect(inlineHtmlToMarkdown('a_b \\$x_1\\$ c_d')).toBe('a\\_b $x_1$ c\\_d') + }) + + test('keeps the leading-hash rule on whichever segment starts the line', () => { + expect(inlineHtmlToMarkdown('# heading \\$x\\$')).toBe('\\# heading $x$') + }) + + test('converts both notations in one run', () => { + expect(inlineHtmlToMarkdown('compare \\$a\\$ to \\(b\\) directly')) + .toBe('compare $a$ to $b$ directly') + }) + + test('withholds a run whose stem delimiters do not pair up, and reports it', () => { + // Pairing is positional, so there is no way to tell which of the three + // delimiters is the stray one. Pairing the first two would splice the prose + // between them into a formula and strip its escaping — `*emphasis*` would + // reach the page as live Markdown. + const { result, runs } = convertStem('broken \\$a and *emphasis* then \\$b\\$ end') + + expect(runs).toEqual(['broken \\$a and *emphasis* then \\$b\\$ end']) + expect(result).toBe('broken \\\\$a and \\*emphasis\\* then \\\\$b\\\\$ end') + }) + + test('withholds an empty stem expression rather than emitting bare $$', () => { + // `$$` is a display-math delimiter pair to most renderers, so emitting it + // would silently change the construct. + const { result, runs } = convertStem('empty \\$\\$ here') + + expect(runs).toEqual(['empty \\$\\$ here']) + expect(result).toBe('empty \\\\$\\\\$ here') + }) + + test('leaves a run with no backslash untouched by the stem scan', () => { + expect(inlineHtmlToMarkdown('a $5 price and a_b')).toBe('a $5 price and a\\_b') + }) + + test('leaves an escaped delimiter as prose, because it is not stem output', () => { + // Asciidoctor emits one backslash for a stem delimiter and passes a source + // `\\$` through as two, so a page writing *about* MathJax arrives here as + // `\\$x_1\\$`. Reading that as math would publish `$x\$` and strip the + // escaping from `_` on the way. + const { result, runs } = convertStem('prose \\\\$x_1\\\\$ here') + + expect(result).toBe('prose \\\\\\\\$x\\_1\\\\\\\\$ here') + expect(runs).toEqual([]) + }) + + test('leaves an escaped paren delimiter as prose the same way', () => { + const { result, runs } = convertStem('prose \\\\(x_1\\\\) here') + + expect(result).toBe('prose \\\\\\\\(x\\_1\\\\\\\\) here') + expect(runs).toEqual([]) + }) + + test('converts a stem expression that shares a run with an escaped literal', () => { + // The escaped pair must not count as a stray delimiter either, or the real + // expression beside it would be withheld as unpaired. + const { result, runs } = convertStem('math \\$x\\$ and literal \\\\$y\\\\$ end') + + expect(result).toBe('math $x$ and literal \\\\\\\\$y\\\\\\\\$ end') + expect(runs).toEqual([]) + }) + test('returns an empty string for empty input', () => { expect(inlineHtmlToMarkdown('')).toBe('') }) diff --git a/tests/unit/upstream-sources.test.ts b/tests/unit/upstream-sources.test.ts index 21189c6b..ee68d7f5 100644 --- a/tests/unit/upstream-sources.test.ts +++ b/tests/unit/upstream-sources.test.ts @@ -74,7 +74,7 @@ describe('resolveUpstream', () => { describe('supportedProjects', () => { test('lists the known projects', () => { - expect(supportedProjects()).toEqual(['boot', 'framework', 'security']) + expect(supportedProjects()).toEqual(['ai', 'boot', 'framework', 'security']) }) }) @@ -462,3 +462,99 @@ describe('resolveUpstream for Spring Security', () => { .toBe('https://docs.spring.io/spring-security/site/docs/6.5.6/api') }) }) + +describe('ai', () => { + test('resolves to its component root under an overlay era', () => { + const upstream = resolveUpstream('ai', '1.0.0') + + expect(upstream.repo).toBe('spring-projects/spring-ai') + expect(upstream.tag).toBe('v1.0.0') + expect(upstream.componentPath).toBe('spring-ai-docs/src/main/antora') + expect(upstream.assembly.descriptor).toBe('overlay') + }) + + test('waits on no published artifact, because the tag carries everything', () => { + // Being tagged upstream is the whole of being buildable: Spring AI publishes + // no content archive, and the era reads no metadata jar. + const upstream = resolveUpstream('ai', '1.0.0') + + expect(upstream.archives).toEqual([]) + expect(upstream.metadataJars).toEqual([]) + expect(requiredArtifactUrls('ai', '1.0.0')).toEqual([]) + }) + + test('generates no attribute, because the build contributes none', () => { + const { assembly } = resolveUpstream('ai', '1.0.0') + if (assembly.descriptor !== 'overlay') + throw new Error('expected an overlay assembly') + + // `resources/antora-resources/antora.yml` is two lines — `version` and + // `prerelease` — and neither is an asciidoc attribute: `version` is the + // descriptor field the overlay writes from the catalog version, and only GA + // versions are built, so `prerelease` is never true. + expect(assembly.generatedAttributes).toEqual({}) + expect(assembly.derivedAttributes).toBeUndefined() + }) + + test('declares no symlink, because the component ships none', () => { + const { assembly } = resolveUpstream('ai', '1.0.0') + if (assembly.descriptor !== 'overlay') + throw new Error('expected an overlay assembly') + + // No mode 120000 blob exists under `spring-ai-docs` at v1.0.0 or v2.0.1, and + // there is no examples tree to reach through one — the corpus contains no + // `include-code::`. + expect(assembly.internalSymlinks).toEqual([]) + }) + + test('checks out the component root alone', () => { + expect(resolveUpstream('ai', '1.0.0').checkoutPaths).toEqual([ + 'spring-ai-docs/src/main/antora', + ]) + }) + + test('pins images and javadoc to the exact version', () => { + const upstream = resolveUpstream('ai', '2.0.1') + + // The reference site collapses a patch to its minor — `/reference/2.0.1/_images/` + // answers a 301 to `/reference/2.0/_images/` — so images come from the tag. + // Spring AI uses Antora's short `modules/ROOT/images`, not `assets/images`. + expect(upstream.imageBase).toBe( + 'https://raw.githubusercontent.com/spring-projects/spring-ai/v2.0.1' + + '/spring-ai-docs/src/main/antora/modules/ROOT/images', + ) + expect(upstream.javadocLocation).toBe('https://docs.spring.io/spring-ai/docs/2.0.1/api') + }) + + test('maps no external components, because every xref stays in-component', () => { + // The only qualified references in the corpus are four `xref:ROOT:` into this + // component's own module. + expect(resolveUpstream('ai', '1.0.0').externalComponents).toEqual({}) + }) + + test('refuses 0.8.x, which Spring never synced to Maven Central', () => { + // The layout is byte-identical at v0.8.0, so the floor is a publication fact, + // as Boot's archive floor is: `org/springframework/ai/spring-ai-bom` begins at + // `1.0.0-M5`, and no consumer can pin a dependency to a 0.8.x that is not + // there. + expect(() => resolveUpstream('ai', '0.8.1')).toThrow(/not buildable/) + expect(() => resolveUpstream('ai', '1.0.0')).not.toThrow() + }) + + test('runs one era with no ceiling, across the 1.x-to-2.x line', () => { + // v2.0.1 keeps v1.0.0's component path, `mvnw process-resources` collector, + // single `ROOT` module and two-line generated template, so 2.x is not a + // second era. + for (const version of ['1.0.0', '1.1.8', '2.0.0', '2.0.1']) { + const upstream = resolveUpstream('ai', version) + expect(upstream.componentPath).toBe('spring-ai-docs/src/main/antora') + expect(upstream.assembly.descriptor).toBe('overlay') + } + }) + + test('orders versions from tags the same way the build does', () => { + expect( + supportedVersionsFromTags('ai', ['v0.8.1', 'v1.0.0', 'v1.1.8', 'v2.0.1', 'v2.1.0-M1']), + ).toEqual(['1.0.0', '1.1.8', '2.0.1']) + }) +})