Repository navigation
INFRA-1882: add a partition API acceptance suite and test package #597
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rbcasperson
wants to merge
4
commits into
main
Choose a base branch
from
infra-1882-acceptance-suite
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+2,527
−0
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
ee6469c
feat(acceptance): INFRA-1882 add a partition API acceptance suite
rbcasperson 99aa9eb
chore(acceptance): INFRA-1882 trim a cross-repo reference from the pu…
rbcasperson efa0660
fix(acceptance): INFRA-1882 mint the feed token inline in a public repo
rbcasperson 4c31e4a
fix(acceptance): INFRA-1882 publish without lifecycle scripts while h…
rbcasperson File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| name: CD — acceptance test package | ||
|
|
||
| # Publishes acceptance/ to the Azure Artifacts "unstructured-npm" feed as | ||
| # @utic/unstructured-api-acceptance. A hub suite that depends on it picks the scenarios up through the | ||
| # "unstructured-acceptance-tests" keyword in acceptance/package.json. | ||
| # | ||
| # The version is `<major>.<minor>` from acceptance/package.json plus, as the patch, the number of | ||
| # commits that have touched acceptance/. Nobody writes the patch by hand, so a merged change to | ||
| # the suite cannot ship nothing by forgetting a bump, and a commit that changes nothing here | ||
| # derives the same number and has nothing to publish. | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - "acceptance/**" | ||
|
|
||
| # Two concurrent runs would race to publish the same version. | ||
| concurrency: | ||
| group: cd-acceptance-package | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| publish: | ||
| name: Publish @utic/unstructured-api-acceptance | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
| permissions: | ||
| contents: read | ||
| id-token: write | ||
| defaults: | ||
| run: | ||
| working-directory: acceptance | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
| with: | ||
| # The patch is a commit count; a shallow clone would count 1 forever. | ||
| fetch-depth: 0 | ||
|
|
||
| - uses: actions/setup-node@v7 | ||
| with: | ||
| node-version-file: acceptance/.nvmrc | ||
| cache: npm | ||
| cache-dependency-path: acceptance/package-lock.json | ||
|
|
||
| # acceptance/.npmrc reads the feed token from AZURE_ARTIFACTS_TOKEN. The install gets a | ||
| # read token so dependency install scripts never hold publish rights. This is a public repo, | ||
| # which GitHub does not allow to use actions from the internal github-workflows repo, so the | ||
| # tokens are minted inline. | ||
| - name: Log in to Azure with the package feed reader | ||
| uses: azure/login@a457da9ea143d694b1b9c7c869ebb04ebe844ef5 # v2.3.0 | ||
| with: | ||
| client-id: a74a6a24-379b-4810-b01a-00707cb7decf | ||
| tenant-id: dcb38522-c289-4665-a9d8-c4995b630804 | ||
| allow-no-subscriptions: true | ||
|
|
||
| - name: Mint package feed reader token | ||
| id: ado_read | ||
| run: | | ||
| token=$(az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 --query accessToken -o tsv) | ||
| echo "::add-mask::${token}" | ||
| echo "AZ_DEVOPS_TOKEN=${token}" >> "${GITHUB_OUTPUT}" | ||
|
|
||
| - name: Install dependencies | ||
| env: | ||
| AZURE_ARTIFACTS_TOKEN: ${{ steps.ado_read.outputs.AZ_DEVOPS_TOKEN }} | ||
| run: npm ci | ||
|
|
||
| # bddgen fails on an unbound step and --list on no collected scenario, so a suite the hub | ||
| # cannot run stops here instead of shipping. No global setup runs, so no secret is fetched. | ||
| - name: Verify the suite builds and binds | ||
| env: | ||
| TEST_ENV: staging | ||
| TEST_SECRETS_ID: "" | ||
| run: | | ||
| npm run build | ||
| npx --no-install bddgen | ||
| npx --no-install playwright test --list | ||
|
|
||
| # The publisher app only admits refs/heads/main. | ||
| - name: Log in to Azure with the package feed publisher | ||
| uses: azure/login@a457da9ea143d694b1b9c7c869ebb04ebe844ef5 # v2.3.0 | ||
| with: | ||
| client-id: 781d0ae1-1f8f-42f6-a7ae-136af0b01d48 | ||
| tenant-id: dcb38522-c289-4665-a9d8-c4995b630804 | ||
| allow-no-subscriptions: true | ||
|
|
||
| - name: Mint package feed publisher token | ||
| id: ado_write | ||
| run: | | ||
| token=$(az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 --query accessToken -o tsv) | ||
| echo "::add-mask::${token}" | ||
| echo "AZ_DEVOPS_TOKEN=${token}" >> "${GITHUB_OUTPUT}" | ||
|
|
||
| - name: Publish to the feed | ||
| env: | ||
| AZURE_ARTIFACTS_TOKEN: ${{ steps.ado_write.outputs.AZ_DEVOPS_TOKEN }} | ||
|
rbcasperson marked this conversation as resolved.
Resolved
|
||
| run: | | ||
| set -euo pipefail | ||
| series="$(node -p "require('./package.json').version.split('.').slice(0, 2).join('.')")" | ||
| version="${series}.$(git rev-list --count HEAD -- .)" | ||
| npm version --no-git-tag-version --allow-same-version "${version}" >/dev/null | ||
| # Every version lands under the 'rc' dist-tag: consumers pin a version or range, so | ||
| # 'latest' is never resolved. --ignore-scripts keeps prepack's build (code from | ||
| # node_modules) away from the publisher token; the verify step already built the steps. | ||
| if output=$(npm publish --tag rc --ignore-scripts 2>&1); then | ||
| echo "$output" | ||
| echo "::notice::Published @utic/unstructured-api-acceptance@${version}" | ||
| elif grep -qiE "EPUBLISHCONFLICT|409 Conflict|cannot publish over|already exists" <<<"$output"; then | ||
| echo "$output" | ||
| echo "::notice::@utic/unstructured-api-acceptance@${version} is already published; nothing to do." | ||
| else | ||
| echo "$output" >&2 | ||
| echo "::error::npm publish failed for @utic/unstructured-api-acceptance@${version}" | ||
| exit 1 | ||
| fi | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| name: CI — acceptance | ||
|
|
||
| # Proves every acceptance scenario binds to a step and is collected. It runs no scenario against an | ||
| # environment: those need staging's secrets and a live partition API, which a pull request check | ||
| # should not. | ||
|
|
||
| on: | ||
| pull_request: | ||
| workflow_dispatch: | ||
|
|
||
| concurrency: | ||
| group: acceptance-${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| contract: | ||
| name: Acceptance contract | ||
| # A fork's pull request gets no OIDC token, so it could not install the framework. | ||
| if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
| permissions: | ||
| contents: read | ||
| id-token: write | ||
| defaults: | ||
| run: | ||
| working-directory: acceptance | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
|
|
||
| - uses: actions/setup-node@v7 | ||
| with: | ||
| node-version-file: acceptance/.nvmrc | ||
| cache: npm | ||
| cache-dependency-path: acceptance/package-lock.json | ||
|
|
||
| # @utic/acceptance-core is on the private Azure Artifacts feed; acceptance/.npmrc reads the | ||
| # token from AZURE_ARTIFACTS_TOKEN. This is a public repo, which GitHub does not allow to use | ||
| # actions from the internal github-workflows repo, so the token is minted inline. | ||
| - name: Log in to Azure with the package feed reader | ||
| uses: azure/login@a457da9ea143d694b1b9c7c869ebb04ebe844ef5 # v2.3.0 | ||
| with: | ||
| client-id: a74a6a24-379b-4810-b01a-00707cb7decf | ||
| tenant-id: dcb38522-c289-4665-a9d8-c4995b630804 | ||
| allow-no-subscriptions: true | ||
|
|
||
| - name: Mint package feed token | ||
| id: ado | ||
| run: | | ||
| token=$(az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 --query accessToken -o tsv) | ||
| echo "::add-mask::${token}" | ||
| echo "AZ_DEVOPS_TOKEN=${token}" >> "${GITHUB_OUTPUT}" | ||
|
|
||
| - name: Install dependencies | ||
| env: | ||
| AZURE_ARTIFACTS_TOKEN: ${{ steps.ado.outputs.AZ_DEVOPS_TOKEN }} | ||
|
rbcasperson marked this conversation as resolved.
Resolved
|
||
| run: npm ci | ||
|
|
||
| # The compile a published test package needs; a type error here would ship no steps. | ||
| - name: Build the acceptance step sources | ||
| run: npm run build | ||
|
|
||
| # bddgen fails on an unbound step, so a feature cannot land depending on a step nobody wrote. | ||
| # Staging, because that is where this suite's scenarios run; bddgen and --list run no global | ||
| # setup, so no secret is fetched. | ||
| - name: Generate Playwright tests from every scenario | ||
| env: | ||
| TEST_ENV: staging | ||
| TEST_SECRETS_ID: "" | ||
| run: npx --no-install bddgen | ||
|
|
||
| - name: List the collected scenarios | ||
| env: | ||
| TEST_ENV: staging | ||
| TEST_SECRETS_ID: "" | ||
| run: npx --no-install playwright test --list | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # unstructured-api | ||
|
|
||
| ## Before committing | ||
|
|
||
| - **If your change alters anything a user can see or do** — the partition API's request/response | ||
| behavior: endpoints, parameters, response fields, status codes and error messages — the | ||
| acceptance suite may assert on it. Before finishing, run | ||
| `cd acceptance && npm ci && npx acceptance-help update-for-product-change` and follow it. | ||
| `npm ci` needs `AZURE_ARTIFACTS_TOKEN` exported; how to mint it is at the top of | ||
| `acceptance/.npmrc`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| @AGENTS.md |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| node_modules/ | ||
| .features-gen/ | ||
| playwright-report/ | ||
| test-results/ | ||
| .auth/ | ||
| .acceptance-suite/ | ||
| .recordings/ | ||
| env/*.local.env | ||
| # Compiled step definitions — build output of `npm run build` (prepack), shipped in the | ||
| # published test package and removed again by postpack. Never committed. | ||
| steps/**/*.js |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| { | ||
| "mcpServers": { | ||
| "playwright": { | ||
| "command": "npx", | ||
| "args": ["@playwright/mcp@0.0.78", "--isolated", "--storage-state", ".auth/state.json"] | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| ; @utic/* — the shared acceptance framework (@utic/acceptance-core) — is published to the | ||
| ; Azure Artifacts "unstructured-npm" feed; all other deps come from the public registry. | ||
| ; The feed token is NEVER written to disk: the _authToken line below interpolates it from the | ||
| ; AZURE_ARTIFACTS_TOKEN environment variable at install time. Export it locally from an Azure | ||
| ; DevOps token (you must be `az login`'d): | ||
| ; export AZURE_ARTIFACTS_TOKEN=$(az account get-access-token \ | ||
| ; --resource 499b84ac-1321-427f-aa17-267ca6975798 --query accessToken -o tsv) | ||
| ; The --resource GUID is required — a plain `az account get-access-token` is ARM-scoped and the | ||
| ; feed rejects it. In CI, export the same var from the OIDC-minted feed token. | ||
| registry=https://registry.npmjs.org/ | ||
| @utic:registry=https://pkgs.dev.azure.com/unstructured/_packaging/unstructured-npm/npm/registry/ | ||
| //pkgs.dev.azure.com/unstructured/_packaging/unstructured-npm/npm/registry/:_authToken=${AZURE_ARTIFACTS_TOKEN} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| 24.18.1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| # Acceptance Tests — Instructions | ||
|
|
||
| The shared conventions ship with `@utic/acceptance-core`, at the version this suite locks. From | ||
| this directory: | ||
|
|
||
| ```bash | ||
| npm ci # installs exactly the locked framework version | ||
| npx acceptance-help conventions # shared rules + Gherkin guidelines; read before changing any scenario or step | ||
| npx acceptance-help update-for-product-change # after a product change that may affect what this suite asserts | ||
| npx acceptance-help # every framework command and guide | ||
| ``` | ||
|
|
||
| This file holds only what is specific to this repo. | ||
|
|
||
| ## This repo | ||
|
|
||
| - Surface: the partition API's request/response behaviour — `POST /general/v0/general` on | ||
| `TEST_API_URL`, authenticated with `TEST_API_KEY` in the `unstructured-api-key` header. The | ||
| scenarios call it directly; there is no UI here. | ||
| - Where it runs: staging by default (`TEST_ENV=staging npm test`). Its key comes from | ||
| `acceptance-tests/staging` in Secrets Manager, which the standing SND `AcceptanceSecretsReadOnly` | ||
| role can read. Not SND: API keys do not work on SND instances, so every scenario carries | ||
| `@skip-snd`. transform-staging and transform-prod do not serve this API and are skipped too. | ||
| - This suite runs no scenario in this repo's CI. The `Acceptance contract` job only checks that | ||
| every step binds and every scenario is collected. Real runs happen in `infra/acceptance`, which | ||
| installs the published `@utic/unstructured-api-acceptance` package. | ||
| - Test files come from `s3://u10d-acceptance-test-data` under `unstructured-api/`, pinned by | ||
| SHA-256. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| @AGENTS.md |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # API keys do not work on SND instances, and this feature authenticates with one. | ||
| # transform-staging and transform-prod do not serve the partition API. | ||
| @platform @freq-hourly @skip-snd @skip-transform-staging @skip-transform-prod | ||
| Feature: Partition a document | ||
| As an API client | ||
| I want to send a PDF to the partition API | ||
| So that I get its content back as structured elements | ||
|
|
||
| Scenario: A partitioned PDF comes back as elements headed by its title | ||
| Given an API client with a valid Unstructured API key | ||
| When the API client partitions the LayoutParser paper PDF with the "fast" strategy | ||
| Then the partition API returns a list of document elements | ||
| And the first element is titled "LayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysis" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.