Repository navigation
Nutrient DWS: grade the API's own Markdown by default, caption chart tables with printed text only - #4
Merged
Conversation
The default body_source is now "hosted": the graded body is the API's own output.markdown, as for most providers here, instead of Markdown rebuilt from the returned elements. The rebuild stays available with body_source "graph" or NUTRIENT_DWS_BODY_SOURCE=graph, and its tests now select it explicitly.
When a chart had no printed caption, the provider wrote the chart's summary field into the table's <caption>. The summary is text DWS generates, not text on the page. Its only effect on scoring was that label words in it let chart rules pass for values the table holds in the wrong row or column. Chart tables now take their caption from the captionIds elements or keep the caption already in htmlTable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Nutrient DWS provider now grades the Markdown the API returns (
body_source: "hosted") by default, and a chart table's<caption>comes only from text printed on the page.Why
Since #2 the request asks for inline formatting, so the hosted Markdown carries the styles the formatting rules score. In understand mode it now scores within 0.02 points of the rebuild (75.53 against 75.55 on CI build 438), and it is what a DWS user gets back.
The caption fallback used the chart's generated
summarysentence when the page had no printed caption. That sentence isn't on the page. On the chart pages where it changed a score, a rule passed only because the summary repeated a label.Changes
body_sourcedefaults to"hosted". The rebuild stays available withbody_source: "graph"orNUTRIENT_DWS_BODY_SOURCE=graph, anddocs/pipelines.mdsays so.summary. Chart tables appended to the hosted Markdown follow the same rule.body_source: "graph"explicitly.Testing
uv run ruff check src testsanduv run ruff format --check src testspass.uv run pytest -q: 5,074 passed.Re-grade
Stored CI responses (build 438, understand, 2,078 files) re-normalized with these commits and graded again: