Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,8 @@ Use the programmatic API when you want to integrate AXIS into an existing test r

`run({ profile })` and `loadConfig(path, { profile })` select a named profile from the config, the same overlay `axis run --profile` applies. `loadConfig` returns the resolved `config` alongside the pre-merge `baseConfig`, for callers that need the suite layout as a whole rather than the active suite.

For per-category scoring without the full `scoreRunResult` pipeline, import `runDeepEval`, `computeCategoryScore` and `aggregateDimension` from `@netlify/axis`. Select your judge adapter, registered with `registerAdapter`, through `runDeepEval`’s `options.judging`. It makes one judge call per category with interactions; setting that category’s `options.weights` value to `0` skips the call. Skipped categories use AXIS’s defaults. Use `computeCategoryScore` for a category score or `aggregateDimension` for a single audit dimension. The package root also exports `computeHeuristicSpeed`, `parseCategoryEvalResponse`, `parseJsonFromText`, and the types `DeepEvalOptions` and `CalibrationParams`. These exports add no scoring behavior changes; existing deep imports keep working.

## Roadmap

Delivered: scenario runner, four-dimension scoring pipeline, baselines with regression detection, repeated runs with representative-run selection and noise-aware regression bands, MCP/skills wiring, custom adapter API, config profiles for running one repo's scenarios under several agent matrices, built-in adapters for Claude Code, Codex, and Gemini.
Expand Down
5 changes: 5 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ export type { AcpAdapterSpec } from "./adapters/base/acp-adapter.js";
export { scoreResults, scoreRunResult, buildScoredOutput } from "./scoring/index.js";
export { ScoringError } from "./scoring/errors.js";
export { buildSparseIndex } from "./scoring/sparse-index.js";
export { runDeepEval, computeHeuristicSpeed, parseCategoryEvalResponse } from "./scoring/deep-eval.js";
export type { DeepEvalOptions } from "./scoring/deep-eval.js";
export { computeCategoryScore, aggregateDimension } from "./scoring/category-score.js";
export type { CalibrationParams } from "./scoring/category-score.js";
export { parseJsonFromText } from "./scoring/parse-json.js";
export { getPromptTemplates, interpolate } from "./scoring/prompt-templates.js";
export type { PromptTemplate, PromptVariable } from "./scoring/prompt-templates.js";
export { categorizeInteraction } from "./transcript/categorize.js";
Expand Down
21 changes: 21 additions & 0 deletions test/unit/scoring/root-exports.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import { describe, expect, it } from "vitest";
import * as axis from "../../../src/index.js";
import { runDeepEval, computeHeuristicSpeed, parseCategoryEvalResponse } from "../../../src/scoring/deep-eval.js";
import { computeCategoryScore, aggregateDimension } from "../../../src/scoring/category-score.js";
import { parseJsonFromText } from "../../../src/scoring/parse-json.js";

describe("package root exports", () => {
it("re-exports the scoring internals that programmatic consumers import", () => {
const expected = {
runDeepEval,
computeHeuristicSpeed,
parseCategoryEvalResponse,
computeCategoryScore,
aggregateDimension,
parseJsonFromText,
};
for (const [name, fn] of Object.entries(expected)) {
expect(axis[name as keyof typeof axis], name).toBe(fn);
}
});
});
Loading