Repository navigation

Aafeedback changes (do not merge) - #604

Draft

ShuxinLin wants to merge 106 commits into

mainfrom

aafeedback_changes

Commits on Sep 26, 2026
- feat(harbor): run AssetOpsBench scenarios as Harbor tasks
  DhavalRepo18
  committed
  b91549f
- adding files
  DhavalRepo18
  committed
  96197eb
Commits on Sep 27, 2026
- revised code
  DhavalRepo18
  committed
  6948500
- Updated
  DhavalRepo18
  committed
  13d5ad3
- revised code
  DhavalRepo18
  committed
  88f1bb8
- adding more models and code for tsfm
  DhavalRepo18
  committed
  ae4d69c
- revised models
  DhavalRepo18
  committed
  27ac496
- chore(harbor): mark preload_models.py executable
  
  DhavalRepo18
  and
  claude
  committed
  f53e383
- revised code
  DhavalRepo18
  committed
  8987013
- revised git ignore and output
  DhavalRepo18
  committed
  abff786
- revised docker ignore
  DhavalRepo18
  committed
  672ec56
- revised code
  DhavalRepo18
  committed
  69ca4af
- revised code
  DhavalRepo18
  committed
  5421771
- revised catalog generator
  DhavalRepo18
  committed
  e119b1f
Commits on Sep 28, 2026
- revised code
  DhavalRepo18
  committed
  bc26dde
- revised code
  DhavalRepo18
  committed
  8edc871
- revised code
  DhavalRepo18
  committed
  e32abb7
- docs(harbor): fix stale paths and run steps in README
  ShuxinLin
  committed
  0c45322
- feat(harbor): generate tasks against an external scenario corpus
  ShuxinLin
  committed
  bba3fa6
- style(harbor): ruff format generate_tasks.py
  ShuxinLin
  committed
  2dd44bc
- feat(harbor): run.sh, the Harbor counterpart of benchmarks/run.sh
  ShuxinLin
  committed
  7be1922
- refactor(harbor): rename corpus to suite
  ShuxinLin
  committed
  533bc5a
- refactor(harbor): rename corpus to suite in contents
  ShuxinLin
  committed
  c453ad7
- fix(harbor): check the model router before a job; retry crashed trials
  ShuxinLin
  committed
  836e9d3
Commits on Sep 29, 2026
- docs(harbor): document generating mini/lite/all profiles
  ShuxinLin
  committed
  958e3d8
- feat(harbor): load .env for StirrupAgent credentials
  ShuxinLin
  committed
  1b9306f
- docs(harbor): correct the shared-data gap for non-open profiles
  ShuxinLin
  committed
  6eaa1d8
- Merge pull request #575 from IBM/feat/harbor-load-dotenv
  ShuxinLin
  authored
  1e1579f
- Merge feature/harbor-integration into feat/harbor-suite-run
  ShuxinLin
  committed
  e45f338
- feat(harbor): mount a private suite with AOB_PRIVATE_DIR
  ShuxinLin
  committed
  cfecba9
- refactor(harbor): infer the suite data dir from --scenario-root
  ShuxinLin
  committed
  1832899
- feat(harbor): run mini/lite/all by mounting the private suite
  ShuxinLin
  committed
  b4b01f3
- Merge pull request #577 from IBM/feat/harbor-private-data
  ShuxinLin
  authored
  3dedb12
- docs(harbor): align smoke tests with the PR comment; record agent runs
  ShuxinLin
  committed
  8459a7b
- revised code
  DhavalRepo18
  committed
  460543f
Commits on Sep 30, 2026
- fix(fmsr): take the FMSR model from FMSR_MODEL_ID, with no built-in default
  
  authored and
  DhavalRepo18
  committed
  a30f036
- fix(fmsr): accept only the llm.routers prefixes, dropping watsonx
  
  authored and
  DhavalRepo18
  committed
  ffa4c38
- fix(harbor): forward FMSR_MODEL_ID and check its router up front
  
  authored and
  DhavalRepo18
  committed
  2a1b421
- fix(fmsr): gate the integration test on the database as well as the LLM
  
  authored and
  DhavalRepo18
  committed
  b77f60f
- fix(fmsr): generate failure modes without an initialised catalog
  
  authored and
  DhavalRepo18
  committed
  534b010
- Merge pull request #583 from IBM/fmsr-model-pin
  ShuxinLin
  authored
  bca6aaa
- Merge branch 'feature/harbor-integration' into harbor-private-mount-docker-md
  ShuxinLin
  authored
  b74b621
- Merge pull request #581 from IBM/harbor-private-mount-docker-md
  ShuxinLin
  authored
  bc8886b
- fix(harbor): run.sh mounts only the private suite's shared/
  ShuxinLin
  committed
  4ee86af
- fix(harbor): build the runtime image from a clean clone of HEAD
  ShuxinLin
  committed
  b7a7d84
- fix(harbor): ship no .git in the runtime image
  ShuxinLin
  committed
  47a806e
- fix(harbor): allowlist task inputs and fail on a missing suite path
  ShuxinLin
  committed
  7f66b87
- Merge pull request #586 from IBM/fix/harbor-run-private-mount
  ShuxinLin
  authored
  ec62c34
- Merge pull request #587 from IBM/fix/harbor-clean-runtime-image
  ShuxinLin
  authored
  326598d
- feat(harbor): pick the runtime image at run time with AOB_RUNTIME_IMAGE
  
  ShuxinLin
  and
  claude
  committed
  e91aa07
- feat(harbor): tag each runtime build with its commit
  
  ShuxinLin
  and
  claude
  committed
  3bb4f12
- fix(harbor): pin the runtime image for a run and guard resumes
  
  ShuxinLin
  and
  claude
  committed
  1ad4eea
- Merge pull request #588 from IBM/feat/harbor-runtime-image-param
  ShuxinLin
  authored
  401197f
- docs(harbor): correct stale docs, docstrings and comments
  
  ShuxinLin
  and
  claude
  committed
  f2d1310
- Merge pull request #590 from IBM/fix/harbor-stale-docs
  ShuxinLin
  authored
  57dff2f
- fix(harbor): give each run.sh job its own name, tasks and exit status
  
  ShuxinLin
  and
  claude
  committed
  2494f1e
- fix(harbor): run.sh retries, code tar, key checks, env file, lock, paths
  
  ShuxinLin
  and
  claude
  committed
  66eb514
- fix(harbor): refuse to resume a run.sh job on another suite
  
  ShuxinLin
  and
  claude
  committed
  d9eb32d
- Merge pull request #591 from IBM/fix/harbor-run-job-identity
  ShuxinLin
  authored
  cb058ab
- fix(test): keep a pre-set COUCHDB_URL in the MCP env test
  
  ShuxinLin
  and
  claude
  committed
  1f68511
- chore(harbor): drop duplicate deps and ignore rules; fix template comments
  
  ShuxinLin
  and
  claude
  committed
  038d900
- fix(tsfm): ship ttm_energy_168_24 from artifacts/tsfm_models
  
  ShuxinLin
  and
  claude
  committed
  8676a3a
- docs(harbor): trim docs and comments, drop stale content
  
  ShuxinLin
  and
  claude
  committed
  2daca75
- fix(harbor): name the code image in the loader's no-tar message
  
  ShuxinLin
  and
  claude
  committed
  581a538
Commits on Oct 1, 2026
- fix(harbor): leave the dev group out of the runtime image
  
  ShuxinLin
  and
  claude
  committed
  2970b5d
- perf(harbor): rebuild the runtime image in seconds after a source edit
  
  ShuxinLin
  and
  claude
  committed
  102646e
- docs(harbor): organise the README by how a run is driven
  
  ShuxinLin
  and
  claude
  committed
  ba120aa
- fix(harbor): keep "__" out of run.sh's per-job task folder
  
  ShuxinLin
  and
  claude
  committed
  8f95ad2
- Merge pull request #558 from IBM/feature/harbor-integration
  ShuxinLin
  authored
  9750df7
- refactor(utilities): drop json_reader and the time tools
  ShuxinLin
  committed
  1ff3dc2
- Merge pull request #599 from IBM/chore/utilities-drop-json-time-tools
  ShuxinLin
  authored
  bfc810f
- refactor(fmsr): remove generate_failure_modes
  
  ShuxinLin
  and
  claude
  committed
  2d5c0d4
- refactor(agent): stop pinning FMSR_MODEL_ID for MCP servers
  
  ShuxinLin
  and
  claude
  committed
  300182c
- chore(env): drop FMSR_MODEL_ID from .env.public
  
  ShuxinLin
  and
  claude
  committed
  178e1fe
- fix(agent): withhold LLM credentials from MCP servers
  
  ShuxinLin
  and
  claude
  committed
  9424277
- chore(suite): add fmsr_fmea scenario profile
  
  ShuxinLin
  and
  claude
  committed
  16cf039
- chore(suite): rename fmsr_fmea profile to aa_v1
  
  ShuxinLin
  and
  claude
  committed
  391447b
- chore(suite): drop FMEA scenarios from aa_v1 for now
  
  ShuxinLin
  and
  claude
  committed
  3f788d5
Commits on Oct 2, 2026
- Add smoke_features.py: gate the feature catalog and extractors on real data
  
  DhavalRepo18
  and
  claude
  committed
  d8f6725
- Merge branch 'main' into aafeedback_changes
  ShuxinLin
  authored
  da19aa2
- Merge branch 'aafeedback_changes' into refactor/fmsr-drop-generate
  ShuxinLin
  committed
  5081706
- Merge pull request #600 from IBM/refactor/fmsr-drop-generate
  ShuxinLin
  authored
  c79262c
- chore(suite): add FMEA scenarios back to aa_v1
  
  ShuxinLin
  and
  claude
  committed
  7c4b27d
Commits on Oct 3, 2026
- This PR is based on v1 run
  DhavalRepo18
  committed
  649f9fb
Commits on Oct 5, 2026
- revised code
  DhavalRepo18
  committed
  b4d8e41
- Merge pull request #613 from IBM/aa-prompt-only-fixed
  ShuxinLin
  authored
  438708f
- Merge pull request #611 from IBM/fmsr_doc_string_change
  ShuxinLin
  authored
  df4901e
- adding tsfm and other scenario
  DhavalRepo18
  committed
  9f25139
- revised scenarios
  DhavalRepo18
  committed
  01895ad
- adding set of wosr
  DhavalRepo18
  committed
  19e289c
- Bring TSFM and IoT server updates
  DhavalRepo18
  committed
  25988aa
- Update aa_v1.yaml
  ShuxinLin
  authored
  6c2b361
- Add values to car list in aa_v1.yaml
  ShuxinLin
  authored
  dc767f6
- Add missing values to aa_v1.yaml
  ShuxinLin
  authored
  04e23ca
- revised scenarios id
  DhavalRepo18
  committed
  639f7eb
- Merge branch 'add-tsfm-and-car-' of https://github.com/IBM/AssetOpsBench into add-tsfm-and-car-
  DhavalRepo18
  committed
  9716e4b
- revised tsfm
  DhavalRepo18
  committed
  3e52a94
- Merge pull request #615 from IBM/add-tsfm-and-car-
  DhavalRepo18
  authored
  657b14f
- Merge pull request #616 from IBM/tsfm_iot_update
  DhavalRepo18
  authored
  d5cea09
- Add pass@k/pass^k, temperature and turn-budget controls to harbor runs
  
  DhavalRepo18
  and
  claude
  committed
  5abbfe8
- Merge pull request #619 from IBM/passk-only
  DhavalRepo18
  authored
  64a30a9
Commits on Oct 6, 2026
- string req.
  DhavalRepo18
  committed
  b0a544b
- adding aa
  DhavalRepo18
  committed
  843a222
- Merge pull request #624 from IBM/fix/identifier-count-collapse
  DhavalRepo18
  authored
  8c59cd9
Commits on Oct 8, 2026
- fix(evaluation): score CAR scenarios on the mode key only
  ShuxinLin
  committed
  f8805be
- Merge pull request #627 from IBM/fix/car-mode-key-only
  ShuxinLin
  authored
  9584e28

Commits

Commits on Sep 26, 2026

Commits on Sep 27, 2026

Commits on Sep 28, 2026

Commits on Sep 29, 2026

Commits on Sep 30, 2026

Commits on Oct 1, 2026

Commits on Oct 2, 2026

Commits on Oct 3, 2026

Commits on Oct 5, 2026

Commits on Oct 6, 2026

Commits on Oct 8, 2026