Aafeedback changes (do not merge)

.dockerignore

@@ -0,0 +1,46 @@
+# Filters the runtime image's build context, which build-runtime-image.sh makes
+# from `git archive HEAD`: committed files not listed here are baked into the
+# image by `COPY . .` in benchmarks/harbor/base-image/Dockerfile.
+
+# Credentials. A bare name matches only at the context root, hence **/.
+**/.env
+**/.env.*
+!**/.env.public
+
+# Host virtualenv: the image builds its own with `uv sync --frozen`.
+.venv/
+
+# Agent-written checkpoints. artifacts/tsfm_models/ ships in the image.
+artifacts/output/
+
+# Harbor output and generated tasks.
+jobs/
+benchmarks/harbor/datasets/
+
+# Caches and local noise.
+**/__pycache__/
+*.py[cod]
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+traces/
+.DS_Store
+
+# Answers: the image is the agent's container. The verifier reads the copies
+# Harbor uploads to /tests after the agent phase.
+src/couchdb/scenarios_data/**/groundtruth*
+src/couchdb/scenarios_data/**/rubric.json
+src/couchdb/scenarios_data/**/reference_answer.json
+src/couchdb/scenarios_data/**/scenario_meta.json
+
+# Local work that is not part of the benchmark.
+reports/
+logs/
+src/tmp/
+.claude/
+notebook/kdd_tutorial/
+artifacts/kdd_tutorial/
+
+# Git data holds every committed blob, answers included. The commit reaches the
+# image as /opt/aob/.aob-commit instead.
+.git

.gitignore

@@ -199,8 +199,16 @@ benchmark/cods_track2/.env.local
 CLAUDE.md
 mcp/couchdb/sample_data/bulk_docs.json
 .env
-mcp/servers/tsfm/artifacts/tsfm_models/
+# Agent-written checkpoints. artifacts/tsfm_models/ is tracked.
+artifacts/output/
 src/tmp/
+trainer_output
 
 # Observability artifacts (OTLP-JSON traces + per-run trajectory JSON).
 traces/
+
+# Generated Harbor tasks; they hold the scenarios' answers.
+benchmarks/harbor/datasets/
+
+# Harbor run output.
+jobs/

README.md

@@ -81,12 +81,12 @@ AssetOpsBench is a **unified framework for developing, orchestrating, and evalua
 | MCP Servers | Important tools |
 |---|---|
 | **IoT** | `sites`, `asset_ids`, `asset_detail`, `assets`, `find_assets_by_sensors`, `installed_sensors`, `measured_sensors`, `latest_reading`, `history`, `sensor_stats` |
-| **FMSR** | `get_failure_modes`, `generate_failure_modes`, `add_failure_modes` |
+| **FMSR** | `get_failure_modes`, `add_failure_modes` |
 | **TSFM** | Tasks/evidence: `list_tasks`, `profile_series`, `characterize_series`, `data_quality`; model catalog: `list_models`, `search_models`, `find_models`, `resolve_model`, `model_template`, `register_model`, `register_finetuned`, `hf_stats`; feature catalog: `list_features`, `search_features`, `extract_features`, `select_features`; run/eval ledger: `recipe_template`, `run_recipe`, `run_tabular_recipe`, `run_plan`, `evaluate`, `list_runs`, `list_results` |
 | **WO** | Read: `list_workorders`, `get_workorder`, `get_workorder_tasks`, `get_workorder_costs`, `get_workorder_actuals_vs_planned`, `get_workorder_kpis`, `get_schedule_calendar`, `get_my_assigned_workorders`, `get_failure_codes`; write: `generate_work_order`, `update_workorder`, `approve_workorder`, `assign_technician`, `close_workorder`, `cancel_workorder` |
 | **Vibration** | `get_vibration_data`, `list_vibration_sensors`, `compute_fft_spectrum`, `compute_envelope_spectrum`, `assess_vibration_severity`, `calculate_bearing_frequencies`, `diagnose_vibration` |
 | *Shared utility server* | |
-| **Utilities** | `json_reader`, `get_sensor_catalog`, `get_asset_catalog`, `get_failure_mode_catalog`, `current_date_time`, `current_time_english` |
+| **Utilities** | `get_sensor_catalog`, `get_asset_catalog`, `get_failure_mode_catalog` |
 
 Five domain servers (IoT, FMSR, TSFM, WO, Vibration) plus one shared utility server, documented in full — arguments, categories, and backing services — in **[docs/mcp-servers.md](./docs/mcp-servers.md)**.
 

artifacts/README.md

@@ -0,0 +1,66 @@
+# artifacts/
+
+Model weights that do not come from the HuggingFace Hub.
+
+```
+artifacts/
+  tsfm_models/      read-only. Committed here, baked into the image.
+  output/
+    tuned_models/   writable. For checkpoints an agent fine-tunes during a trial. Never committed.
+```
+
+## tsfm_models/ - shipped checkpoints
+
+Four TinyTimeMixer checkpoints (`ttm_512_96`, `ttm_512_720`, `ttm_1536_96`,
+`ttm_1536_720`) and `ttm_energy_168_24`, a TTM fine-tuned for energy load
+forecasting. A fine-tuned checkpoint that ships with the repo belongs here, not
+in `output/`. An optional `meta.json` beside the weights records what they
+cannot (domain, lineage, training data) for
+`benchmarks/harbor/scripts/generate_model_catalog.py`.
+
+Each entry is a `save_pretrained` directory (`config.json` plus weights) that a
+catalog card points at:
+
+```json
+"source": "local_artifact",
+"hf_repo": null,
+"artifact_path":    "artifacts/tsfm_models/ttm_512_96",
+"model_checkpoint": "artifacts/tsfm_models/ttm_512_96",
+"params": { "model_path": "artifacts/tsfm_models/ttm_512_96" }
+```
+
+Keep the location fields in step. Only `params.model_path` is read at load
+time, but the agent copies this shape when it registers its own cards.
+
+The path resolves against the working directory: the repo root locally,
+`/opt/aob` in the container. A missing directory is treated as a Hub repo id,
+and the resulting error does not mention the directory, so check instead:
+
+```bash
+uv run python benchmarks/harbor/scripts/preload_models.py --check
+```
+
+That resolves every active card, Hub and local alike, and exits non-zero when
+anything would fail at fit time.
+
+### What belongs here
+
+Public, redistributable weights only: check both the base model's licence and
+what the checkpoint was fine-tuned on. A model tuned on internal or customer
+data belongs in the private set, whatever the base licence says.
+
+Plain git is fine at these sizes (0.15 MB to 20 MB); use Git LFS if a file
+approaches 50 MB.
+
+## output/tuned_models/ - agent output
+
+For checkpoints an agent fine-tunes during a trial: `run_recipe`'s `save_to`
+names the directory (any path works), and `register_finetuned` points a new
+card at it. It lives in the trial container and is discarded with it;
+`.gitignore` and `.dockerignore` both exclude it.
+
+Cards for these models carry `created_by: "agent.tsfm.finetune"`, which is what
+`preload_models.py --check` uses to skip them.
+
+If you mount this root from the host, give each trial its own subdirectory, or
+parallel trials will overwrite each other.

artifacts/tsfm_models/ttm_1536_720/config.json

@@ -0,0 +1,64 @@
+{
+  "adaptive_patching_levels": 3,
+  "architectures": [
+    "TinyTimeMixerForPrediction"
+  ],
+  "categorical_vocab_size_list": null,
+  "context_length": 1536,
+  "d_model": 384,
+  "d_model_scale": 3,
+  "decoder_adaptive_patching_levels": 0,
+  "decoder_d_model": 256,
+  "decoder_d_model_scale": 2,
+  "decoder_mode": "common_channel",
+  "decoder_num_layers": 2,
+  "decoder_raw_residual": false,
+  "distribution_output": "student_t",
+  "dropout": 0.4,
+  "dtype": "float32",
+  "enable_forecast_channel_mixing": false,
+  "exogenous_channel_indices": null,
+  "expansion_factor": 2,
+  "fcm_context_length": 1,
+  "fcm_gated_attn": true,
+  "fcm_mix_layers": 3,
+  "fcm_prepend_past": true,
+  "fcm_prepend_past_offset": null,
+  "fcm_use_mixer": true,
+  "frequency_token_vocab_size": 8,
+  "gated_attn": true,
+  "head_dropout": 0.4,
+  "huber_delta": 1,
+  "init_embed": "pytorch",
+  "init_linear": "pytorch",
+  "init_processing": true,
+  "init_std": 0.02,
+  "loss": "mse",
+  "mask_value": 0,
+  "masked_context_length": null,
+  "mode": "common_channel",
+  "model_type": "tinytimemixer",
+  "norm_eps": 1e-05,
+  "norm_mlp": "LayerNorm",
+  "num_input_channels": 1,
+  "num_layers": 2,
+  "num_parallel_samples": 100,
+  "num_patches": 12,
+  "patch_last": true,
+  "patch_length": 128,
+  "patch_stride": 128,
+  "positional_encoding_type": "sincos",
+  "post_init": false,
+  "prediction_channel_indices": null,
+  "prediction_filter_length": null,
+  "prediction_length": 720,
+  "quantile": 0.5,
+  "resolution_prefix_tuning": false,
+  "scaling": "std",
+  "self_attn": false,
+  "self_attn_heads": 1,
+  "stride_ratio": 1,
+  "transformers_version": "4.57.6",
+  "use_decoder": true,
+  "use_positional_encoding": false
+}

artifacts/tsfm_models/ttm_1536_96/config.json

@@ -0,0 +1,64 @@
+{
+  "adaptive_patching_levels": 3,
+  "architectures": [
+    "TinyTimeMixerForPrediction"
+  ],
+  "categorical_vocab_size_list": null,
+  "context_length": 1536,
+  "d_model": 384,
+  "d_model_scale": 3,
+  "decoder_adaptive_patching_levels": 0,
+  "decoder_d_model": 256,
+  "decoder_d_model_scale": 2,
+  "decoder_mode": "common_channel",
+  "decoder_num_layers": 2,
+  "decoder_raw_residual": false,
+  "distribution_output": "student_t",
+  "dropout": 0.4,
+  "dtype": "float32",
+  "enable_forecast_channel_mixing": false,
+  "exogenous_channel_indices": null,
+  "expansion_factor": 2,
+  "fcm_context_length": 1,
+  "fcm_gated_attn": true,
+  "fcm_mix_layers": 3,
+  "fcm_prepend_past": true,
+  "fcm_prepend_past_offset": null,
+  "fcm_use_mixer": true,
+  "frequency_token_vocab_size": 8,
+  "gated_attn": true,
+  "head_dropout": 0.4,
+  "huber_delta": 1,
+  "init_embed": "pytorch",
+  "init_linear": "pytorch",
+  "init_processing": true,
+  "init_std": 0.02,
+  "loss": "mse",
+  "mask_value": 0,
+  "masked_context_length": null,
+  "mode": "common_channel",
+  "model_type": "tinytimemixer",
+  "norm_eps": 1e-05,
+  "norm_mlp": "LayerNorm",
+  "num_input_channels": 1,
+  "num_layers": 2,
+  "num_parallel_samples": 100,
+  "num_patches": 12,
+  "patch_last": true,
+  "patch_length": 128,
+  "patch_stride": 128,
+  "positional_encoding_type": "sincos",
+  "post_init": false,
+  "prediction_channel_indices": null,
+  "prediction_filter_length": null,
+  "prediction_length": 96,
+  "quantile": 0.5,
+  "resolution_prefix_tuning": false,
+  "scaling": "std",
+  "self_attn": false,
+  "self_attn_heads": 1,
+  "stride_ratio": 1,
+  "transformers_version": "4.57.6",
+  "use_decoder": true,
+  "use_positional_encoding": false
+}

artifacts/tsfm_models/ttm_512_720/config.json

@@ -0,0 +1,64 @@
+{
+  "adaptive_patching_levels": 3,
+  "architectures": [
+    "TinyTimeMixerForPrediction"
+  ],
+  "categorical_vocab_size_list": null,
+  "context_length": 512,
+  "d_model": 192,
+  "d_model_scale": 3,
+  "decoder_adaptive_patching_levels": 0,
+  "decoder_d_model": 128,
+  "decoder_d_model_scale": 2,
+  "decoder_mode": "common_channel",
+  "decoder_num_layers": 2,
+  "decoder_raw_residual": false,
+  "distribution_output": "student_t",
+  "dropout": 0.4,
+  "dtype": "float32",
+  "enable_forecast_channel_mixing": false,
+  "exogenous_channel_indices": null,
+  "expansion_factor": 2,
+  "fcm_context_length": 1,
+  "fcm_gated_attn": true,
+  "fcm_mix_layers": 3,
+  "fcm_prepend_past": true,
+  "fcm_prepend_past_offset": null,
+  "fcm_use_mixer": true,
+  "frequency_token_vocab_size": 5,
+  "gated_attn": true,
+  "head_dropout": 0.4,
+  "huber_delta": 1,
+  "init_embed": "pytorch",
+  "init_linear": "pytorch",
+  "init_processing": true,
+  "init_std": 0.02,
+  "loss": "mse",
+  "mask_value": 0,
+  "masked_context_length": null,
+  "mode": "common_channel",
+  "model_type": "tinytimemixer",
+  "norm_eps": 1e-05,
+  "norm_mlp": "LayerNorm",
+  "num_input_channels": 1,
+  "num_layers": 2,
+  "num_parallel_samples": 100,
+  "num_patches": 8,
+  "patch_last": true,
+  "patch_length": 64,
+  "patch_stride": 64,
+  "positional_encoding_type": "sincos",
+  "post_init": false,
+  "prediction_channel_indices": null,
+  "prediction_filter_length": null,
+  "prediction_length": 720,
+  "quantile": 0.5,
+  "resolution_prefix_tuning": false,
+  "scaling": "std",
+  "self_attn": false,
+  "self_attn_heads": 1,
+  "stride_ratio": 1,
+  "transformers_version": "4.57.6",
+  "use_decoder": true,
+  "use_positional_encoding": false
+}