Add databricks-serverless-storage-check skill

[GabbysCode]

Summary

Adds databricks-serverless-storage-check as a nested sub-skill under skills/databricks-serverless-migration/ — physically reflecting the parent/sub-skill hierarchy. The skill ships an executable preflight scanner detecting the antipattern where serverless tasks share state through /local_disk0, /tmp, or trustedTemp paths. This is the failure mode behind INTERNAL_ERROR: [Errno 13] Permission denied: '/local_disk0/.../trustedTemp.../...', where a parent task writes to local disk and a child task on a different node cannot read it.

Complementary to databricks-serverless-migration (which covers single-notebook migration and correctly recommends /local_disk0/tmp for intra-task scratch). This sub-skill covers the cross-task case.

New location

skills/
└── databricks-serverless-migration/
    ├── SKILL.md
    ├── references/
    └── databricks-serverless-storage-check/   ← new sub-skill here
        ├── SKILL.md
        ├── agents/openai.yaml
        ├── scripts/preflight.py
        ├── scripts/test_preflight.py
        ├── references/
        └── eval/

Contents

Path	Purpose
skills/databricks-serverless-migration/databricks-serverless-storage-check/SKILL.md	When-to-use, quick start, output interpretation, the core rule, related skills
.../agents/openai.yaml	Codex marketplace metadata
.../scripts/preflight.py	Stdlib-only AST + regex scanner with 5 input modes, --json flag, exit codes 0/1/2
.../scripts/test_preflight.py	7 self-tests, no third-party deps
.../references/pattern-catalog.md	Full table for FANOUT001–006 + ENV001 with examples, fixes, and detection rules
.../references/remediation-guide.md	Before/after code for Volumes / Workspace / taskValues / pipeline-downstream handoffs
.../eval/ground_truth.yaml	4 SkillForge test cases (3 positive triggers + 1 boundary) — stf lint clean (0 errors, 0 warnings)
.../eval/thinking_instructions.md	L4 reasoning criteria (Efficiency, Clarity, Recovery, Completeness, Hierarchy Awareness, Scanner Output Hygiene)
.../eval/output_instructions.md	L5 output criteria with expected artifacts, mandatory facts, negative signals, per-case acceptance bar
.../eval/manifest.yaml	SkillForge eval config

Supporting changes

• scripts/skills.py: iter_skill_dirs() extended to yield one level of nested skill directories, so the manifest generator and validator discover sub-skills alongside top-level skills
• skills/databricks-serverless-migration/SKILL.md: Sub-skills table added; relative links updated to databricks-serverless-storage-check/SKILL.md
• manifest.json: Regenerated (skill count unchanged: 9)

Detection rules

ID	Severity	Detects
FANOUT001	Blocker	Local-disk path passed to dbutils.notebook.run, taskValues.set, or job-task parameter
FANOUT002	Blocker	Child notebook reads from /local_disk0 or /tmp via widget, parameter, or taskValues.get
FANOUT003	Warning	DAB job with multiple sibling tasks referencing the same local-disk path
FANOUT004	Warning	pipeline_task immediately downstream of a notebook_task that wrote to local temp
FANOUT005	Info	dbutils.fs.cp local-to-local inside a notebook invoked by a multi-task job (heuristic)
FANOUT006	Blocker	Hardcoded BSI signature /local_disk0/spark-*/trustedTemp/... anywhere in source
ENV001	Info	--run-id mode only: routes ENVIRONMENT_SETUP_ERROR.PYTHON_NOTEBOOK_ENVIRONMENT to support escalation

Validation

• python3 scripts/skills.py validate — Everything is up to date.
• python3 .../scripts/test_preflight.py — 7/7 passing
• stf lint .../databricks-serverless-storage-check — 0 errors, 0 warnings

Checklist

• python3 scripts/skills.py validate passes
• scripts/skills.py iter_skill_dirs() updated for nested skill discovery
• SKILL_METADATA entry present in scripts/skills.py
• agents/openai.yaml hand-authored
• Self-tests pass (7/7)
• SKILL.md body under 250 lines (149 lines)
• Frontmatter description includes trigger phrases: trustedTemp, local_disk0, permission denied, fan-out, cross-task
• stf lint passes (0 errors, 0 warnings)
• SkillForge eval scaffolded: ground_truth.yaml (4 cases), thinking_instructions.md, output_instructions.md, manifest.yaml
• Relative SKILL.md links updated for new nested path
• Signed off per DCO

[dustinvannoy-db]

We will reach out directly about possible consolidation of this PR and one of our Field Eng maintainers will add a review once we feel its ready.

[jacksandom]

Hi @GabbysCode - can you restructure to match the other skills. So for example, your markdown files should be references for databricks-serverless-migration. Consider if you need 3 md files for this or if we can consolidate?

[jacksandom]

Should be able to remove this and assets as it is captured in the main skill.