[NoopDog]

Closes #718

What changed

• Three more SGDP genomes reach a donor, on a recorded genotype check.
  • src/meta_disco/sources/sgdp_genotype_checks.yaml records a bcftools gtcheck run. Each sample's chr22 gVCF was compared with every genome of SGDP's public release (278) and HGDP's (929). For each sample the record holds the genome it matched (or none) and the two best scores per release.
  • make import-sgdp-entities links a sample when either check passes:
    • sex check: SGDP's sex agrees with T2T_CHRY's karyotype (275 samples, as before);
    • genotype check: the record matches the sample to the genome its tables pair it with (3 more).
  • A recorded non-match leaves a sample out whatever its sex says.
  • The three:
    • S_Palestinian-2 becomes donor hgdp:HGDP00725, sex M;
    • S_Norwegian-1 and B_Australian-3 get donors with no sex.
  • S_Naxi-2 stays out: it isn't in SGDP's file, and it matches no public genome.
• Both checks are declared rules: code_rules.IDENTITY_CHECKS (sgdp_sex_agrees, sgdp_genotype_matches). They are listed in the rules report.
• Every import says why it linked or left out each sample, in outcomes.json beside its evidence: the pairing, each check's finding, and which checks confirmed it. No evidence reader takes the file as evidence (it isn't .ndjson).
• Reconcile reports donor reach:
  • entities.donor_reach, per dataset: files with samples, files whose every sample has a donor, files with one sample and no donor, and files with several samples where some have no donor;
  • entities.samples.no_donor: each sample a file names that has no donor, with what its importer recorded.
• Table cells are read more carefully:
  • A blank cell disagrees with nothing.
  • Two tables naming different genomes are tables_disagree.
  • Two stated karyotypes make the sex check karyotypes_disagree, which the genotype record can still confirm.
• SGDP's U sex gets an authored value-map row, phenotypic_sex.sgdp_u (declares: {}: reviewed, and no claim).
• Docs:
  • docs/studies/sgdp-genome-identity.md, a new note: the chain from file to person, what T2T_CHRY holds, both checks, the method, the controls, and each genome's verdict.
  • Contract 2.10 amended, plus CLAUDE.md and the Makefile.

Why

#559 linked a sample only where SGDP's sex and T2T_CHRY's karyotype agree, because the sample-to-genome pairing rests on one table. Four genomes failed that check. A genotype comparison answers the real question — is this the same person? — directly. Three of the four are, and one isn't anyone in the public releases.

For S_Palestinian-2, the content sides with SGDP's M: the sample's own chrX shows one X. That makes T2T_CHRY's XX the odd one out. I think it may be a male cell line that lost its Y; that isn't checked.

Assumptions I made

• The genotype record is a measurement, not a source. It adds no value to a record: it confirms or vetoes the pairing T2T_CHRY's tables state. The importer reads only sgdp_id and matched. The scores are there for a person, and only the record's keys are checked, not their values.
• The record is tied to the catalog it measured (anvil15). The import refuses another catalog until a person confirms the record still holds there. SGDP genotype record: pin the measured gVCFs by md5 instead of the catalog #737 would replace this with pinning each measured gVCF by md5.
• The measurement isn't re-runnable from the repo yet. The note's §4 and §6 give the exact steps. Re-runnable genotype check for the SGDP record (gtcheck, optionally a genome-wide panel) #738 adds a target that re-measures the record and fails on any difference.
• gtcheck can't read a gVCF directly (its ALTs carry <NON_REF>). So each sample is written as plain genotypes at the reference's sites, with reference blocks read as 0/0, before scoring.
• Several importers may write outcomes for one dataset. Reconcile keeps one per source per sample, so a second importer is never refused.
• The two U donors get no sex.

How to verify

Definition of done from #718:

• Each of the four is either confirmed by a check both sides state, then linked as Capture the donors and biosamples AnVIL leaves out of T2T_CHRY, MAGE and nhp_dGTEx, and link them across datasets #559 links the others, or left out with the reason recorded where reconcile reports it. Steps 1–3.
• The count of SGDP files without a donor in reconcile_report.json matches what this issue leaves. Step 3.

1. Read the note: docs/studies/sgdp-genome-identity.md §4–§5. You should see gtcheck mismatches:
   • the same person differs at 0–1 of about 25,000 HGDP sites and 33–60 of about 30,800 SGDP sites;
   • every other genome differs at 1.98% of sites or more;
   • S_Naxi-2's best match is 2.6% (S_Naxi-3), so it has none.
2. Import: make import-sgdp-entities ARGS="--entity-root /tmp/718_e" (network; or omit ARGS to write the shared root). It should print:

   linked: 278
     confirmed by sgdp_sex_agrees: 275
     confirmed by sgdp_genotype_matches: 3
   left out: 1
     simons_data_sample_273 (S_Naxi-2): not_in_sgdp_file, sex not_checked, genotype no_match
   

   The generation folder holds outcomes.json with one entry per sample.
3. Reconcile with that evidence (make reconcile, or scripts/reconcile.py --run <copy of a run> --entity-root <root holding anvil, anvil_published and the new sgdp>). In reconciled/reconcile_report.json:
   • entities.donor_reach.ANVIL_T2T_CHRY.one_sample_without_donor is 118 (was 448), S_Naxi-2's single-sample files;
   • several_samples_some_without_donor is 62,310: the joint-called files and the SGDP workspace tars, all of which name S_Naxi-2;
   • entities.samples.no_donor.ANVIL_T2T_CHRY lists simons_data_sample_273 with its outcome;
   • donors.ndjson.gz gains hgdp:HGDP00725, sgdp:S_Norwegian-1 and sgdp:B_Australian-3;
   • every file artifact is identical to the same run reconciled on main.
4. make rules-report lists sgdp_sex_agrees and sgdp_genotype_matches under the sample and donor rules.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VF7tbFa5ZTmYWZ6JWKiBBS
🐦 Built with cc-claude-tools

[Copilot]

🟡 Changes recommended

Missing-donor diagnostics omit sample identifiers when no importer outcome exists, leaving affected samples counted but unidentified.

1 open issue

• Preserve identifiers for samples without importer outcomes · New

What changed in this PR

Extends SGDP donor linking to accept recorded genotype matches and adds donor-reach reporting to reconciliation, addressing #718.

Changes:

• Adds genotype-based identity checks and per-sample import outcomes.
• Reports file-level donor reach and missing-donor diagnostics.
• Updates rule reporting, tests, and identity-check documentation.

File	Description
tests/​test_sgdp_entities.py	Tests genotype checks, outcomes, and donor reach.
tests/​test_entities.py	Tests donor-reach counts and diagnostics.
tests/​test_code_rules.py	Validates identity-check declarations.
src/​meta_disco/​sources/​sgdp_genotype_checks.yaml	Records four samples’ genotype verdicts.
src/​meta_disco/​sgdp_entities.py	Applies identity checks and writes outcomes.
src/​meta_disco/​rules/​value_map.yaml	Treats SGDP’s unknown sex as reviewed without a claim.
src/​meta_disco/​reconcile.py	Passes file counts into entity reconciliation.
src/​meta_disco/​reconcile_samples.py	Counts files by dataset and sample list.
src/​meta_disco/​reconcile_entities.py	Builds donor-reach and missing-donor reports.
src/​meta_disco/​entity_evidence.py	Reads and writes outcome sidecars.
src/​meta_disco/​code_rules.py	Declares sex and genotype identity checks.
scripts/​generate_rules_report.py	Includes identity checks in rule reports.
Makefile	Updates SGDP import help and comments.
docs/​studies/​sgdp-genome-identity.md	Documents identity evidence, methods, and verdicts.
docs/​claims-contract.md	Updates linking and reporting requirements.
CLAUDE.md	Updates SGDP architecture guidance.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The new donor links depend on manually recorded genotype measurements that require domain-expert validation and are not yet reproducible through repository tooling.

No open issues 🎉

1 resolved since last review

• Preserve identifiers for samples without importer outcomes

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The additional donor links rely on manually recorded genotype measurements that need domain-expert validation, with reproducible measurement tooling deferred to #738.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The donor identity changes rely on manually recorded genotype measurements that need human validation, with repository-based remeasurement deferred to #738.

No open issues 🎉

_{Review effort: Balanced}