Skip to content

remeta gene: bare std::invalid_argument: stod when a condition lead is out of range #11

Description

@sklasfeld-p4

Summary

  • Component: remeta gene, conditional mode (--condition-list / --condition-htp)
  • Version: v0.11.2-9462889 (source checkout: /home/sklasfeld/remeta, commit 9462889, matches the container banner)
  • Severity: fatal — SIGABRT, exit 134

REMETA gene fails when conditioning is on and the condition list's first well-formed entry sits beyond the last record on that contig in some cohort's --condition-htp. Any earlier entry whose position has a record at or after it disarms the abort, so the same set of leads passes or fails depending on list order.

Call chain

Step Location What happens
1 src/run_htp.cpp:213 logs loading conditional variants...
2 src/run_htp.cpp:216 calls util::get_htp_variants(condition_list_file, condition_htp_files, chr)
3 src/util.cpp:79 per lead, per cohort: htp_reader.seek(cpra[0], pos)
4 src/io/bgz_reader.cpp:224-253 BgzReader::seek — the defect is here
5 src/util.cpp:81-82 while (rec.pos <= pos && !htp_reader.eof()) { rec = htp_reader.readrec(); ... }
6 src/io/bgz_reader.cpp:179-180 readline() returns this->seek_buffer, which is ""
7 src/io/htpv4_reader.cpp:205 readrec parses the empty line; stod("") throws

If the abort is in this path, src/run_htp.cpp:220 (found N variants to condition on in study i)
never prints. Presence or absence of those lines in <--out>.gene.log is the cheapest way to
localize the crash.

Root cause: check_eof() is called one line before seek_set is set

src/io/bgz_reader.cpp:239-253:

  int ret = tbx_bgzf_itr_next(this->bgzf, this->bgzf_tbx, this->bgzf_itr,
                              &this->buffer);
  if (ret == -1) {              // query matched nothing
    this->at_eof = true;        // line 242 — correct
    this->seek_buffer = "";     // line 243
  } else if (ret < -1) {
    throw runtime_error("tbx_bgzf_itr_next error");
  } else {
    stringstream ss(this->buffer.s);
    getline(ss, this->seek_buffer);
  }

  this->check_eof();            // line 251
  this->seek_set = true;        // line 252  <-- one line too late

src/io/bgz_reader.cpp:284-291:

void BgzReader::check_eof() {
  if (this->seek_set && this->seek_buffer == "") {     // line 285
    this->at_eof = true;
  } else if (!this->seek_set && bgzf_peek(this->bgzf) == -1) {
    this->at_eof = true;
  } else {
    this->at_eof = false;                              // line 290
  }
}

On a reader's first seek seek_set is still false, so the guard at line 285 cannot fire.
Control reaches the bgzf_peek branch, and unless the underlying stream is physically at EOF,
line 290 resets at_eof back to false — discarding the correct value just set at line 242.

Consequences, in order:

  1. HTPv4Reader::eof() (src/io/htpv4_reader.hpp) reports "not at end".
  2. src/util.cpp:81 enters its loop and calls readrec().
  3. readrec → BgzReader::readline (src/io/bgz_reader.cpp:172). seek_set is true by now, so
    line 180 takes line = this->seek_buffer — the empty string set at line 243.
  4. readrec parses "". Every >> extraction fails, leaving each token empty.

Trigger conditions

The abort requires a tabix query that matches nothing. src/io/bgz_reader.cpp:248 queries
tbx_itr_queryi(tbx, tid, position - 1, HTS_POS_MAX) — i.e. [pos, end-of-contig], open ended.

  • It matches whenever any record exists at or after pos on that contig. Proximity to the
    nearest record is irrelevant: a lead in a gene desert lands on the next record and is handled
    correctly.
  • It matches nothing only when pos > (last record's position on that contig in that file).
  • A contig missing from the index fails earlier and differently: tbx_name2id < 0 →
    runtime_error("called seek with invalid contig") (src/io/bgz_reader.cpp:233-236)

Only the first seek on a given reader can trigger it. From the second seek onward seek_set is
already true, the guard at line 285 fires, at_eof stays true, and src/util.cpp:81 skips the
loop — remeta simply does not find that lead. get_htp_variants builds one reader per cohort
(src/util.cpp:69), so in practice it is the first well-formed variant ID in the condition list
that decides whether the run aborts. Malformed IDs are skipped at src/util.cpp:73-76 and do not
count.

Confirmed against v0.11.2-9462889 on a two-cohort fixture with two leads, both absent from the
HTP, differing only in position — 1:60000 before every record, 1:140000 after the last one:

condition list result
1:60000 alone PASS — and silently not conditioned on
1:140000 alone ABORT, what(): stod
1:60000, then 1:140000 PASS — the fatal lead is disarmed
1:140000, then 1:60000 ABORT — the same two leads, only the order changed

Two consequences worth stating plainly:

  • Survival depends on clumping output order. The same data can abort or not between runs if
    that order is not stable.
  • The non-fatal case is not harmless either. A lead whose seek matches a record that is not
    the lead is silently not conditioned on — no warning, and the run degrades to marginal for that
    lead.

The practical scenario. Condition leads come from LD clumping against a common/imputed
reference panel, while --htp for the gene tests is an exome/PAV HTP. Exome variants stop well
before the end of a chromosome, so a common lead beyond the last PAV row matches nothing. The
effective ceiling is the minimum over cohorts of the last row's position, since any one
cohort's empty query is enough.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions