Summary
- Component: remeta
gene, conditional mode (--condition-list / --condition-htp)
- Version:
v0.11.2-9462889 (source checkout: /home/sklasfeld/remeta, commit 9462889, matches the container banner)
- Severity: fatal — SIGABRT, exit 134
REMETA gene fails when conditioning is on and the condition list's first well-formed entry sits beyond the last record on that contig in some cohort's --condition-htp. Any earlier entry whose position has a record at or after it disarms the abort, so the same set of leads passes or fails depending on list order.
Call chain
| Step |
Location |
What happens |
| 1 |
src/run_htp.cpp:213 |
logs loading conditional variants... |
| 2 |
src/run_htp.cpp:216 |
calls util::get_htp_variants(condition_list_file, condition_htp_files, chr) |
| 3 |
src/util.cpp:79 |
per lead, per cohort: htp_reader.seek(cpra[0], pos) |
| 4 |
src/io/bgz_reader.cpp:224-253 |
BgzReader::seek — the defect is here |
| 5 |
src/util.cpp:81-82 |
while (rec.pos <= pos && !htp_reader.eof()) { rec = htp_reader.readrec(); ... } |
| 6 |
src/io/bgz_reader.cpp:179-180 |
readline() returns this->seek_buffer, which is "" |
| 7 |
src/io/htpv4_reader.cpp:205 |
readrec parses the empty line; stod("") throws |
If the abort is in this path, src/run_htp.cpp:220 (found N variants to condition on in study i)
never prints. Presence or absence of those lines in <--out>.gene.log is the cheapest way to
localize the crash.
Root cause: check_eof() is called one line before seek_set is set
src/io/bgz_reader.cpp:239-253:
int ret = tbx_bgzf_itr_next(this->bgzf, this->bgzf_tbx, this->bgzf_itr,
&this->buffer);
if (ret == -1) { // query matched nothing
this->at_eof = true; // line 242 — correct
this->seek_buffer = ""; // line 243
} else if (ret < -1) {
throw runtime_error("tbx_bgzf_itr_next error");
} else {
stringstream ss(this->buffer.s);
getline(ss, this->seek_buffer);
}
this->check_eof(); // line 251
this->seek_set = true; // line 252 <-- one line too late
src/io/bgz_reader.cpp:284-291:
void BgzReader::check_eof() {
if (this->seek_set && this->seek_buffer == "") { // line 285
this->at_eof = true;
} else if (!this->seek_set && bgzf_peek(this->bgzf) == -1) {
this->at_eof = true;
} else {
this->at_eof = false; // line 290
}
}
On a reader's first seek seek_set is still false, so the guard at line 285 cannot fire.
Control reaches the bgzf_peek branch, and unless the underlying stream is physically at EOF,
line 290 resets at_eof back to false — discarding the correct value just set at line 242.
Consequences, in order:
HTPv4Reader::eof() (src/io/htpv4_reader.hpp) reports "not at end".
src/util.cpp:81 enters its loop and calls readrec().
readrec → BgzReader::readline (src/io/bgz_reader.cpp:172). seek_set is true by now, so
line 180 takes line = this->seek_buffer — the empty string set at line 243.
readrec parses "". Every >> extraction fails, leaving each token empty.
Trigger conditions
The abort requires a tabix query that matches nothing. src/io/bgz_reader.cpp:248 queries
tbx_itr_queryi(tbx, tid, position - 1, HTS_POS_MAX) — i.e. [pos, end-of-contig], open ended.
- It matches whenever any record exists at or after
pos on that contig. Proximity to the
nearest record is irrelevant: a lead in a gene desert lands on the next record and is handled
correctly.
- It matches nothing only when
pos > (last record's position on that contig in that file).
- A contig missing from the index fails earlier and differently:
tbx_name2id < 0 →
runtime_error("called seek with invalid contig") (src/io/bgz_reader.cpp:233-236)
Only the first seek on a given reader can trigger it. From the second seek onward seek_set is
already true, the guard at line 285 fires, at_eof stays true, and src/util.cpp:81 skips the
loop — remeta simply does not find that lead. get_htp_variants builds one reader per cohort
(src/util.cpp:69), so in practice it is the first well-formed variant ID in the condition list
that decides whether the run aborts. Malformed IDs are skipped at src/util.cpp:73-76 and do not
count.
Confirmed against v0.11.2-9462889 on a two-cohort fixture with two leads, both absent from the
HTP, differing only in position — 1:60000 before every record, 1:140000 after the last one:
| condition list |
result |
1:60000 alone |
PASS — and silently not conditioned on |
1:140000 alone |
ABORT, what(): stod |
1:60000, then 1:140000 |
PASS — the fatal lead is disarmed |
1:140000, then 1:60000 |
ABORT — the same two leads, only the order changed |
Two consequences worth stating plainly:
- Survival depends on clumping output order. The same data can abort or not between runs if
that order is not stable.
- The non-fatal case is not harmless either. A lead whose seek matches a record that is not
the lead is silently not conditioned on — no warning, and the run degrades to marginal for that
lead.
The practical scenario. Condition leads come from LD clumping against a common/imputed
reference panel, while --htp for the gene tests is an exome/PAV HTP. Exome variants stop well
before the end of a chromosome, so a common lead beyond the last PAV row matches nothing. The
effective ceiling is the minimum over cohorts of the last row's position, since any one
cohort's empty query is enough.
Summary
gene, conditional mode (--condition-list/--condition-htp)v0.11.2-9462889(source checkout:/home/sklasfeld/remeta, commit9462889, matches the container banner)REMETA gene fails when conditioning is on and the condition list's first well-formed entry sits beyond the last record on that contig in some cohort's --condition-htp. Any earlier entry whose position has a record at or after it disarms the abort, so the same set of leads passes or fails depending on list order.
Call chain
src/run_htp.cpp:213loading conditional variants...src/run_htp.cpp:216util::get_htp_variants(condition_list_file, condition_htp_files, chr)src/util.cpp:79htp_reader.seek(cpra[0], pos)src/io/bgz_reader.cpp:224-253BgzReader::seek— the defect is heresrc/util.cpp:81-82while (rec.pos <= pos && !htp_reader.eof()) { rec = htp_reader.readrec(); ... }src/io/bgz_reader.cpp:179-180readline()returnsthis->seek_buffer, which is""src/io/htpv4_reader.cpp:205readrecparses the empty line;stod("")throwsIf the abort is in this path,
src/run_htp.cpp:220(found N variants to condition on in study i)never prints. Presence or absence of those lines in
<--out>.gene.logis the cheapest way tolocalize the crash.
Root cause:
check_eof()is called one line beforeseek_setis setsrc/io/bgz_reader.cpp:239-253:src/io/bgz_reader.cpp:284-291:On a reader's first seek
seek_setis stillfalse, so the guard at line 285 cannot fire.Control reaches the
bgzf_peekbranch, and unless the underlying stream is physically at EOF,line 290 resets
at_eofback tofalse— discarding the correct value just set at line 242.Consequences, in order:
HTPv4Reader::eof()(src/io/htpv4_reader.hpp) reports "not at end".src/util.cpp:81enters its loop and callsreadrec().readrec→BgzReader::readline(src/io/bgz_reader.cpp:172).seek_setistrueby now, soline 180 takes
line = this->seek_buffer— the empty string set at line 243.readrecparses"". Every>>extraction fails, leaving each token empty.Trigger conditions
The abort requires a tabix query that matches nothing.
src/io/bgz_reader.cpp:248queriestbx_itr_queryi(tbx, tid, position - 1, HTS_POS_MAX)— i.e.[pos, end-of-contig], open ended.poson that contig. Proximity to thenearest record is irrelevant: a lead in a gene desert lands on the next record and is handled
correctly.
pos > (last record's position on that contig in that file).tbx_name2id< 0 →runtime_error("called seek with invalid contig")(src/io/bgz_reader.cpp:233-236)Only the first seek on a given reader can trigger it. From the second seek onward
seek_setisalready
true, the guard at line 285 fires,at_eofstaystrue, andsrc/util.cpp:81skips theloop — remeta simply does not find that lead.
get_htp_variantsbuilds one reader per cohort(
src/util.cpp:69), so in practice it is the first well-formed variant ID in the condition listthat decides whether the run aborts. Malformed IDs are skipped at
src/util.cpp:73-76and do notcount.
Confirmed against
v0.11.2-9462889on a two-cohort fixture with two leads, both absent from theHTP, differing only in position —
1:60000before every record,1:140000after the last one:1:60000alone1:140000alonewhat(): stod1:60000, then1:1400001:140000, then1:60000Two consequences worth stating plainly:
that order is not stable.
the lead is silently not conditioned on — no warning, and the run degrades to marginal for that
lead.
The practical scenario. Condition leads come from LD clumping against a common/imputed
reference panel, while
--htpfor the gene tests is an exome/PAV HTP. Exome variants stop wellbefore the end of a chromosome, so a common lead beyond the last PAV row matches nothing. The
effective ceiling is the minimum over cohorts of the last row's position, since any one
cohort's empty query is enough.