test(TEST-006): IPO pipeline end-to-end scenario test

[DoRmAmMu1997]

Ticket scope (TEST-006)

Goal: lock the seams BETWEEN the IPO stages. Each stage already has a thorough isolated suite (test_ipo_sebi_ingestion, test_scan_ipo_filings_job, test_ipo_document_downloader, test_ipo_manual_repository, test_ipo_ratio_engine, test_ipo_scorecard, test_ipo_verdict, test_ipo_repository) — but every one of them starts from a hand-built fixture, so a contract drift between two stages (e.g. ingestion persisting a URL shape the downloader rejects, or the extraction verifying against different bytes than the download wrote) would pass all of them and only fail in production.

In scope

• One new scenario file, tests/test_ipo_pipeline_e2e.py, walking a single company end-to-end on real repository functions and a real database:
  1. Ingest — the real run_scan_ipo_filings job loop + the real ingest_filings, with only the SEBI HTTP fetch programmed. Asserts the issue's company identity is derived (title normalization → "Example Limited" / key "example limited"), status RHP_FILED, document NOT_DOWNLOADED.
  2. Download — the real download_document two-transaction orchestration around the real download_document_file, with HTTP/DNS doubled (the downloader suite's fake-session pattern: detail page → iframe → PDF). Asserts the exact request sequence and the on-disk content-addressed bytes.
  3. Manual extraction — the real submit_manual_extraction, verified against the stage-2 cache (not a hand-planted file).
  4. Ratios — an admin records the price band (update_issue), then get_latest_ipo_ratios runs the pure engine on the latest revision. Spot values are hand-checkable: EPS 20, ROE 12.5%, P/E 12.1; EPS reconciliation not materially different.
  5. Score + verdict — evaluate_issue with the seven-factor input whose source_documents must be URLs stage 1 actually registered → "Recommended", get_latest_recommendation round-trip.
• Provenance-continuity lock: the faked PDF's SHA-256 must surface unchanged in the download result → document row → manual revision → ratio analysis. Any stage silently re-reading different bytes or skipping verification breaks the chain.
• Parametrized over both conftest engines (in-memory and file-backed with production SQLite pragmas), per the house testing conventions.

Out of scope (deliberate)

• No production code changes; no existing test modified. Fakes replicate the downloader suite's doubles rather than importing across test modules (cross-test imports are brittle).
• No UI stage — the admin IPO page has its own render suite; this ticket is about the backend pipeline.
• Failure-path e2e permutations (download failure → extraction rejection etc.) stay in the per-stage suites where they are already covered.

This is ticket 5 of a 10-ticket second improvement wave (TEST-007 #98, REF-003 #99, AI-006 #100, IPO-006 #101, TEST-006, PERF-002, DEPLOY-004, QUAL-006, QUAL-007, DOC-003), each shipped as its own PR off main.

Gates (all green, local, Python 3.13)

Gate	Result
pytest -q --cov=backend --cov=screeners --cov=ui --cov-fail-under=87	1,388 passed, 1 skipped; coverage 88.13%
New scenario	2 passed (in-memory + file-backed)
pre_commit validate-config	clean
compileall	clean
ruff check	clean
mypy (119 files)	clean
bandit	clean
pip_audit -r constraints.txt	no known vulnerabilities (one transient PyPI timeout, clean on retry)

constraints.txt / pyproject.toml diff vs main: empty (as required).

Review

/code-review: Approve — the scenario drives real functions at every stage (the only fakes are HTTP, DNS, clocks, and the audit sink); assertions are hand-computable; the fixture parametrization uses request.getfixturevalue so both engines run the identical walk. /security-review: no findings (test-only diff; the scenario additionally exercises the SSRF gate, content-addressed verification, and actor normalization end-to-end).

🤖 Generated with Claude Code

[DoRmAmMu1997]

Self-review (posted as COMMENT — the bot runs as the PR author and cannot APPROVE). Verdict: Approve.

Checks performed on the scenario:

1. Real functions at every stage — the only doubles are true externals: the SEBI HTTP fetch (job fetcher=), the downloader's HTTP session + DNS resolver (the downloader suite's own pattern), fixed clocks, and the audit sink. ingest_filings, download_document, download_document_file, submit_manual_extraction, get_latest_ipo_ratios, evaluate_issue, and get_latest_recommendation all run for real against a real schema.
2. Provenance chain is the core lock — the PDF digest is asserted at four hand-offs (download result → document row → manual revision → ratio analysis). This is the assertion that fails if any stage skips verification or silently re-reads different bytes; no per-stage suite can catch that class.
3. Assertions are hand-computable — 10 crore PAT / 50 lakh shares → EPS 20; net worth 80 crore → ROE 12.5%; upper band 242 → P/E 12.1 (documented in the payload builder's beginner note). Decimal equality in Python is numeric, so quantization (20 vs 20.0000) cannot cause false failures.
4. Derived identity asserted, not copied — the issue's company name/key come from normalize_company_identity on the filing title, so the test locks the title-normalization seam into ingestion too.
5. Both engines — parametrized over the in-memory and file-backed (production-pragma) conftest factories via request.getfixturevalue; both pass. The file-backed run also exercises real fsync/os.replace filesystem behavior in stage 2.
6. No cross-test imports — the HTTP doubles replicate the downloader suite's shape locally instead of importing from tests/test_ipo_document_downloader.py; importing between test modules couples suites and breaks collection isolation.
7. Suites unmodified — no existing test touched; the IPO docstring-policy guard passes on the new module (every definition, including nested fakes, is documented).

Gates: 1,388 passed + 1 skipped, coverage 88.13% (floor 87); ruff/mypy(119)/bandit/pre-commit/compileall clean; pip-audit clean on retry after one transient PyPI read-timeout; constraints/pyproject diff vs main empty. Security review: no findings (test-only; the walk additionally exercises the SSRF gate and content-addressed verification end-to-end).

[DoRmAmMu1997]

Codex review plan:

I found no actionable standalone issue in TEST-006. I will rerun the two-engine IPO pipeline scenario and perform a test-only security review, then post a no-findings verdict. No branch change is planned unless the scenario exposes a regression.

[DoRmAmMu1997]

Completion verdict

Complete — no findings. This is a completion comment, not an approval.

No follow-up code changes were required. The IPO pipeline was rerun against both supported database engines; 2 end-to-end tests passed, and the review found no remaining actionable findings.