diff --git a/research/publication-metadata-reuse/PROOF.md b/research/publication-metadata-reuse/PROOF.md new file mode 100644 index 0000000000..320cd3e066 --- /dev/null +++ b/research/publication-metadata-reuse/PROOF.md @@ -0,0 +1,58 @@ +# Acceptance equivalence and implementation limits + +Assume the same stable working tree, publication commit and ordinary successful +Git object reads as the original check. Both implementations inventory the same +regular committed files, compare every working byte, and complete the blob +iterator's final Git-status check before returning success. + +The original check then calls `publication_sources()`, which reads the size and +bytes of `commit:package/SOURCES.json` through two additional Git processes. +The candidate retains that same immutable blob from the existing inventory +comparison. It retains its bytes only when their size is at most the existing +16 MiB metadata limit. For an oversized blob it retains only its size, then +rejects after the complete comparison iterator finishes. A missing metadata file +still rejects only when history checking was requested. + +Both paths check the missing/size conditions, parse the same JSON bytes and call +exactly the same metadata/history validator. Therefore successful runs have +identical complete result dictionaries and the same declared pin acceptance. +The standalone `source_history()` still authenticates its own metadata through +`publication_sources()`. The new internal helper is called only with JSON from +these committed-byte boundaries. + +Malformed JSON is parsed after the complete byte comparison, as before. Working +metadata drift and a final blob-reader failure still precede parsing. All +metadata shape, row/count limits, exact commit IDs, raw ancestry, shallow history, +replacement/graft isolation, retained Git errors and timeout checks are unchanged. +The candidate no longer performs the two duplicate Git reads, so failures injected +only into those removed operations have no counterpart. This is an acceptance +argument on successful immutable reads, not an identical subprocess trace claim. +Concurrent checkout mutation remains outside the original stable-checkout contract. + +Eight new tests use actual committed fixture bytes. They cover no metadata refetch, +the standalone reader, drift before parsing, malformed committed JSON, final-reader +failure before parsing, committed versus later working bytes, exact size bounds, +absence of cross-call caching and explicit older publications. Existing history, +byte, execute-mode and replacement tests remain in the registered suite. Finite +controls corroborate the argument; they are not a proof-kernel formalization. + +## Code constraints + +The additional retained payload is at most 16 MiB, local to one call, with no new +loop or subprocess. The original blob reader still allocates each complete blob; +its existing allocation is outside this change. Python runtime allocation is a +managed-runtime deviation from fixed allocation. Existing input bounds, finite +file inventories and Git timeouts apply. The now 58-line check function is kept +as one inventory/comparison transaction; splitting it would introduce a second +owner for the captured metadata. No recursive call is added. Explicit `require` +checks remain active under optimized Python. + +The optional benchmark loads one fixed, pinned historical Python module through +importlib. This bounded baseline-loading exception adds no dynamic dispatch to +the production preflight. It times complete `check()` calls and requires full +result equality for every sample. Profile measurements are not included among +the reported timing samples. Timing numbers do not predict a mathematical replay +or physical multiplier speedup. + +No selected cold replay, Lean, Docker, physical multiplier or supplier admission +was run. Independent human and formal proof-kernel review are unclaimed. diff --git a/research/publication-metadata-reuse/README.md b/research/publication-metadata-reuse/README.md new file mode 100644 index 0000000000..c026873f38 --- /dev/null +++ b/research/publication-metadata-reuse/README.md @@ -0,0 +1,86 @@ +# Reuse authenticated publication metadata + +The complete publication byte/history check can avoid two duplicate Git +processes. `check()` already reads the committed `SOURCES.json` blob and compares +it with the working file. It now retains those immutable bytes for the subsequent +source-history validation. The standalone `source_history()` interface and every +source-commit resolution and ancestry query remain. + +This is a measured verifier improvement with a scoped equivalence argument and +finite executable controls. It supplies no mathematical construction, supplier +admission, complete multiplication theorem or new kappa. No predecessor +mathematical verifier is run by this package. + +## Evidence + +On CPython 3.12.15, macOS arm64, nine alternating baseline/candidate complete +checks per package gave: + +| Unchanged package | Baseline median, range | Candidate median, range | Reduction | +| --- | --- | --- | --- | +| `stage-zero-forced-translation` | 121.65 ms, 117.86–132.99 | 99.36 ms, 97.92–102.72 | 18.33% | +| `p104-helper470-twin-rung-removal` | 120.33 ms, 118.24–126.17 | 101.49 ms, 98.93–103.96 | 15.66% | + +All 36 timed checks returned identical complete receipts within their package; +no timed call failed. Each call checks the full package/input inventory, exact +working/committed bytes, committed source metadata, shallow-history guard and +all declared source-commit ancestry. These are complete `check()` function times +in a warm Python process. Imports, baseline extraction and CLI startup are outside +the timed region. Mathematical replay is outside the measured workload. + +A separate profile attributed 109 of 130 ms to synchronous Git subprocess +calls, including 21 ms in the duplicate metadata reader. For these one-pin +packages, history-specific Git calls fall from five to three and total Git +processes from nine to seven. The host had 18 logical CPUs and load averages +5.76, 3.64 and 2.80 at orientation; other work was active. Alternating samples +have separated ranges. There is no cross-host speed claim. + +[PROOF.md](PROOF.md) gives the acceptance argument, bounded retention, failure +ordering, controls and implementation limits. The registered verifier runs the +new tests plus retained history, byte, executable-mode and replacement controls. +Its generated stdout receipt binds all executed test files and their shared +scripts by SHA256, including the exporter and scaffold imported by replacement +controls. [TASK.json](TASK.json) records the scope and next question. + +## Reproduce + +From the repository root: + +```sh +/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/publication-metadata-reuse/verify.py +/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -O -W error -B research/publication-metadata-reuse/verify.py +/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/research_package.py --package research/publication-metadata-reuse +/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -B scripts/check_research_commit.py --package research/publication-metadata-reuse --check-source-history +``` + +Elsewhere substitute CPython 3.12. Git and full source history are required. +Tests create and remove only synthetic temporary repositories. They do not +change project refs or execute predecessor mathematical packages. + +For optional timing, first materialize both benchmark packages and their +`VERIFY.json` inputs. Use an existing directory outside the checkout: + +```sh +/Users/nick/git/integer-mult-bounds/.venv/bin/python3.12 -I -W error -B research/publication-metadata-reuse/benchmark.py --scratch /absolute/external/scratch --repeats 9 +``` + +The benchmark obtains the baseline from immutable Git objects at +`721c03822c421b7f7fcbb2598993b947c4d88419`, compares complete receipts, records +every sample and source hash, and removes its temporary baseline module. +Timing is optional and has no CI speed threshold. + +## Open next questions + +Does the same saving remain measurable on a second operating system and Git +implementation? Run the same complete-check benchmark on unchanged fixtures, +with imports outside both arms. A second host and materialized inputs are +required. Joint ancestry batching is separate open work. + +## Credit + +This contribution was produced by Nick Petalas. Credit ywz65 for the publication +byte preflight, Nick Petalas for its original source-history check, roger-sato +for immutable Git source isolation, Douglas Colkitt for the package runner and +the retained source-reader contributors. Reusing immutable bytes is an ordinary +technique; the contribution is its bounded integration, failure controls and +measured complete-check improvement. Apache-2.0. Existing notices are unchanged. diff --git a/research/publication-metadata-reuse/SOURCES.json b/research/publication-metadata-reuse/SOURCES.json new file mode 100644 index 0000000000..6858893ff3 --- /dev/null +++ b/research/publication-metadata-reuse/SOURCES.json @@ -0,0 +1,24 @@ +{ + "schema_version": 1, + "base_commit": "721c03822c421b7f7fcbb2598993b947c4d88419", + "files": [ + { + "path": "certificates/selected-result.json", + "usage": "provenance", + "git_blob": "d59b5035e7b4cbaa9d688a1d32dd0d6eb79c466a", + "sha256": "9b8d0eea95b3bf0dfcb7938a732a6439b87a2685dec2a5ab9e525b3b34debbfa" + }, + { + "path": "docs/research/agent-task-catalog.json", + "usage": "provenance", + "git_blob": "3404fbdd5e7529a0674eef26114b5071e4586cec", + "sha256": "ead6cdc9d0af6d58e8a49744d8ec8c7c15e13d2ba93c276c434bcf9b51b93a81" + }, + { + "path": "scripts/check_research_commit.py", + "usage": "provenance", + "git_blob": "dc0be6c743f79b3131e674d92c81b75b069812c9", + "sha256": "63026ec3805f2a3236cc071fc78cf1c54bc6528c43e2f3d82548dc68bf346bce" + } + ] +} diff --git a/research/publication-metadata-reuse/TASK.json b/research/publication-metadata-reuse/TASK.json new file mode 100644 index 0000000000..707628312a --- /dev/null +++ b/research/publication-metadata-reuse/TASK.json @@ -0,0 +1,61 @@ +{ + "schema_version": 1, + "task_id": "replay-runtime", + "priority_revision": 8, + "base_commit": "721c03822c421b7f7fcbb2598993b947c4d88419", + "original_card": { + "id": "replay-runtime", + "question": "Can one measured replay bottleneck be sped up without dropping coverage?", + "starting_points": [ + "scripts/run_isolated_tests.py", + "scripts/verify_replay_coverage.py", + "certificates/selected-result.json" + ], + "methods": [ + "Profile one isolated replay and optimize the measured bottleneck.", + "Design source-bound reuse with explicit dependency invalidation and negative controls." + ], + "acceptance_gate": "Report before/after wall time on the same fixture, retained checks and invalidation behavior.", + "first_step": { + "command": [ + "python3", + "-B", + "scripts/research_selected_baseline.py" + ], + "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY", + "inspect": "Resolve the current selected review, separate complex/ordinary BIT/admitted BIT/outer budgets, and compare the card against live handoffs before starting. Historical ceilings require exact source/model equivalence.", + "scope": "Source-binding orientation only; not a scientific replay or all-size proof. Materialize selected inputs if this checkout is sparse." + }, + "first_experiment": "Measure one unchanged fixture, preserve check coverage, and test one optimization with invalidation controls.", + "bottleneck_link": "Faster trustworthy replays increase review throughput and let mechanism candidates be checked sooner.", + "allowed_variations": [ + "Profile one stage or design one source-bound cache boundary." + ], + "scope_change_triggers": [ + "Skipping coverage or comparing different fixtures to claim speedup." + ], + "overlap_terms": [ + "replay runtime", + "cache invalidation", + "isolated tests" + ] + }, + "scope_status": "refined", + "actual_scope": "Reuse the committed SOURCES.json payload already authenticated by the complete publication byte/history check, without changing ancestry operations.", + "scope_change_reason": "The replay-runtime card is narrowed to duplicate publication metadata reads; it is a tooling improvement, not a mathematical replay.", + "bottleneck_link": "Publication preflight is required before research submission; avoiding duplicate immutable-source reads reduces that wait while preserving its evidence contract.", + "budget_minutes": 60, + "stop_rule": "Stop at the first reproducible complete-check improvement with identical receipts and passing rejection controls; otherwise publish a bounded negative finding.", + "outcome": "completed", + "evidence": "Nine alternating runs per arm on each of two real packages; all 36 complete receipts match within package. Median reductions 18.33% and 15.66%. Hash-bound controls and optional benchmark emit stdout receipts.", + "author": "Nick Petalas", + "ai_assistance": "Independent human and formal proof-kernel review are unclaimed.", + "open_questions": [ + { + "question": "Does the same complete-check improvement remain measurable on a second operating system and Git implementation?", + "why_it_matters": "The local improvement does not establish cross-host timing behavior.", + "first_step": "Run benchmark.py on the same two unchanged package inventories, with nine alternating samples and complete receipt equality.", + "blocked_by": "A second host and matching materialized fixtures." + } + ] +} diff --git a/research/publication-metadata-reuse/VERIFY.json b/research/publication-metadata-reuse/VERIFY.json new file mode 100644 index 0000000000..95bf63c0a3 --- /dev/null +++ b/research/publication-metadata-reuse/VERIFY.json @@ -0,0 +1,44 @@ +{ + "schema_version": 1, + "status": "ready", + "python": "3.12", + "requirements": [], + "inputs": [ + "scripts/check_research_commit.py", + "scripts/research_package.py", + "scripts/export_research_package.py", + "scripts/new_research_package.py", + "tests/test_research_publication_metadata_reuse.py", + "tests/test_research_source_history.py", + "tests/test_research_commit_bytes.py", + "tests/test_research_commit_modes.py", + "tests/test_research_git_replacements.py" + ], + "commands": [ + { + "argv": [ + "{python}", + "-I", + "-W", + "error", + "-B", + "{package}/verify.py" + ], + "timeout_seconds": 120, + "checks": "Run new immutable-metadata handoff controls and retained history, byte, mode and replacement tests; emit a hash-bound coverage receipt." + }, + { + "argv": [ + "{python}", + "-I", + "-O", + "-W", + "error", + "-B", + "{package}/verify.py" + ], + "timeout_seconds": 120, + "checks": "Repeat all same controls under optimized Python; explicit validation must remain active." + } + ] +} diff --git a/research/publication-metadata-reuse/benchmark.py b/research/publication-metadata-reuse/benchmark.py new file mode 100644 index 0000000000..65f327db79 --- /dev/null +++ b/research/publication-metadata-reuse/benchmark.py @@ -0,0 +1,84 @@ +#!/usr/bin/env python3 +"""Compare complete publication checks, with imports and profiling outside timing.""" +import argparse +import hashlib +import importlib.util +import json +from pathlib import Path +import platform +import statistics +import subprocess +import sys +import tempfile +import time + +ROOT = Path(__file__).resolve().parents[2] +BASE = '721c03822c421b7f7fcbb2598993b947c4d88419' +PACKAGES = ('research/stage-zero-forced-translation', + 'research/p104-helper470-twin-rung-removal') + + +def load_module(name, path): + spec = importlib.util.spec_from_file_location(name, path) + if spec is None or spec.loader is None: + raise ValueError('cannot load checker: ' + str(path)) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def compare(baseline, candidate, package, repeats): + samples = {'baseline': [], 'candidate': []} + receipt = None + for _ in range(repeats): + for arm, checker in (('baseline', baseline), ('candidate', candidate)): + start = time.perf_counter() + result = checker.check(ROOT, package, BASE, check_source_history=True) + samples[arm].append(time.perf_counter() - start) + if receipt is None: + receipt = result + if result != receipt: + raise ValueError('complete publication receipts differ') + summary = {} + for arm, values in samples.items(): + summary[arm] = {'seconds': values, 'median_seconds': statistics.median(values), + 'range_seconds': [min(values), max(values)]} + summary['fraction_reduced'] = 1 - (summary['candidate']['median_seconds'] / + summary['baseline']['median_seconds']) + return {'package': package, 'complete_receipt': receipt, 'samples': summary, + 'successful_checks': 2 * repeats, 'errors': 0} + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--scratch', type=Path, required=True) + parser.add_argument('--repeats', type=int, default=9, choices=range(5, 32)) + args = parser.parse_args() + if not args.scratch.is_dir(): + parser.error('--scratch must name an existing external directory') + if args.scratch.resolve().is_relative_to(ROOT): + parser.error('scratch must be outside the checkout') + sys.path.insert(0, str(ROOT / 'scripts')) + import research_package as runner + original = subprocess.run( + ['git', 'show', BASE + ':scripts/check_research_commit.py'], cwd=ROOT, + env=runner.git_source_env(), capture_output=True, check=True, timeout=30).stdout + current = ROOT / 'scripts/check_research_commit.py' + with tempfile.TemporaryDirectory(dir=args.scratch) as temporary: + path = Path(temporary) / 'baseline.py' + path.write_bytes(original) + baseline = load_module('baseline_publication_checker', path) + candidate = load_module('candidate_publication_checker', current) + results = [compare(baseline, candidate, name, args.repeats) for name in PACKAGES] + report = {'schema': 'publication-metadata-reuse-benchmark/v1', 'base_commit': BASE, + 'python': sys.version, 'platform': platform.platform(), + 'baseline_sha256': hashlib.sha256(original).hexdigest(), + 'candidate_sha256': hashlib.sha256(current.read_bytes()).hexdigest(), + 'runner_sha256': hashlib.sha256((ROOT / 'scripts/research_package.py').read_bytes()).hexdigest(), + 'timed_region': 'complete check(); imports and baseline extraction excluded', + 'order': 'baseline,candidate repeated', 'results': results} + print(json.dumps(report, sort_keys=True)) + + +if __name__ == '__main__': + main() diff --git a/research/publication-metadata-reuse/verify.py b/research/publication-metadata-reuse/verify.py new file mode 100644 index 0000000000..3fd3a75fba --- /dev/null +++ b/research/publication-metadata-reuse/verify.py @@ -0,0 +1,39 @@ +#!/usr/bin/env python3 +"""Run finite publication controls and emit a source-bound receipt.""" +import hashlib +import json +from pathlib import Path +import sys +import unittest + +ROOT = Path(__file__).resolve().parents[2] +PATTERNS = ('test_research_publication_metadata_reuse.py', + 'test_research_source_history.py', 'test_research_commit_bytes.py', + 'test_research_commit_modes.py', 'test_research_git_replacements.py') + + +def main(): + loader = unittest.TestLoader() + suite = unittest.TestSuite() + for pattern in PATTERNS: + tests = loader.discover(str(ROOT / 'tests'), pattern=pattern) + if tests.countTestCases() == 0: + raise ValueError('no controls found: ' + pattern) + suite.addTests(tests) + count = suite.countTestCases() + result = unittest.TextTestRunner(stream=sys.stderr, verbosity=1).run(suite) + if not result.wasSuccessful() or result.testsRun != count: + raise SystemExit(1) + names = ['scripts/check_research_commit.py', 'scripts/research_package.py', + 'scripts/export_research_package.py', 'scripts/new_research_package.py', + 'research/publication-metadata-reuse/verify.py'] + names.extend('tests/' + pattern for pattern in PATTERNS) + receipt = {'schema': 'publication-metadata-reuse-controls/v1', 'tests_run': count, + 'skipped': [(str(test), reason) for test, reason in result.skipped], + 'source_sha256': {name: hashlib.sha256((ROOT / name).read_bytes()).hexdigest() + for name in names}} + print('PASS_PUBLICATION_METADATA_REUSE ' + json.dumps(receipt, sort_keys=True)) + + +if __name__ == '__main__': + main() diff --git a/scripts/check_research_commit.py b/scripts/check_research_commit.py index dc0be6c743..571ec97809 100644 --- a/scripts/check_research_commit.py +++ b/scripts/check_research_commit.py @@ -58,7 +58,11 @@ def source_history(root, package, commit): This is declared history availability, not full SOURCES.json validation or discovery of imports, archive references or runtime dependencies. """ - sources = publication_sources(root, package, commit) + return _source_history_from_metadata(root, publication_sources(root, package, commit), commit) + + +def _source_history_from_metadata(root, sources, commit): + """Validate metadata obtained from the immutable publication blob.""" require(isinstance(sources, dict), 'SOURCES.json must be an object') base = sources.get('base_commit') require(isinstance(base, str) and re.fullmatch(r'[0-9a-f]{40,64}', base), @@ -121,6 +125,7 @@ def check(root, package, ref='HEAD', check_source_history=False, check_modes=Fal ', '.join(sorted(set(entries) ^ expected))) names = sorted(entries) require(names, 'empty package inventory') + source_payload, source_size = None, None with closing(git_blobs(root, [entries[name] for name in names])) as blobs: # Advance the blob iterator once past its last yield, so its final # Git exit-status check runs before reporting success. @@ -134,11 +139,17 @@ def check(root, package, ref='HEAD', check_source_history=False, check_modes=Fal 'working/commit bytes differ: '+name+' (working SHA256 '+ hashlib.sha256(working).hexdigest()+', commit SHA256 '+ hashlib.sha256(committed).hexdigest()+')') + if check_source_history and name == package + '/SOURCES.json': + source_size = len(committed) + source_payload = committed if source_size <= MAX_SOURCE_METADATA_BYTES else None result = {'commit':commit, 'files':len(names), 'package':package} if check_modes: result['executable_modes_checked'] = True if check_source_history: - result['source_history_commits'] = source_history(root, package, commit) + require(source_size is not None, 'missing publication SOURCES.json') + require(source_size <= MAX_SOURCE_METADATA_BYTES, 'source metadata byte limit exceeded') + result['source_history_commits'] = _source_history_from_metadata( + root, json.loads(source_payload), commit) return result diff --git a/tests/test_research_publication_metadata_reuse.py b/tests/test_research_publication_metadata_reuse.py new file mode 100644 index 0000000000..0773576704 --- /dev/null +++ b/tests/test_research_publication_metadata_reuse.py @@ -0,0 +1,140 @@ +"""Actual-publication controls for reusing authenticated source metadata.""" +import json +from pathlib import Path +import subprocess +import sys +import tempfile +import unittest +from unittest.mock import patch + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / 'scripts')) +import check_research_commit as checker +sys.path.pop(0) + + +class PublicationMetadataReuseTests(unittest.TestCase): + def setUp(self): + temporary = tempfile.TemporaryDirectory() + self.addCleanup(temporary.cleanup) + self.root = Path(temporary.name) + self.package = 'research/fixture' + self.path = self.root / self.package + self.path.mkdir(parents=True) + self.git('init', '-q') + self.git('config', 'user.name', 'Fixture') + self.git('config', 'user.email', 'fixture@example.invalid') + (self.path / 'VERIFY.json').write_text('{"inputs":[]}\n', encoding='utf-8') + self.git('add', '.') + self.git('commit', '-qm', 'fixture base') + self.base = self.git('rev-parse', 'HEAD').strip() + self.publish({'base_commit': self.base, 'files': []}) + + def git(self, *args): + return subprocess.run( + ['git', '-c', 'maintenance.auto=false', '-c', 'gc.auto=0', *args], + cwd=self.root, capture_output=True, text=True, check=True, timeout=30).stdout + + def publish(self, sources): + raw = json.dumps(sources).encode('utf-8') + (self.path / 'SOURCES.json').write_bytes(raw) + self.git('add', '.') + self.git('commit', '-qm', 'fixture publication') + return self.git('rev-parse', 'HEAD').strip() + + def check(self, ref='HEAD', history=True): + return checker.check(self.root, self.package, ref, check_source_history=history) + + def test_complete_check_avoids_metadata_refetch_and_preserves_receipt(self): + head = self.git('rev-parse', 'HEAD').strip() + expected = checker.source_history(self.root, self.package, head) + with patch.object(checker, 'publication_sources', side_effect=AssertionError('refetch')): + result = self.check() + self.assertEqual(result, {'commit': head, 'files': 2, 'package': self.package, + 'source_history_commits': expected}) + + def test_standalone_reader_keeps_its_original_boundary(self): + head = self.git('rev-parse', 'HEAD').strip() + with patch.object(checker, 'history_git', wraps=checker.history_git) as calls: + self.assertEqual(checker.source_history(self.root, self.package, head), [self.base]) + self.assertEqual([c.args[1:3] for c in calls.call_args_list[:2]], + [('cat-file', '-s'), ('cat-file', 'blob')]) + with patch.object(checker, 'history_git', wraps=checker.history_git) as calls: + self.check() + self.assertTrue(calls.call_args_list) + self.assertNotIn('cat-file', [c.args[1] for c in calls.call_args_list]) + + def test_working_drift_is_rejected_before_metadata_parsing(self): + (self.path / 'SOURCES.json').write_bytes(b'{invalid') + with self.assertRaisesRegex(ValueError, 'bytes differ.*SOURCES.json'): + self.check() + + def test_malformed_committed_json_is_only_parsed_when_requested(self): + (self.path / 'SOURCES.json').write_bytes(b'{invalid') + self.git('add', '.') + self.git('commit', '-qm', 'malformed fixture') + self.assertEqual(self.check(history=False)['files'], 2) + with self.assertRaises(json.JSONDecodeError): + self.check() + + def test_reader_final_failure_precedes_parsing_of_captured_metadata(self): + self.publish(None) + original = checker.git_blobs + + def fail_after_payloads(root, specs): + yield from original(root, specs) + raise ValueError('final blob-reader failure') + + with patch.object(checker, 'git_blobs', side_effect=fail_after_payloads): + with self.assertRaisesRegex(ValueError, '^final blob-reader failure$'): + self.check() + + def test_retained_metadata_is_committed_not_a_later_working_read(self): + original = checker.git_blobs + sources = self.path / 'SOURCES.json' + before = sources.read_bytes() + + def change_after_comparison(root, specs): + yield from original(root, specs) + sources.write_text('{"base_commit":"' + '1' * 40 + '","files":[]}', + encoding='utf-8') + + try: + with patch.object(checker, 'git_blobs', side_effect=change_after_comparison): + self.assertEqual(self.check()['source_history_commits'], [self.base]) + finally: + sources.write_bytes(before) + self.assertEqual(self.git('status', '--porcelain'), '') + + def test_exact_metadata_size_limit_and_final_reader_completion(self): + size = (self.path / 'SOURCES.json').stat().st_size + with patch.object(checker, 'MAX_SOURCE_METADATA_BYTES', size): + self.assertEqual(self.check()['source_history_commits'], [self.base]) + original = checker.git_blobs + completed = [] + + def mark_completion(root, specs): + yield from original(root, specs) + completed.append(True) + + with patch.object(checker, 'git_blobs', side_effect=mark_completion): + with patch.object(checker, 'MAX_SOURCE_METADATA_BYTES', size - 1): + with self.assertRaisesRegex(ValueError, 'source metadata byte limit'): + self.check() + self.assertEqual(completed, [True]) + + def test_no_cross_call_cache_and_explicit_older_publication(self): + old = self.git('rev-parse', 'HEAD').strip() + raw = (self.path / 'SOURCES.json').read_bytes() + self.check() + self.publish({'base_commit': '1' * 40, 'files': []}) + with self.assertRaisesRegex(ValueError, 'unavailable source commit'): + self.check() + (self.path / 'SOURCES.json').write_bytes(raw) + self.assertEqual(self.check(old)['source_history_commits'], [self.base]) + with self.assertRaisesRegex(ValueError, 'bytes differ.*SOURCES.json'): + self.check() + + +if __name__ == '__main__': + unittest.main()