12.16 Issue 59 Vulnerability Ledger Refresh Implementation Plan¶

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Replace the stale dependency-vulnerability snapshot with a reproducible, alias-aware, four-surface ledger whose current counts are statically reconciled without turning Issue #59 into the automated enforcement work reserved for Issue #60.

Architecture: The existing D10 documentation verifier will parse one explicitly headed “Current accepted advisories” subsection and fail closed when its summary, advisory table, or total is missing or inconsistent; historical rows elsewhere in the document will be ignored. Four fresh pip-audit observations provide review evidence for the combined runtime, Torch attribution, documentation lock, and Atlas-contract test environment, while canonical Markdown publishes the reviewed result to the repository, generated site, and native wiki.

Tech Stack: Python 3.11, pip-audit 2.10.0, pytest, regular expressions, Markdown, scripts.verify_repo, MkDocs Material, and the repository's site/wiki projection pipeline.

12.16.1 Global constraints¶

Keep every dependency declaration unchanged; Issue #62 owns the coordinated Torch upgrade and Issue #63 owns dependency hash locking.
Do not add a CI job, workflow edit, checked-in JSON audit artifact, suppression file, pip-audit --ignore-vuln, or machine-readable accepted baseline; Issue #60 owns enforcement.
Audit the combined runtime from requirements.txt and torch-requirements.txt together, and audit torch-requirements.txt, docs-requirements.txt, and atlas-contract-requirements.txt as separately named surfaces.
Use python -m pip_audit, --strict, --vulnerability-service pypi, JSON output, --aliases on, --desc off, and --progress-spinner off for every final observation.
Use --disable-pip only for the complete universal hashed docs-requirements.txt lock.
Exit 0 and exit 1 are complete observations; any other exit, missing output, or malformed JSON invalidates the observation.
Preserve raw feed records, including duplicate primary IDs with different fix metadata. Aliases are metadata on a record, not additional findings.
Do not claim that feed absence proves remediation, non-reachability, or an upstream fix.
Never load an untrusted model or checkpoint while assessing reachability.
Do not start or contact Atlas, JupyterHub, Ollama, ComfyUI, Docker Compose, or another service. Never use containerized Ollama.
Keep generated site/wiki trees ignored; edit canonical sources only.
Every production change follows red-green-refactor and receives a fresh review before the next task starts.

12.16.2 Task 1: Scope D10 reconciliation to the current accepted snapshot¶

Files: - Modify: scripts/verify_repo.py - Modify: tests/test_verify_repo.py - Modify: this plan

Interfaces: - Produces: _DEPENDENCY_CURRENT_SNAPSHOT_RE, which extracts the body beneath the exact current heading ### 6.1.1.2 Current accepted advisories and stops at the next H3, H2, or end of file. - Preserves: finding ID D10.dependency_ledger_count and the existing Atlas gitlink reconciliation. - Consumes later: Task 2 writes the canonical summary and advisory tables inside the extracted current body and may place parser-compatible historical tables outside it.

[x] Step 1: Add historical-isolation and fail-closed tests

Add a small fixture builder beside the existing D10 tests in tests/test_verify_repo.py:

def _dependency_snapshot(*, summary_count=2, advisory_rows=None):
    rows = advisory_rows or [
        "| `torch` | `PYSEC-2025-41` | 1 | `2.6.0` |",
        "| `torch` | `PYSEC-2025-191` | 1 | `2.7.1rc1` |",
    ]
    return (
        "# 6.1 Dependency Contracts\n\n"
        "## 6.1.1 Audit Snapshot\n\n"
        "### 6.1.1.2 Current accepted advisories\n\n"
        f"Result: {summary_count} known vulnerabilities across one resolved package.\n\n"
        "| Package | Manifest Constraint | Audited Resolved Version | Finding Count | Current Disposition |\n"
        "| --- | --- | ---: | ---: | --- |\n"
        f"| `torch` | `torch==2.4.1` | `2.4.1` | {summary_count} | Accepted temporarily. |\n\n"
        "| Package | Advisory ID | Feed Records | Fix Versions |\n"
        "| --- | --- | ---: | --- |\n"
        + "\n".join(rows)
        + "\n"
    )

Add this helper and the focused tests below:

def _d10_count_findings(tmp_path, text):
    repo = _temp_repo(tmp_path)
    docs = repo / "docs"
    docs.mkdir()
    (docs / "dependency-contracts.md").write_text(text, encoding="utf-8")
    verify_repo = _load_verify_module()
    return [
        finding
        for finding in verify_repo._dependency_ledger_findings(repo)
        if finding.id == "D10.dependency_ledger_count"
    ]


def test_docs_d10_ignores_parser_compatible_historical_rows(tmp_path):
    historical = (
        "\n### 6.1.1.3 Historical reconciliation\n\n"
        "| Package | Manifest Constraint | Audited Resolved Version | Finding Count | Disposition |\n"
        "| --- | --- | ---: | ---: | --- |\n"
        "| `nltk` | `nltk>=3.9.3` | `3.9.4` | 7 | Archived. |\n\n"
        "| Package | Advisory ID | Feed Records | Fix Versions |\n"
        "| --- | --- | ---: | --- |\n"
        "| `nltk` | `CVE-2099-9999` | 7 | none listed |\n"
    )
    assert _d10_count_findings(tmp_path, _dependency_snapshot() + historical) == []


def test_docs_d10_flags_missing_current_advisory_section(tmp_path):
    text = _dependency_snapshot().replace(
        "### 6.1.1.2 Current accepted advisories",
        "### 6.1.1.2 Historical advisories",
    )
    findings = _d10_count_findings(tmp_path, text)
    assert any("section is missing" in finding.message for finding in findings)


def test_docs_d10_flags_malformed_current_summary_table(tmp_path):
    text = _dependency_snapshot().replace(
        "| `torch` | `torch==2.4.1` | `2.4.1` | 2 | Accepted temporarily. |\n",
        "",
    )
    findings = _d10_count_findings(tmp_path, text)
    assert any("summary table" in finding.message for finding in findings)


def test_docs_d10_flags_malformed_current_advisory_table(tmp_path):
    text = _dependency_snapshot(advisory_rows=["| no parseable advisory row |"])
    findings = _d10_count_findings(tmp_path, text)
    assert any("advisory table" in finding.message for finding in findings)


def test_docs_d10_flags_current_package_count_drift(tmp_path):
    rows = ["| `torch` | `PYSEC-2025-41` | 1 | `2.6.0` |"]
    findings = _d10_count_findings(
        tmp_path, _dependency_snapshot(summary_count=2, advisory_rows=rows)
    )
    assert any("torch advisory feed-record count" in finding.message for finding in findings)


def test_docs_d10_flags_current_total_count_drift(tmp_path):
    text = _dependency_snapshot().replace(
        "Result: 2 known vulnerabilities", "Result: 3 known vulnerabilities"
    )
    findings = _d10_count_findings(tmp_path, text)
    assert any("advisory feed-record total" in finding.message for finding in findings)

The historical test must append an H3 named ### 6.1.1.3 Historical reconciliation containing both a five-column package row and a four-column CVE-2099-9999 advisory row. It must assert no D10 count finding. Missing/malformed tests must assert a D10 finding whose message names the absent current summary or advisory table; they must not pass vacuously because no count was parsed.

[x] Step 2: Run the focused tests RED

pytest -p no:cacheprovider tests/test_verify_repo.py -q -k 'docs_d10 and (historical or missing_current or malformed_current or current_package_count or current_total_count)'

Expected: historical isolation fails because the existing regex scans the whole document, while missing/malformed current-section cases fail because the existing verifier silently accepts an empty parse. Record exact failures in .superpowers/sdd/task-1-report.md.

[x] Step 3: Implement a bounded current-section parser

Add this structural matcher near the existing D10 constants:

_DEPENDENCY_CURRENT_SNAPSHOT_RE = re.compile(
    r"^###[ \t]+6[.]1[.]1[.]2[ \t]+Current accepted advisories[ \t]*\r?$"
    r"(?P<body>.*?)(?=^#{2,3}[ \t]|\Z)",
    re.MULTILINE | re.DOTALL,
)

Inside _dependency_ledger_findings, initialize findings before count parsing. Extract exactly one body. When the heading is absent, emit D10.dependency_ledger_count with message current accepted-advisories section is missing; when the package summary table or advisory table has no parseable rows, emit the same ID with a specific malformed-table message. Parse package_counts, advisory_counts, and Result: N known vulnerabilities only from the bounded body. Continue into the independent Atlas submodule SHA checks even when the current section is missing.

Keep the current row grammar deliberately narrow:

package_rows = re.findall(
    r"^\| `([^`]+)` \| `[^`]+` \| `[^`]+` \| (\d+) \|", body, re.M
)
advisory_rows = re.findall(
    r"^\| `([^`]+)` \| `(?:PYSEC|CVE)-[^`]+` \| (\d+) \|", body, re.M
)

Do not add JSON parsing, audit execution, aliases, or policy logic to scripts/verify_repo.py; D10 is a static documentation-integrity check only.

[x] Step 4: Run Task 1 GREEN and mutation checks

pytest -p no:cacheprovider tests/test_verify_repo.py -q -k docs_d10
pytest -p no:cacheprovider tests/test_verify_repo.py -q
ruff check --no-cache scripts/verify_repo.py tests/test_verify_repo.py
git diff --check

Temporarily remove the H3 bound from _DEPENDENCY_CURRENT_SNAPSHOT_RE; the historical-isolation test must fail. Restore it, temporarily rename the current heading in the fixture, and confirm the missing-section test fails closed. Restore the source exactly and rerun the focused set green.

[x] Step 5: Update the task record and commit

Mark only Task 1 checkboxes complete and overwrite .superpowers/sdd/task-1-report.md with RED, GREEN, mutation, scope, and concern evidence. Then commit only the Task 1 files:

git add scripts/verify_repo.py tests/test_verify_repo.py \
  docs/superpowers/plans/2026-08-12-issue-59-vulnerability-ledger-implementation-plan.md
git diff --cached --check
git commit -m "test: scope vulnerability ledger reconciliation"

12.16.3 Task 2: Capture the four-surface audit and publish the reviewed ledger¶

Files: - Modify: docs/dependency-contracts.md - Modify: SECURITY.md - Modify: CHANGELOG.md - Modify: tests/test_check_docs.py - Modify: tests/test_build_docs.py - Modify: tests/test_wiki.py - Modify: this plan

Interfaces: - Consumes: Task 1's exact H3 boundary ### 6.1.1.2 Current accepted advisories. - Produces: a dated four-surface observation, current accepted-package/advisory tables, an alias-aware historical reconciliation, exact rerun commands, and a published manual-versus-CI boundary. - Preserves: docs/dependency-contracts.md as the sole canonical dependency ledger; generated generated/site/dependency-contracts.md and generated/wiki/6-1-Dependency-ledger.md remain ignored validation outputs.

[x] Step 1: Add documentation-contract tests RED

In tests/test_check_docs.py, add a test that reads the real canonical files and requires:

ledger = (REPO_ROOT / "docs/dependency-contracts.md").read_text(encoding="utf-8")
security = (REPO_ROOT / "SECURITY.md").read_text(encoding="utf-8")
changelog = (REPO_ROOT / "CHANGELOG.md").read_text(encoding="utf-8")

assert "### 6.1.1.1 Reproducible four-surface audit" in ledger
assert "### 6.1.1.2 Current accepted advisories" in ledger
assert "### 6.1.1.3 Alias-aware historical reconciliation" in ledger
assert "2026-08-12" in ledger
assert "--disable-pip -r docs-requirements.txt" in ledger
assert "Absent from the 2026-08-12 snapshot; archived audit provenance only" in ledger
assert "does not claim an automated vulnerability-baseline gate" in security
assert "Four-surface vulnerability ledger refresh" in changelog

Add this manifest-sequence test to tests/test_check_docs.py:

def test_real_manifest_declares_issue_59_design_and_implementation_records():
    manifest = load_manifest(REPO_ROOT / "docs/manifest.yaml", REPO_ROOT)
    records = next(section for section in manifest.sections if section.id == "design-records")
    start = next(
        index
        for index, child in enumerate(records.children)
        if child.id == "issue-59-vulnerability-ledger-design"
    )
    assert [
        (child.number, child.source) for child in records.children[start : start + 2]
    ] == [
        (
            "12.15",
            "docs/superpowers/specs/2026-08-12-issue-59-vulnerability-ledger-design.md",
        ),
        (
            "12.16",
            "docs/superpowers/plans/2026-08-12-issue-59-vulnerability-ledger-implementation-plan.md",
        ),
    ]

Add this site projection test to tests/test_build_docs.py:

def test_real_manifest_projects_current_vulnerability_snapshot_to_site(tmp_path):
    manifest = load_manifest(REPO_ROOT / "docs/manifest.yaml", REPO_ROOT)
    out = tmp_path / "generated/site"
    render_site(manifest, REPO_ROOT, out, trusted_output_root=tmp_path)
    ledger = (out / "dependency-contracts.md").read_text(encoding="utf-8")
    assert "### 6.1.1.2 Current accepted advisories" in ledger
    assert "2026-08-12" in ledger
    assert "archived audit provenance only" in ledger

Add the equivalent wiki test to tests/test_wiki.py:

def test_real_manifest_projects_current_vulnerability_snapshot_to_wiki(tmp_path):
    manifest = load_manifest(REPO_ROOT / "docs/manifest.yaml", REPO_ROOT)
    out = tmp_path / "generated/wiki"
    render_wiki(manifest, REPO_ROOT, out, trusted_output_root=tmp_path)
    ledger = (out / "6-1-Dependency-ledger.md").read_text(encoding="utf-8")
    assert "### 6.1.1.2 Current accepted advisories" in ledger
    assert "2026-08-12" in ledger
    assert "archived audit provenance only" in ledger

Do not hard-code advisory IDs or mutable counts in projection tests; D10 owns table reconciliation.

[x] Step 2: Run documentation tests RED

pytest -p no:cacheprovider tests/test_check_docs.py tests/test_manifest.py -q \
  -k 'vulnerability or issue_59 or manifest_sequence'

Expected: the canonical ledger still contains the 2026-07-04 two-manifest snapshot and lacks the four-surface, alias-reconciliation, and changelog markers. Record exact failures before editing canonical prose.

[x] Step 3: Create an isolated evidence directory and record immutable metadata

Run from the repository root without modifying the active environment:

AUDIT_DIR="$(mktemp -d /private/tmp/ml-eng-lab-issue59-audit.XXXXXX)"
git rev-parse HEAD
python --version
python -m pip_audit --version
uname -s
uname -m
shasum -a 256 requirements.txt torch-core-requirements.txt torch-requirements.txt \
  docs-requirements.txt atlas-contract-requirements.txt

Copy the exact UTC timestamp, commit SHA, platform, Python, pip-audit, and manifest hashes into the task report. Do not add $AUDIT_DIR, its JSON files, resolver logs, or a generated hash file to Git.

[x] Step 4: Run the exact four final audits

Run each command separately and record its exit immediately. Accept only 0 or 1:

python -m pip_audit -r requirements.txt -r torch-requirements.txt \
  --strict --vulnerability-service pypi --format json \
  --aliases on --desc off --progress-spinner off \
  --output "$AUDIT_DIR/runtime.json"

python -m pip_audit -r torch-requirements.txt \
  --strict --vulnerability-service pypi --format json \
  --aliases on --desc off --progress-spinner off \
  --output "$AUDIT_DIR/torch.json"

python -m pip_audit --disable-pip -r docs-requirements.txt \
  --strict --vulnerability-service pypi --format json \
  --aliases on --desc off --progress-spinner off \
  --output "$AUDIT_DIR/docs.json"

python -m pip_audit -r atlas-contract-requirements.txt \
  --strict --vulnerability-service pypi --format json \
  --aliases on --desc off --progress-spinner off \
  --output "$AUDIT_DIR/atlas-contract.json"

Expected from the design observation: runtime and Torch exit 1; docs and Atlas-contract exit 0. A different valid result must be reviewed and transcribed from its emitted JSON rather than forced to these preliminary counts. Any exit above 1, missing JSON, or malformed JSON stops the task before documentation edits.

[x] Step 5: Validate and reconcile every emitted record

Use this read-only command to load each JSON file and report dependency count, vulnerable-package count, raw feed-record count, package/version, primary ID, aliases, and fix versions:

AUDIT_DIR="$AUDIT_DIR" python -c 'import json, os, pathlib; root = pathlib.Path(os.environ["AUDIT_DIR"]); files = ("runtime.json", "torch.json", "docs.json", "atlas-contract.json"); [(lambda deps, name: print(name, {"dependencies": len(deps), "vulnerable_packages": sum(bool(d.get("vulns")) for d in deps), "feed_records": sum(len(d.get("vulns", [])) for d in deps), "records": [(d["name"], d["version"], v["id"], v.get("aliases", []), v.get("fix_versions", [])) for d in deps for v in d.get("vulns", [])]}))(json.loads((root / name).read_text(encoding="utf-8"))["dependencies"], name) for name in files]'

The review must establish these preliminary identities or document valid feed drift:

former CVE-2025-3000 is the alias of current primary PYSEC-2025-194;
former CVE-2025-3730 is the alias of current primary PYSEC-2026-1970;
former CVE-2026-31221 is the alias of current primary PYSEC-2026-3043;
PYSEC-2026-2286 / CVE-2026-24747 is genuinely new;
the former NLTK record is absent after the ranged requirement resolves to the observed newer version, which is resolver drift rather than a committed pin change;
duplicate Torch primary IDs remain multiple feed records when the feed emits distinct entries.

Record both raw feed-record totals and unique alias-aware identity totals in the task report. Do not collapse raw duplicates in the canonical D10 tables.

[x] Step 6: Replace §6.1.1 with the current four-surface ledger

Keep ## 6.1.1 Audit Snapshot, then use exactly these H3 boundaries:

### 6.1.1.1 Reproducible four-surface audit
### 6.1.1.2 Current accepted advisories
### 6.1.1.3 Alias-aware historical reconciliation
### 6.1.1.4 Enforcement boundary

The reproducibility subsection must include the immutable metadata from Step 3, manifest hashes, all four copyable commands, per-surface dependency/vulnerable-package/feed-record results, and the resolver caveat that open ranges make this snapshot evidence rather than a reproducible lock.

The current subsection must contain:

Result: N known vulnerabilities across M resolved packages. using the combined-runtime raw totals;
a five-column package summary whose Finding Count values sum to N;
a current advisory table whose first four columns remain Package, Advisory ID, Feed Records, and Fix Versions, followed by Audited Version, Aliases, and Surface;
explicit accepted-risk reasoning and revisit triggers for Torch and Lightning, including the prohibition on untrusted pickle-backed checkpoints and coordinated-stack upgrade requirement.

The historical subsection must classify every former/current identity as retained, re-keyed, absent, or genuinely new. For each absent historical record use the exact disclaimer prefix:

Absent from the 2026-08-12 snapshot; archived audit provenance only, not proof of remediation,
reachability, or an upstream fix.

Keep parser-compatible historical rows outside 6.1.1.2 so Task 1 proves they cannot inflate current totals. The enforcement subsection must state that Issue #60 owns a future checked-in baseline and CI gate and Issue #62 owns the coordinated Torch upgrade.

[x] Step 7: Align security policy and durable history

In SECURITY.md §13.6, state that the canonical ledger is manually refreshed from explicit repository install surfaces and that the repository does not claim an automated vulnerability-baseline gate. Preserve the actionable-uncertainty, coordinated-upgrade, and Atlas ownership boundaries.

Under CHANGELOG.md → Unreleased → Changed, add a Four-surface vulnerability ledger refresh entry describing the dated runtime/Torch/docs/Atlas-contract observation, alias-aware historical reconciliation, static current-table integrity check, and explicit deferral of automated gating to Issue #60. Do not duplicate transient advisory counts in README.

[x] Step 8: Run Task 2 GREEN and inspect all three surfaces

pytest -p no:cacheprovider tests/test_check_docs.py tests/test_manifest.py \
  tests/test_verify_repo.py -q -k 'vulnerability or dependency_ledger or manifest'
python scripts/verify_repo.py --check docs --fast
make docs-check
make docs-wiki
ruff check --no-cache scripts/verify_repo.py tests/test_verify_repo.py tests/test_check_docs.py
git diff --check

Inspect the generated site and wiki dependency pages and confirm the current marker, surface table, all current advisory rows, historical disclaimer, and internal links are present. Confirm git status --short contains no generated output. Temporarily add one historical advisory row and confirm D10 stays green; temporarily decrement a current feed-record total and confirm D10 fails; restore both mutations and rerun green.

[x] Step 9: Run the complete repository gate

make test
make verify
make lint
git diff --check

Expected: every command exits 0; only already-documented platform/backend skips are acceptable. Record exact pass/skip totals and warnings in .superpowers/sdd/task-2-report.md.

[x] Step 10: Update the task record and commit

Mark only Task 2 checkboxes complete and overwrite .superpowers/sdd/task-2-report.md with audit metadata, commands/exits, raw/unique reconciliation, RED/GREEN/mutation evidence, surface inspection, scope, and concerns. Commit only the Task 2 files:

git add docs/dependency-contracts.md SECURITY.md CHANGELOG.md tests/test_check_docs.py \
  tests/test_build_docs.py tests/test_wiki.py \
  docs/superpowers/plans/2026-08-12-issue-59-vulnerability-ledger-implementation-plan.md
git diff --cached --check
git commit -m "docs: refresh dependency vulnerability ledger"

12.16.4 Controller integration and completion gate¶

After both task commits pass independent spec and quality review:

Run the final four audit commands again only if the feed observation or requirement manifests changed after Task 2; otherwise retain the reviewed observation and hashes.
Run make test, make verify, make lint, make docs-check, make docs-wiki, and git diff --check from a clean feature branch.
Push codex/issue-59-vulnerability-ledger and open a ready PR into develop referencing #59.
Wait for every required check, review the complete PR diff, merge, and delete the remote feature branch.
Open a second ready PR from develop into main, wait for every required check, review, and merge.
If the release merge SHA is not already in develop, open and merge a main → develop sync PR. Prove main^{tree} == develop^{tree} and main is an ancestor of develop.
Check every satisfied Issue #59 acceptance box, add verification/PR evidence, close the issue, move its project item to Done, and update parent #53 without closing it while later backlog items remain.
Delete merged local/remote feature and sync branches, prune remotes, confirm one intended worktree, no open merged PRs, clean main/develop, and no Atlas/Ollama/ComfyUI service was started.

12.16.5 Completion criteria¶

Issue #59 is complete only when the reviewed four-surface snapshot is canonical on all three documentation surfaces, current D10 reconciliation fails closed without counting history, all required tests and live PR checks pass, the feature and release PRs are merged, develop and main are tree-equivalent, issue/project/parent bookkeeping is current, and merged branch/worktree state is clean.