Add maintainer-only acquisition catalogue health checks #71

New issue

Open

opened 2026-09-27 01:20:41 -04:00 by JackFrostbyte · 0 comments

JackFrostbyte commented

2026-09-27 01:20:41 -04:00

Owner

Goal

Create a development-only maintainer tool that periodically or on demand audits the health of reviewed acquisition catalogue entries. Users' offline installation must not depend on this tool or its database.

Boundaries

Treat version-controlled reviewed recipes, rights metadata and source records as authoritative. Maintain a development-only SQLite health/history database for derived inspection results and scan timestamps, never a second editable source catalogue. Allow a clean rebuild of the health database from versioned catalogue inputs and documented inspection history.
Inspect review expiry, withdrawal, recorded version, declared jurisdictions, rights/adapter eligibility, official host policy, documented expected type/size/checksum, and links where permissible.
For a source eligible for automated acquisition, only send permitted bounded requests to reviewed official hosts, revalidate each redirect/destination and obey network timeouts/limits. No credential use, browser automation, scraping, bypassing manual-only rules or unreviewed sites. Manual-only sources are flagged for human verification, not automatically downloaded.
Classify results distinctly: healthy; link unavailable or HTTP failure; unexpected redirect/host; version possibly changed; checksum mismatch; eligibility/review expired; withdrawn; manual review required; transient/inconclusive network failure. Distinguish transport observations from verified publisher version changes.
A mismatched checksum or changed document must never be auto-approved or written into the trusted catalogue. Surface it for review and a human-controlled recipe update.
Produce deterministic JSON and human-readable reports with inspection timestamp, per-source reasons, actionable diagnostics and source IDs. Strip tokens, secret-bearing URL components and private user paths. Non-public user-local content must not be collected.
Keep CI tests completely offline using controlled fixtures/fake transports; any real publisher check is an explicit, policy-respecting maintainer operation.

Deliverables

Maintainer-only CLI, development-only SQLite audit/history database, and machine-readable/human-readable reports. Document schema, migrations and safe local storage.
Documentation covering review workflow, audit frequency and safe remediation of missing/changed publisher content.
Positive/negative tests for health states, redaction, expired/withdrawn/manual sources, redirect rules, checksum mismatch and inconclusive responses.

Acceptance criteria

The reviewed repository catalogue is authoritative; audit state cannot implicitly rewrite accepted checksums or rights decisions.
The maintainer SQLite database records dated inspections and source/version status history without becoming authoritative for rights or checksums.
Results distinguish unavailable, changed, blocked and inconclusive sources with actionable reasons.
Manual-only, retired, expired and unapproved sources are never silently fetched.
Tests do not contact live third-party sites; live audit is opt-in and policy-constrained.
Output is safe to share with contributors after explicit review for sensitive information.

Coordination

Depends on #43, #44, #45 and the trusted single-source acquisition foundation #46. Operational integration with #72 is a future consumer, not a prerequisite for this issue.

Out of scope

Bulk user acquisition, automatic approval of new source versions, credentials, public mirroring, user-local processing and user-distributed background telemetry.

## Goal Create a **development-only maintainer tool** that periodically or on demand audits the health of reviewed acquisition catalogue entries. Users' offline installation must not depend on this tool or its database. ## Boundaries - Treat version-controlled reviewed recipes, rights metadata and source records as authoritative. Maintain a **development-only SQLite health/history database** for derived inspection results and scan timestamps, never a second editable source catalogue. Allow a clean rebuild of the health database from versioned catalogue inputs and documented inspection history. - Inspect review expiry, withdrawal, recorded version, declared jurisdictions, rights/adapter eligibility, official host policy, documented expected type/size/checksum, and links where permissible. - For a source eligible for automated acquisition, only send **permitted bounded requests** to reviewed official hosts, revalidate each redirect/destination and obey network timeouts/limits. No credential use, browser automation, scraping, bypassing manual-only rules or unreviewed sites. Manual-only sources are flagged for human verification, not automatically downloaded. - Classify results distinctly: healthy; link unavailable or HTTP failure; unexpected redirect/host; version possibly changed; checksum mismatch; eligibility/review expired; withdrawn; manual review required; transient/inconclusive network failure. Distinguish transport observations from verified publisher version changes. - A mismatched checksum or changed document must **never** be auto-approved or written into the trusted catalogue. Surface it for review and a human-controlled recipe update. - Produce deterministic JSON and human-readable reports with inspection timestamp, per-source reasons, actionable diagnostics and source IDs. Strip tokens, secret-bearing URL components and private user paths. Non-public user-local content must not be collected. - Keep CI tests completely offline using controlled fixtures/fake transports; any real publisher check is an explicit, policy-respecting maintainer operation. ## Deliverables - Maintainer-only CLI, development-only SQLite audit/history database, and machine-readable/human-readable reports. Document schema, migrations and safe local storage. - Documentation covering review workflow, audit frequency and safe remediation of missing/changed publisher content. - Positive/negative tests for health states, redaction, expired/withdrawn/manual sources, redirect rules, checksum mismatch and inconclusive responses. ## Acceptance criteria - [ ] The reviewed repository catalogue is authoritative; audit state cannot implicitly rewrite accepted checksums or rights decisions. - [ ] The maintainer SQLite database records dated inspections and source/version status history without becoming authoritative for rights or checksums. - [ ] Results distinguish unavailable, changed, blocked and inconclusive sources with actionable reasons. - [ ] Manual-only, retired, expired and unapproved sources are never silently fetched. - [ ] Tests do not contact live third-party sites; live audit is opt-in and policy-constrained. - [ ] Output is safe to share with contributors after explicit review for sensitive information. ## Coordination Depends on #43, #44, #45 and the trusted single-source acquisition foundation #46. Operational integration with #72 is a future consumer, not a prerequisite for this issue. ## Out of scope Bulk user acquisition, automatic approval of new source versions, credentials, public mirroring, user-local processing and user-distributed background telemetry.

JackFrostbyte added this to the User-Acquired Content Overlay v0.1 milestone

2026-09-27 01:20:41 -04:00

JackFrostbyte referenced this issue

2026-09-27 01:20:42 -04:00

Implement resilient batch acquisition and incremental retry #72

JackFrostbyte referenced this issue

2026-09-27 01:20:42 -04:00

Run a ten-source user-overlay end-to-end pilot for v0.1 #73

JackFrostbyte referenced this issue

2026-09-27 01:20:42 -04:00

Implement safe user-side acquisition and manual import #46

JackFrostbyte added a new dependency