Implement resilient batch acquisition and incremental retry #72

New issue

Open

opened 2026-09-27 01:20:42 -04:00 by JackFrostbyte · 0 comments

JackFrostbyte commented

2026-09-27 01:20:42 -04:00

Owner

Goal

Orchestrate Issue #46's individually trusted acquisition operations into resumable collection acquisition. Independent failures should not abort all other eligible sources, and future retries should not redownload unchanged validated artifacts.

Requirements

Accept a declared collection of reviewed recipe IDs, a user-declared jurisdiction and required explicit acknowledgements; reuse #46's adapter and eligibility policies without an alternate downloader.
Revalidate existing user-local artifacts against pinned snapshots, checksums, sizes, types, provenance and compatible current recipe/version decisions. A recipe/version change is a review decision, not permission to accept a new hash automatically.
Continue processing other independent eligible entries after a missing source, transient download failure, checksum mismatch or failed manual import, but never mark an invalid/partial entry as successful or indexable.
Save resumable user-local state with per-entry status (available, existing/reused, needs manual action, changed/review required, missing, invalid, failed, retryable) and exact safe reasons. Handle interrupted runs, repeats and concurrent/conflicting execution.
Retry only missing/failed/changed entries after eligibility and trusted publisher metadata have been rechecked. Retain deterministic records of completed results and refuse silently destructive replacements.
Show a prominently visible completeness summary: expected vs valid content, missing/invalid/changed IDs, safe failure reasons and next actions; allow exporting a redacted diagnostic report for project support.
Provide dry-run mode with zero persistent writes and zero network acquisition. Do not accidentally include user-local originals or absolute private paths in support exports or bundled core.
No third-party networking in automated tests. Exercise failures and retries with local test servers or fake transports only.

Deliverables

Batch acquisition CLI or tightly scoped orchestration above #46.
Resumable per-source tracking and machine-readable/user-facing completeness report.
Documentation and tests for repeated runs, partial success, retries, changed versions and security boundaries.

Acceptance criteria

A simulated collection with independently failing sources completes all remaining eligible acquisitions and reports every omission.
A repeated run validates and reuses unchanged complete files and touches only eligible missing/changed entries.
A checksum mismatch, interrupted transfer or invalid source cannot be promoted, reused or incorrectly reported complete.
A recipe update or source withdrawal cannot silently replace or revalidate a previously accepted artifact.
Completeness results and exported failure reports are actionable, with no credentials or private user content leaked.
Core data and mandatory core builds retain their stricter failure/reproducibility guarantees.

Dependencies and exclusions

Depends on #46. Coordinates with #71, #47, #48 and the final pilot #73; it must not implement those other issues' processing/indexing/maintainer audit work or expand the allowed acquisition adapters.

## Goal Orchestrate Issue #46's individually trusted acquisition operations into **resumable collection acquisition**. Independent failures should not abort all other eligible sources, and future retries should not redownload unchanged validated artifacts. ## Requirements - Accept a declared collection of reviewed recipe IDs, a user-declared jurisdiction and required explicit acknowledgements; reuse #46's adapter and eligibility policies without an alternate downloader. - Revalidate existing user-local artifacts against pinned snapshots, checksums, sizes, types, provenance and compatible current recipe/version decisions. A recipe/version change is a **review decision**, not permission to accept a new hash automatically. - Continue processing other independent eligible entries after a missing source, transient download failure, checksum mismatch or failed manual import, but never mark an invalid/partial entry as successful or indexable. - Save resumable **user-local** state with per-entry status (available, existing/reused, needs manual action, changed/review required, missing, invalid, failed, retryable) and exact safe reasons. Handle interrupted runs, repeats and concurrent/conflicting execution. - Retry only missing/failed/changed entries **after** eligibility and trusted publisher metadata have been rechecked. Retain deterministic records of completed results and refuse silently destructive replacements. - Show a prominently visible completeness summary: expected vs valid content, missing/invalid/changed IDs, safe failure reasons and next actions; allow exporting a redacted diagnostic report for project support. - Provide dry-run mode with zero persistent writes and zero network acquisition. Do not accidentally include user-local originals or absolute private paths in support exports or bundled core. - No third-party networking in automated tests. Exercise failures and retries with local test servers or fake transports only. ## Deliverables - Batch acquisition CLI or tightly scoped orchestration above #46. - Resumable per-source tracking and machine-readable/user-facing completeness report. - Documentation and tests for repeated runs, partial success, retries, changed versions and security boundaries. ## Acceptance criteria - [ ] A simulated collection with independently failing sources completes all remaining eligible acquisitions and reports every omission. - [ ] A repeated run validates and reuses unchanged complete files and touches only eligible missing/changed entries. - [ ] A checksum mismatch, interrupted transfer or invalid source cannot be promoted, reused or incorrectly reported complete. - [ ] A recipe update or source withdrawal cannot silently replace or revalidate a previously accepted artifact. - [ ] Completeness results and exported failure reports are actionable, with no credentials or private user content leaked. - [ ] Core data and mandatory core builds retain their stricter failure/reproducibility guarantees. ## Dependencies and exclusions Depends on #46. Coordinates with #71, #47, #48 and the final pilot #73; it must not implement those other issues' processing/indexing/maintainer audit work or expand the allowed acquisition adapters.

JackFrostbyte added this to the User-Acquired Content Overlay v0.1 milestone

2026-09-27 01:20:42 -04:00

JackFrostbyte referenced this issue

2026-09-27 01:20:42 -04:00

Run a ten-source user-overlay end-to-end pilot for v0.1 #73

JackFrostbyte referenced this issue

2026-09-27 01:20:42 -04:00

Add maintainer-only acquisition catalogue health checks #71

JackFrostbyte referenced this issue

2026-09-27 01:20:42 -04:00

Implement safe user-side acquisition and manual import #46

JackFrostbyte referenced this issue