Prevent local-content contamination and validate the complete overlay flow #50

New issue

Open

opened 2026-07-31 01:25:52 -04:00 by JackFrostbyte · 0 comments

JackFrostbyte commented

2026-07-31 01:25:52 -04:00

Owner

Goal

Prove end to end that user-side acquisition expands a local installation without
contaminating Arkive's redistributable bundle, release process, or diagnostics.

Required scenarios

Validate a reviewed fixture recipe.
Acquire from a local test server or manually import a fixture.
Produce local-only documents and chunks.
Build the separate local index.
Search core and local indexes together.
Resolve exact citations from both domains.
Build the portable core bundle and prove no local byte, record, checksum, index,
log, or path is included.
Withdraw/expire the fixture recipe and verify future acquisition fails closed
while existing local provenance remains honest.
Run the flow under the constrained-resource profile from Issue #28.

Release guardrails

Explicit allowlist of core release inputs.
Hard failure on local-overlay paths or local-only records.
Source-control ignores for default local paths.
Secret scanning and log redaction checks.
Support/diagnostic bundles exclude local source text and sensitive provenance by
default.
User documentation warns that backups or exports containing the overlay may not
be redistributable.

Deliverables

End-to-end test suite.
Release contamination tests.
Portable deployment documentation.
User-facing legal/distribution notices and backup/export warnings.
Maintainer runbook for recipe withdrawal and incident response.

Acceptance criteria

The complete local workflow passes without internet access.
The core bundle is byte-for-byte unaffected by presence of local content.
Release jobs fail when deliberately presented with local-only inputs.
No credential or sensitive local provenance appears in logs or diagnostics.
Constrained-resource execution remains within documented limits.
User notices are shown at acquisition and export/backup boundaries.

Out of scope

Publishing a production recipe catalogue.
Legal approval of individual third-party targets.

Roadmap clarification: recovery-path contamination validation

Extend overlay-contamination coverage to include previously validated file reuse, repeated acquisition, interrupted and partial batch downloads (#72), incremental processing (#47), and incremental local-index rebuilding (#48).
Check that temporary quarantine files, failed-source reports, per-user path information and diagnostic data remain outside bundled-core paths, redistributable artifacts, core indexes and support bundles. Maintain strict core validation even when a user's optional overlay has a reported partial build.
The full 10-source operational validation is tracked separately in #73; #50 retains responsibility for storage isolation and adversarial contamination tests.

Additional acceptance criteria

Retry and incremental paths cannot contaminate core input, derived data, manifests, indexes or release artifacts.
Partial optional-overlay completion remains visible without relaxing strict core release validation.

## Goal Prove end to end that user-side acquisition expands a local installation without contaminating Arkive's redistributable bundle, release process, or diagnostics. ## Required scenarios 1. Validate a reviewed fixture recipe. 2. Acquire from a local test server or manually import a fixture. 3. Produce local-only documents and chunks. 4. Build the separate local index. 5. Search core and local indexes together. 6. Resolve exact citations from both domains. 7. Build the portable core bundle and prove no local byte, record, checksum, index, log, or path is included. 8. Withdraw/expire the fixture recipe and verify future acquisition fails closed while existing local provenance remains honest. 9. Run the flow under the constrained-resource profile from Issue #28. ## Release guardrails - Explicit allowlist of core release inputs. - Hard failure on local-overlay paths or local-only records. - Source-control ignores for default local paths. - Secret scanning and log redaction checks. - Support/diagnostic bundles exclude local source text and sensitive provenance by default. - User documentation warns that backups or exports containing the overlay may not be redistributable. ## Deliverables - End-to-end test suite. - Release contamination tests. - Portable deployment documentation. - User-facing legal/distribution notices and backup/export warnings. - Maintainer runbook for recipe withdrawal and incident response. ## Acceptance criteria - [ ] The complete local workflow passes without internet access. - [ ] The core bundle is byte-for-byte unaffected by presence of local content. - [ ] Release jobs fail when deliberately presented with local-only inputs. - [ ] No credential or sensitive local provenance appears in logs or diagnostics. - [ ] Constrained-resource execution remains within documented limits. - [ ] User notices are shown at acquisition and export/backup boundaries. ## Out of scope - Publishing a production recipe catalogue. - Legal approval of individual third-party targets. ---  ### Roadmap clarification: recovery-path contamination validation - Extend overlay-contamination coverage to include previously validated file reuse, repeated acquisition, interrupted and partial batch downloads (#72), incremental processing (#47), and incremental local-index rebuilding (#48). - Check that temporary quarantine files, failed-source reports, per-user path information and diagnostic data remain outside bundled-core paths, redistributable artifacts, core indexes and support bundles. Maintain strict core validation even when a user's optional overlay has a reported partial build. - The full 10-source operational validation is tracked separately in #73; #50 retains responsibility for storage isolation and adversarial contamination tests. **Additional acceptance criteria** - [ ] Retry and incremental paths cannot contaminate core input, derived data, manifests, indexes or release artifacts. - [ ] Partial optional-overlay completion remains visible without relaxing strict core release validation.

JackFrostbyte added this to the User-Acquired Content Overlay v0.1 milestone

2026-07-31 01:25:52 -04:00

JackFrostbyte added the

area/deployment

type/testing

labels

2026-07-31 01:25:52 -04:00

JackFrostbyte added a new dependency

2026-07-31 01:25:55 -04:00

#45 Define the bundled-core and user-local storage/index boundary

JackFrostbyte added a new dependency

2026-07-31 01:25:55 -04:00