Implement safe user-side acquisition and manual import #46

New issue

Open

opened 2026-07-31 01:25:50 -04:00 by JackFrostbyte · 0 comments

JackFrostbyte commented

2026-07-31 01:25:50 -04:00

Owner

Goal

Implement the minimum trusted acquisition layer that consumes reviewed declarative
recipes and writes only to the user-local overlay.

Initial supported modes

manual_import
- the user obtains the file through the publisher's normal workflow;
- Arkive validates and imports the user-selected file locally.
official_direct_download
- HTTPS only;
- reviewed, allowlisted official/publisher/institutional hosts;
- bounded redirects to approved hosts;
- no login, paywall, CAPTCHA, session-cookie harvesting, or access-control bypass.

Official authenticated APIs may be added later only through a dedicated reviewed
adapter and credential-handling design.

Required safeguards

Closed adapter allowlist; no arbitrary command execution.
HTTPS and hostname validation.
Redirect limits and redirect-host allowlist.
Timeouts, maximum compressed/uncompressed sizes, and content-type checks.
Checksum/signature verification when available.
Archive path-traversal, symlink, decompression-bomb, and nested-archive defenses.
Atomic writes and quarantine before validation.
No secrets in recipe files, logs, manifests, exceptions, or telemetry.
Dry-run/explain mode showing what Arkive would contact and store.
Refusal of expired, withdrawn, unsupported-jurisdiction, or contradictory recipes.
Clear user acknowledgement and local-only notice before acquisition.

Deliverables

Recipe validator and trusted adapters.
CLI for recipe inspection, dry run, direct acquisition, and manual import.
Local acquisition provenance record.
Unit and integration tests using local fixtures/test servers only.
Security documentation and threat model.

Acceptance criteria

No valid recipe can execute arbitrary code.
Network tests do not depend on live third-party sites.
Partial or invalid downloads never become approved local inputs.
Credentials and authenticated browser sessions are not scraped or persisted.
Withdrawn/expired recipes fail closed.
All successful outputs are created only in the user-local store.

Out of scope

Circumvention of any technical or contractual access control.
Browser automation for paid/login-bound content.
Public proxying, mirroring, or hosting of acquired files.

## Goal Implement the minimum trusted acquisition layer that consumes reviewed declarative recipes and writes only to the user-local overlay. ## Initial supported modes 1. `manual_import` - the user obtains the file through the publisher's normal workflow; - Arkive validates and imports the user-selected file locally. 2. `official_direct_download` - HTTPS only; - reviewed, allowlisted official/publisher/institutional hosts; - bounded redirects to approved hosts; - no login, paywall, CAPTCHA, session-cookie harvesting, or access-control bypass. Official authenticated APIs may be added later only through a dedicated reviewed adapter and credential-handling design. ## Required safeguards - Closed adapter allowlist; no arbitrary command execution. - HTTPS and hostname validation. - Redirect limits and redirect-host allowlist. - Timeouts, maximum compressed/uncompressed sizes, and content-type checks. - Checksum/signature verification when available. - Archive path-traversal, symlink, decompression-bomb, and nested-archive defenses. - Atomic writes and quarantine before validation. - No secrets in recipe files, logs, manifests, exceptions, or telemetry. - Dry-run/explain mode showing what Arkive would contact and store. - Refusal of expired, withdrawn, unsupported-jurisdiction, or contradictory recipes. - Clear user acknowledgement and local-only notice before acquisition. ## Deliverables - Recipe validator and trusted adapters. - CLI for recipe inspection, dry run, direct acquisition, and manual import. - Local acquisition provenance record. - Unit and integration tests using local fixtures/test servers only. - Security documentation and threat model. ## Acceptance criteria - [ ] No valid recipe can execute arbitrary code. - [ ] Network tests do not depend on live third-party sites. - [ ] Partial or invalid downloads never become approved local inputs. - [ ] Credentials and authenticated browser sessions are not scraped or persisted. - [ ] Withdrawn/expired recipes fail closed. - [ ] All successful outputs are created only in the user-local store. ## Out of scope - Circumvention of any technical or contractual access control. - Browser automation for paid/login-bound content. - Public proxying, mirroring, or hosting of acquired files.

JackFrostbyte added this to the User-Acquired Content Overlay v0.1 milestone

2026-07-31 01:25:50 -04:00

JackFrostbyte added the

area/pipeline

type/implementation

labels

2026-07-31 01:25:50 -04:00

JackFrostbyte added a new dependency

2026-07-31 01:25:53 -04:00

#43 Define jurisdiction and target eligibility rules for user-side acquisition

JackFrostbyte added a new dependency

2026-07-31 01:25:53 -04:00