Arkive/arkive
1
0
Fork 0
Code Issues 18 Pull requests 1 Projects Releases Packages Wiki Activity Actions

[Duplicate] Define acquisition recipe and local-use rights schemas #36

New issue
Closed
opened 2026-07-31 01:21:14 -04:00 by JackFrostbyte · 1 comment
JackFrostbyte commented 2026-07-31 01:21:14 -04:00
Owner
Copy link

Goal

Define machine-readable contracts for reviewed user-side acquisition without
turning descriptive source provenance into executable code.

Design requirements

Keep origin.acquisition_path descriptive

origin.acquisition_path remains human-readable provenance. It must not contain
shell commands, executable code, credentials, cookies, tokens, or an implicit
permission decision.

Add a distinct acquisition recipe contract

Define a versioned recipe schema that can represent:

  • recipe ID and version;
  • source ID and reviewed target;
  • acquisition mode: manual_import, official_direct_download, or a future
    explicitly reviewed official API adapter;
  • approved hosts and redirect hosts;
  • expected media/content types and size limits;
  • optional stable checksums or publisher signatures;
  • required user acknowledgement;
  • review evidence and jurisdictions;
  • last-reviewed, expiry/recheck, and withdrawn fields;
  • adapter type and adapter-specific parameters from a closed allowlist;
  • post-download integrity and provenance requirements.

Recipes must be declarative data consumed by trusted Arkive adapters, not arbitrary
Python, shell, JavaScript, templates, or commands.

Add a local-use rights profile

Represent independently:

  • redistribution status;
  • local-copy/local-processing status;
  • automation status;
  • terms/contract review status;
  • authentication requirement;
  • technical access-control status;
  • conditions and evidence;
  • reviewed jurisdictions;
  • reviewer and review lifecycle.

Record local provenance

Define a local acquisition record containing the recipe/version, acquisition time,
final URL, checksums, byte size, adapter/tool version, and user acknowledgement.
Do not require storage of secrets, account identifiers, order receipts, or other
sensitive data.

Inheritance

All documents, chunks, and indexes derived from a user-local acquisition must
inherit a machine-enforced local-only distribution scope.

Deliverables

  • Versioned JSON Schemas.
  • Valid and invalid fixtures.
  • Schema documentation and examples.
  • Migration/compatibility notes for the current source schema.
  • ADR note explaining why provenance, permission, recipe, and runtime records are
    separate concepts.

Acceptance criteria

  • Existing bundled-source records remain valid or have a documented migration.
  • Unknown permission states fail closed.
  • Arbitrary code and secrets cannot be represented by a valid recipe.
  • Recipe withdrawal and expiry are representable.
  • Derived local artifacts inherit local_only.
  • Validation tests cover contradictory states and missing evidence.

Out of scope

  • Network access.
  • Download execution.
  • Search implementation.
## Goal Define machine-readable contracts for reviewed user-side acquisition without turning descriptive source provenance into executable code. ## Design requirements ### Keep `origin.acquisition_path` descriptive `origin.acquisition_path` remains human-readable provenance. It must not contain shell commands, executable code, credentials, cookies, tokens, or an implicit permission decision. ### Add a distinct acquisition recipe contract Define a versioned recipe schema that can represent: - recipe ID and version; - source ID and reviewed target; - acquisition mode: `manual_import`, `official_direct_download`, or a future explicitly reviewed official API adapter; - approved hosts and redirect hosts; - expected media/content types and size limits; - optional stable checksums or publisher signatures; - required user acknowledgement; - review evidence and jurisdictions; - last-reviewed, expiry/recheck, and withdrawn fields; - adapter type and adapter-specific parameters from a closed allowlist; - post-download integrity and provenance requirements. Recipes must be declarative data consumed by trusted Arkive adapters, not arbitrary Python, shell, JavaScript, templates, or commands. ### Add a local-use rights profile Represent independently: - redistribution status; - local-copy/local-processing status; - automation status; - terms/contract review status; - authentication requirement; - technical access-control status; - conditions and evidence; - reviewed jurisdictions; - reviewer and review lifecycle. ### Record local provenance Define a local acquisition record containing the recipe/version, acquisition time, final URL, checksums, byte size, adapter/tool version, and user acknowledgement. Do not require storage of secrets, account identifiers, order receipts, or other sensitive data. ### Inheritance All documents, chunks, and indexes derived from a user-local acquisition must inherit a machine-enforced local-only distribution scope. ## Deliverables - Versioned JSON Schemas. - Valid and invalid fixtures. - Schema documentation and examples. - Migration/compatibility notes for the current source schema. - ADR note explaining why provenance, permission, recipe, and runtime records are separate concepts. ## Acceptance criteria - [ ] Existing bundled-source records remain valid or have a documented migration. - [ ] Unknown permission states fail closed. - [ ] Arbitrary code and secrets cannot be represented by a valid recipe. - [ ] Recipe withdrawal and expiry are representable. - [ ] Derived local artifacts inherit `local_only`. - [ ] Validation tests cover contradictory states and missing evidence. ## Out of scope - Network access. - Download execution. - Search implementation.
JackFrostbyte commented 2026-07-31 01:35:49 -04:00
Author
Owner
Copy link

Closing this issue as a duplicate of #44.

Issues #36 and #44 were created with the same roadmap
content during recovery from an interrupted automation run. The later set
#43–#50 is canonical because Issue #34's finalized roadmap body, roadmap comment,
and dependency graph reference that set.

No implementation work was completed in this duplicate issue.

<!-- arkive-roadmap-duplicate-cleanup-v1 --> Closing this issue as a duplicate of #44. Issues #36 and #44 were created with the same roadmap content during recovery from an interrupted automation run. The later set #43–#50 is canonical because Issue #34's finalized roadmap body, roadmap comment, and dependency graph reference that set. No implementation work was completed in this duplicate issue.
JackFrostbyte changed title from Define acquisition recipe and local-use rights schemas to [Duplicate] Define acquisition recipe and local-use rights schemas 2026-07-31 01:35:49 -04:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
feature/issue-17-reproducibility-tests
develop
No results found.
Labels
Clear labels
  
area/deployment
Portable packaging and constrained-environment validation

  
area/governance
Project rules, policies, contributor workflow

  
area/metadata
Schemas, manifests, validation contracts

  
area/pipeline
Import, extraction, normalization, chunking

  
area/search
Indexing, retrieval, citation resolution

  
type/data
Source selection, metadata, fixtures, and corpus work

  
type/design
Architecture, policy, contracts, and ADR work

  
type/implementation
Production code or tooling

  
type/testing
Reproducibility, integration, evaluation, and benchmark work

No labels
area/deployment
area/governance
area/metadata
area/pipeline
area/search
type/data
type/design
type/implementation
type/testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Arkive/arkive#36
No description provided.