Arkive/arkive
1
0
Fork 0
Code Issues 18 Pull requests 1 Projects Releases Packages Wiki Activity Actions

[Duplicate] Process user-acquired content as local-only derived artifacts #39

New issue
Closed
opened 2026-07-31 01:21:15 -04:00 by JackFrostbyte · 1 comment
JackFrostbyte commented 2026-07-31 01:21:15 -04:00
Owner
Copy link

Goal

Run validated user-acquired inputs through Arkive's existing processing adapters
without weakening bundle eligibility or duplicating the pipeline.

Requirements

  • The processing entry point receives an explicit core or local_overlay context.
  • Local inputs produce local-only document, chunk, and processing-manifest records.
  • Distribution scope is inherited and cannot be overridden by a CLI convenience flag.
  • Source, document, chunk, checksum, lineage, and adapter-version validation remains
    equivalent to the bundled pipeline.
  • Local failures, logs, and temporary files remain inside approved local paths.
  • Reprocessing is deterministic when the source bytes and tool versions are unchanged.
  • Recipe withdrawal prevents new acquisitions but does not falsify existing provenance;
    the UI/CLI marks affected local records accordingly.
  • A local artifact cannot satisfy or replace a bundled-core fixture.

Deliverables

  • Processing-context contract.
  • Local-overlay output layout.
  • CLI integration with the Issue #16 processing command.
  • Validation and reproducibility tests.
  • Documentation for deletion and reprocessing.

Acceptance criteria

  • One processing implementation serves both domains.
  • Every local derived artifact is traceable to a local acquisition record.
  • Local-only scope propagates through all derived records.
  • Core release/build inputs reject local artifacts.
  • Tests prove equivalent lineage and checksum guarantees across both domains.

Out of scope

  • Local index generation.
  • Search federation.
## Goal Run validated user-acquired inputs through Arkive's existing processing adapters without weakening bundle eligibility or duplicating the pipeline. ## Requirements - The processing entry point receives an explicit `core` or `local_overlay` context. - Local inputs produce local-only document, chunk, and processing-manifest records. - Distribution scope is inherited and cannot be overridden by a CLI convenience flag. - Source, document, chunk, checksum, lineage, and adapter-version validation remains equivalent to the bundled pipeline. - Local failures, logs, and temporary files remain inside approved local paths. - Reprocessing is deterministic when the source bytes and tool versions are unchanged. - Recipe withdrawal prevents new acquisitions but does not falsify existing provenance; the UI/CLI marks affected local records accordingly. - A local artifact cannot satisfy or replace a bundled-core fixture. ## Deliverables - Processing-context contract. - Local-overlay output layout. - CLI integration with the Issue #16 processing command. - Validation and reproducibility tests. - Documentation for deletion and reprocessing. ## Acceptance criteria - [ ] One processing implementation serves both domains. - [ ] Every local derived artifact is traceable to a local acquisition record. - [ ] Local-only scope propagates through all derived records. - [ ] Core release/build inputs reject local artifacts. - [ ] Tests prove equivalent lineage and checksum guarantees across both domains. ## Out of scope - Local index generation. - Search federation.
JackFrostbyte commented 2026-07-31 01:35:52 -04:00
Author
Owner
Copy link

Closing this issue as a duplicate of #47.

Issues #39 and #47 were created with the same roadmap
content during recovery from an interrupted automation run. The later set
#43–#50 is canonical because Issue #34's finalized roadmap body, roadmap comment,
and dependency graph reference that set.

No implementation work was completed in this duplicate issue.

<!-- arkive-roadmap-duplicate-cleanup-v1 --> Closing this issue as a duplicate of #47. Issues #39 and #47 were created with the same roadmap content during recovery from an interrupted automation run. The later set #43–#50 is canonical because Issue #34's finalized roadmap body, roadmap comment, and dependency graph reference that set. No implementation work was completed in this duplicate issue.
JackFrostbyte changed title from Process user-acquired content as local-only derived artifacts to [Duplicate] Process user-acquired content as local-only derived artifacts 2026-07-31 01:35:52 -04:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
feature/issue-17-reproducibility-tests
develop
No results found.
Labels
Clear labels
  
area/deployment
Portable packaging and constrained-environment validation

  
area/governance
Project rules, policies, contributor workflow

  
area/metadata
Schemas, manifests, validation contracts

  
area/pipeline
Import, extraction, normalization, chunking

  
area/search
Indexing, retrieval, citation resolution

  
type/data
Source selection, metadata, fixtures, and corpus work

  
type/design
Architecture, policy, contracts, and ADR work

  
type/implementation
Production code or tooling

  
type/testing
Reproducibility, integration, evaluation, and benchmark work

No labels
area/deployment
area/governance
area/metadata
area/pipeline
area/search
type/data
type/design
type/implementation
type/testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Arkive/arkive#39
No description provided.