Arkive/arkive
1
0
Fork 0
Code Issues 18 Pull requests 1 Projects Releases Packages Wiki Activity Actions

[Duplicate] Generate and validate a separate local-content index #40

New issue
Closed
opened 2026-07-31 01:21:15 -04:00 by JackFrostbyte · 1 comment
JackFrostbyte commented 2026-07-31 01:21:15 -04:00
Owner
Copy link

Goal

Generate a search index and manifest for the user-local overlay without merging its
files or build provenance into the redistributable core index.

Requirements

  • Reuse the selected search backend and index-manifest concepts from Issues #20–#22.
  • Store local index files and manifests only in the local overlay.
  • Include source/document/chunk identifiers, acquisition record, recipe/version,
    tool versions, timestamps, and input checksums needed for local validation.
  • Detect stale, missing, withdrawn, or corrupt local inputs.
  • Rebuild or remove only the affected local index data.
  • Never make local index files inputs to a core release.
  • Support an installation with only the core index or only locally imported content.

Deliverables

  • Local index build command.
  • Local index manifest and validation.
  • Deterministic fixture-based tests.
  • Staleness/corruption behavior.
  • Storage and resource-usage documentation.

Acceptance criteria

  • Core and local manifests are independently valid.
  • Local index creation cannot mutate the core index.
  • Stale and corrupt local content is detected before retrieval.
  • Withdrawn recipes are visibly represented without silently deleting user data.
  • Tests use redistributable fixtures and no live third-party downloads.

Out of scope

  • Federated ranking and result presentation.
## Goal Generate a search index and manifest for the user-local overlay without merging its files or build provenance into the redistributable core index. ## Requirements - Reuse the selected search backend and index-manifest concepts from Issues #20–#22. - Store local index files and manifests only in the local overlay. - Include source/document/chunk identifiers, acquisition record, recipe/version, tool versions, timestamps, and input checksums needed for local validation. - Detect stale, missing, withdrawn, or corrupt local inputs. - Rebuild or remove only the affected local index data. - Never make local index files inputs to a core release. - Support an installation with only the core index or only locally imported content. ## Deliverables - Local index build command. - Local index manifest and validation. - Deterministic fixture-based tests. - Staleness/corruption behavior. - Storage and resource-usage documentation. ## Acceptance criteria - [ ] Core and local manifests are independently valid. - [ ] Local index creation cannot mutate the core index. - [ ] Stale and corrupt local content is detected before retrieval. - [ ] Withdrawn recipes are visibly represented without silently deleting user data. - [ ] Tests use redistributable fixtures and no live third-party downloads. ## Out of scope - Federated ranking and result presentation.
JackFrostbyte commented 2026-07-31 01:35:53 -04:00
Author
Owner
Copy link

Closing this issue as a duplicate of #48.

Issues #40 and #48 were created with the same roadmap
content during recovery from an interrupted automation run. The later set
#43–#50 is canonical because Issue #34's finalized roadmap body, roadmap comment,
and dependency graph reference that set.

No implementation work was completed in this duplicate issue.

<!-- arkive-roadmap-duplicate-cleanup-v1 --> Closing this issue as a duplicate of #48. Issues #40 and #48 were created with the same roadmap content during recovery from an interrupted automation run. The later set #43–#50 is canonical because Issue #34's finalized roadmap body, roadmap comment, and dependency graph reference that set. No implementation work was completed in this duplicate issue.
JackFrostbyte changed title from Generate and validate a separate local-content index to [Duplicate] Generate and validate a separate local-content index 2026-07-31 01:35:54 -04:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
feature/issue-17-reproducibility-tests
develop
No results found.
Labels
Clear labels
  
area/deployment
Portable packaging and constrained-environment validation

  
area/governance
Project rules, policies, contributor workflow

  
area/metadata
Schemas, manifests, validation contracts

  
area/pipeline
Import, extraction, normalization, chunking

  
area/search
Indexing, retrieval, citation resolution

  
type/data
Source selection, metadata, fixtures, and corpus work

  
type/design
Architecture, policy, contracts, and ADR work

  
type/implementation
Production code or tooling

  
type/testing
Reproducibility, integration, evaluation, and benchmark work

No labels
area/deployment
area/governance
area/metadata
area/pipeline
area/search
type/data
type/design
type/implementation
type/testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Arkive/arkive#40
No description provided.