Federate search and citations across core and local stores #49

New issue

Open

opened 2026-07-31 01:25:51 -04:00 by JackFrostbyte · 0 comments

JackFrostbyte commented

2026-07-31 01:25:51 -04:00

Owner

Goal

Allow one local Arkive query to search the redistributable core and the optional
user-local overlay while preserving provenance and legal/distribution boundaries.

Requirements

Query each index independently and merge results at the application layer.
Every result visibly identifies bundled_core or user_local.
Citations resolve to the correct store and exact passage.
Local citations include local acquisition provenance and recipe/version where
appropriate, without exposing credentials or sensitive account data.
Missing or disconnected local storage degrades cleanly to core-only search.
Ranking must not hide the distribution domain.
Export/copy actions warn when selected material includes local-only content.
Diagnostic output never uploads or embeds local source text by default.

Deliverables

Federated query interface.
Domain-aware result and citation models.
CLI presentation and filters (core, local, all).
Mixed-domain fixtures and tests.
Documentation for offline and portable deployments.

Acceptance criteria

A result's domain is unambiguous from search through citation resolution.
Core-only operation remains unchanged when no local overlay exists.
Local-only citations never resolve against bundled files.
Search refuses stale/corrupt local indexes according to existing validation rules.
Export warnings are covered by tests.

Out of scope

LLM/RAG answer generation.
Redistributing the user's local overlay.

## Goal Allow one local Arkive query to search the redistributable core and the optional user-local overlay while preserving provenance and legal/distribution boundaries. ## Requirements - Query each index independently and merge results at the application layer. - Every result visibly identifies `bundled_core` or `user_local`. - Citations resolve to the correct store and exact passage. - Local citations include local acquisition provenance and recipe/version where appropriate, without exposing credentials or sensitive account data. - Missing or disconnected local storage degrades cleanly to core-only search. - Ranking must not hide the distribution domain. - Export/copy actions warn when selected material includes local-only content. - Diagnostic output never uploads or embeds local source text by default. ## Deliverables - Federated query interface. - Domain-aware result and citation models. - CLI presentation and filters (`core`, `local`, `all`). - Mixed-domain fixtures and tests. - Documentation for offline and portable deployments. ## Acceptance criteria - [ ] A result's domain is unambiguous from search through citation resolution. - [ ] Core-only operation remains unchanged when no local overlay exists. - [ ] Local-only citations never resolve against bundled files. - [ ] Search refuses stale/corrupt local indexes according to existing validation rules. - [ ] Export warnings are covered by tests. ## Out of scope - LLM/RAG answer generation. - Redistributing the user's local overlay.

JackFrostbyte added this to the User-Acquired Content Overlay v0.1 milestone

2026-07-31 01:25:51 -04:00

JackFrostbyte added the

type/implementation

area/search

labels

2026-07-31 01:25:51 -04:00

JackFrostbyte added a new dependency

2026-07-31 01:25:54 -04:00

#48 Generate and validate a separate local-content index

JackFrostbyte added a new dependency

2026-07-31 01:25:54 -04:00