Arkive/arkive
1
0
Fork 0
Code Issues 14 Pull requests Projects Releases Packages Wiki Activity Actions

Evaluate offline full-text search backends and record the decision #20

New issue
Closed
opened 2026-07-24 19:38:19 -04:00 by JackFrostbyte · 1 comment
JackFrostbyte commented 2026-07-24 19:38:19 -04:00
Owner
Copy link

Goal

    Evaluate suitable lightweight lexical search backends against the pilot corpus and retrieval fixtures, then record the selected backend in an architecture decision record.

    ## Why

    Arkive needs a backend chosen from measured project requirements: offline operation, deterministic builds, Unicode behavior, citation metadata, constrained resources, and long-term maintainability.

    ## Scope

    - Identify a small set of plausible offline full-text search backends.

  • Build minimal throwaway evaluations against the actual five-document corpus and M2-01 fixtures.

  • Measure or compare deterministic build behavior, Python integration, CPU use, RAM use, index size, query latency, Unicode handling, filters, portability, citation metadata support, and maintenance burden.

  • Document licensing and redistribution implications of each candidate dependency.

  • Record the selected backend and rejected alternatives in a new ADR.

  • Define the architectural boundary between the search core and presentation.

  • Delete or clearly isolate disposable evaluation code that is not part of the selected implementation.

      ## Out of scope

  - Production index-builder implementation.

  • Vector or hybrid retrieval.

  • LLM evaluation.

  • A user interface.

  • Premature optimization for a large corpus.

      ## Dependencies

  - #19 — Define retrieval evaluation fixtures and success criteria

  ## Acceptance criteria

  - At least two credible backends are compared against the same fixtures.

  • The decision is supported by recorded evidence rather than popularity alone.

  • The ADR covers offline use, determinism, resources, Unicode, citation support, portability, licensing, and maintenance.

  • The selected backend can preserve Arkive source, document, and chunk identifiers.

  • No vector or LLM dependency is introduced.

      ## Validation

  - Run the M2-01 fixture set against each evaluated backend.

  • Record reproducible evaluation commands and environment details.

  • Verify the selected backend operates with networking disabled.

  • Review and approve the ADR.

      ## Suggested branch

  `feature/issue-20-search-backend-adr`

  <!-- arkive-planning-automation:2026-07-24-v3:M2-02 -->
## Goal Evaluate suitable lightweight lexical search backends against the pilot corpus and retrieval fixtures, then record the selected backend in an architecture decision record. ## Why Arkive needs a backend chosen from measured project requirements: offline operation, deterministic builds, Unicode behavior, citation metadata, constrained resources, and long-term maintainability. ## Scope - Identify a small set of plausible offline full-text search backends. - Build minimal throwaway evaluations against the actual five-document corpus and M2-01 fixtures. - Measure or compare deterministic build behavior, Python integration, CPU use, RAM use, index size, query latency, Unicode handling, filters, portability, citation metadata support, and maintenance burden. - Document licensing and redistribution implications of each candidate dependency. - Record the selected backend and rejected alternatives in a new ADR. - Define the architectural boundary between the search core and presentation. - Delete or clearly isolate disposable evaluation code that is not part of the selected implementation. ## Out of scope - Production index-builder implementation. - Vector or hybrid retrieval. - LLM evaluation. - A user interface. - Premature optimization for a large corpus. ## Dependencies - #19 — Define retrieval evaluation fixtures and success criteria ## Acceptance criteria - At least two credible backends are compared against the same fixtures. - The decision is supported by recorded evidence rather than popularity alone. - The ADR covers offline use, determinism, resources, Unicode, citation support, portability, licensing, and maintenance. - The selected backend can preserve Arkive source, document, and chunk identifiers. - No vector or LLM dependency is introduced. ## Validation - Run the M2-01 fixture set against each evaluated backend. - Record reproducible evaluation commands and environment details. - Verify the selected backend operates with networking disabled. - Review and approve the ADR. ## Suggested branch `feature/issue-20-search-backend-adr` <!-- arkive-planning-automation:2026-07-24-v3:M2-02 -->
JackFrostbyte commented 2026-09-04 01:48:55 -04:00
Author
Owner
Copy link

Completed by PR #68.

SQLite FTS5 3.51.2 and Tantivy 0.26.0 were evaluated against Arkive’s actual five-document, 36-chunk pilot corpus using the unchanged Issue #19 M2-01 retrieval fixture.

Both candidates passed all 10 fixture cases, respected metadata filters and zero-result requirements, preserved Arkive source/document/chunk and citation identity, and operated fully offline. The evaluation also recorded determinism, build and query performance, CPU and memory use, index size, Unicode behavior, Python integration, portability, maintenance, licensing, and redistribution implications.

ADR-0004 selects SQLite FTS5 for Arkive’s offline lexical search core based on the recorded pilot evidence. Tantivy remains documented as a credible alternative if future representative larger-corpus measurements establish requirements that SQLite cannot meet.

The reproducible evaluation harness and full comparison report are retained under docs/evaluations/. No production search implementation, vector or hybrid retrieval, UI work, or Issue #21 index-manifest work was included.

Issue #20 is complete, and Issue #21 is now unblocked.

Completed by PR #68. SQLite FTS5 3.51.2 and Tantivy 0.26.0 were evaluated against Arkive’s actual five-document, 36-chunk pilot corpus using the unchanged Issue #19 M2-01 retrieval fixture. Both candidates passed all 10 fixture cases, respected metadata filters and zero-result requirements, preserved Arkive source/document/chunk and citation identity, and operated fully offline. The evaluation also recorded determinism, build and query performance, CPU and memory use, index size, Unicode behavior, Python integration, portability, maintenance, licensing, and redistribution implications. ADR-0004 selects SQLite FTS5 for Arkive’s offline lexical search core based on the recorded pilot evidence. Tantivy remains documented as a credible alternative if future representative larger-corpus measurements establish requirements that SQLite cannot meet. The reproducible evaluation harness and full comparison report are retained under `docs/evaluations/`. No production search implementation, vector or hybrid retrieval, UI work, or Issue #21 index-manifest work was included. Issue #20 is complete, and Issue #21 is now unblocked.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
develop
No results found.
Labels
Clear labels
  
area/deployment
Portable packaging and constrained-environment validation

  
area/governance
Project rules, policies, contributor workflow

  
area/metadata
Schemas, manifests, validation contracts

  
area/pipeline
Import, extraction, normalization, chunking

  
area/search
Indexing, retrieval, citation resolution

  
type/data
Source selection, metadata, fixtures, and corpus work

  
type/design
Architecture, policy, contracts, and ADR work

  
type/implementation
Production code or tooling

  
type/testing
Reproducibility, integration, evaluation, and benchmark work

No labels
area/deployment
area/governance
area/metadata
area/pipeline
area/search
type/data
type/design
type/implementation
type/testing
Milestone
Clear milestone
No items
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
JackFrostbyte
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
#21 Define index manifest schema v0.1
Arkive/arkive
Depends on
#19 Define retrieval evaluation fixtures and success criteria
Arkive/arkive
Reference
Arkive/arkive#20
No description provided.