Add the complete AI-independent offline proof-loop test #26

New issue

Open

opened 2026-07-24 19:38:22 -04:00 by JackFrostbyte · 0 comments

JackFrostbyte commented

2026-07-24 19:38:22 -04:00

Owner

Goal

    Automate the complete path from validated pilot records through index construction, known searches, citation resolution, and exact source-passage opening without network access or an LLM.

    ## Why

    The core Arkive proof loop must be demonstrated as one integrated, repeatable system rather than as separately passing components.

    ## Scope

    - Validate the complete five-document pilot corpus.

Build the lexical index and index manifest in an isolated test workspace.
Run the M2-01 query fixtures.
Verify expected ranked chunks and filter behavior.
Resolve every expected citation.
Open or load the correct authoritative source passage and verify exact text and offsets.
Exercise no-result queries and stale-index refusal.
Ensure the test does not start, import, call, or require an LLM.
Ensure the test succeeds with networking disabled.
Keep test artifacts isolated from committed production outputs.
```
  ## Out of scope

  - Portable packaging.
```
Resource benchmarking.
Embeddings, vector search, or generated answers.
Translation or speech.

A graphical interface.

  ## Dependencies

  - #19 — Define retrieval evaluation fixtures and success criteria

#22 — Implement deterministic full-text index generation
#23 — Implement the offline search library and CLI
#24 — Implement citation resolution and exact passage display

#25 — Detect stale, mismatched, missing, or corrupted indexes

  ## Acceptance criteria

  - One automated workflow validates, builds, searches, resolves, and verifies exact passages.

All M2-01 expected cases pass.
No-result cases return no misleading passage.
Stale or corrupted indexes are refused.
The workflow requires no network connection or AI component.

Repeated proof-loop runs produce the same deterministic results.

  ## Validation

  - Run the complete unit-test and proof-loop suites in an isolated clean workspace.

Disable network access during the proof-loop test.
Run the proof loop twice and compare results.

Confirm no model file, inference server, embedding service, or remote API is available.

  ## Suggested branch

  `feature/issue-26-offline-proof-loop-test`

  <!-- arkive-planning-automation:2026-07-24-v3:M2-08 -->

## Goal Automate the complete path from validated pilot records through index construction, known searches, citation resolution, and exact source-passage opening without network access or an LLM. ## Why The core Arkive proof loop must be demonstrated as one integrated, repeatable system rather than as separately passing components. ## Scope - Validate the complete five-document pilot corpus. - Build the lexical index and index manifest in an isolated test workspace. - Run the M2-01 query fixtures. - Verify expected ranked chunks and filter behavior. - Resolve every expected citation. - Open or load the correct authoritative source passage and verify exact text and offsets. - Exercise no-result queries and stale-index refusal. - Ensure the test does not start, import, call, or require an LLM. - Ensure the test succeeds with networking disabled. - Keep test artifacts isolated from committed production outputs. ## Out of scope - Portable packaging. - Resource benchmarking. - Embeddings, vector search, or generated answers. - Translation or speech. - A graphical interface. ## Dependencies - #19 — Define retrieval evaluation fixtures and success criteria - #22 — Implement deterministic full-text index generation - #23 — Implement the offline search library and CLI - #24 — Implement citation resolution and exact passage display - #25 — Detect stale, mismatched, missing, or corrupted indexes ## Acceptance criteria - One automated workflow validates, builds, searches, resolves, and verifies exact passages. - All M2-01 expected cases pass. - No-result cases return no misleading passage. - Stale or corrupted indexes are refused. - The workflow requires no network connection or AI component. - Repeated proof-loop runs produce the same deterministic results. ## Validation - Run the complete unit-test and proof-loop suites in an isolated clean workspace. - Disable network access during the proof-loop test. - Run the proof loop twice and compare results. - Confirm no model file, inference server, embedding service, or remote API is available. ## Suggested branch `feature/issue-26-offline-proof-loop-test`

JackFrostbyte added this to the Offline Search Proof Loop v0.1 milestone

2026-07-24 19:38:22 -04:00

JackFrostbyte added the

type/testing

area/search

labels

2026-07-24 19:38:22 -04:00

JackFrostbyte added a new dependency

2026-07-24 20:10:50 -04:00

#19 Define retrieval evaluation fixtures and success criteria

JackFrostbyte added a new dependency

2026-07-24 20:10:50 -04:00