Validate the proof loop under constrained resources #28

New issue

Open

opened 2026-07-24 19:38:23 -04:00 by JackFrostbyte · 0 comments

JackFrostbyte commented

2026-07-24 19:38:23 -04:00

Owner

Goal

    Measure and document the portable non-AI proof loop under a constrained CPU-only Linux environment representative of Arkive's low-power target.

    ## Why

    Arkive's later retrieval and local-AI decisions need a measured baseline for storage, memory, latency, CPU use, startup, and offline behavior.

    ## Scope

    - Define a reproducible constrained VM or container profile when real target hardware is unavailable.

Use CPU-only operation and no required discrete GPU.
Measure source and processed-corpus size, index size, bundle size, and installed size.
Measure idle RAM, startup peak RAM, search RAM, and citation-resolution RAM.
Measure startup time and query latency across the M2-01 fixture set.
Measure CPU utilization using a documented method.
Confirm behavior with networking disabled.
Record host hardware, constrained environment, software versions, commands, and measurement limitations.

Publish the results as the baseline for the future low-power RAG milestone.

  ## Out of scope

  - Optimizing or changing the search backend solely to improve benchmark numbers.

GPU benchmarking.
LLM, embedding, translation, or speech benchmarking.
Claims about battery life or power draw without appropriate measurement hardware.

Production performance guarantees.

  ## Dependencies

  - #27 — Create a portable non-AI proof-loop bundle

  ## Acceptance criteria

  - The constrained profile and benchmark method are reproducible.

All required size, RAM, CPU, startup, and latency measurements are recorded.
The complete proof loop remains functional without network access.
Results identify limitations and avoid unsupported power-consumption claims.
The baseline is suitable for comparison with later vector and local-LLM work.

Milestone 2 completion criteria are reviewed and recorded.

  ## Validation

  - Recreate the documented constrained environment.

Run the complete M2-01 fixture set and citation resolution.
Repeat measurements enough to report a documented representative value or range.
Run with networking disabled.

Review the final report against the Milestone 2 completion criteria.

  ## Suggested branch

  `feature/issue-28-constrained-benchmark`

  <!-- arkive-planning-automation:2026-07-24-v3:M2-10 -->

## Goal Measure and document the portable non-AI proof loop under a constrained CPU-only Linux environment representative of Arkive's low-power target. ## Why Arkive's later retrieval and local-AI decisions need a measured baseline for storage, memory, latency, CPU use, startup, and offline behavior. ## Scope - Define a reproducible constrained VM or container profile when real target hardware is unavailable. - Use CPU-only operation and no required discrete GPU. - Measure source and processed-corpus size, index size, bundle size, and installed size. - Measure idle RAM, startup peak RAM, search RAM, and citation-resolution RAM. - Measure startup time and query latency across the M2-01 fixture set. - Measure CPU utilization using a documented method. - Confirm behavior with networking disabled. - Record host hardware, constrained environment, software versions, commands, and measurement limitations. - Publish the results as the baseline for the future low-power RAG milestone. ## Out of scope - Optimizing or changing the search backend solely to improve benchmark numbers. - GPU benchmarking. - LLM, embedding, translation, or speech benchmarking. - Claims about battery life or power draw without appropriate measurement hardware. - Production performance guarantees. ## Dependencies - #27 — Create a portable non-AI proof-loop bundle ## Acceptance criteria - The constrained profile and benchmark method are reproducible. - All required size, RAM, CPU, startup, and latency measurements are recorded. - The complete proof loop remains functional without network access. - Results identify limitations and avoid unsupported power-consumption claims. - The baseline is suitable for comparison with later vector and local-LLM work. - Milestone 2 completion criteria are reviewed and recorded. ## Validation - Recreate the documented constrained environment. - Run the complete M2-01 fixture set and citation resolution. - Repeat measurements enough to report a documented representative value or range. - Run with networking disabled. - Review the final report against the Milestone 2 completion criteria. ## Suggested branch `feature/issue-28-constrained-benchmark`

JackFrostbyte added this to the Offline Search Proof Loop v0.1 milestone

2026-07-24 19:38:23 -04:00

JackFrostbyte added the

type/testing

area/deployment

labels

2026-07-24 19:38:23 -04:00

JackFrostbyte added a new dependency

2026-07-24 20:10:51 -04:00

#27 Create a portable non-AI proof-loop bundle

JackFrostbyte referenced this issue

2026-07-31 01:21:16 -04:00