Arkive/arkive

Fork 0

Define the bundled-core and user-local storage boundary #59

Merged

JackFrostbyte merged 4 commits from feature/issue-45-storage-boundary into develop

2026-08-11 02:18:23 -04:00

McJuniorstein commented

2026-08-09 23:04:23 -04:00

Member

Summary

Adds docs/storage-boundary.md, specifying the hard separation between the redistributable core and the user-local overlay: canonical locations, ownership, manifests and identifiers, the shared pipeline's distribution context, lifecycle handling, release and export behaviour, and container and portable deployments.
Extends the validator's contamination check to reject any record carrying distribution_scope: user_local found under data/ — documents and chunks, not only local acquisition records.
Adds .gitignore entries for the conventional overlay root directories and the sentinel marker, deliberately excluding record filenames.
Adds tests/test_storage_boundary.py with eight regression tests.
Resulting behaviour: the boundary is specified well enough for Issues #46 through #50 to implement against, and the one input path that exists today is checked.

Acceptance criteria

Section 9 of the document maps each criterion to its mechanism; summarised here.

No command can silently promote a local artifact into the bundled core — the runtime holds no writable handle to the core (§3); the pipeline receives its distribution context explicitly and refuses cross-root paths (§5); release tooling performs a positive contamination check and aborts (§7.1). Enforced today by the validator for the core store.
Core builds do not enumerate or hash user-local content — core manifest generation takes the core root as its sole input and has no parameter that could name the local root (§4.1).
User-local data paths are ignored by version control and excluded from releases — the default root is outside the repository (§2.2), .gitignore covers a misconfigured root (§7.3), release tooling aborts on contamination (§7.1), and exports exclude the overlay by default (§7.2).
Runtime results can always identify their originating domain — every cross-boundary reference is a (distribution_scope, identifier) pair, and every document and chunk record carries distribution_scope (§4.2).
Container and non-container deployments follow the same boundary — the boundary lives in the code path rather than the deployment shape, and all three deployment forms run the same startup validation (§8).
The design reuses one processing pipeline rather than two divergent implementations — a single implementation takes an explicit distribution context (§5).

Verification

git diff --check
python pipeline/validate/validate_records.py — PASS, committed corpus unchanged
python -m unittest discover -s tests -v — 150 tests passing, up from 142
python -m ruff check src pipeline tests
python -m ruff format --check src pipeline tests
Rendered Markdown reviewed — 5 relative links checked, 0 broken
Complete diff inspected

Scope confirmation

No unrelated production behaviour changed
No metadata contract changed — no schema file is touched
No unreviewed source or generated data added
No acquisition, index generation, or federated ranking implemented
No test tooling, dependency, or CI infrastructure introduced

Design decisions worth reviewing

The read-only core handle is the primary guarantee, not a policy statement. A process that never holds a writable handle to the core cannot write into it regardless of caller intent. Everything else in §3 and §5 is defence layered on top of that.

The distribution context is passed, never inferred. §5 states this as a rule with its reasoning: inference from a path, filename, or ambient state is precisely how a local artifact ends up treated as core. This is the constraint most likely to be eroded later by a convenience helper that "figures out" the domain, so it is written as a rule rather than a convention.

Identifiers are not renamed to avoid collision. Local artifacts keep the same doc_ / chk_ patterns; a cross-boundary reference carries the domain alongside the identifier instead. Renaming would have made local records structurally different from core ones for no benefit and broken the shared pipeline's uniformity.

The overlay stores a recipe snapshot, and this resolves a constraint created by PR #56. A local acquisition record pins the recipe_version it used, and the validator requires that version to match the referenced recipe. Comparing against the shipped catalogue would invalidate every existing acquisition record the moment Arkive revised a recipe. §2.4 therefore has the overlay retain a verbatim copy of the recipe at the version used, and local validation resolve against that snapshot. The shipped catalogue remains authoritative for what may be acquired next; the snapshot records what was acquired then.

Stale indexes are excluded from search rather than queried. §6.4. A stale index returns citations that may point at text no longer at those offsets, and a wrong citation is worse than a missing result in a project whose central claim is that every answer resolves to a real passage.

A footgun caught during implementation

The first .gitignore draft listed bare record filenames (local_acquisition.json, recipe-snapshot.json). git check-ignore --no-index confirmed those patterns silently shadow the acquisition fixtures added in PR #56: the already-tracked file survives, but any new fixture with that name would be invisible to git add.

The entries were narrowed to the conventional root directories and the sentinel marker only, and GitignoreBoundaryTests now asserts that record filenames stay un-ignored while the marker stays ignored. The comment in .gitignore records why.

Test approach

tests/test_storage_boundary.py points validator.DATA_DIR at a temporary tree rather than writing contamination fixtures into the real data/. A fixture placed in data/ would itself be the thing the guard exists to prevent, so the tests must not create one.

Coverage includes both directions: user_local documents, chunks, and acquisition records under data/ are rejected; bundled_core records, records omitting the field entirely, and user_local records outside data/ all pass. The overlay is legitimate — only its presence inside the core store is not.

Note for a follow-up issue

Not addressed here, and out of scope under the tooling rule in AGENTS.md: the test harness has accumulated some duplication worth consolidating. Three test modules bootstrap the validator through spec_from_file_location while five import the package normally, a fixture helper is borrowed across unrelated TestCase classes, DATA_DIR is patched by hand, and there is no coverage measurement. The last one is the substantive gap — an unreachable validator rule was found by reasoning during the PR #56 review rather than by tooling, and at 838 lines with five rule families that question is no longer answerable by inspection. A separate issue will be raised.

Linked issue

Closes #45

## Summary - Adds `docs/storage-boundary.md`, specifying the hard separation between the redistributable core and the user-local overlay: canonical locations, ownership, manifests and identifiers, the shared pipeline's distribution context, lifecycle handling, release and export behaviour, and container and portable deployments. - Extends the validator's contamination check to reject **any** record carrying `distribution_scope: user_local` found under `data/` — documents and chunks, not only local acquisition records. - Adds `.gitignore` entries for the conventional overlay root directories and the sentinel marker, deliberately excluding record filenames. - Adds `tests/test_storage_boundary.py` with eight regression tests. - Resulting behaviour: the boundary is specified well enough for Issues #46 through #50 to implement against, and the one input path that exists today is checked. ## Acceptance criteria Section 9 of the document maps each criterion to its mechanism; summarised here. - [x] **No command can silently promote a local artifact into the bundled core** — the runtime holds no writable handle to the core (§3); the pipeline receives its distribution context explicitly and refuses cross-root paths (§5); release tooling performs a positive contamination check and aborts (§7.1). Enforced today by the validator for the core store. - [x] **Core builds do not enumerate or hash user-local content** — core manifest generation takes the core root as its sole input and has no parameter that could name the local root (§4.1). - [x] **User-local data paths are ignored by version control and excluded from releases** — the default root is outside the repository (§2.2), `.gitignore` covers a misconfigured root (§7.3), release tooling aborts on contamination (§7.1), and exports exclude the overlay by default (§7.2). - [x] **Runtime results can always identify their originating domain** — every cross-boundary reference is a `(distribution_scope, identifier)` pair, and every document and chunk record carries `distribution_scope` (§4.2). - [x] **Container and non-container deployments follow the same boundary** — the boundary lives in the code path rather than the deployment shape, and all three deployment forms run the same startup validation (§8). - [x] **The design reuses one processing pipeline rather than two divergent implementations** — a single implementation takes an explicit distribution context (§5). ## Verification - [x] `git diff --check` - [x] `python pipeline/validate/validate_records.py` — PASS, committed corpus unchanged - [x] `python -m unittest discover -s tests -v` — **150 tests** passing, up from 142 - [x] `python -m ruff check src pipeline tests` - [x] `python -m ruff format --check src pipeline tests` - [x] Rendered Markdown reviewed — 5 relative links checked, 0 broken - [x] Complete diff inspected ## Scope confirmation - [x] No unrelated production behaviour changed - [x] No metadata contract changed — no schema file is touched - [x] No unreviewed source or generated data added - [x] No acquisition, index generation, or federated ranking implemented - [x] No test tooling, dependency, or CI infrastructure introduced ## Design decisions worth reviewing **The read-only core handle is the primary guarantee, not a policy statement.** A process that never holds a writable handle to the core cannot write into it regardless of caller intent. Everything else in §3 and §5 is defence layered on top of that. **The distribution context is passed, never inferred.** §5 states this as a rule with its reasoning: inference from a path, filename, or ambient state is precisely how a local artifact ends up treated as core. This is the constraint most likely to be eroded later by a convenience helper that "figures out" the domain, so it is written as a rule rather than a convention. **Identifiers are not renamed to avoid collision.** Local artifacts keep the same `doc_` / `chk_` patterns; a cross-boundary reference carries the domain alongside the identifier instead. Renaming would have made local records structurally different from core ones for no benefit and broken the shared pipeline's uniformity. **The overlay stores a recipe snapshot, and this resolves a constraint created by PR #56.** A local acquisition record pins the `recipe_version` it used, and the validator requires that version to match the referenced recipe. Comparing against the *shipped* catalogue would invalidate every existing acquisition record the moment Arkive revised a recipe. §2.4 therefore has the overlay retain a verbatim copy of the recipe at the version used, and local validation resolve against that snapshot. The shipped catalogue remains authoritative for what may be acquired next; the snapshot records what was acquired then. **Stale indexes are excluded from search rather than queried.** §6.4. A stale index returns citations that may point at text no longer at those offsets, and a wrong citation is worse than a missing result in a project whose central claim is that every answer resolves to a real passage. ## A footgun caught during implementation The first `.gitignore` draft listed bare record filenames (`local_acquisition.json`, `recipe-snapshot.json`). `git check-ignore --no-index` confirmed those patterns silently shadow the acquisition fixtures added in PR #56: the already-tracked file survives, but any *new* fixture with that name would be invisible to `git add`. The entries were narrowed to the conventional root directories and the sentinel marker only, and `GitignoreBoundaryTests` now asserts that record filenames stay un-ignored while the marker stays ignored. The comment in `.gitignore` records why. ## Test approach `tests/test_storage_boundary.py` points `validator.DATA_DIR` at a temporary tree rather than writing contamination fixtures into the real `data/`. A fixture placed in `data/` would itself be the thing the guard exists to prevent, so the tests must not create one. Coverage includes both directions: `user_local` documents, chunks, and acquisition records under `data/` are rejected; `bundled_core` records, records omitting the field entirely, and `user_local` records *outside* `data/` all pass. The overlay is legitimate — only its presence inside the core store is not. ## Note for a follow-up issue Not addressed here, and out of scope under the tooling rule in `AGENTS.md`: the test harness has accumulated some duplication worth consolidating. Three test modules bootstrap the validator through `spec_from_file_location` while five import the package normally, a fixture helper is borrowed across unrelated `TestCase` classes, `DATA_DIR` is patched by hand, and there is no coverage measurement. The last one is the substantive gap — an unreachable validator rule was found by reasoning during the PR #56 review rather than by tooling, and at 838 lines with five rule families that question is no longer answerable by inspection. A separate issue will be raised. ## Linked issue Closes #45

McJuniorstein added 1 commit

2026-08-09 23:04:24 -04:00

Define the bundled-core and user-local storage boundary aa8c63ce41

Specify the architectural separation between Arkive's redistributable
core and content a user acquires on their own system, and enforce the
part of it that is enforceable today.

Locations and ownership:
- the local overlay root lives outside the repository, the installed
  bundle, and every release input path, defaulting to a platform user
  data directory and configurable through ARKIVE_LOCAL_ROOT
- a configured root inside, or a parent of, any of those is a startup
  error rather than a warning
- the local root is user-owned and mode 0700; the runtime opens the core
  read-only, which is the primary mechanical guarantee against promotion

One pipeline, explicit context:
- a single processing implementation serves both domains and receives an
  explicit distribution context rather than inferring one from paths,
  filenames, or ambient state, since inference is where silent promotion
  enters
- a context reads and writes only within its own root, and a bundled_core
  context is not writable at runtime

Manifests, identifiers, and citations:
- per-domain manifests, with core manifest generation taking the core
  root as its only input so it cannot enumerate user-local content
- identifiers are unique within a domain, and every cross-boundary
  reference is a (distribution_scope, identifier) pair, so a merged
  result set always names the domain it resolves in

Provenance:
- the overlay stores a verbatim snapshot of the recipe version used, so a
  pinned local acquisition stays verifiable offline and does not fail
  validation when the shipped catalogue is later revised

Lifecycle:
- deletion cascades from an acquisition to its derived artifacts and
  index entries
- recipe withdrawal stops future acquisition, records and surfaces the
  reason, and never deletes a user's existing files
- a stale index is excluded from search rather than queried, because a
  wrong citation is worse than a missing result

Release, export, and diagnostics:
- release tooling performs a positive contamination check and aborts on a
  marker file, a user_local record, a local provenance file, or a path
  inside the local root
- exports exclude the overlay by default; support bundles never include
  local content, filenames, titles, or URLs

Enforcement added here: the validator now rejects any record carrying
distribution_scope user_local found under data/, covering documents and
chunks as well as local acquisition records.

The gitignore entries deliberately list only the conventional root
directories and the sentinel marker. Bare record filenames such as
local_acquisition.json are not ignored, because they are legitimate test
fixture names and a bare pattern silently shadows them; a regression test
asserts this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

JackFrostbyte commented

2026-08-10 00:47:05 -04:00

Owner

I found two items I would like corrected before merge:

1. Blocking: symlink escape in the core contamination check

_under_data_dir() currently uses:

path.resolve().relative_to(DATA_DIR)

Because Path.resolve() follows symbolic links, a metadata file whose directory entry is inside data/ but which is a symlink to a target outside data/ is classified as outside the core. For example:

data/sources/example/documents.json -> /some/local/path/documents.json

would not trigger _reject_local_content() solely on the basis of being a user_local record under the core tree.

That weakens the hard-boundary/fail-closed guarantee this issue is intended to establish.

Please either reject symlinked core metadata records outright, or make the containment check account for both the path's location in the core namespace and its resolved target. Please also add a regression test covering a user_local record reached through a symlink located under data/.

My preference is to reject symlinked metadata records in the bundled core unless Arkive has an explicit use case for supporting them.

2. Cross-platform permissions

docs/storage-boundary.md defines a Windows local root under %LOCALAPPDATA%, but §3 universally specifies mode 0700. 0700 is a POSIX permission model and does not provide the corresponding protection on Windows.

Please make the architecture platform-specific here:

POSIX systems: user-owned local root with mode 0700.
Windows: equivalent user-private protection through Windows filesystem ACL/DACL semantics.

The actual Windows implementation can remain downstream, but Issue #45 is defining the ownership/permission contract that implementation will follow.

Minor documentation cleanup: §9 says “every record carries distribution_scope”. The schemas currently put distribution_scope on document and chunk records. I suggest matching the more precise language already used in §4.2: document/chunk records carry the scope, and cross-domain references carry (distribution_scope, identifier).

Other than these points, I found the PR well scoped and consistent with the existing architecture. Once the symlink boundary and cross-platform permission language are corrected, I expect this to be ready for merge.

Thanks for the thorough implementation. The overall architecture matches Issue #45 and ADR-0002 well, and I agree with the main design choices: separate stores/manifests, explicit distribution context, domain-qualified references, retained recipe snapshots, stale-index exclusion, and keeping the later acquisition/index/release work in Issues #46–#50. I found two items I would like corrected before merge: **1. Blocking: symlink escape in the core contamination check** `_under_data_dir()` currently uses: ```python path.resolve().relative_to(DATA_DIR) ``` Because `Path.resolve()` follows symbolic links, a metadata file whose directory entry is inside `data/` but which is a symlink to a target outside `data/` is classified as outside the core. For example: ```text data/sources/example/documents.json -> /some/local/path/documents.json ``` would not trigger `_reject_local_content()` solely on the basis of being a `user_local` record under the core tree. That weakens the hard-boundary/fail-closed guarantee this issue is intended to establish. Please either reject symlinked core metadata records outright, or make the containment check account for both the path's location in the core namespace and its resolved target. Please also add a regression test covering a `user_local` record reached through a symlink located under `data/`. My preference is to reject symlinked metadata records in the bundled core unless Arkive has an explicit use case for supporting them. **2. Cross-platform permissions** `docs/storage-boundary.md` defines a Windows local root under `%LOCALAPPDATA%`, but §3 universally specifies mode `0700`. `0700` is a POSIX permission model and does not provide the corresponding protection on Windows. Please make the architecture platform-specific here: * POSIX systems: user-owned local root with mode `0700`. * Windows: equivalent user-private protection through Windows filesystem ACL/DACL semantics. The actual Windows implementation can remain downstream, but Issue #45 is defining the ownership/permission contract that implementation will follow. **Minor documentation cleanup:** §9 says “every record carries `distribution_scope`”. The schemas currently put `distribution_scope` on document and chunk records. I suggest matching the more precise language already used in §4.2: document/chunk records carry the scope, and cross-domain references carry `(distribution_scope, identifier)`. Other than these points, I found the PR well scoped and consistent with the existing architecture. Once the symlink boundary and cross-platform permission language are corrected, I expect this to be ready for merge.

JackFrostbyte referenced this pull request

2026-08-10 01:02:59 -04:00

Consolidate the test harness and add coverage measurement #60

McJuniorstein referenced this pull request from a commit

2026-08-10 01:38:01 -04:00

Reject symlinks in the bundled core and correct the permission contract

McJuniorstein added 1 commit

2026-08-10 01:38:01 -04:00

Reject symlinks in the bundled core and correct the permission contract a9034c4bf7

Addresses the review on PR #59.

Blocking: symlink escape in the contamination check.

`_under_data_dir()` resolved the path before testing containment, and
`Path.resolve()` follows symbolic links. A record whose directory entry
sat inside data/ but whose target lay outside it was therefore classified
as outside the core, so `_reject_local_content()` never fired on it. That
defeated the boundary this issue exists to establish.

Two fixes, because there were two escapes:

- containment is now judged by namespace as well as by resolved target. A
  record counts as core when either its lexical location or its resolved
  target sits under the core root. A record's domain follows the namespace
  it is published in, not wherever its bytes happen to live, and for a
  boundary check the safe direction to fail is towards treating something
  as core.
- symbolic links under the core root are rejected outright rather than
  followed. This closes the second escape found while testing the first: a
  recursive walk does not descend into a symlinked directory, so records
  beneath one were never examined at all rather than being examined and
  passed.

Writing the regression test for the symlinked-directory case also exposed
an unrelated defect: `validate_tree()` returned success as soon as no
record files were discovered, before reporting accumulated problems. A
tree whose only content was a symlinked directory therefore reported
"No record files found" and exited zero. The early return is now taken
only when there is genuinely nothing to report.

Five regression tests cover a user_local record reached through a symlink,
a bundled_core record reached the same way, a symlinked parent directory,
real core files still validating, and symlinks outside the core remaining
untouched.

Documentation corrections:

- section 3 now states user-private local storage as a contract with a
  per-platform mechanism, since POSIX mode 0700 has no meaning on Windows
  and must not stand in for an ACL. An implementation must verify the
  protection it actually applied, and refuse to write local content into a
  root it could not make private.
- section 9 now matches the precise language in section 4.2: document and
  chunk records carry `distribution_scope`, and cross-domain references
  carry the `(distribution_scope, identifier)` pair.
- section 7.1 and the section 11 summary record the namespace-containment
  and no-symlinks rules.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

McJuniorstein commented

2026-08-10 01:38:29 -04:00

Author

Member

All three addressed in a9034c4.

1. Symlink escape — confirmed, and there were two of them.

The finding was right. _under_data_dir() called Path.resolve() before testing containment, and resolve() follows symbolic links, so a record whose directory entry sat inside data/ but whose target lay outside it was classified as outside the core and skipped the guard entirely. Demonstrated directly against the old and new logic:

symlinked record under data/:
  old _under_data_dir -> False   (guard silently skipped)
  new _under_data_dir -> True    (guard fires)

Containment is now judged by namespace as well as resolved target: a record counts as core when either its lexical location or its resolved target sits under the core root. A record's domain follows the namespace it is published in rather than wherever its bytes live, and for a boundary check the safe direction to fail is towards treating something as core.

Taking your stated preference, symbolic links under the core root are also rejected outright rather than followed. That turned out to matter for a second reason I did not anticipate: writing the regression test for a symlinked directory showed that rglob does not descend into one, so records beneath it were never examined at all. Not a bad record passing validation — a whole subtree silently invisible. Both escapes are closed by the same rule.

A third defect fell out of writing that test. validate_tree() returned success as soon as no record files were discovered, before reporting accumulated problems. A tree whose only content was a symlinked directory therefore printed "No record files found" and exited zero, with the symlink problem already recorded and discarded. The early return is now taken only when there is genuinely nothing to report. That bug predates this PR but is squarely in the path this issue is about, so it is fixed here rather than deferred.

Five regression tests: a user_local record reached through a symlink, a bundled_core record reached the same way, a symlinked parent directory, real core files still validating cleanly, and symlinks outside the core remaining untouched — the overlay is free to use them.

2. Cross-platform permissions.

Section 3 now states the requirement as a contract — the local root must be private to the user running Arkive — with the mechanism given per platform: POSIX ownership plus mode 0700, and on Windows an ACL granting the owning account only, with inheritance disabled so a permissive parent ACL is not carried in. The text says explicitly that 0700 has no meaning on Windows and must not stand in for an ACL.

I added one requirement you did not ask for, and will remove it if you would rather it waited for the implementation issue: an implementation must verify the protection it actually applied rather than assume the creation call achieved it, and must refuse to write local content into a root it could not make private. A silently world-readable local root is the failure this section exists to prevent, and it is the kind of thing that is easy to assume and rarely checked.

3. distribution_scope wording.

Section 9 now uses the section 4.2 phrasing: document and chunk records carry distribution_scope, and cross-domain references carry the (distribution_scope, identifier) pair.

Section 7.1 and the section 11 summary also now record the namespace-containment rule and the no-symlinks rule, so the document describes the boundary as it is actually enforced.

Validation

git diff --check clean
metadata validator PASS, committed corpus unchanged
155 tests passing, up from 150
ruff check and ruff format --check clean

On sequencing: understood, and I agree with the ordering. #60 next once this merges, before the overlay implementation issues. Your refinements there are all sensible — branch coverage rather than line coverage, the corrected test-count criterion, uncovered branches being audited rather than assumed dead, real package imports rather than hiding the spec_from_file_location workaround behind a helper, and a recorded baseline with no threshold. The symlink cases added here will be part of that audit.

All three addressed in `a9034c4`. **1. Symlink escape — confirmed, and there were two of them.** The finding was right. `_under_data_dir()` called `Path.resolve()` before testing containment, and `resolve()` follows symbolic links, so a record whose directory entry sat inside `data/` but whose target lay outside it was classified as outside the core and skipped the guard entirely. Demonstrated directly against the old and new logic: ``` symlinked record under data/: old _under_data_dir -> False (guard silently skipped) new _under_data_dir -> True (guard fires) ``` Containment is now judged by **namespace as well as resolved target**: a record counts as core when either its lexical location or its resolved target sits under the core root. A record's domain follows the namespace it is published in rather than wherever its bytes live, and for a boundary check the safe direction to fail is towards treating something as core. Taking your stated preference, symbolic links under the core root are also **rejected outright** rather than followed. That turned out to matter for a second reason I did not anticipate: writing the regression test for a symlinked *directory* showed that `rglob` does not descend into one, so records beneath it were never examined at all. Not a bad record passing validation — a whole subtree silently invisible. Both escapes are closed by the same rule. **A third defect fell out of writing that test.** `validate_tree()` returned success as soon as no record files were discovered, before reporting accumulated problems. A tree whose only content was a symlinked directory therefore printed "No record files found" and exited zero, with the symlink problem already recorded and discarded. The early return is now taken only when there is genuinely nothing to report. That bug predates this PR but is squarely in the path this issue is about, so it is fixed here rather than deferred. Five regression tests: a `user_local` record reached through a symlink, a `bundled_core` record reached the same way, a symlinked parent directory, real core files still validating cleanly, and symlinks *outside* the core remaining untouched — the overlay is free to use them. **2. Cross-platform permissions.** Section 3 now states the requirement as a contract — the local root must be private to the user running Arkive — with the mechanism given per platform: POSIX ownership plus mode `0700`, and on Windows an ACL granting the owning account only, with inheritance disabled so a permissive parent ACL is not carried in. The text says explicitly that `0700` has no meaning on Windows and must not stand in for an ACL. I added one requirement you did not ask for, and will remove it if you would rather it waited for the implementation issue: an implementation must *verify* the protection it actually applied rather than assume the creation call achieved it, and must refuse to write local content into a root it could not make private. A silently world-readable local root is the failure this section exists to prevent, and it is the kind of thing that is easy to assume and rarely checked. **3. `distribution_scope` wording.** Section 9 now uses the section 4.2 phrasing: document and chunk records carry `distribution_scope`, and cross-domain references carry the `(distribution_scope, identifier)` pair. Section 7.1 and the section 11 summary also now record the namespace-containment rule and the no-symlinks rule, so the document describes the boundary as it is actually enforced. **Validation** - `git diff --check` clean - metadata validator PASS, committed corpus unchanged - **155 tests** passing, up from 150 - `ruff check` and `ruff format --check` clean On sequencing: understood, and I agree with the ordering. #60 next once this merges, before the overlay implementation issues. Your refinements there are all sensible — branch coverage rather than line coverage, the corrected test-count criterion, uncovered branches being audited rather than assumed dead, real package imports rather than hiding the `spec_from_file_location` workaround behind a helper, and a recorded baseline with no threshold. The symlink cases added here will be part of that audit.

McJuniorstein referenced this pull request

2026-08-10 01:49:36 -04:00

Consolidate the test harness and add coverage measurement #60

JackFrostbyte added 2 commits

2026-08-11 02:16:11 -04:00