spillwavesolutions/research-knowledge-capture
v0.2.9MIT
Research Knowledge Capture (RKC): ingest deep-research dumps and articles into an OKF ContentPack of areas, subjects, claims, evidence, and findings. Layer 0 of the research second brain.
Changelog
0.2.9 — 2026-09-19
Symlinked-root patch. Closes #35 via #37.
- Engine parity holds on a symlinked root (#35).
rgprints resolved paths, andsearch()passed the caller's root through unresolved, so_relcould not make the rg hits relative. It fell intoexcept ValueErrorand returned the full absolute path with a doubled leading slash, and the two engines then reported different paths for the same file. That is the parity contract 0.2.8 shipped to guarantee, and the caller could not tell the wrong path from a real one. The root is the cause, not_rel: both engines derive their paths from it, sosearch()resolves it once. One syscall per call, not one per hit. _relno longer catchesValueError. The old branch did not recover, it returned a wrong value.- Not macOS-only.
/varis a symlink to/private/varthere, so everytempfileroot reproduces it. A symlinked checkout or a container bind mount is the same shape on Linux. CI has neither, which is why it shipped. - New
tools/ci-local.shmirrors every step of.github/workflows/ci.yml. Its first check parses every workflow and fails when the glob finds none. PKC #82 was an unparseableci.ymlthat failed at startup with zero jobs, and a workflow cannot check itself. - Tests:
test_symlinked_root_agrees_across_enginessearches through a real symlink.
0.2.8 — 2026-09-19
Retrieval-ladder patch: parse once, engine parity, fail-closed rg override, rg test coverage. Closes #30, #31, #32 via #33.
rkc_search.pyscan path parses each file once (closes #30).iter_okfalready parsed every file to yield it;search()parsed each again (measured 2.0×, ~2.9 s vs 99 ms for rg on a 3k-file tree). The scan rung is what hosts without ripgrep run.rkc_validate.pyparses the tree once per run;validate()andspine_issues()share it (load_tree).- Search parity (#31): the filename stem no longer enters the haystack (
hay_title,hay_id). A titleless node whose filename matched the query was found byscanbut invisible to thergprefilter. Filenames are not content; display still falls back to the stem. find_rg()(#32) honoursRKC_RG_PATH/OKF_RG_PATH/SECOND_BRAIN_RG_PATHand fails closed when the override is unusable — the same rule as research-graph and PKC, so one env var means one thing across plugins.- Scan search skips unparsable files instead of aborting, matching the rg path.
- Tests:
tests/fixtures/fake_rg.py(ported from PKC) and rg-vs-scan parity tests; the rg engine had no coverage.
0.2.7 — 2026-09-12
Query-time retrieval, PKC/SAC/DEKC parity.
agents/research-retriever.md— isolated retriever. Scores and deep-walks off the parent. Returns a summary card only./research-retrieveskill + command. Parent spawns the retriever; do not pack inline for Q&A.rkc_pack.py --summary— compact card-friendly pack (seed, hops/nodes/tokens, spine counts, lead nodes, gaps).--tinyis hops=1 / max_nodes=8.--max-tokensfail-closed (default 32000). Seed may be an id, path, or stem.rkc_search.py+/research-search— git-nativerg→ scan ladder. AND terms, type filter, limit 5,--json. No Chroma, BM25, or Kuzu. Layer 1 may feed seeds later.research-capturerdelegates Q&A retrieval toresearch-retriever.
0.2.6 — 2026-08-25
--dry-rundoes not createresearch/catalogs/ingest-index.jsonor its parent directories. Closes #25.- Empty, whitespace-only, and frontmatter-only sources are skipped and reported.
--min-source-bytesdefaults to 1. Closes #26.
0.2.5 — 2026-08-25
- Subject title is derived from the
--subjectslug (or--subject-title), not the first source filename.rkc_spine.pyuses the same rule when it creates a missing Subject. Closes #23.
0.2.4 — 2026-08-25
- Intra-run claim merge no longer calls
add_linkon a node whose path is stillNone. A second file that repeats a claim from the first no longer crashes extract with'NoneType' object has no attribute 'read_text'.
0.2.3 — 2026-08-25
- Quote date-like YAML scalars.
as_of/timestamp/captured_atstay strings through parse and write. Closes #16 and #17. parse_okfwrapsyaml.safe_loadinParseErrorthat names the file.rkc_validate.pyreports it and continues. Closes #6.- Default
--extractor-versiontoEXTRACTOR_VERSION. Same bytes under another version are reported, not duplicated, unless--allow-reextract. Idempotent--extractskips the extractor. Closes #18. index.md/README.mdtitles becomeparent/filename. Closes #19.
0.2.2 — 2026-08-24
rkc_spine.py//research-spinerepairs Area → Subject → Task on a tree ingested by 0.2.0.--link-tasks,--list-subjects,--area-map. Idempotent.
0.2.1 — 2026-08-24
Bulk-ingest hardening from the first 2,997-file corpus (#13).
Blockers:
- Ingest lookup is O(1) via
research/catalogs/ingest-index.json. Rebuild with--rebuild-index(sources + tasks only). #1 - Heuristic extract skips sources over 200 KB (
--force-largeto override), caps candidates, and matches near-duplicates through a token index instead of all-pairs Jaccard. #2 - Long subject slugs keep a prefix plus an 8-hex digest of the full value so distinct names no longer collapse. #3
Major:
pyyamlis a required dependency._mini_yamlunescapes\\n/\\t/\\"/\\\\and warns when the fallback runs. #4_needs_quotetreats a trailing colon as quoted. Ingest writes throughwrite_okf. #5rkc_validate.pyreports the unparsable path and continues. #6- SourceDocuments store
origin_path.index.md/README.mdtitles use the parent directory.--source-kindis accepted. #7 - Archive-only ingest creates the Subject node. #8
- Ingest writes
has_task.--areacreates a ResearchArea and writeshas_subject.--spinefails validation on a missing ladder. #9
Minor:
- Heuristic extractor drops Markdown/numbered headings and
filecite/citeturnexport artifacts. #10 - Per-file progress on stderr, ingest-key resume,
--errors-file(does not abort the run). Intra-tree--workersis not added: concurrent writers race, andpkill -forphans multiprocessing children. Shard by Subject instead. #11 - Vendor is free text. Convention now includes
perplexityandunknown. #12
0.2.0 — 2026-08-22
Phase 2 extractor.
rkc_segment.py— global line/char locators, heading/paragraph packing.rkc_extract.py— heuristic extract + fail-closed overlay JSON.claim_keymerge attaches Evidence to existing Claims; near-matchsame_as;contradictsas a new draft.- Accepted / reviewed / verified nodes are never auto-superseded.
- Required PR summary under
research/catalogs/pr-summaries/(not an OKF type). - Ingest key is
sha256(bytes|prompt_hash|extractor_version).--extractflag. /research-extractskill + command. ADR 007.
0.1.0 — 2026-08-22
Phase 0–3 bootstrap.
- Eight nouns, twelve rels, JSON schemas, registry.
- Five-host packaging (Claude Code, Grok Build, Codex, Cursor, Agent Plugins 1.0).
/research-ingest,/research-pack,/research-validate.- Northstar / Lumenfield fiction sample + eval fixtures (spine, claim_key, quote verify, unknown rel).
- ADRs 001–006. Compact PRD under
docs/prd/.