Turning coding sessions into an Obsidian knowledge base

A coding session often contains one useful decision buried in several attempts and corrections. Copying the transcript into Obsidian preserves all of it, including the wrong turns. Asking a model to edit the vault directly makes the opposite problem possible: a concise but unsupported note can overwrite a carefully maintained page.

My local compiler separates those operations. Capture records evidence. A worker asks the model for a JSON proposal. Deterministic validation and a separate semantic review happen before a local writer touches Markdown.

Keep the queue outside the synchronized vault

View the pipeline at full size.

The storage split looks like this; the runtime location is local and deliberately outside iCloud:

local runtime/
  queue.sqlite3       capture and worker state
  captures/           captured input
  extractions/        validated intermediate output
  rejected/           proposals that did not pass
  transactions/       incomplete write journals
  committed/          completed journals
  credentials/        local credential material

Obsidian vault/
  daily/              normalized, redacted source evidence
  knowledge/
    concepts/         source-linked technical notes
    connections/      links between established ideas
  wiki/               human-maintained documentation

Capture hooks enqueue quickly; the periodic worker does the expensive processing. Normalized evidence excludes tool output and private reasoning, and records the source role, message identity and capture date. A user correction should not become indistinguishable from an earlier assistant guess.

Chunk checkpoints allow a failed extraction to resume. Three failures hold a chunk for inspection rather than repeatedly spending on the same input. The local writer lock prevents two workers on this machine from writing simultaneously. It is not a distributed lock across computers sharing an iCloud vault.

Ask for a proposal with a narrow shape

The model gets no filesystem tools. A change proposal contains articles and optional enrichments of selected wiki pages. Here is a minimal illustrative article proposal; its quote must occur verbatim in the selected source:

{
  "articles": [
    {
      "path": "concepts/deployment-ownership.md",
      "title": "Deployment ownership",
      "summary": "The owning repository's pipeline performs deployment.",
      "body": "An agent prepares a reviewable revision. Deployment follows the owning repository's approval and CI policy.",
      "evidence": [{"quote": "Keep deployment in the owning pipeline."}]
    }
  ],
  "enrichments": []
}

The validator first checks the schema. Article paths must then match:

r"(?:concepts|connections)/[a-z0-9][a-z0-9-]*\.md"

It rejects duplicate paths, empty fields and a body that supplies its own frontmatter. The compiler owns metadata such as source links and timestamps. Existing articles must have been read into the selected context and must still match that read before replacement is considered.

Require real evidence, then review what it means

The evidence check in the implementation is small:

def verify_evidence(evidence, source):
    if not evidence or any(
        not item["quote"].strip() or item["quote"] not in source
        for item in evidence
    ):
        raise ValueError("Missing or non-verbatim source evidence")

This catches invented or empty quotations. It does not prove that a quote supports the article, that the speaker was correct, or that the statement still applies. That is why schema and substring checks are followed by a semantic review. The resulting note retains the evidence and its origin so a reader can inspect the claim later.

Path validation is a separate boundary. After the allowed article-name pattern, the storage helper checks containment and rejects symlink traversal:

from pathlib import Path


def safe_path(base, relative):
    rel = Path(relative)
    if rel.is_absolute() or not rel.parts or any(
        part in ("..", ".") or part.startswith(".") for part in rel.parts
    ):
        raise ValueError(f"Unsafe path: {relative}")
    base = base.resolve()
    target = base / rel
    if target.resolve() != target or not target.resolve().is_relative_to(base):
        raise ValueError(f"Symlink or escaping path: {relative}")
    return target

These checks prevent a proposal from choosing ../outside.md, a hidden configuration directory or an existing symlink. They assume the local runtime and vault are under the operator's control; they are not a filesystem sandbox against a hostile process swapping paths concurrently.

Download the guard examples and their self-tests. They use temporary directories and require Python 3.10 or later:

python3 knowledge-guards.py -v

The tests exercise missing evidence, a fabricated quote, escaping paths, symlinks and the journal's concurrent-edit decision. They do not call a model or need access to a real vault.

Journal a write before applying it

Before changing files, the writer saves a transaction containing each target's previous and proposed content. Recovery uses three cases:

Current file contents	Recovery action
Equal the proposed after value	Already applied; skip this entry
Equal the recorded before value	Apply the proposed write
Equal neither value	Stop and preserve the journal for inspection

The last case protects a human edit made between the original read and recovery. Blindly replaying the model's output would lose that edit. Individual file replacements are atomic, but a multi-file transaction can be partially applied before interruption. The journal makes that state recoverable; it does not make the whole batch one atomic filesystem operation.

Wiki enrichment has a narrower write surface: the designated ## Lessons Learned (AI-compiled) section. A compile may enrich at most three selected wiki pages, with one to four plain-text lessons per page. Each enrichment must cite articles produced by that compile, and its target must still match the version that was read. The rest of the human-maintained page remains outside the generated section.

Inspect the compiler without spending on a new run

In the compiler checkout, these commands inspect plans, structure and local health:

# Search the local knowledge index without asking a model.
python3 scripts/query.py --search "GitLab SSH"

# Inspect pending compilation without applying writes.
python3 scripts/compile.py --dry-run

# Structural checks and local diagnostics; no live provider probe.
python3 scripts/lint.py --structural-only
python3 scripts/manage.py doctor

These are commands for my compiler repository, not built-in Obsidian commands. The downloadable example above contains only the small guards, not the complete compiler or its runtime data.

The useful output is a note whose path, origin and write history can be inspected. I still review the resulting knowledge: a source-linked model proposal is easier to challenge, but a citation alone cannot make it correct.