Nestor G Pestelos Jr · Writing · Print

Build the Source of Truth Before the Index

Published August 18, 2026. Revised September 4, 2026.

TL;DR

The default move for AI memory is a vector database first. Below context-window scale, that is backwards. Build a human-readable, editable source of truth, then layer search, graph structure, and agent access on top as rebuildable derivations.

The August 18, 2026 version is at build-source-of-truth-before-index-20260818. The September 3, 2026 version is at build-source-of-truth-before-index-20260903.

In 1970, E. F. Codd wrote the paper that founded the relational database. Section 1.2.2 establishes that an index is a redundant component of the data representation. An index is not central to the system and not authoritative. It is a performance artifact that the system can create and destroy without breaking anything that depends on it.

Three traditions, one architecture

The Pragmatic Programmer's Tip 25 argues from software craft: keep knowledge in plain text because plain text never becomes obsolete.

Doug McIlroy's Unix philosophy: write programs to handle text streams because text is a universal interface.

The build order for 2026

shyam's lesson from building long-term memory for an AI agent: start with a source of truth humans can read and edit. Next add search. Then add graph structure. Finally give agents access. A markdown wiki remains correctable. A vector database seeded with arbitrary chunks remains an un-auditable black box.

Boris Cherny, creator of Claude Code, makes the search half of the argument (relayed by Aakash Gupta). Simple Unix search (grep, find, ripgrep) beats RAG pipelines because search inspects the source of truth directly.

Andrej Karpathy runs the identical build order for his knowledge base: raw documents compiled into a markdown wiki, an LLM as maintainer, and zero RAG at roughly 100 articles and 400,000 words.

Two companies tried it backwards

Sourcegraph shipped Cody on embeddings, then deprecated them in February 2024. Code left local perimeters for third-party embedding. The index demanded constant upkeep past 100,000 repositories. Sourcegraph replaced embeddings with BM25 search across raw code.

Pinecone's metadata limits forced two-step retrieval at Confident AI. Their separated vector store lacked synchronization with the primary data source. They folded vectors back into PostgreSQL. Both accounts are vendor-reported and qualitative, without independent replication.

Where the derivation earns its keep

Cursor recorded semantic search over its embeddings index at 12.5% higher question-answering accuracy than grep alone, with code retention in A/B tests rising 2.6% on repositories over 1,000 files. These numbers are Cursor's internal benchmarks without independent confirmation.

Cursor's index is a Merkle tree synchronized with the local working copy. Chunks are cached by hash and evicted immediately when a file changes on disk.

A 2025 study found long-context models beat chunk-based RAG on accuracy until the corpus outgrows the window. Aggregations like "find the top five themes across all notes" need a derived knowledge graph. BM25 catches exact identifiers; vector search catches paraphrases. Avi Chawla notes robust retrieval combines both.

Before reaching for a vector database

  1. Does a human-readable, editable version of this knowledge exist? If not, that is the missing substrate.
  2. Does the corpus fit inside the model's context window? If so, retrieval is premature.
  3. Do questions target single documents or corpus-wide aggregation? Lookups need search; aggregation needs a graph.
  4. Can every index be regenerated from raw files with zero data loss? If not, you built an accidental second source of truth.

In my own vault, every note is plain markdown, tracked by git, editable by hand. Full-text and vector indexes run on top, rebuilt from source files whenever they drift. This is not a speculative architecture. It is running right now under the text you are reading.

Sources

Back to top