Nestor G Pestelos Jr · Writing · Print

Build the Source of Truth Before the Index

Published August 18, 2026. Revised September 3, 2026. This is the September 3, 2026 version. The current version is at build-source-of-truth-before-index.

TL;DR

The default move for AI memory is a vector database first. Below context-window scale, that is backwards. Build a human-readable, editable source of truth, then layer search, graph structure, and agent access on top as rebuildable derivations.

The August 18, 2026 version is at build-source-of-truth-before-index-20260818.

In 1970, E. F. Codd wrote the paper that founded the relational database. Section 1.2.2 establishes that an index is a redundant component of the data representation. An index is not central to the system and not authoritative. It is a performance artifact that the system can create and destroy without breaking anything that depends on it.

Fifty-six years later, the default move for building AI memory is to skip straight to the redundant component: chunk everything, embed it, query by similarity, and never build the substrate the index was supposed to index.

Three traditions, one architecture

The Pragmatic Programmer's Tip 25 argues from software craft: keep knowledge in plain text because plain text never becomes obsolete. It remains parseable by tools nobody has invented yet.

Doug McIlroy's Unix philosophy ends on a parallel principle: write programs to handle text streams because text is a universal interface. Binary formats couple readers to the program that wrote them. Plain text carries no such coupling. It can be inspected, filtered, diffed, and repaired by hand when surrounding tooling fails.

Database theory, software craft, and Unix design converge on one architecture: a durable, human-readable substrate, with every index built on top as a derivation you can discard and rebuild.

The build order for 2026

shyam's lesson from building long-term memory for an AI agent restates the sequence: start with a source of truth humans can read and edit. Next add search. Then add graph structure. Finally give agents access. A markdown wiki remains correctable. A vector database seeded with arbitrary chunks remains an un-auditable black box.

Boris Cherny, creator of Claude Code, makes the search half of the argument from inside a production coding agent (relayed by Aakash Gupta). Simple Unix search—grep, find, ripgrep—beats RAG pipelines for code comprehension because search inspects the source of truth directly instead of an out-of-sync secondary store.

Andrej Karpathy runs the identical build order for his knowledge base: raw documents compiled into a markdown wiki, an LLM as maintainer, and zero RAG at roughly 100 articles and 400,000 words. The model's context window already holds the entire corpus.

Two companies tried it backwards

Sourcegraph shipped Cody on embeddings, then deprecated them in February 2024. Code left local perimeters for third-party embedding, the index demanded constant upkeep, and operational complexity grew unsustainable past 100,000 repositories. Sourcegraph replaced embeddings with BM25 search across raw code. The source of truth was sufficient all along; the index was the fragile dependency.

Confident AI encountered a separate wall. Pinecone's metadata limits forced two-step retrieval, and their separated vector store lacked synchronization mechanisms with the primary data source. They folded vectors back into PostgreSQL, eliminating desync risk by construction. Both accounts are vendor-reported and qualitative, without independent replication.

Where the derivation earns its keep

The counter-case has measured evidence. Cursor recorded semantic search over its embeddings index at 12.5% higher question-answering accuracy than grep alone, with code retention in A/B tests rising 2.6% on repositories over 1,000 files. These numbers are Cursor's internal benchmarks without independent confirmation.

Cursor's index is a Merkle tree synchronized with the local working copy. Chunks are cached by hash and evicted immediately when a file changes on disk. This is not an independent data store. It is the disciplined derivation shyam described, fully funded. Sourcegraph and Cursor met the same fork; only Cursor maintained the synchronization engine.

The architectural boundary is explicit:

  1. Context-window capacity: A 2025 study on long-context models versus chunk retrieval found long-context models beat chunk-based RAG on accuracy. That advantage flips once a corpus outgrows the window, where input costs scale linearly and indexing becomes mandatory.
  2. Whole-corpus synthesis: Aggregations like "find the top five themes across all notes" require a derived knowledge graph, because single chunks cannot answer global questions.
  3. Lexical versus semantic recall: BM25 catches exact identifiers that embeddings blur, while vector search captures paraphrases BM25 misses. Avi Chawla notes that robust retrieval combines both.

Add the index and the graph only when query requirements demand them. Until then, leave them out.

Before reaching for a vector database

  1. Does a human-readable, editable version of this knowledge exist? If not, that is the missing substrate.
  2. Does the corpus fit inside the model's context window? If so, retrieval is premature.
  3. Do questions target single documents or corpus-wide aggregation? Lookups need search; aggregation needs a graph.
  4. Can every index be regenerated from raw files with zero data loss? If not, you built an accidental second source of truth.

In my own vault, every note is plain markdown: readable by any editor, tracked by git, and editable by hand. Full-text and vector indexes run on top, rebuilt from source files whenever they drift. This is not a speculative architecture. It is running right now under the text you are reading.

Sources

Back to top