Nestor G Pestelos Jr · Writing · Print

Build the Source of Truth Before the Index

Published August 18, 2026. This is the original version published on that date. The current version is at build-source-of-truth-before-index.

TL;DR

The default move for AI memory is a vector database first. Below context-window scale, that is backwards: build a human-readable, editable source of truth, then layer search, graph structure, and agent access on top as rebuildable derivations.

In 1970, E. F. Codd wrote the paper that founded the relational database. Section 1.2.2 states: "From an informational standpoint, an index is a redundant component of the data representation." An index isn't central to the system, and it isn't authoritative. It's a performance artifact, something the system creates and destroys without breaking anything that depends on it.

Fifty-six years later, the default move for building AI memory is to skip straight to the redundant component. Chunk everything, embed it, query by similarity, and never build the thing the index was supposed to index.

Three traditions, one architecture

The Pragmatic Programmer's Tip 25 makes the case from software craft: "Keep Knowledge in Plain Text... Plain text won't become obsolete." Plain text stays parseable by tools nobody has invented yet.

Doug McIlroy's three-sentence Unix philosophy ends on a parallel line: "Write programs to handle text streams, because that is a universal interface." A binary format couples every reader to the program that wrote it. Text carries no such coupling. It can be inspected, filtered, diffed, and repaired by hand when the tooling around it fails.

Database theory, software craft, and Unix design point to one architecture: a durable, human-readable, editable substrate, with everything else built on top as a derivation you can throw away and rebuild.

The build order, restated for 2026

shyam's lesson from building long-term memory for an AI agent states the 2026 version in one sentence: "Don't start with a vector database. Start with a source of truth humans can read and edit. Then add search. Then add graph structure. Then give agents access to it." A markdown wiki someone can read and edit stays correctable. A vector database seeded with random chunks is a black box nobody can audit.

Boris Cherny, who built Claude Code, makes the search half of the same argument from inside a production coding agent, in an account relayed by Aakash Gupta. Simple Unix search, grep, find, ripgrep, beats RAG pipelines for code understanding, because search runs on the source of truth instead of a secondary store that has to stay synchronized with it.

Andrej Karpathy runs the identical build order for his own knowledge base: raw documents compiled into a markdown wiki, an LLM as maintainer, and no RAG at roughly 100 articles and 400,000 words, because the model's context window already holds it all.

Two companies tried it backwards

Sourcegraph shipped Cody on embeddings, then deprecated them in February 2024. Code had to leave the building for a third party to embed it. The index needed constant upkeep. It grew complex and resource-intensive past 100,000 repositories.

The replacement is search over the code itself, ranked by BM25, because the source of truth had been enough all along and the index was the fragile extra piece.

Confident AI hit a different wall. Pinecone's metadata cap forced a two-step retrieval, and the separated vector store, in their own words, "lacks mechanisms to ensure data synchronization with the primary datasource." They folded the vectors back into their existing Postgres database, removing the desync risk by construction. Both accounts are vendor-run and qualitative, with no independent numbers or replication published.

The steelman test

The steelman has real numbers behind it. Cursor measured semantic search over its own embeddings index at 12.5% higher question-answering accuracy than grep alone, and code retention in A/B tests rose too, up to 2.6% on codebases over 1,000 files. The numbers are Cursor's own, on Cursor's own benchmark, with no independent replication found.

That index is a Merkle tree kept in sync with the local working copy, chunks cached by hash, a result dropped the moment the client can't prove the file is still on disk. None of that is a second, independent store. It's the same disciplined derivation shyam described, paid for. Sourcegraph and Cursor ran into the same fork; only one of them kept funding the sync work.

The thesis has limits too. A 2025 comparison of long-context models against chunk-based retrieval found long-context models generally beat chunk-based RAG on accuracy. That result should flip once the corpus outgrows the context window: past that size, cost scales directly with input size, and an index stops being optional.

Whole-corpus questions are the other edge. Something like "what are the five biggest themes across everything I've written" needs a derived knowledge graph, because no single chunk can answer that kind of question on its own.

Plain search has its own gap. BM25 catches the exact term a vector search can blur past; a vector search catches the paraphrase BM25 misses. Avi Chawla's read is that the best search layer runs both and combines the ranks. Add the index and the graph once the questions demand them. Until then, leave them out.

Before reaching for a vector database

  1. Does a human-readable, editable version of this knowledge exist anywhere? If not, that is the missing layer.
  2. Does the corpus fit inside a model's context window? If yes, retrieval may not be needed yet.
  3. Are the questions single-document lookups, or whole-corpus aggregation? Lookups want search. Aggregation wants a graph.
  4. When an index gets added, can it be rebuilt from the source of truth alone, with nothing lost? If not, a second source of truth just got built by accident.

I made this bet in my own notes before any of the X threads above existed. Every file is plain markdown, readable by any editor, diffable by git, correctable by hand. Search runs on top of that, full-text and vector retrieval alike, and gets rebuilt from the files whenever it drifts. Nothing about that is a plan for some future architecture. It is what is running right now, underneath the sentence you are reading.

Sources

Back to top