---
title: Growth path
description: "Four stages, each with the trigger that unlocks it. Stage 1 is the only one to build now."
---

# Growth path

Build stage 1. Do not build stage 2 until its trigger fires. Each trigger is an
observation, not a hunch.

```text
  STAGE 1 — NOW                                        under $0.01/mo
  ┌──────────────────────────────────────────────────────────────┐
  │ Private S3 bucket, versioning, SSE, one IAM user.            │
  │ Markdown + YAML frontmatter. Scheduled aws s3 sync into a    │
  │ local mirror. Agents use native Read / Grep / Glob.          │
  │ Nothing custom.                                              │
  └──────────────────────────────┬───────────────────────────────┘
         TRIGGER: a query returns more hits than an agent can
         usefully scan, or the file count passes ~2,000, or the
         listing alone starts eating context.
                                 ▼
  STAGE 2 — INDEX                                                $0
  ┌──────────────────────────────────────────────────────────────┐
  │ A build script walks the local files and writes              │
  │ transcripts.db — SQLite, FTS5, BM25 ranking, frontmatter as  │
  │ columns. Derived and disposable; rebuilt from S3 in seconds. │
  │ Files stay the source of truth. Agent gets one more tool.    │
  └──────────────────────────────┬───────────────────────────────┘
         TRIGGER: three logged recall misses that survived
         multi-query grep attempts.
                                 ▼
  STAGE 3 — SEMANTIC                                    pennies/mo
  ┌──────────────────────────────────────────────────────────────┐
  │ Chunk + embed. Vectors in sqlite-vec beside the FTS5 index,  │
  │ or S3 Vectors if they must be shared. Hybrid BM25 + vector,  │
  │ both feeding one agent tool.                                 │
  └──────────────────────────────┬───────────────────────────────┘
         TRIGGER: a NON-agent consumer — a client-facing app, a
         dashboard, concurrent humans needing sub-second queries.
                                 ▼
  STAGE 4 — SERVER                                       $0–$25/mo
  ┌──────────────────────────────────────────────────────────────┐
  │ Postgres. Loaded from files. Files still source of truth.    │
  └──────────────────────────────────────────────────────────────┘
```

## Reading the triggers

**Stage 2 is a ranking problem, not a speed problem.** Grep over this corpus is
instant and stays instant. What breaks is that grep returns hits in file order
with no relevance signal, so 200 hits give an agent no basis to pick. FTS5's
BM25 fixes that for nothing.

**Stage 3 is a recall failure, not a size.** Someone asks about a moment using
words the transcript does not contain. Grep cannot find it. But a competent
agent generates several query variants before giving up, and that recovers most
vocabulary misses. So the trigger is *repeated* misses that survive multiple
queries. **Log them. Three real ones justify embeddings.** Embedding cost is a
rounding error and must not drive the decision — the index infrastructure is
the cost.

**Stage 4 is never an agent's request.** An agent asking for Postgres is an
agent that would rather have files.

## Do not build

Iceberg or S3 Tables — cost inversion on small objects, and they buy ACID,
schema evolution and time travel that append-only blobs never use. OpenSearch,
any generation. Kendra. DynamoDB — no full-text search and you would build
search on top anyway. A managed knowledge base. A chunking pipeline. A vector
index before a recall miss is observed.

## What grep genuinely cannot do

1. **Vocabulary mismatch.** The word is not in the document.
2. **Ranking under many hits.** No relevance order.
3. **Synthesis across more text than fits in context.**
4. **Predicates on facts that live only in the prose** — "meetings where the
   deal size exceeded a threshold" is not greppable.

Point 4 argues for **extraction**, not storage: an LLM pass writing structured
fields into frontmatter. Do the extraction into the files. The database stays
optional after that.