Cerebras Knowledge, their internal knowledge base, serves 15,000+ questions per day from humans, automations, and agents after just 3 months. The architecture is a masterclass in practical enterprise RAG: meet data where it lives, unify via a single schema, and let the orchestrator (human UI or MCP client) handle the intelligence.
A single Postgres table, many sources
The core is deliberately boring. One Postgres table holds embeddings, raw summaries, and typed metadata from every source - Slack threads, wiki pages, code repositories, netlists, and custom databases. Every row follows the same schema regardless of origin. New connectors are just Python plugin scripts that teams submit as PRs, emitting rows shaped like the embeddings table.
This means the query surface is identical for every source. No special casing, no separate indexes per platform.
Hybrid search for unstructured Slack
Slack was the hardest source and the most important - it’s where real-time engineering discussions happen. Raw vector search failed because:
- Short filler messages (“sounds good, thanks!”) rank high in cosine similarity
- Exact error strings need lexical matching, not semantic
- Thread meaning depends on the full conversation, not isolated messages
The fix is a four-signal hybrid that fuses at query time via Reciprocal Rank Fusion (weight / (60 + rank), default weight 60, smoothing 60):
- Full-text search (Postgres GIN index) - catches literal error tokens, flag names, host names
- Embedding search - catches paraphrase across different vocabulary
- IDF - rare tokens beat filler; “sounds good” scores near zero once term rarity is considered
- Age decay - newer threads win ties; 8-month-old answers about infrastructure that no longer exists get penalized
No single signal is trusted alone. RRF makes consensus across retrievers matter more than any one strong vote.
Slack ingestion: Socket Mode + LLM distillation
A Slack bot in Socket Mode receives every message event over a persistent WebSocket (no polling, no rate-limit burn). The ingest consumer doesn’t save individual messages - it resolves the thread, re-fetches the full conversation, and writes the entire thread as one row. A reply re-pulls the parent and all siblings.
Distillation: An LLM extracts structured data from each thread:
- A one-line searchable question
- Summary and resolution
- Systems and code references
The original transcript is not embedded directly. Normalizing into a consistent format significantly improved accuracy.
Bursting: Important messages inside long threads were still missed. Bursting solves this: consecutive messages from the same author are grouped, prepended with thread topic as context, scored against a threshold (IDF >= 4.0, >= 200 chars, has reactions), and embedded separately. This makes the one tangent message that contains the actual answer findable on its own.
Code embeddings via CocoIndex
Code repositories (some 40GB+) are vectorized using CocoIndex, an open-source embedding framework. It uses language-specific regex boundaries ordered coarse-to-fine (class -> method -> block). A single file may generate multiple embeddings at different specificity levels.
CocoIndex tracks sync metadata in Postgres and re-embeds only changed chunks per commit. Repository onboarding moved to team-submitted config files with allowlist/denylist at the file-path level.
Query pipeline: planner > executor > synthesizer
Every query runs through three stages:
- Planner - a lightweight LLM pass inspects the query and active project, then chooses which tools matter (code_search, vector_search, Slack_search, ripgrep, PR_search, expert_search)
- Executor - fans out tool calls in parallel, normalizes results into a shared evidence schema with scores, recency, and source hints
- Synthesizer - final LLM pass produces the answer with citations, caveats, and cross-source synthesis
Reranking applies after the executor: RRF fusion, dedup at source level, cap per-file results, then a small reranker model scores each candidate 0-10, keeping the top 10. Winners get neighboring context injected back (e.g., wiki sections split by chunking are reunified).
MCP integration: simple primitives, no orchestration inside
MCP exposes each retrieval primitive as a direct tool (code_search, vector_search) - intentionally simple, LLM-free, with narrow structured inputs/outputs. The client (Claude Code or any MCP-compatible agent) owns orchestration: it decides which tools to call, in what order, and how to assemble results.
The retrieval layer does not depend on LLM decisions to serve requests. This is the right architectural boundary: retrieval is a fast, cheap service; intelligence lives in the caller.
Projects for scoped search
“Search everything everywhere” broke at Cerebras scale. Projects bundle data sources (specific Slack channels, repos, databases) per team/initiative. Sources can belong to multiple projects. New engineers pick a default project during onboarding and get high-signal answers without learning the org chart first.
Lessons worth keeping
- One table, any source - a single Postgres embeddings table with a common schema handles Slack threads, code, databases, netlists. Simplicity beats specialisation.
- Embed the distillation, not the raw text - normalising threads into structured question/summary/resolution via an LLM outperforms embedding raw conversations.
- Hybrid search > pure vector - full-text + vector + IDF + age decay fused via RRF beats any single signal. No single scorer is trusted alone.
- MCP tools should be dumb - expose fast, LLM-free retrieval primitives. Let the client orchestrate. Don’t bake intelligence into the retrieval layer.
- Projects scale relevance - bundle sources per team. Default projects during onboarding eliminate the learn-the-org-chart tax.
- Bursting recovers lost signal - individual messages from long threads, grouped by author and scored against thresholds, catch answers that thread-level summaries miss.
Related
- Retrieval-Augmented Generation (RAG) - the general RAG pipeline that this knowledge base is a production instance of
- L8 Principal’s Agentic Engineering Setup - another perspective on MCP tools as simple primitives that an agent orchestrates externally