Reference architecture: retrieval-augmented generation (RAG)

RAG grounds a language model in an organisation's own content. It is two systems: an ingestion pipeline that keeps an index fresh, and a query path that has to be fast, safe and observable.

Unstructured sourcesDocuments, PDFs, wiki and web pages, images, audio, video

Unstructured data pathChunked, embedded and indexed

1. Parse & extractOCR, transcription, dedup, quality checks
2. ClassifyType, topic, sensitivity, PII redaction
3. Chunk & enrichStructure-aware chunking, summaries
4. Metadata & accessProvenance, filters, ACLs per chunk
5. EmbeddingsChunks to vectorsversioned model
6. Search indexVector + keyword

Supports semantic and hybrid retrieval

→RetrievalRouting, hybrid search, re-ranking, generation

Structured sourcesDatabases, tables, data warehouse, data catalogue

Structured data pathQueried in place, under governance

1. Schema & profilingTables, fields, relationships, quality checks
2. Classify & describeDomain, sensitivity, ownership, business descriptions
3. Governed query layerSQL access, views, row and column policies
4. Structured accessExact lookups, aggregations, analytical queries

No row-level embeddings; schema descriptions indexed for routing

Shared ingestion controls

• Orchestration
• Validation
• Incremental sync and schedules
• Lineage and provenance
• Access control
• Lifecycle, re-indexing and deletion

Online request flowFast, safe and observable

1. User or appQuestion or task
2. API gatewayAuthentication, quotas, rate limits
3. Input guardrailsPrompt injection, PII, policy checks
4. RouterUnstructured, structured or both
5. RetrieveHybrid search and governed SQL, filtered by the caller's permissions
6. Re-rankMost relevant context first
7. GenerateGrounded answer with citations and output checks

Permissions enforced at retrieval: the model never sees content the user cannot

Observability and evaluation

• End-to-end tracing
• Retrieval quality
• Answer faithfulness
• Latency per stage
• Cost per request
• User feedback loop

Key design decisions

• Chunking: split on document structure, not fixed sizes, and keep source and section metadata for citations and access control.
• Hybrid retrieval: combine vector and keyword search, then re-rank. Pure vector search misses exact terms such as product codes.
• Permissions: filter by the caller's entitlements at retrieval time, so the model never sees content the user cannot.
• Versioning: treat the embedding model and index as a versioned pair. Changing one means re-indexing.
• Evaluation: keep a golden question set and measure retrieval quality and answer faithfulness on every change.