Enterprise RAG architecture that respects permissions and can prove its answers
We design enterprise RAG architecture that grounds AI answers in your own content: permission-aware retrieval, cited sources, fresh indexes and measured answer quality, built alongside your engineers so the system is yours.
Platform architecture
Why RAG demos work and enterprise RAG stalls
A retrieval-augmented generation demo takes an afternoon: chunk some PDFs, embed them, put a chat box on top. Enterprise RAG architecture is a different problem. The content lives in SharePoint, Confluence, file shares, ticketing systems and databases, each with its own permissions, formats and owners, and most of it was never written to be retrieved by a machine.
The failures are predictable. A user sees a paragraph from a document they were never allowed to open. An answer cites last year's policy because the index was loaded once and never refreshed. Tables and scanned forms turn into noise during parsing. Nobody can say whether answer quality improved or regressed after the last change, because there is no evaluation set.
Security and legal teams then stop the rollout, and they are right to. The fix is not a better prompt or a bigger model. It is an architecture that treats retrieval as a governed data product: identity-aware, observable, refreshed on a schedule, and tested against questions your people actually ask.
What you get
Knowledge source inventory
A map of the repositories in scope, their owners, permission models, formats, volumes and change rates, with a ranking of which sources carry the most value and the least risk for the first production release.
RAG reference architecture
The target design for ingestion, parsing, chunking, embedding, indexing, retrieval, re-ranking and generation, with decision records for vector store, hybrid search, embedding model and hosting choices, written so your architects can defend each one.
Permission-aware retrieval design
How source access controls are captured at ingestion and enforced at query time, so every result is filtered by the caller's identity and group membership. Includes handling for revoked access, shared links and sensitive labels.
Evaluation harness and golden set
A curated set of real questions with expected sources and answers, plus automated scoring for retrieval recall, groundedness and citation accuracy. It runs on every change so quality becomes a number, not an opinion.
Production ingestion pipelines
Built with your engineers: connectors, incremental sync, document parsing for tables and scans, metadata enrichment and deletion propagation, with monitoring that shows index freshness and failed documents per source.
The target architecture for enterprise retrieval
A production RAG system is a data pipeline, a search system and a model application at once. These are the components we design and build, independent of which cloud or model provider you choose.
- Source connectors and incremental sync
- Connectors that pull content, metadata and access control lists from each repository, detect changes and deletions, and keep a record of what was indexed, when and from which version.
- Parsing and chunking layer
- Format-specific parsing for PDFs, Office files, HTML, tables and scanned images, with chunking that follows document structure such as headings and sections rather than fixed character counts.
- Hybrid index
- Vector and keyword search together, in a store such as pgvector, OpenSearch, Azure AI Search or Pinecone, with metadata fields for source, date, classification and permissions to support filtering.
- Policy-aware retrieval service
- A single retrieval API that resolves the caller's identity, applies permission and classification filters before ranking, re-ranks candidates and returns passages with source links for citation.
- Generation and citation layer
- Prompt templates, model routing through your LLM gateway, answer grounding instructions and citation formatting, with a clear fallback when retrieved evidence is too weak to answer.
- Evaluation and observability
- Offline evaluation against the golden set, online feedback capture, and traces that link each answer to the query, retrieved chunks, model and prompt version for debugging and audit.
How we deliver it
Assess
We review the target use cases, the content behind them and how it is secured today. We test a sample of real documents through parsing and retrieval to find the quality and permission problems before anyone commits to a design.
Blueprint
We define the enterprise RAG architecture: sources, pipelines, index design, identity propagation, model routing and evaluation. Each major choice gets a decision record with the options considered, trade-offs and the conditions that would change it.
Build
Our architects work inside your team to ship the first production knowledge assistant or retrieval API, including ingestion, access filtering, citations, the evaluation harness and dashboards, then extend the pattern to the next sources.
Govern
We set up content ownership, refresh service levels, evaluation gates in release pipelines and a review process for new sources, so adding a repository is a routine change with known controls rather than a new project.
What changes
- Answers cite the specific source passages they rely on, so users and reviewers can check them.
- Users only retrieve content they are already entitled to see in the source system.
- Answer quality is measured on every release, and regressions are caught before users find them.
- Your team owns the pipelines, index and evaluation harness, and can add new sources without us.
Related services
- Generative AI Platform ArchitectureLLM gateway, model routing, evaluations and cost controls that let teams use multiple models safely, observably and within budget.
- Data Architecture for AIData foundations for AI workloads: unstructured data pipelines, semantic layers, lineage, quality and access control for models and agents.
- AI Security ArchitectureSecurity for LLM apps and agents: threat modeling, prompt injection and data leakage controls, agent identity and red teaming.
RAG and Knowledge Architecture: common questions
What is enterprise RAG architecture?
Enterprise RAG architecture is the design of systems that retrieve relevant passages from an organization's own content and give them to a language model to ground its answer. At enterprise scale it covers connectors, parsing, chunking, hybrid search, permission filtering, citations, freshness and evaluation, not just an embedding model and a vector database.
How do you keep RAG from exposing documents users should not see?
Capture each document's access control list at ingestion and filter results by the caller's identity at query time, before anything reaches the model. Group membership is resolved from your identity provider, deletions and permission changes propagate on a schedule, and sensitive labels can exclude content entirely. Relying on the prompt to hide content is not a control.
Do we need a dedicated vector database for RAG?
Not always. Many enterprises start with vector search in a system they already run, such as PostgreSQL with pgvector, OpenSearch or their cloud's search service. A dedicated vector database makes sense at high scale or with demanding latency and filtering needs. Hybrid keyword and vector search usually matters more for answer quality than the choice of store.
How do you measure whether a RAG system is accurate?
Build a golden set of real questions with known correct sources and answers, then score retrieval recall, groundedness and citation accuracy automatically. Run it on every change to prompts, chunking, models or indexes. Add user feedback and sampled human review in production. Without a golden set, teams cannot tell whether a change helped or hurt.
When should we use RAG instead of fine-tuning?
Use RAG when answers depend on facts that change, must be cited, or must respect per-user permissions, which covers most enterprise knowledge use cases. Fine-tuning suits teaching a model a format, style or narrow task. The two combine well: a tuned model can still retrieve current, permissioned content at query time.
Bring us the architecture question you are stuck on.
Kelsey, our AI assistant, can answer questions and book you a 30-minute briefing with a senior architect in about two minutes, by voice or by typing.
- Vendor-neutral advice. We resell nothing.
- Senior architects on the call, not a sales team.
- Mutual NDA available before we talk specifics.
Book with Kelsey
She finds a time that works, books it on our calendar and sends the invite. No forms.
By talking or chatting with Kelsey, you agree we may text and email you about your request. Message and data rates may apply. Reply STOP to opt out. Terms and Privacy.
Prefer a person? Call (301) 301-5765.