Multi-Tenant RAG

The Multi-Tenant RAG & Agentic Operations Assistant is a full-stack system I built to explore what it takes to move retrieval-augmented generation beyond a single-user demo. It gives organizations a private knowledge workspace where users can ingest operational documents, ask grounded questions, inspect the evidence behind an answer, and turn selected requests into tenant-scoped tasks or incident summaries.

The key engineering problem was not simply connecting an LLM to a vector database. It was making identity, retrieval, tools, conversation state, and observability agree on the same tenant boundary while keeping the agent useful when a model provider or retrieval stage is imperfect.

The product problem

An internal knowledge assistant has to solve several problems at once. Documents must be private to the organization that supplied them. Retrieval must be reproducible and filterable. Answers need source attribution. Actions must operate on the correct organization's records. Users need continuity across conversations, while administrators need enough metadata to understand latency, retrieval quality, model usage, and failure.

Many RAG prototypes handle only the happy path: embed a file, run similarity search, send the top chunks to a model. I built this project as the larger operational system around that loop—authentication, ingestion state, retrieval strategies, guardrails, business tools, streaming, persistence, evaluation, and a usable frontend.

What I built

Organization and identity boundaries

The relational control plane models organizations as tenants with their own users, provider settings, document quotas, query limits, conversations, tasks, and incidents. Users authenticate with JWTs that carry tenant identity, while FastAPI dependencies resolve and validate the current user and tenant before protected routes execute.

Tenant ownership is repeated at each meaningful storage boundary rather than inferred from the UI. SQLAlchemy queries include the tenant key, document chunks persist it alongside the parent document, Redis retrieval keys include it, and Qdrant payload filters require it for search and deletion.

This excerpt from the vector layer captures that non-negotiable invariant:

conditions = [
    FieldCondition(
        key="tenant_id",
        match=MatchValue(value=tenant_id),
    )
]
 
for key, value in (filter_conditions or {}).items():
    conditions.append(
        FieldCondition(key=key, match=MatchValue(value=value))
    )
 
search_filter = Filter(must=conditions)

The same constraint protects document deletion and whole-tenant vector cleanup. It is intentionally impossible for an optional metadata filter to replace the mandatory tenant filter.

Document ingestion and lifecycle management

Users can upload PDF, DOCX, and text documents through the product. The backend stores the file and metadata, tracks uploaded, processing, completed, and failed states, extracts text using format-specific readers, chunks it with configurable overlap, embeds each passage, and writes both relational chunk records and Qdrant points.

The management surface supports document listing, tags and metadata, pagination, deletion, and targeted batch reprocessing. Reprocessing preserves the order of requested IDs, reports missing records, and selects candidates only from the active tenant. UTC normalization and timestamps make date filters consistent between PostgreSQL and vector metadata.

Tenant-filtered retrieval

The retrieval service composes four independently replaceable stages:

• A provider-aware embedding service, with MiniLM as the local default
• Qdrant similarity search with tenant, document, tag, type, and date/range filters
• Redis caching with tenant identity and search parameters encoded into the cache key
• An optional reranker for cross-encoder experiments

Search retrieves one additional point to calculate cursor-like pagination without returning vectors to the application. Results are normalized into a provider-independent shape containing source, document, chunk, page, score, and safe metadata.

Intent-aware RAG strategies

The agent classifies each request as informational, analytical, action-oriented, or in need of clarification. Classification uses a structured LLM response with a rule-based fallback so a provider failure does not eliminate routing altogether.

For research questions, users can choose between two strategies:

• Direct decomposes the question and searches the original query plus up to four focused subqueries.
• Informed first retrieves initial context, generates a short evidence summary, then uses that evidence to create better subqueries. It falls back to direct retrieval when the first pass finds nothing.

Contexts are deduplicated by document and chunk, with the strongest score retained. The final prompt is citation-focused, receives only the trimmed evidence set, and returns explicit source information to the interface. When no passage meets the threshold, the system says so rather than generating an unsupported answer.

Streamed, inspectable execution

The FastAPI streaming endpoint emits structured Server-Sent Events for processing status, classified intent, retrieved contexts, action results, final answer, guardrail report, and completion. The Next.js conversation experience consumes those events progressively, allowing a user to see what the system retrieved and whether a tool ran without waiting for one opaque response payload.

Every completed turn persists more than chat text. Query records capture tenant, user, strategy, subqueries, processing time, retrieval threshold, provider, model, context count, source attribution, guardrail state, and conversation turn. Conversation messages preserve the evidence and action metadata needed to reconstruct how the answer was produced.

Bounded operational tools

Action requests do not translate into arbitrary function calls. The planner maps to a small explicit registry:

• Create a task with validated title, priority, tags, description, and optional due date
• List the tenant's open tasks
• Summarize recent incidents over a bounded timeframe

Unsupported tool names return a structured unsupported result. Every service method receives the active tenant ID, and the streamed guardrail report surfaces failed or unsupported tool outcomes. The tasks and incidents also have direct tenant-scoped REST APIs for normal product workflows.

System architecture

The application uses Python 3.11+, FastAPI, Pydantic, SQLAlchemy, PostgreSQL, Redis, Qdrant, sentence-transformers, OpenAI and Anthropic adapters, plus a TypeScript frontend built with Next.js 14, React, SWR, Zod, Tailwind, and Radix UI.

A replaceable provider layer

The LLM abstraction supports OpenAI and Anthropic behind the same text and RAG response contracts. Embeddings can run locally with all-MiniLM-L6-v2 or use a remote provider. Vector, embedding, cache, reranking, conversation, and operational services are injected independently, which keeps tests from requiring a live model or vector database for every code path.

A complete local topology

Docker Compose provisions PostgreSQL 16, Redis 7, Qdrant, the FastAPI application, and the Next.js frontend with health-aware dependencies and persistent volumes. The backend and frontend Dockerfiles mirror the package manifests, while the Makefile collects installation, infrastructure, seeding, linting, formatting, testing, and development commands.

Three sample organizations—healthcare, security, and manufacturing scenarios—provide isolated documents, tasks, incidents, and evaluation questions. They are deliberately synthetic and make it possible to exercise cross-domain retrieval without placing real operational data in the repository.

Product experience

The frontend includes organization login and signup, tenant switching, a conversation list, a streaming conversation detail view, document management, task views, and an administrative surface. SWR hooks own cache-aware data access; Zod and TypeScript mirror backend contracts; reusable Radix-based controls provide accessible interaction foundations.

The conversation view exposes strategy selection, source snippets, subquery traces, retrieval scores, guardrail warnings, and tool outcomes. That visibility is important: an assistant used for operations should show the evidence and action boundary, not ask users to trust an animated typing indicator.

Evaluation, testing, and current limits

The backend contains 22 test functions across authentication, API smoke coverage, conversation persistence and title generation, document extraction and chunking, batch reprocessing, tenant isolation, and seed-data management. Test doubles replace embeddings and Qdrant where appropriate, while service tests use a lightweight relational fixture.

An evaluation script runs a labelled query set against the retrieval stack and records quality-oriented output for experimentation. Optional reranking, similarity thresholds, strategy selection, and retrieval traces make relevance changes measurable rather than purely subjective.

I treat the following as explicit next steps rather than finished claims:

• Move heavy ingestion and evaluation from request-adjacent execution into durable background jobs
• Add dedicated frontend unit and browser suites beyond the implemented backend coverage
• Strengthen action confirmation and policy controls before broadening the tool registry
• Add production-grade migrations and deployment observability around the local-first topology

What this project demonstrates

This project demonstrates my ability to design an AI feature as a multi-tenant product system. The retrieval algorithm matters, but so do the identity model, storage boundaries, failure modes, action constraints, UI transparency, and operational record left after an answer.

It required work across Python services, typed APIs, relational and vector data design, document processing, authentication, LLM orchestration, streaming protocols, React product development, containerized infrastructure, and test design. Most importantly, it shows how I turn a probabilistic model interaction into a bounded, inspectable workflow.

If you are building a private knowledge platform, an operations copilot, or a multi-tenant AI product, review the repository or reach out by email.