Multi-Tenant RAG: SaaS Architecture to Serve 1000 Clients from One System

Building a RAG chatbot for one client is straightforward. Running it for 1000 clients simultaneously on the same infrastructure while guaranteeing total data isolation? That's an entirely different challenge. This guide details the architectures, patterns, and pitfalls of multi-tenant RAG, based on Ailog's experience serving hundreds of clients from shared infrastructure.

TL;DR

• Multi-tenancy = one RAG system serving multiple clients with data isolation
• Three isolation strategies: namespace/partition, dedicated collection, dedicated database
• The critical choice: tradeoff between isolation, cost, and operational complexity
• Qdrant: payload filtering (most common), separate collections, or separate instances
• Cost: from $0.50/tenant/month (shared) to $50/tenant/month (dedicated)
• Security: data leakage between tenants is risk number 1

Why Multi-Tenancy?

Without multi-tenancy, each new client requires dedicated infrastructure. It's an operational and financial nightmare.

Dedicated infrastructure (1 system per client):
┌──────────┐ ┌──────────┐ ┌──────────┐     ┌──────────┐
│ Client A │ │ Client B │ │ Client C │ ... │Client 999│
│ ┌──────┐ │ │ ┌──────┐ │ │ ┌──────┐ │     │ ┌──────┐ │
│ │Vector│ │ │ │Vector│ │ │ │Vector│ │     │ │Vector│ │
│ │  DB  │ │ │ │  DB  │ │ │ │  DB  │ │     │ │  DB  │ │
│ ├──────┤ │ │ ├──────┤ │ │ ├──────┤ │     │ ├──────┤ │
│ │ LLM  │ │ │ │ LLM  │ │ │ │ LLM  │ │     │ │ LLM  │ │
│ └──────┘ │ │ └──────┘ │ │ └──────┘ │     │ └──────┘ │
└──────────┘ └──────────┘ └──────────┘     └──────────┘
     ❌ Doesn't scale. Linear cost.

Multi-tenant (1 system, N clients):
┌─────────────────────────────────────────────────────┐
│                  Shared RAG System                    │
│                                                       │
│  ┌─────────────────────────────────────────────────┐ │
│  │              Vector Database                     │ │
│  │  ┌─────┐ ┌─────┐ ┌─────┐         ┌─────┐      │ │
│  │  │  A  │ │  B  │ │  C  │  ...    │ 999 │      │ │
│  │  └─────┘ └─────┘ └─────┘         └─────┘      │ │
│  └─────────────────────────────────────────────────┘ │
│  ┌─────────────────────────────────────────────────┐ │
│  │              LLM Gateway (shared)                │ │
│  └─────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
     ✅ Scales. Sub-linear cost.

The Three Isolation Strategies

Strategy 1: Namespace / Payload Filtering

All tenant data lives in the same collection, differentiated by a tenant_id field.

DEVELOPERpython
from qdrant_client import QdrantClient
from qdrant_client.models import (
    Distance, VectorParams, PointStruct,
    Filter, FieldCondition, MatchValue
)

client = QdrantClient(url="http://localhost:6333")

# 1. Create shared collection
client.create_collection(
    collection_name="shared_knowledge",
    vectors_config=VectorParams(
        size=1536,
        distance=Distance.COSINE
    )
)

# 2. Index documents with tenant_id
def index_document(tenant_id: str, doc_id: str,
                   embedding: list, content: str, metadata: dict):
    """Index a document with tenant_id as payload."""
    client.upsert(
        collection_name="shared_knowledge",
        points=[
            PointStruct(
                id=doc_id,
                vector=embedding,
                payload={
                    "tenant_id": tenant_id,
                    "content": content,
                    **metadata
                }
            )
        ]
    )

# 3. Search with tenant filter
def search_tenant(tenant_id: str, query_embedding: list,
                  top_k: int = 5) -> list:
    """Search limited to tenant's documents."""
    results = client.search(
        collection_name="shared_knowledge",
        query_vector=query_embedding,
        query_filter=Filter(
            must=[
                FieldCondition(
                    key="tenant_id",
                    match=MatchValue(value=tenant_id)
                )
            ]
        ),
        limit=top_k
    )
    return results

Pros: Simple, economical, no managing multiple collections. Cons: Logical isolation only, degraded performance with very large indexes, data leak risk if filtering bug.

Strategy 2: Dedicated Collection per Tenant

Each tenant gets their own collection within the same vector database instance.

DEVELOPERpython
class CollectionPerTenantStrategy:
    """One Qdrant collection per tenant."""

    def __init__(self, qdrant_url: str):
        self.client = QdrantClient(url=qdrant_url)

    def create_tenant(self, tenant_id: str, vector_size: int = 1536):
        """Create a dedicated collection for a new tenant."""
        collection_name = f"tenant_{tenant_id}"
        self.client.create_collection(
            collection_name=collection_name,
            vectors_config=VectorParams(
                size=vector_size,
                distance=Distance.COSINE
            )
        )
        return collection_name

    def delete_tenant(self, tenant_id: str):
        """Delete all data for a tenant."""
        collection_name = f"tenant_{tenant_id}"
        self.client.delete_collection(collection_name)

    def search(self, tenant_id: str, query_embedding: list,
               top_k: int = 5):
        """Search within a tenant's collection."""
        collection_name = f"tenant_{tenant_id}"
        return self.client.search(
            collection_name=collection_name,
            query_vector=query_embedding,
            limit=top_k
        )

Pros: Good isolation, predictable per-tenant performance, easy data deletion. Cons: More collections to manage, memory overhead, collection count limits.

Strategy 3: Dedicated Database per Tenant

Each tenant gets their own vector database instance. Reserved for enterprise accounts.

DEVELOPERpython
class DedicatedInstanceStrategy:
    """One dedicated Qdrant instance per tenant (or pool)."""

    def __init__(self):
        self.instances = {}

    def provision_tenant(self, tenant_id: str,
                         tier: str = "standard") -> str:
        """Provision a dedicated instance."""
        config = self._get_tier_config(tier)

        instance_url = self._deploy_instance(
            tenant_id=tenant_id,
            cpu=config["cpu"],
            memory=config["memory"],
            storage=config["storage"]
        )

        self.instances[tenant_id] = {
            "url": instance_url,
            "client": QdrantClient(url=instance_url),
            "tier": tier
        }
        return instance_url

    def _get_tier_config(self, tier: str) -> dict:
        """Configuration per service tier."""
        tiers = {
            "starter": {"cpu": "0.5", "memory": "512Mi", "storage": "5Gi"},
            "standard": {"cpu": "1", "memory": "2Gi", "storage": "20Gi"},
            "enterprise": {"cpu": "4", "memory": "8Gi", "storage": "100Gi"},
        }
        return tiers[tier]

Pros: Total isolation, guaranteed performance, regulatory compliance. Cons: High cost, operational complexity, underutilized resources.

Strategy Comparison

Decision Matrix

Number of tenants?
├── > 5,000 → Namespace/Filtering (required)
├── 500 - 5,000 → Dedicated Collection
├── < 500
│   ├── Strict GDPR / sensitive data → Dedicated Instance
│   ├── Limited budget → Namespace/Filtering
│   └── Standard → Dedicated Collection
└── < 10 (enterprise accounts) → Dedicated Instance

Complete Multi-Tenant Architecture

DEVELOPERpython
from enum import Enum
from typing import Optional

class IsolationLevel(Enum):
    SHARED = "shared"           # Namespace/Filtering
    COLLECTION = "collection"    # Collection per tenant
    DEDICATED = "dedicated"      # Dedicated instance

class MultiTenantRAGService:
    """Multi-tenant RAG service with automatic routing."""

    def __init__(self):
        self.shared_client = QdrantClient(url="http://qdrant-shared:6333")
        self.tenant_registry = {}

    def register_tenant(self, tenant_id: str, plan: str,
                        isolation: IsolationLevel = None):
        """Register a new tenant."""
        if isolation is None:
            isolation = self._determine_isolation(plan)

        config = {
            "plan": plan,
            "isolation": isolation,
        }

        if isolation == IsolationLevel.COLLECTION:
            collection_name = f"tenant_{tenant_id}"
            self.shared_client.create_collection(
                collection_name=collection_name,
                vectors_config=VectorParams(
                    size=1536, distance=Distance.COSINE
                )
            )
            config["collection"] = collection_name

        elif isolation == IsolationLevel.DEDICATED:
            instance_url = self._provision_dedicated(tenant_id, plan)
            config["instance_url"] = instance_url
            config["client"] = QdrantClient(url=instance_url)

        self.tenant_registry[tenant_id] = config

    def _determine_isolation(self, plan: str) -> IsolationLevel:
        """Determine isolation level based on plan."""
        plan_mapping = {
            "free": IsolationLevel.SHARED,
            "starter": IsolationLevel.SHARED,
            "pro": IsolationLevel.COLLECTION,
            "business": IsolationLevel.COLLECTION,
            "enterprise": IsolationLevel.DEDICATED,
        }
        return plan_mapping.get(plan, IsolationLevel.SHARED)

    async def query(self, tenant_id: str, question: str,
                    query_embedding: list) -> list:
        """Search documents for a specific tenant."""
        config = self.tenant_registry[tenant_id]
        isolation = config["isolation"]

        if isolation == IsolationLevel.SHARED:
            return self.shared_client.search(
                collection_name="shared_knowledge",
                query_vector=query_embedding,
                query_filter=Filter(must=[
                    FieldCondition(
                        key="tenant_id",
                        match=MatchValue(value=tenant_id)
                    )
                ]),
                limit=5
            )
        elif isolation == IsolationLevel.COLLECTION:
            return self.shared_client.search(
                collection_name=config["collection"],
                query_vector=query_embedding,
                limit=5
            )
        elif isolation == IsolationLevel.DEDICATED:
            return config["client"].search(
                collection_name="knowledge",
                query_vector=query_embedding,
                limit=5
            )

    async def delete_tenant_data(self, tenant_id: str):
        """Delete all tenant data (GDPR)."""
        config = self.tenant_registry[tenant_id]
        isolation = config["isolation"]

        if isolation == IsolationLevel.SHARED:
            self.shared_client.delete(
                collection_name="shared_knowledge",
                points_selector=Filter(must=[
                    FieldCondition(
                        key="tenant_id",
                        match=MatchValue(value=tenant_id)
                    )
                ])
            )
        elif isolation == IsolationLevel.COLLECTION:
            self.shared_client.delete_collection(config["collection"])
        elif isolation == IsolationLevel.DEDICATED:
            self._deprovision_dedicated(tenant_id)

        del self.tenant_registry[tenant_id]

Performance Isolation

Data isolation isn't enough. A resource-hungry tenant must not degrade performance for others.

Per-Tenant Rate Limiting

DEVELOPERpython
from datetime import datetime, timedelta
from collections import defaultdict

class TenantRateLimiter:
    """Rate limiter per tenant."""

    def __init__(self):
        self.limits = {
            "free": {"rpm": 10, "rpd": 100, "tokens_per_day": 50000},
            "starter": {"rpm": 30, "rpd": 1000, "tokens_per_day": 500000},
            "pro": {"rpm": 100, "rpd": 10000, "tokens_per_day": 5000000},
            "enterprise": {"rpm": 500, "rpd": 100000, "tokens_per_day": 50000000},
        }
        self.usage = defaultdict(lambda: {
            "minute": [], "day": [], "tokens_today": 0
        })

    async def check_and_consume(self, tenant_id: str,
                                 plan: str, tokens: int = 0) -> bool:
        """Check and consume quota."""
        limits = self.limits[plan]
        usage = self.usage[tenant_id]
        now = datetime.utcnow()

        usage["minute"] = [
            t for t in usage["minute"]
            if now - t < timedelta(minutes=1)
        ]
        usage["day"] = [
            t for t in usage["day"]
            if now - t < timedelta(days=1)
        ]

        if len(usage["minute"]) >= limits["rpm"]:
            return False
        if len(usage["day"]) >= limits["rpd"]:
            return False
        if usage["tokens_today"] + tokens > limits["tokens_per_day"]:
            return False

        usage["minute"].append(now)
        usage["day"].append(now)
        usage["tokens_today"] += tokens
        return True

Limits per Plan

Cost Optimization

Per-Tenant Cost Model

Total cost per tenant = Storage + Compute + LLM + Infra

Storage:
├── Vectors: $0.10 / 1M vectors / month
├── Payloads: $0.05 / GB / month
└── Backup: $0.02 / GB / month

Compute:
├── Search: $0.0001 / query
├── Indexing: $0.001 / document
└── Reranking: $0.0005 / query

LLM:
├── Embedding: $0.02 / 1M tokens (text-embedding-3-small)
├── Generation: $0.60 / 1M output tokens (GPT-4o-mini)
└── CRAG evaluation: $0.001 / query

Shared infra (distributed):
├── Qdrant: $200/mo / 100 tenants = $2/tenant
├── API Gateway: $100/mo / 100 tenants = $1/tenant
└── Monitoring: $50/mo / 100 tenants = $0.50/tenant

Estimated Cost per Tenant Profile

Security: Preventing Data Leaks

Risk Number 1: Cross-Tenant Data Leak

DEVELOPERpython
class TenantSecurityMiddleware:
    """Security middleware to prevent cross-tenant data leaks."""

    def __init__(self):
        self.audit_logger = AuditLogger()

    async def validate_request(self, request, tenant_id: str):
        """Validate that the request is legitimate for this tenant."""
        auth_tenant = self._extract_tenant_from_auth(request)
        if auth_tenant != tenant_id:
            self.audit_logger.log_security_event(
                "TENANT_MISMATCH",
                f"Auth tenant {auth_tenant} != request tenant {tenant_id}"
            )
            raise SecurityError("Tenant mismatch")

        tenant = await self._get_tenant(tenant_id)
        if not tenant or tenant.status != "active":
            raise SecurityError("Tenant not found or inactive")

        self.audit_logger.log_access(tenant_id, request.path)
        return True

    async def validate_response(self, response, tenant_id: str):
        """Verify no data from another tenant leaks."""
        if hasattr(response, "documents"):
            for doc in response.documents:
                doc_tenant = doc.metadata.get("tenant_id")
                if doc_tenant and doc_tenant != tenant_id:
                    self.audit_logger.log_security_event(
                        "DATA_LEAK_PREVENTED",
                        f"Doc from {doc_tenant} nearly served to {tenant_id}"
                    )
                    response.documents.remove(doc)
        return response

Multi-Tenant Security Checklist

FAQ

How do you handle migrating a tenant from one plan to another?

Plan migration (e.g., shared to dedicated collection) requires copying data from one storage space to another. Proceed in three steps: (1) create the new collection, (2) batch copy the data, (3) switch routing. During migration, both spaces coexist. Verify all data is correctly copied before deleting the old space.

How do you horizontally scale when a single Qdrant server isn't enough?

Qdrant supports native sharding and replication. Configure a Qdrant cluster with N nodes, and collections are automatically distributed. For multi-tenant, sharding by tenant_id is ideal: a tenant's data lives on the same shard, optimizing filtering performance. Pinecone and Weaviate offer similar managed solutions.

How do you guarantee complete deletion of a tenant's data (GDPR)?

With the "dedicated collection" strategy, it's simple: delete the collection. With "namespace/filtering," it's more complex: you must delete all points with the tenant_id, then verify no residue remains in caches or backups. Keep a deletion certificate with the date and number of deleted documents for GDPR compliance.

What's the impact of tenant count on search performance?

With namespace/filtering, search performance degrades when the index exceeds 10M vectors (all tenants combined). The search must filter through all vectors before returning results. With dedicated collections, each tenant has a small independent index, so performance is constant regardless of tenant count.

How do you accurately bill tenants?

Track three metrics per tenant: (1) number of indexed documents (storage), (2) number of queries (compute), (3) number of LLM tokens consumed (generation). Store these metrics in a billing table updated in real-time. Billing can be flat-rate (plan limits) or usage-based (pay-per-query).

Conclusion

Multi-tenant architecture is the foundation of any viable RAG SaaS. The isolation strategy choice directly impacts your platform's costs, performance, and security.

Key takeaways:

1. Start with namespace/filtering for free and starter plans, it's the most economical
2. Dedicated collections for pro plans: good isolation/cost tradeoff
3. Dedicated instances only for enterprise accounts with regulatory requirements
4. Rate limiting is essential to protect the platform from abuse
5. Cross-tenant security auditing must be automated and continuous

Ailog is natively designed as multi-tenant, with strict data isolation and guaranteed performance for each client. Create your account and deploy your RAG chatbot in minutes on shared and secure infrastructure.

Resources

• Qdrant Multi-tenancy - Official documentation
• Pinecone Namespaces - Namespace isolation
• RAG Security & Compliance - Complete RAG security
• Production Deployment - Production deployment
• RAG Cost Optimization - Cost optimization
• GDPR & Chatbot - GDPR compliance