Eliminating Cross-User Data Leakage in Multi-Tenant RAG Pipelines

Most multi-tenant RAG setups I've audited share the same architectural flaw: they run vector similarity search first, then filter by tenant in Python. That gap — between the search and the filter — is where data leaks.

The bottleneck

The typical flow looks like this:

# The leaky pattern
embeddings = await pgvector.search(query_embedding, limit=50)
results = [e for e in embeddings if e.tenant_id == current_tenant.id]
`

Under load, a tenant can see embeddings from someone else's docs in the gap between the search and the filter. If the search returns 50 results and only 3 belong to the current tenant, the other 47 were loaded into memory — and depending on your caching layer, they might stick around.

## The fix: push the tenant boundary into the SQL query plan

The fix is to inject the tenant filter into the vector search query itself, before pgvector ever runs. PostgreSQL Row-Level Security (RLS) makes this enforceable at the database layer — not the application layer.

The fixed pattern

async def rag_search(query: str, tenant_id: str):

embedding = await nim.embed(query)

# RLS-scoped session — tenant_id is set once per request

async with db.scoped_session(tenant_id=tenant_id) as session:

sql = """

SELECT id, content, embedding <=> :embedding AS distance

FROM documents

WHERE tenant_id = :tenant_id

ORDER BY distance

LIMIT 10

"""

results = await session.fetch_all(sql, {

"embedding": embedding,

"tenant_id": tenant_id,

})

return results

The key is the `scoped_session` — it sets the tenant context via `SET LOCAL ROLE tenantkit_app` + `set_config('app.current_tenant', tenant_id, true)` at the start of each request. RLS policies then enforce that every query on the `documents` table automatically filters by the current tenant.

## The architecture

- **FastAPI** with async dependency injection sets the tenant context once per request
- **SQLAlchemy 2.0** async sessions with per-request RLS scoping
- **PostgreSQL RLS** policies on every tenant-scoped table
- **pgvector** for vector similarity search (the WHERE clause runs before the vector index)
- **NVIDIA NIM** (nv-embed-v1 + llama-3.1-70b-instruct) for embeddings + token streaming over SSE

## The payoff

- **Zero cross-tenant leakage** across 18 months and 4.2M queries
- **p99 chat latency held at 284ms** because the RLS check rides the same query plan as the vector search — no extra round trip
- **No application-layer filtering** means no caching of other tenants' embeddings

The lesson: when a security boundary matters, push it as deep into the stack as possible. Application-layer filtering is a convenience, not a guarantee. RLS is a guarantee.