Deciding what metadata to capture and store alongside each chunk during ingestion — source, date, permissions, category — is a design decision made once at ingestion time, but used constantly at query time for filtering, citation, and access control.
Common Metadata Fields
| Field | Used For |
|---|---|
| Source document / URL | Citations — showing users where an answer came from |
| Section / heading | More precise citations, and context for the chunk's role in the source |
| Date created/updated | Freshness filtering — excluding outdated content, see RAG Failure Modes |
| Access permissions / tenant ID | Security — enforced via metadata filtering at query time |
| Document type/category | Narrowing search to relevant content types |
Example — Storing Metadata at Ingestion
vector_db.upsert(
id=chunk_id,
vector=embedding,
metadata={
"source_document": "refund-policy-v3.pdf",
"section": "Sale Items",
"last_updated": "2026-06-01",
"tenant_id": "customer-42",
"document_type": "policy"
}
)This metadata is what a later query-time filter (see Metadata Filtering) can act on — but only if it was captured during ingestion; metadata can't be reconstructed retroactively without re-processing the source.
Design This Before Ingesting at Scale
Adding a new metadata field after already ingesting millions of chunks means either re-ingesting everything, or living with inconsistent metadata across old and new content — worth thinking through what filtering, citation, and access-control needs you'll have before a large-scale ingestion run, not after.
Practical Use Case
A multi-tenant SaaS RAG product absolutely requires tenant/customer ID metadata from day one — retrofitting proper access-control metadata onto an already-running system serving multiple customers is a much bigger, riskier undertaking than designing it in from the start.
Common Mistakes
- Not capturing metadata that will later be needed for filtering or citations, requiring a costly re-ingestion to add it retroactively
- Inconsistent metadata field naming/values across different ingestion runs or document sources, making filtering unreliable
- Treating metadata as an afterthought rather than a core part of the ingestion design, especially for security-critical fields like tenant/access permissions
Interview Relevance
"What metadata would you capture when ingesting documents for a multi-tenant RAG system?" — tenant/customer ID (for access control), source and section (for citations), and freshness date (for filtering outdated content) are the expected core answers.
Practice Question
Design the metadata schema for ingesting a company's internal wiki into a RAG system that needs to support citations, freshness filtering, and department-based access control.
