RAG that actually works in production
Notes from shipping a medical AI platform
Vector search is the easy part. The hard part is chunking, freshness, and knowing when the model should refuse to answer.
Chunking is where quality dies
Naive fixed-size chunks destroy semantic boundaries — a definition ends up split from its example, and retrieval fetches the wrong half. We segment by heading structure first, then rebalance chunks toward a target token count without crossing heading boundaries. Retrieval accuracy jumped materially the day we did this.
Freshness beats cleverness
Users notice a stale answer faster than a slightly-worse one. Every source document has an updated_at, every embedding row carries a source_version, and a nightly job re-embeds anything whose source has drifted. Retrieval joins on version so we never mix old chunks with new ones.
Teach the model to refuse
A RAG system that always answers will hallucinate. We inject retrieved passages with clear delimiters and instruct the model to answer 'I don't have that in my sources' when the top-k similarity is below a threshold. Refusals are a feature, not a bug.
Silent JWT refresh without the thundering herd
The 401 handler you copy-paste from Stack Overflow will fire N parallel refresh calls the moment your access token expires. Here's the fix.
The Firestore listener leak nobody talks about
onSnapshot is easy to add and easy to forget. In a SPA, forgetting means memory leaks and phantom writes.