Client engagement · 2026 (ongoing) · Anonymized

A chatbot for an organization’s multinational program. Users ask natural-language questions; the system retrieves from a curated knowledge corpus and answers using a major LLM provider. Hosted on GCP on Cloud Run.

Outcomes

10.1 s p95 time to first token, at 40 concurrent users ↓ from 160.1 s before tuning
16.7 s p95 to a complete streamed answer ↓ from 164.9 s
200 / 200 requests served under that load, with zero fallback replies
2.3 min median commit-to-deployed pipeline 1.8 min typical, 6.8 min at p90

Measured July 2026. Latency figures come from our own load harness driving 40 concurrent virtual users through the live API; the pipeline figure is measured across 132 successful production runs. The engagement is ongoing, so these move.

The headline improvement was not model choice — it was discovering that naive concurrency was converting into provider 429s, each costing 5–30 s of backoff. Client-side rate limiting coordinated service-wide, tuned to sit deliberately under the provider’s per-bucket ceiling, turned a two-and-a-half-minute worst case into ten seconds.

The client has since extended scope twice — once to add another language to the retrieval and answer paths, and once for a verification feature that checks a submitted application against the program’s eligibility and completeness rules, then reports back what is missing or non-compliant.

That second one is a different shape of problem from question answering. A chatbot can be usefully approximate; a correctness check cannot, because being confidently wrong about a rule is worse than declining to answer. So the rules live in a hand-authored, reviewable layer that the check evaluates against, with explicit references pulling in the exact source documents — not inferred from the corpus and hoped for.

High-level architecture

High-level architecture of the enterprise AI chatbot A Vite app and a FastAPI service on Cloud Run handle the request path; the API runs a chat orchestrator that calls an external LLM provider. Cloud SQL holds users and a pgvector corpus, fed by Cloud Run jobs from knowledge sources in GCS. Secret Manager and Cloud Logging back the API. Google Cloud · single region Request path Cloud Run · app Vite SPA · login + MFA Cloud Run · api FastAPI · HTTPS LB Chat orchestrator planner → retrieval → answer LLM provider Claude · streamed Platform Secret Manager session keys · DB creds third-party API keys Cloud Logging health + status reporting Data & batch Cloud SQL · users auth · MFA · sessions Cloud SQL · pgvector chunks · embeddings Cloud Run jobs migrate · ingest · admin Cloud Storage knowledge sources auth RAG

A browser SPA (login + MFA) talks to a FastAPI service over an HTTPS load balancer. The API runs a LangGraph orchestrator that plans the query, embeds it (Vertex AI), retrieves from a hybrid pgvector + full-text store, and streams the answer from Claude. Knowledge files live in GCS and are ingested by Cloud Run jobs; auth state sits in a separate Cloud SQL instance, with secrets and health reporting handled by Secret Manager and Cloud Logging.

What we built

  • Streaming chat UI — TypeScript SPA (Vite, Tailwind) with login, MFA enrollment, idle session timeout, feedback capture, and an optional retrieval/debug panel for operators
  • RAG API — Python FastAPI with LangGraph orchestration: query contextualization, pipeline routing across corpus namespaces, hybrid full-text + vector retrieval (RRF fusion), and Claude answer generation over SSE
  • Knowledge pipeline — document ingest (extract → chunk → embed → store), glossary-aware multilingual answers (English-authored corpus, multiple non-English query paths), and tunable chunk/embedding config
  • Application verification — a curated rule layer, authored and reviewed rather than inferred, that a submission can be checked against for eligibility and completeness, with deterministic references into the source documents behind each rule
  • Security posture — JWT sessions, rate-limited login with lockout, CORS/TLS hardening, region-pinned data (us-central1), and production flags that strip debug payloads
  • Infrastructure — Terraform on GCP (Cloud Run, Cloud SQL / pgvector, GCS, Vertex AI embeddings), GitHub Actions CI, golden-question evaluation suites for English and multilingual quality

Retrieval, in detail

“Vector search” undersells it. Every query runs hybrid retrieval — Postgres full-text and pgvector similarity searched as separate channels, then fused with reciprocal rank fusion — because lexical and semantic search fail on different questions, and a coach asking about a specific form field needs the lexical hit that embeddings alone will miss.

Around that sits the machinery that decides what to search:

  • Namespace routing — the corpus is split into distinct retrieval namespaces (core program documents, supplementary material, translations), and a router classifies each question into the right branch before any retrieval happens
  • Query contextualization — multi-turn questions are rewritten against conversation history, so “what about the second one?” resolves to something retrievable
  • Doc-type-aware chunking — chunks are sized to be self-contained semantic units (a Q&A pair, a form tab, a rubric row group) rather than uniform token windows, with title and section-path prefixes carried into the embedding
  • Multilingual without frontend i18n — query-language detection, a translations namespace, per-document glossary indexing, and reserved translation slots inside the result set, against an English-authored corpus

The corpus is 60 documents across four languages, spanning DOCX, PDF and plain text.

Tuned, not hardcoded

Retrieval quality lives in the tuning, so the tuning is a reviewable artifact rather than scattered constants. A single settings file carries roughly two dozen knobs — RRF constant, candidate over-fetch multiplier, per-document chunk caps, quality-floor gating, translation reserve, document-version preference, chunk size bands, embedding model and dimensions — and they are split into two classes with different blast radii:

  • Retrieval, generation, language and model settings deploy with the API alone
  • Chunking and embedding settings trigger a full corpus re-index in CI, because changing them invalidates every stored vector

Model selection is per orchestrator step, not global: each of the six steps names its own model, so a cheap fast model handles query rewriting while a stronger one handles routing and answer generation. Each step declares an equivalent on a second provider, giving cross-provider failover rather than a single vendor dependency.

Proving it before go-live

  • Configurable load harness — one runner drives golden-question, multi-turn conversation, and stress modes against the live API, parameterized by virtual users, request interval, rounds, stagger, and endpoint. It emits per-request JSONL (status, time to first byte, total, fallback flag) so a run is analyzable after the fact rather than a pass/fail feeling. Every number in the outcomes above came out of it.
  • Continuous evaluation — golden-question, multilingual, and multi-turn conversation suites run as scheduled CI workflows, plus a duplicate-chunk check on the corpus. 66 English and 25 multilingual golden questions, so a retrieval-tuning change that quietly regresses answers is caught by the pipeline, not by a user.
  • Go-live readiness — an end-to-end health report exercises the real user path (login, MFA, chat, retrieval) against a deployed environment as a single command.

Stack at a glance

Layer Choice
Frontend Vite + TypeScript SPA
API FastAPI, LangGraph, Anthropic Claude
Embeddings Google Vertex AI
Vector / search PostgreSQL + pgvector (hybrid FTS)
Auth Separate Postgres (users, MFA, sessions)
Cloud GCP Cloud Run · Terraform

Details of the client organization and complete application architecture are withheld under NDA. Want something in this shape for your team? Get in touch.