Our incumbent NLP-based chatbot suffered from slow language detection (3+ seconds), high drop-off rates when users hit low-confidence or unsupported intents, and 20+ second response latencies that frustrated customers. The probabilistic drift of LLMs made intent classification unreliable at scale, and we lacked the guardrails necessary to protect PII and filter toxic content before go-live. With 40+ products across 7 systems and growing user expectations, we needed a resilient, sub-second conversational AI platform that could handle multi-turn dialogue, personalization, and compliance from day one.
I owned the AI platform development strategy and organized platform steerco discussions to align stakeholders. I designed a sub-second multi-model ensemble system for language detection and introduced hierarchical intent mapping to reduce disambiguation. To handle edge cases, I devised HITL (human-in-the-loop) fallback and escalation strategies for low-confidence, risky, or unsupported intents. I developed an 11-layered context engine to enable true multi-turn conversation and integrated 8 fragmented customer propensity models into an evolving "cDNA" for 6-layered personalization. I designed an agentic guardrail system with a closed feedback loop for toxicity filtering, PII protection, and response grounding, and deployed an automated red-teaming pipeline integrated into our test suite. I balanced cost, latency, and response quality through prompt and context engineering, and established baseline metrics from the incumbent NLP bot to track LLM impact via 8 self-serve dashboards. I also prepared a launch playbook to align stakeholders for cascading sign-offs.
Chose a multi-model ensemble over a single large model for language detection to optimize for speed and cost. Introduced hierarchical intent mapping instead of flat classification to improve scalability and reduce disambiguation rates. Opted for HITL fallback strategies rather than fully automated handling of low-confidence intents to balance user experience with safety. Prioritized prompt and context engineering over model fine-tuning to achieve sub-5s latency while controlling costs. Researched multiple guardrail providers and drove technical feasibility meetings to select and procure the best solution rather than building in-house. Deployed automated regression testing and red-teaming pipelines early to de-risk go-live.
built the product