Applied AI • 2024 • Current Role

Platform AI Assistant

A RAG assistant that lets 150+ internal teams answer their own integration questions. The build took weeks; adoption took months.

Role Associate Director, Product Management
Industry PaaS / SaaS
Client Current Role
Context Applied AI
Platform AI Assistant

Support Work Nobody Owned

The customer identity platform I lead sits behind a large enterprise product portfolio, and the teams building on it kept asking the same questions. How do I wire up SSO. Which API do I call. Why is this configuration failing. My team was fielding five to eight unsolicited pings a day and losing hours every week to consultation calls: untracked work that pulled senior people away from the roadmap.

Nobody asked me to fix it. It wasn't anyone's job, including mine; I run identity, not AI. I built it because the waste was visible and unowned.

Answers That Stay in Sync With the Docs

The assistant retrieves from our platform documentation and runs on the company's internal generative AI platform, so teams can self-serve on components, API integrations, configuration options, and how-to guides.

The first version indexed static PDFs. We then wired the pipeline directly into Confluence, so every documentation update re-indexes automatically and the assistant stays in sync with the latest API changes without manual re-indexing.

I owned the evaluation criteria and the quality bar, monitored response accuracy, and tuned retrieval against real usage rather than benchmark scores. That last part is where most of the actual work was.

Adoption Stalled at First

Adoption stalled. I had built a tool, which is not the same as launching a product. People had a working habit, which was to ask a colleague, and the assistant was an unfamiliar alternative with no track record.

What fixed it was treating the rollout as its own product. I sat with teams to see where they actually got stuck. I walked them through real questions instead of demos. On troubleshooting calls I started using the assistant live, asking their actual question in front of them and getting the answer on the spot, which proved the point better than any explanation could. I found a champion in each business line, and every failure went back into retrieval, so each fix showed up in the next answers.

The Numbers, and the Lesson Behind Them

Support requests to the platform team fell from more than 60 a week to about 10 to 12 on average. The organization recovered more than 50 hours a week of manual effort, and the assistant is now used across 150+ internal teams: developers, product managers, customer care, and delivery. New products onboard to the platform without the support load growing alongside them.

The build took weeks. Adoption took months, and that was the real work.

Key Insights & Takeaways

  • Shipping a tool is not launching a product. Adoption needed onboarding, a feedback loop, and a champion in every business line.
  • The fastest way to earn trust was using the assistant live on a troubleshooting call, on the team's own question.
  • Retrieval tuned on real usage beats retrieval tuned on benchmarks.
Let's Connect

Looking for a similar outcome?

Let's discuss how we can modernize your product operations or accelerate your AI roadmap.