Applied AI 2024 S&P Global

SAM Assist

A RAG assistant that lets 150+ internal teams answer their own integration questions. The build took weeks; adoption took months.

Role Associate Director, Product Management
Industry PaaS / SaaS
Client S&P Global
Context Applied AI
SAM Assist

Support Work Nobody Owned

The SAM platform is the identity layer behind S&P Global's product portfolio, and the teams building on it kept asking the same questions. How do I wire up SSO. Which API do I call. Why is this configuration failing. My engineers were fielding five to eight unsolicited pings a day and losing hours every week to consultation calls: untracked work that pulled senior people away from the roadmap.

Nobody asked me to fix it. It wasn't anyone's job, including mine; I run identity, not AI. I built it because the waste was visible and unowned.

Answers That Stay in Sync With the Docs

SAM Assist is a retrieval-augmented assistant trained on our platform documentation, built on S&P's internal Spark Assist platform, so teams can self-serve on components, API integrations, configuration options, and how-to guides.

The first version indexed static PDFs. We then wired the pipeline directly into Confluence, so every documentation update re-indexes automatically and the assistant stays in sync with the latest API changes without manual retraining.

I owned the evaluation criteria and the quality bar, monitored response accuracy, and tuned retrieval against real usage rather than benchmark scores. That last part is where most of the actual work was.

Nobody Used It at First

Adoption stalled completely. I had built a tool and announced it, which is not the same as launching a product. People had a working habit, which was to ask a colleague, and the assistant was an unfamiliar alternative with no track record.

What fixed it was treating the rollout as its own product. I sat with teams to see where they actually got stuck. I walked them through real questions instead of demos. On troubleshooting calls I started using the assistant live, asking their actual question in front of them and getting the answer on the spot, which proved the point better than any explanation could. I found a champion in each business line, and every failure went back into retrieval so the answers visibly improved.

The Numbers, and the Lesson Behind Them

Support tickets fell 65%. The organization recovered more than 50 hours a week of manual effort, and the assistant is now used across 150+ internal teams: developers, product managers, customer care, and delivery. New products onboard to the platform without the support load growing alongside them.

The build took weeks. Adoption took months, and that was the real work.

Key Insights & Takeaways

  • Announcing a tool is not launching a product. Adoption needed onboarding, a feedback loop, and a champion in every business line.
  • The fastest way to earn trust was using the assistant live on a troubleshooting call, on the team's own question.
  • Retrieval tuned on real usage beats retrieval tuned on benchmarks.
Let's Connect

Looking for a similar outcome?

Let's discuss how we can modernize your product operations or accelerate your AI roadmap.