How Spotify's ROSIE framework measures sales training impact

When was the last time you felt completely confident in your sales training measurement report?

If you hesitated, you’re not alone. My name is Meredith Ayan, and I head up sales training and enablement for Spotify. I’m not here as an expert who has measurement down and is going to tell you all the super technical ways you can do this. Instead, I want to show you a beginner's guide to proving the impact and the measurement of training.

The pressure on us is immense. We’re constantly asked to prove our ROI, which can feel overwhelming. But here’s the good news: you can’t fall off the floor, right? If you're not doing much formal measurement at all, you're not going to do any harm by starting somewhere. You may as well experiment and try a few things.

That’s where this framework comes in. It’s for all of us, whether you have sophisticated BI dashboards or you’re a smaller team trying to make sense of it all.

To help, I’d like to introduce you to a framework called ROSIE…

Meet ROSIE: Your new framework for sanity and success

ROSIE is an acronym that represents the five core categories of things you can and should be measuring:

Retention
Output
Sentiment
Impact
Engagement

These five pillars give us a holistic, 360-degree view of our training programs' effectiveness. Instead of getting stuck on one intimidating metric, this framework allows us to build a more complete and powerful story about the value we create.

Let’s break them down.

R is for retention: Are they actually remembering anything?

This is the most fundamental question. We’re in the business of teaching people information and changing their behaviour, but if they aren’t retaining what we’re trying to teach them, nothing else matters.

The great thing about measuring retention is that you don’t need a sales ops team to help you. There are many ways, both formal and informal, to test for knowledge.

Formal measurement tactics

At Spotify, we use a few key tools to formally check for understanding. You’re likely familiar with many of them:

Knowledge checks and quizzes

These are straightforward but powerful. After a series of sessions, we’ll send out a quiz. Crucially, we frame this not as a test of the sellers, but as a check on our work. We have a ton of empathy for our sellers; we throw so much information at them and expect them to remember everything.

The quiz tells us if we’re doing a good job of making sure the information is landing. No one gets in trouble for a low score; if anyone should, it’s me. We then share the collective results in our team meetings and highlight areas where the group scored lower, which gives us a chance to revisit those topics.

Pitch-offs and certifications

These are fun, active ways to see if the messaging has been retained. Sometimes called "stand and delivers," these exercises allow sellers to practice and demonstrate their grasp of a new product narrative in a safe environment.

Informal measurement tactics

Beyond formal tests, you can learn a lot just by observing sellers in their natural habitat.

Listening in

Tools like Gong are invaluable for this. You can listen to calls to hear if the specific messaging you’ve taught is actually showing up in real meetings with customers.

Riding along

This is one of the simplest yet most effective methods. Just ask a seller, "Would you mind if I rode along to one of your meetings?".

While you’re there, you can mentally check if they’re remembering and applying the things you’ve been teaching them. It’s a low-pressure way to get direct insight.

Role-playing and meeting drop-ins

Whether through structured role-playing or just dropping into team meetings, these informal checks give you a pulse on how well your key concepts are being absorbed.

If retention is low, you know you have a foundational problem to solve before you can ever hope to see an impact on business results.

How Spotify's ROSIE framework measures what really matters in sales training