Skip to main content

Product ops process - experimentation

 

The product ops process is a high-level framework that could help you identify how your product teams work, introduce changes in a controlled way, improve things and optimise your organisation for higher productivity and job satisfaction. 

In the last post, I suggested what the first stage of the process might be - surveying. It's when you aim to understand how your product teams work, how your entire organisation works, discover problems, turn them into products and make those products successful on the market. In this post, I'll cover the central phase of the product ops process - experimentation. 

Experimentation  

At this stage, you need to introduce a change and measure if the new way improved things. 

Goal

Find better ways of working. Improve effectiveness and job satisfaction for your product org. 

What you could do

Well, almost anything. At this stage, you should have very few limitations. You should be able to tinker with any process, tool, or habit. 

Don't try to be a scientist 

Scientists are the coolest but we're dealing with much simpler matters. Avoid overthinking and over-engineering your experiments. If you find yourself disproportionally planning rather than running your experiments - stop and reflect. First of all, you have a safety net. If your change didn't work - you can always return to the previous version. Secondly, you want to run more experiments rather than super-precise experiments. Thirdly, you'd never be 100% sure and it's a good thing, you should keep reasonably doubting your processes at all times, that's how you will keep looking for how to make things even better. 

Design an experiment

Pick the smallest thing possible. It's better to run two tests than a more complicated test. If possible, isolate a thing you will change - one particular part of a process, a single approach to using a tool, or certain behaviour such as one meeting or a piece of documentation. Describe how the thing is right now and how you want it to be. Define a way to measure the change - make sure it's possible to measure. Don't start an experiment if you're not sure you'll be able to measure the results! 

Run the experiment and measure the change

Implement the planned change and see if it brings the improvement you anticipated. If not - note down the results and move on. Don't try to fix "failed" experiments, better run new ones. Also, no such a thing as a "failed" experiment, it could only fail if you didn't do it. Every experiment brings you the most important outcome - new information. So even if an experiment didn't result in the improvement you were aiming for, you're still just getting a bit closer to finding something that would. 

Present the experiment outcomes and adopt the change

When an experiment brought positive results and you've validated the change, you need to present the results to the wider group who will be affected by the change. As not everyone took part in the experimentation, they might be wondering why all of a sudden something changes. You need to explain to them the experiment you've run and the results you've received. Don't be categoric here, some things might work for one group of people but not for another. You might need to run consecutive experiments to find changes beneficial to your entire product org and for each team in particular. 

Summary 

At the experimentation stage, you need to introduce a change in a process, behaviour, tool use and so on. You need to know what's the current state of things (you've done it on the surveying step). Then you design an experiment defining where you want to be and how you'll measure if you're getting there. You run an experiment (or maybe multiple experiments), you get the results and learn from the findings.
 
You need to present the positive changes to your team and help them adopt the new behaviours. Don't worry if everything will not be perfect right away. First, there is no perfect. Second, you'll get things even better in the last stage of the process ops process - optimisation

Popular posts from this blog

Product Vision: an elevator pitch for your product

On this blog, I write a lot about making data-driven decisions . But what if you just starting to think about your product? You have a vague idea and nothing more. No point to go for prototyping or even talking to customers as you don't know yet who to talk to and what to talk about. In such situation - start from creating a product vision.

7 steps of Product Discovery

Before building a product - how do you know what product to build? While building a product - how do you know what features are the most valuable? After you've built a product - how do you know if to tune stuff or add a new one?

3 ways to prioritize a Product Discovery backlog

Discovering the right product is a vital part of a product development process. To do that effectively best product teams use a Product Discovery process. This process foresees you having a Discovery Backlog with a list of ideas, concepts, and hypothesis in need of validation. But how to decide what ideas to validate first ?