How To Put Learnings Into Action

This is part of a series of reflections inspired by my Reforge Growth course. I'm part of a cohort of marketers learning how to develop a systematic approach to solving growth challenges. Find the whole series here.

Once you've generated your ideas, prioritized your backlog, and completed your experiment, you have to learn from them.

In every team and organization, you're squeezed for resources: Time, money, or people. When it comes to growth, you have to deal with high expectations (bringing in a certain amount of leads, revenue, or signups) without a lot of resources to make things happen. Working efficiently is essential—so every experiment needs to count.

Related: Learning to Fail Is The Hardest Thing To Learn

Because less than 25% of experiments will work, it's not just successes that can teach you. You must learn from your failures, too. While success is awesome because we’ll grow more, failed experiments can help you navigate to success.

An experiment is only a failure if you fail to learn from it.
— Brian Balfour

How to Analyze Your Experiments

There are three ways to analyze your experiment:

  • Success/Failure
  • Accuracy
  • The Why

Most people stop at the first: Determining whether or not an experiment was a success or a failure. That's fairly rudimentary if you've set up your hypothesis correctly. For example, if your hypothesis was, "Signup-to-activation rate will increase by changing the color of our call-to-action button," either the rate increased, decreased, or stayed the same. And in the case of growth, if it stayed the same or if it decreased, that still counts as a failure.

From there, it's important to determine accuracy. How far off were your results? Were they statistically significant? If they're close, you may have hit on something—or there may be another factor at play impacting your experiment.

That's why the third way, finding the why, is so important. 90% of all of your learnings should come from the why. What was it about your experiment that made it a success or a failure? What would you do differently? What about your user or customer impacted this result?

Finding the "Why"

You don't have to find a scenic overlook to think about your growth experiments, but you should, if stock photos are any indication.

You don't have to find a scenic overlook to think about your growth experiments, but you should, if stock photos are any indication.

Finding the why comes down to thinking qualitatively about your experiment. It's not enough to consider the numbers—those numbers often represent people (users, customers, or subscribers) and therefore don't always act rationally. 

To analyze the why, you can ask yourself a few questions:

  • What are the inputs to this result? What caused this?
  • What segments influence the outcome? Were there specific user segments where we performed better or worse?
  • What was the user thinking or feeling that may have led to this outcome?
  • What additional experiments can I run to validate these results?

If it's a failure, make sure you narrow in on the specific piece that didn't behave as expected—that will serve as your starting point for the next experiment.

Apply Your Learnings

Books and cleverness can only take you so far; you must make the leap yourself.

Books and cleverness can only take you so far; you must make the leap yourself.

Once you've isolated your learnings (and, more importantly, written them down), you must apply them to your experimental backlog. What you've learned shouldn't exist in a vacuum; rather, they should inform your iterative growth strategy. 

If you've succeeded...

  • Have you hit on a gold mine? Should you double down?
  • How can you apply this learning to something else in the funnel?
  • What should you adjust or change in the backlog to accommodate this new piece of information?

If you've failed...

  • What are some alternative hypotheses you can try to influence this metric or outcome?
  • What should you adjust or change in the backlog to accommodate this new piece of information?

No matter what, you must tie your experiments back into your growth process and inform your new ideas. Otherwise, what's the point of experimenting? You'll go around and around in a loop that never ends, taking a scattershot approach at growth and hoping something will stick.

If you're looking for predictive, deterministic growth (and if you don't think you're looking for it, just go ask your executive team!), then you must document and apply any learnings to future endeavors.

You can find the full series here.