The Growth Process

You have a metric to move, and as a result of your research have a comprehensive understanding of both the current experience and what users need/expect. Armed with this data, you’ll create experiment ideas that will later be transformed into hypotheses and designs.

The scope and type of experiment ideas can vary wildly; from tiny optimizations (e.g. changing the color of a button) to totally new experiences (e.g. a new onboarding journey). This step describes two entirely different ideation processes that can be used for two very different types of problems. In practice many ideas will fall somewhere between these extremes and it will be up to the team to understand how to find a balance between these processes.

A. Optimizing existing funnel

In some situations, the existing journey/feature works fairly well, but you believe it can be greatly improved:

• Your funnel showed that many steps are completed at high rates, but others aren’t.
• Your journey matches, some, but not all of the needs of your users.
• Pain points can be resolved with relative ease, and aren’t prolific.

In these situations you should focused on optimizations. Instead of creating a totally new experience, your experiments will be individual features or improvements. When creating optimization experiments:

• Ensure you have the appropriate volume for running the experiment. Typically, small changes (optimizations) will result in relatively small changes to metrics, so you will need relatively large sample sizes to run the experiments in a reasonable timeframe. Note that while these changes may be small, the value may be high. A small improvement that affects an action that occurs millions of times still has a large overall impact on the product.
• When running your ideation session use any format that suits your preference (e.g. disrupt, brainstorming session, ad hoc ideation).
• Ensure that your ideation focuses on a specific and narrow topic. You should have a specific step in your funnel that you aim to move and the corresponding qualitative data (pain points, user needs, etc.). While your ideas may be broad or even bizarre, it should be very clear why/how they’ll affect your specific metric. Gather as many unique ideas as possible.
• Prioritise the experiment ideas by how likely they are to succeed (using e.g. ICE). Consider how much confidence you have in the idea being successful because it clearly addresses your research findings and the amount of technical risk/difficult for implementation.
• Form hypotheses and capture chosen ideas and begin designing. One experiment idea may have many different designs/treatments. Do not aim for perfection. Your design is ‘good enough’ when you’re confident that you’re genuinely testing the idea and you’ll gain clear insights even if the experiment fails.

B. Create a new journey

In some situations it doesn’t make sense to optimize existing work:

• The existing user journey is inadequate. You’ll know this is the case when your funnel has a lot of dramatic drop-offs on most steps, the user pain points are numerous, and/or the journey doesn’t reflect the users’ needs.
• There is no existing user journey. You may want to create a totally new journey for a behavior that doesn’t occur at all, or only exists via work-arounds.
• Research has suggested that an alternate journey may be effective for a specific segment that the product doesn’t currently cater to.
• You want a large-scale change (15%+ differences), not incremental improvements.

In these situations, the best way to move your metric is large multi-screen change to the experience. This process will be design-lead, and follow our existing design processes.

• Designer leads creating a new journey; ideation, sketching, prototypes, etc. based on the previous research. Your design should directly solve the solutions and and pain points identified during research.
• These experiments will be physically very large, resulting in technical risk (bugs, product changes that break experiments, etc.) and a relatively high development cost. As a result, there is a greater burden of ensuring that there is a high level of confidence that these experiments will be successful. Perform user testing and iterate on the design in advance of the build phase.
• These designs should be sparred internally to get buy-in from product teams and stakeholders.