We’ve just released a new topic on supercooldata: The Aha Moment.

Put simply, The Aha Moment is the point in time when a user sees the value in your product.

It is a moment in the user journey after initial acquisition where an action or set of actions is highly correlated to retention. Our new tutorial on The Aha Moment goes through how you typically approach this type of analysis with SQL and more generally how to find events that correlate highly with actions in future weeks.

How do I find my Aha Moment?

The first step is to identify a set of suitable candidate user actions that allow your users to achieve the core value that your product provides.

User interviews

The best place to develop great hypotheses about what’s driving retention is to talk to your users. Anecdotal evidence is a great way to start, some good questions you can ask are:

• What were you trying to achieve by using the product?
• Why did you come back?
• When did the product click for you?

User data

The next step is to verify the user feedback using real data. Perhaps the users you spoke with are outliers and not representative of most users.

For each candidate user action, produce a graph similar to the one below which shows the relationship between how many actions a user might have taken in their first day or week and their retention in the following week.

We can see that as the frequency of the event increases, the retention rate of users also increases but at a diminishing rate. This shows that this event is correlated with retention.

You can test different events and determine which is a better candidate.

There are three other factors to consider when deciding what your product’s Aha Moment should be:

1. Is the action rare among users?

The event might correlate highly with retention but only a very small percentage of users ever perform it.

2. Is the action performed by most retained users?

If only a small percentage of retained users actually perform an action, then it might not be a good candidate for the Aha Moment.

3. Does it make logical sense to be causal?

I don’t want to get into the whole correlation vs causation thing here, you can have a read more elsewhere. However, there has to be a good level of logical reasoning to justify the Aha Moment, not just numbers.

For example: In Facebook’s case, if users have more friends, they’re more likely to get notifications which are interesting and so retention will be higher.

This is a decent logical explanation for why more friends drives more retention.

Multiple user actions

If there aren’t any suitable individual candidates for the Aha Moment, then you might want to start testing combinations of user actions. Go back to your user interviews and analyse what retained users did together.

Generally, if you want to test multiple user actions, it’s best to have a data scientist on board as they can produce a predictive model which allows them to determine which combinations are most predictive.

You don’t want more than 2 or 3 different user actions to define the Aha Moment, it’s meant to be simple and easy to understand for the whole business.

Time to Aha Moment

Once you’ve identified Aha Moments which have been validated with data, you want to make sure these actually cause retention.

To determine if there’s a causal relationship and not just a correlation, you’ll need to run an A/B test.

For Facebook’s product, this experiment might involve adding your friends in the on-boarding flow before you can actually use the product.

The idea is that you want to reduce how long it takes for your users to find the Aha Moment of your product before they give up and churn.

Examples of experiments you might want to try include:

• Feature flags to hide features meant only for power users
• Adding in user setting selection to the on-boarding flow to personalise the product
• Adding in the Aha Moment to the on-boarding flow
• Making critical features more prominent with tooltips or other UI features

Our New topic: The Aha Moment

We’ve just released a new topic on supercooldata which takes you through how to identify suitable candidates for the Aha Moment using SQL.

You can take a look at the full list of tutorials available by signing in here.