How to Measure Adoption of new Product-feature release

A PM is releasing a new feature (Comments) and is creating the tracking plan. These are the following things we need to achieve for completing the analytics for measuring the adoption of the new comment feature:

1. Create a tracking plan, minimally including the events to the instrument.
2. Metrics to help us answer the questions below.
3. A proposed dashboard to chart our progress.

Are customers finding and using all the comments functionalities?

Our comment feature functionalities can be measured by the following things:

1. The total and a unique number of users using the comment feature.
2. Time elapsed between landing on the Finance Public page and selecting comment action after selecting the cell.
3. Time-taken by Users for Navigating to already commented cells from the top toolbar.

What should we track for adoption?

For tracking comment feature adoption we can look around the new users who are commenting successfully after landing on the Finance Public page.

1. Total % of unique users commenting overtime for the first time.
2. Total % of users commenting daily, monthly for the first time.

Is the UX good enough for users?

Few metrics which can help:

1. Average time user is taking to select the box and post a comment.
2. Average time the user is taking to reply to comment after getting tagged.
3. Number of successful attempts/Total Attempts of commenting

For the actual knowledge of the experience of users, the numbers are not going to give us the full picture since the numbers will be giving us quantitative measures only.

Following methods should be devised for the full picture:

1. Conducting interviews with users (Not Power Users cohort).

2. Collect feedback after conducting a survey.

3. Just a simple feature pop up which asks “How are you liking the comment feature” and analyse the results.

How might we identify different comment users?

For the identification of users, we can create multiple funnels depending on what we want to achieve.
Based on their Usage behaviour:
1. Cohorts of users who are using most, least and in between, drill down by clients.
Based on their archetype:
1. Users with their id are tracked and we can map them with their user type.

How are comments driving other key actions within the app?

Here we have to first define what do we mean by key actions within the app.
Let’s say the following are the main key actions:
1. Highlighting cell

2. Sharing the table

Now that we have a set of key actions, let’s name it Key actions Group. After this, we can do a comparative analysis of the % of the use of these key actions before and after the release of the comment feature.

1. Metric to be compared week-by-week, month-by-month,

Are customers who use comments more valuable to us?

For finding whether the users who are commenting are valuable to us or not we have to first define what’s valuable to us and turn it into actions done by users.
Let’s say the following are the metrics that are valuable to us:

Metric 1: Number of different features used/Total number of features
Metric 2: Number of new unique users increased (WAU/MAU)
Metric 3: Total unique Number of tagged users

After defining these metrics we can do a comparative analysis of the change and analyse the results based on different properties like :
1. Clients

2. User type

3. Type of Feature.

Let’s go through the tracking and dashboard planning:

https://amitb0007.medium.com/event-tracking-plan-for-measuring-adoption-of-the-feature-release-a65cd8f308db

TBC ….
Thanks!