I’ve mostly been working on consumer facing products throughout my PM journey, ranging from growth, monetisation to community engagement.
Post Carousell’s re-organisation, I’ve been working on the Post Orders journey, which looks at the customer journey when someone completes a purchase on Carousell.
It’s still somewhat consumer facing, but some of my key metrics now revolve around solving for internal stakeholders. This includes:
- Reduce the time taken for customer support to resolve dispute tickets,
- Improve the number of tickets responded within 24 hours.
Some things I’m reflecting on after a quarter into the role:
How do we accurately measure the improvements made to our internal tools?
We improved our internal members’ workflows and measured the total time taken to resolve dispute tickets. But measuring the time saved accurately can be challenging, especially if the task or process involves multiple steps or variables. Variables include:
- Changes in support team’s staffing hours,
- Rise in dispute tickets (due to reasons outside of Post Orders) which leads to slower resolution times
This meant seeing little to no change in the total time taken to resolve disputes even after we had prioritised solving some of the biggest pain points of the customer support team. (Oof!)
I also realised with an internal facing product domain, I had to influence changes in legacy / long standing workflows in order to start measuring metrics that we haven’t tracked before.
For instance, we have never tracked the different types of resolution outcomes a dispute case ends up with. Is it a full Return & Refund, or is it a Partial Refund, or an Exchange of Goods? Needing to track that meant I had to influence the team to record that outcome as part of their dispute workflow.
To that, I realised I should approach an internal facing product domain vs a consumer facing domain a little differently.
Don’t get me wrong, we are still here to deliver value to our users, but the approach may differ.
Throughout my previous consumer facing domains, the broad(ish) product process I usually go with starts off with user research and data analysis. Finding out key user problems is the most crucial aspect; following that would be workshops to brainstorm and align on problems and solutions, prioritisation, before rolling out features as AB experiments. It continues on with conclusion of experiments depending on outcome, and we decide whether to iterate on the features or not.
But, there seems to be little upside in doing AB experiments when focusing on making internal tools more efficient. Truth be told, I felt uneasy not ‘validating’ my work with AB experiments, as it was more common in my growth product domains. To that, I ask the question: Why do we need to do AB experiments in the first place?
Do I run AB experiments when serving internal stakeholders?
AB tests are meant for us to validate hypotheses based off assumptions — it’s a data-informed method for us to optimise our product decisions and roadmap.
When you are internal facing, this method may not be as useful in validating those hypotheses - now, you have direct access to your customers, who are your fellow colleagues. You can easily drop them a Slack DM to ask about their workflows. You can get access to video recordings of how colleagues are using your tools, and quickly roll out solutions once you’ve observed certain micro behaviours.
The methods used for measuring the outcomes then now lean heavily towards feedback surveys, user interviews and usability testing.
Instead of dwelling on the experimental method, I learned to send internal surveys to fellow colleagues asking about their experience with using the platform. “How satisfied are you with using X platform? Why? What’s the biggest pain points you faced when using X platform?” (sorta like an internal CSAT.)
With that, I can iterate even quicker than spending time to set up an experiment.
I’ve learnt that I should (and can!) reach out more to my fellow colleagues to poke on workflows and ask about their pain points. If they have a problem I can quickly solve, it’s also a quick win for both of us.
Being more specific about measuring a metric
I also learnt to be more specific in determining the time saved metric (though this may not solely apply to an internal facing product domain)
Given how I’ve not significantly reduced time taken despite a quarter of working on major support team pain points, I’ve learnt to determine the baseline of time taken for specific workflows that we want to improve.
To that, I worked with the customer support team to track our existing average time taken from a specific Step X to Step Y, instead of looking at the whole workflow. This helps me gather a baseline to see if the improvements targeted at this specific funnel really moved the needle.
Learnings
- Different product domains call for different product management approaches. There is no ‘one size fits all’, but more of what fits your domain and stakeholders best.
- Don’t get too hung up about not following the common PM approach. Not doing AB experiments? If there’s little upside, why lament about NOT doing it? Do what’s most impactful for your product area, not what’s most common.
- Be more specific about measuring metrics co-owned by internal stakeholders, and poke hard in getting the right metrics. You are here to help make their lives easier.
- Metrics not moving? They are not your only indicators of success. Talk to your users, get qualitative feedback on whether their workflows have been improving. These feedback are equally as valuable as looking at metrics.
Comments