Is Your Data Deluding You? Driving Real Results with Relevant Metrics

May 18, 2017

In his 2014 remarks delivered to the graduating class at the USC Marshall School of Business, Elon Musk distilled decades of experience launching successful startups down to five minutes, lending us a rare glimpse into the product development methodologies of disruptive companies like Tesla and SpaceX. As we look to develop ideas and products that drive deep engagement with our customers, one nugget stands out – “focus on signal over noise,” he told the hundreds of young entrepreneurs about to embark on their journeys, adding that he doesn’t spend any money on advertising for Tesla.

It’s one of the chief challenges we face as we build our ideas into businesses – how do we drown out the noise to focus on signals – to make business decisions and create products that, in a sense, are so irresistible that they sell themselves?

For startups, that’s a proposition complicated by the pressure to show traction and gain a foothold before we slump into irrelevance. As such, we can have a tendency to make business decisions on how to further develop and roll out packaged software based on metrics like downloads, registered users and raw pageviews – numbers that present a flash of the growth potential we believe to be huge. But recent history (which in our space could be defined as the span of a week) is littered with promising ideas that failed to matrix with the insight necessary to fuel sustainable growth. I think it’s a failure due in part to the over-reliance on these so-called vanity metrics – numbers that give us a pretty good idea about whether or not we have a good idea, but can’t provide any direction on how to execute it.

How can we gain insight into what helps us improve the experience that customers have with our applications while boosting engagement and allowing us to meet our business goals? Let’s look at the aspects we can measure to help us make insight-driven business decisions.

Feature Usage

Here’s a question every startup needs to answer: how do you define active users? Metrics calculated with web analytics tools – like pageviews, visits, bounce rate, time spent, and referral traffic – give us clues about the health of our website and can even give us a decent view of our audience, but do they help us determine how active our users are?

Software usage analytics delivers data and insight into what happens once users decide to give your product a try. By drilling into how a live user is actually leveraging the software as he moves through a particular workflow, you can correlate activity – down to the number of minutes per session – with a host of factors like geography, operating system, language and version to perform detailed analysis on feature usage. You can see where a user is moving outside of the system to accomplish a particular process, and figure out whether that’s an isolated incident for one customer, or something happening across the customer base.

Robust usage data helps ensure that nuanced information is read the right way. Perhaps users aren’t leveraging a key feature in a new release. Is it because they don’t need it, or is it because they don’t know about it? With data that allows you to drill into usage patterns, you can introduce users to killer functionality that is currently overlooked, and may make all the difference.

In turn, being able to clearly see obstacles or patterns with usage – especially by active user engagement minutes – gives you valuable information to reach out to users and ask targeted questions on how the workflow or particular feature could be improved. It also gives you invaluable information to target pitch and marketing campaigns – as you provide relevant narratives on actual usage scenarios.


Measuring downloads doesn’t lend information on whether users continue to be engaged with your product after their initial interest. And some of the greatest insights can come from potential customers who were so frustrated – or simply just not overly impressed – with your product that they abandoned it altogether.

Let’s take a common example of that free, 30-day trial offer to use your product. Breaking down a lost customer by looking at the events that led to their exit – as granular as the number of sessions they spent with you down to the number of minutes in each of those sessions – will lend insight on where users are losing engagement with your product.

Perhaps, as was the case with a leading provider of PDF management tools, users really want to use your product, but can’t. By leveraging usage analytics, the business was able to drill into key details on the user journey – how long users worked with the product before walking away and what the characteristics of those users were, including machine, memory capacity, region, version, edition and operating system. Armed with this information, the company was able to determine thousands of potential users were getting stuck with the configuration wizard, abandoning the software before they’d even had the opportunity to use it. With changes to the wizard’s usability, software conversions quickly spiked.


Once a product is launched, how does your team measure adoption success? Is it by number of downloads? Is it by uptake of new functionality? Usage analytics give you visibility into patterns that allow you to quickly capitalize on usage trends or remedy roadblocks in adoption, as well as continuously improve the product in the process.

As you mature in your analytics’ use, you can comb through data to find usage trends, and ask questions not just about how features are being used, but why they’re being used in particular patterns, processes or workflows. For instance, say two, complementary features aren’t gaining equal traction in the user base. With usage analytics, you discover that one of the features appeals to users right away, but the other appeals to a subset of users some six months after initial deployment. By gaining a more complete picture of your user persona, you can optimize functionality in ways that will help each unique group of users better accomplish their jobs.

This foundation of information will allow you to achieve what is on every product manager’s mind these days – deeper user engagement. With the right information, you can target education and training to help users get more value from their installations, increase adoption and boost your reputation as a partner to your customers.

Leverage analytics to build irresistible products

The right data drives deeper customer focus, eases tough decisions, and helps you develop not only products that customers want, but those they don’t even know they want. Leveraging usage analytics will allow you to make good technical decisions and operational decisions that will lead to irresistible products.

VP Software Analytics

Keith is Revulytics’ VP, Software Analytics and was the co-founder and CEO of Trackerbird Software Analytics before the company was acquired by Revulytics in 2016. Following the acquisition, Keith joined the Revulytics team and is now responsible for the strategic direction and growth of the Usage Analytics business within the company. Prior to founding Trackerbird, Keith held senior product roles at GFI Software where he was responsible for the product roadmap and revenue growth for various security products in the company's portfolio. Keith also brings with him 10 years of IT consultancy experience in the SMB space. Keith has a Masters in Computer Science from the University of Malta, specializing in high performance computing.