Microsoft AppInsights or Usage Intelligence?

Make Sure Your Software Usage Analytics Solution Was Designed For the Job

Most software companies want to know more about how their customers and trial users are engaging with their software. Increasingly, they realize raw data isn’t enough: they need actionable insights for optimizing product development, increasing adoption, and growing sales.

The solution has a name: software usage analytics. But it’s still relatively new, and there are key differences in how potential solutions work. If you want a solution that’s manageable, scalable, cost-effective, and quick to deliver ROI, you need to understand those differences upfront.

Software usage analytics has unique characteristics and requirements. We know, because we built Usage Intelligence specifically for the job. For example, we provided native support for distributed applications on Windows, Macintosh, and Linux platforms, and extensive out-of-the-box tools for:

Capturing granular information about application usage and software and hardware platforms and environments
Surfacing user behavior patterns to find usability problems, improve roadmaps, and refine sales and marketing
Delivering in-app messages to reach users and educate them with timely, relevant information

As we speak to software companies, some ask whether they can do the same job with Microsoft Application Insights on the Azure cloud platform (now known as Azure Monitor). It’s a fair question, because Application Insights is a powerful tool for tracking and troubleshooting Azure cloud applications. However, it focuses largely on performance, not usability – and it simply wasn’t designed to instrument desktop applications.

We know developers who’ve tried to use Microsoft Application Insights for software usage analytics on distributed client software. They say it’s a lot like building their own custom telemetry and analytics systems from the ground up. They’ve had to dedicate significant engineering resources to collect, visualize, and analyze usage data, and as their requirements become more complex, their investments in development and maintenance grew significantly. They weren’t expecting it to be nearly so difficult or costly.

To see why, let’s walk through the software usage analytics lifecycle, and compare solutions based on Usage Intelligence vs. Microsoft Application Insights.

Planning

Plenty of planning knowledge is built into Usage Intelligence. Out of the box, Revenera:

Captures and tracks the usage metrics and KPIs that software companies typically need
Contains full client, server, and communications infrastructure to support usage analytics
Offers flexible reports designed for diverse professionals on both the business and technical sides of a software business
Can evolve and scale to meet advanced analytics requirements that typically arise as companies and their user communities grow

Microsoft Application Insights handles aspects of infrastructure associated with its underlying cloud platform, and stores data in an Azure database. But consider all you still have to plan on your own:

How tracking code will be instrumented in your client software – and who’ll build and integrate it
What data to collect, and how and when to send it
What metrics to report on, combine, analyze, and visualize
Where you’ll find SQL, Tableau, and/or other technical expertise to build your reports
How to make your reports easily consumable by diverse decision-makers – from executives to product managers and development leads
How you’ll provide for offline data management and some aspects of network connectivity
How you’ll maintain a system that will probably contain extensive custom code
How you’ll budget for Microsoft cloud services as you grow to exceed current thresholds for data collection, retention, and storage
How you’ll make sure none of this affects user experience

Client Instrumentation and Telemetry

Both Usage Intelligence and Microsoft Application Insights let you instrument your client applications via an API that can natively support multiple languages. However, because Revenera’s solution was purpose-built for software usage analytics, it includes several crucial capabilities absent from Microsoft’s.

For example, since the Usage Intelligence SDK automatically generates and manages machine fingerprints and user installation IDs, it’s easy to build unique user profiles, capture usage trends for each installation, and link individual installations to download sources and marketing campaigns.

Revenera also provides automatic licensing validation, and an easy customer opt-in/opt-out mechanism. In addition, its lightweight SDK was designed from the ground up to avoid application bloat or performance impact.

When it comes to delivering telemetry, both Revenera and Microsoft provide APIs – but, once more, the differences reflect Revenera’s targeted focus on desktop usage analytics. Only Revenera provides built-in mechanisms to:

Utilize a straightforward call-home protocol
Provide reliable, secure caching for machines without internet connectivity
Aggregate telemetry for scheduled delivery to minimize “call home” traffic
Give your developers full control of when to sync and track
Optimize data collection with no impact to application performance

Depending on the complexity of your telemetry, building comparable functionality for Microsoft Application Insights may take months.

Data Collection, Management, and Integration

Revenera gives you the flexibility to maximize the value of your usage data, but Microsoft establishes limits that can prove costly. For example, Revenera offers unlimited analytics and reporting data export via a Web API, and lets you export raw event data. In contrast, Microsoft charges for its Continuous Export feature (only available with paid subscription). Moreover, you can only export directly to Microsoft Azure, which requires paid storage and an Azure subscription. Only after you’ve exported data to Azure can you re-export it to your own premises-based systems.

Revenera retains telemetry data for at least a year (extensible to five years upon request), whereas Microsoft retains it for only 90 days. If you want to retain your Microsoft Application Insights data longer to understand trends over time, you must implement a process to regularly export it.

Reporting

Out of the box, Revenera includes many flexible reports designed specifically to help software companies answer key questions about how users engage with their products. All these reports can be customized, segmented and filtered to drill down into specifics, without requiring users to learn any reporting syntax. Revenera’s reports include details about:

Microsoft Application Insights dashboards provide significant data on web application performance and failure. However, if you want reporting on how your customers are using your distributed client applications, you need to build most of it from scratch, or create custom dashboards within Microsoft’s Power BI analytics engine, potentially at added cost.

Since Revenera’s built-in reports reflect the experience of scores of software companies, they satisfy most stakeholders. You won’t need SQL skills to build new reports requested by your developers, product managers, marketers, or anyone else.

Revenera also provides User Flow reporting, allowing you to understand the specific paths that users take within your application. This is the only report of its kind, allowing you to visualize the user journey. Not only do you get the benefit of the same drill-down analysis which is featured in other Revenera reports, but this report is available for any tracked feature out-of-the-box.

If you want to capture more custom data or build specific reports to answer new questions, the Usage Intelligence client and reporting framework supports this natively. For example, you might collect specific string data or name-value pairs whenever an event occurs, to capture more detailed user environment or application status parameters.

Revenera also supports integration with third-party frameworks with little additional development. Moreover, our Web Reporting API permits real-time reporting in JSON or graphical formats. All this means it’s easy to integrate usage intelligence with your other business systems, no matter who you bought them from.

In-application Messaging

Getting the right message to the right user at the right time is crucial to building customer adoption, receiving critical feature usage feedback, and promoting continued revenue streams, whether via upgrades or ongoing subscriptions. Often, the best time to communicate with your users is when they’re engaging with your software. For example, you may want to:

Send targeted messages about the value of key features that a user hasn’t tried, and explain how to use them
Request feedback about specific features during beta
Encourage upgrade opportunities with targeted offers

A full software usage analytics solution should include robust in-app messaging for distributed client software. Usage Intelligence does. Our ReachOut feature lets you customize messages to different segments of users, and then deliver them either via an automated pop-up, or in specific message areas built into your product’s UI. It can even present quick surveys and forward users’ answers to your survey framework of choice. With ReachOut, you can:

Precisely target in-app messaging based on geographical location, product details, generic product engagement, specific feature usage, environment information, or other properties you define
Automate delivery and message display, or control delivery manually
Test campaigns before rolling them out to entire segments or user communities
Automatically message all users whenever a relevant upgrade becomes available to them.

Microsoft Application Insights contains none of these features. To build equivalent functionality capable of building effective, contextually relevant messages, you’d first have to match message delivery with usage intelligence data collected from each installation. Revenera ReachOut does that out of the box.

Less Visible Costs

As we’ve seen, choosing Microsoft Application Insights is likely to drive significant development costs related to client instrumentation, telemetry, caching, data integration, and reporting. You avoid these costs with Revenera.

But there are other costs to consider. For example, a Revenera subscription includes free support; Microsoft offers no free support. Revenera subscriptions charge a flat monthly fee based on monthly active users, regardless of their activity. With Microsoft’s basic plan, costs increase proportionally with event data. That’s a disincentive to use Microsoft Application Insights for deep research about customer usage. Microsoft’s Enterprise plan sets data collection limits per node, with cost directly dependent on how many concurrent users send telemetry every hour. Customer usage patterns or increased activity in certain time zones can impact your costs, and may be difficult to predict.

Comparing Solution Effectiveness

Costs are only one side of the equation. The other is effectiveness: what does it take to get actionable, useful answers?

Consider an example. Software developers often need to understand whether specific features are being used, how they’re being used, who is using them, and whether usability problems might be interfering.

Out of the box, Usage Intelligence captures all the information most software companies need to answer questions like these. It can segment and filter granular usage information based on software editions and versions, user hardware and software environments, and any custom-defined property. You can assess the number and duration of attempts to use a feature; related features used or left unused, and many other factors. All this helps you quickly pinpoint problems and opportunities related to new features, so you can precisely target development investments.

Since Microsoft Application Insights offers no comparable out-of-the-box data capture or reporting, software companies must first identify the metrics they want to collect, without the benefit of experience across many software vendors and products. Since Microsoft’s solution is designed for cloud rather than distributed applications, you need to build client services to collect data. Once your data is being collected in an Azure-based database or data warehouse, you must construct SQL reports to perform all the necessary data mining.

Given enough time, resources, and skills, you could do all that – but it might take months or years to start receiving useful analytics. In a fast-moving business environment, that’s an eternity.

Of course, getting analytics up-and-running is only the beginning. When analytics become available, multiple stakeholders quickly recognize its value. They rapidly start asking new questions and requesting new reports. Even more questions arise as products, competitors, and markets evolve.

Many of these questions are anticipated by Revenera’s out-of-the-box data capture and reporting. In contrast, with Microsoft Application Insights, new reports must be custom-built – and if these reports require new source data, costly changes to client instrumentation may be needed.

The Value of a Purpose-Built Solution

To summarize, Microsoft Application Insights excels at measuring the performance of Microsoft Azure cloud applications. You can overcome its shortcomings in tracking distributed Windows, macOS, or Linux applications, but you’ll need to dedicate significant development and extensive ongoing support. Chances are, you’d rather focus those scarce resources on building great customer features and experiences – not on building analytics infrastructure your customers will never see.