This is part seven in a seven-part series on how to use Apteligent to implement a collection of best practices from industry leaders called: “7 Best Practices for Optimizing Mobile Apps.”
Track the Right Metrics
Across our diverse set of customers, a commonality amongst the most successful apps has been a strict focus on continually measuring and monitoring the right data. A focus of this series has been the importance of understanding apps from the user’s perspective using the “Hierarchy of Abandonment” (See Figure 1).
Figure 1: “The Hierarchy of Abandonment”
To measure customer experience, companies have identified a series of metrics for each of these needs:
- Overall Crash Rate < 0.25%
- Crash Rate of Critical Userflows (e.g., “Login”) < 0.25%
- App Load Time < 2 seconds
- Critical Userflows < 2 seconds
- Userflow Dependent APIs: < 1 second latency, < 0.25% error rate
- Monitor Request Volume and Data Sent/Received
However, identifying and measuring these metrics is not enough. It’s important to continually monitor these metrics for changes, especially during key events such as the release of an update to an app, or the release of a new operating system version. In this guide, we will walk through several Apteligent dashboards companies can use to monitor and continually improve each of these metrics.
Use Apteligent’s Release Console to Monitor Releases
One of the most pivotal moments for the success of an app is during a release of a new app or a new version of an app. In interviews with product managers and engineering managers, they stated the importance of monitoring key metrics such as crash rate in real-time during the first 24 to 48 hours after a release.
Apteligent’s Release Console is a dashboard customers use to monitor the success of a release of their app, and make decisions on whether to roll back to a previous release or release a hotfix to fix a bug.
Figure 2: Use Apteligent’s Release Console to track success metrics in real-time
When there’s a new release of an app, Apteligent will automatically determine the app version of latest release of the application as well as the previous release. (Note: the app version of the latest release can also be manually selected by clicking on the settings page.) Use the release console to track the crash rate and adoption of the latest version of an app. Within a few hours of release, the crash rate is a great early indicator on the stability of a release. If the crash rate of the latest release is lower than the previous release, it is a good sign that the release will be successful. Use the app loads and active users metrics on the dashboard to measure the adoption of the latest release. Successful releases will see users quickly adopting the new release of your app.
Use Apteligent’s Crash Trends to Measure Key Metrics Over Time
In addition to closely monitoring new app releases, it’s important to see how key user experience metrics are trending over the course of weeks and months. Many product managers and engineering managers use Apteligent’s “Crash Trends” page to ensure their app’s user experience is improving over time.
Figure 3: Monitor an app’s crash rate across versions
One of the most used graphs on the “Crash Trends” page measures the crash rate (the % of app loads that crashed) of the app broken down by app version. With each release of an app, the crash rate should decrease over time, with a goal of achieving a 0.25% crash rate1.
To properly monitor this crash rate metric, create an alert on Crash Percentage by hovering over the chart and clicking on the alert icon:
Figure 4: Create an alert on crash rate to monitor stability
Use Apteligent’s Userflows to Monitor Critical Flows in An App
Even if an app has a low overall crash rate, if an app fails during a key moment such as “sign up” or “check out”, it can have a big impact on engagement and revenue. Use Apteligent to monitor the success rate of critical userflows across time and app versions.
Figure 5: Use the Userflow Summary dashboard to monitor userflow success rates over time and across app versions
Use the “Userflows Summary” dashboard to trend userflow success rates over time. Filter the dashboard with a 7-day or 30-day view and make sure no critical userflows have a failure rate above 0.25%. Use the app version filter on the dashboard to ensure the latest release of your app has failure rates below 0.25% as well.
Building and maintaining a great mobile app takes a lot of time and effort. One bad release can turn a 5-star app into a 1-star app. Stay on top of bad user experiences by monitoring key metrics that reflect app experience from a user’s perspective. Monitor these app experience metrics during key moments such as an app store release, but also over time and across app versions. Use Apteligent’s Release Console dashboard to ensure healthy app store releases and fix issues quickly. Use the Crash Trends page to understand how your metrics change over time and set alerts on key metrics such as crash rate to stay on top of issues. Finally, monitor the user experience of key flows in your app using the Userflows Summary page.