key: cord-0028096-410x74bf
authors: Kaveladze, Benjamin T; Wasil, Akash R; Bunyi, John B; Ramirez, Veronica; Schueller, Stephen M
title: User Experience, Engagement, and Popularity in Mental Health Apps: Secondary Analysis of App Analytics and Expert App Reviews
date: 2022-01-31
journal: JMIR Hum Factors
DOI: 10.2196/30766
sha: 33fc95ca9516551f5a4076cfb439a6f9db221efe
doc_id: 28096
cord_uid: 410x74bf

BACKGROUND: User experience and engagement are critical elements of mental health apps’ abilities to support users. However, work examining the relationships among user experience, engagement, and popularity has been limited. Understanding how user experience relates to engagement with and popularity of mental health apps can demonstrate the relationship between subjective and objective measures of app use. In turn, this may inform efforts to develop more effective and appealing mental health apps and ensure that they reach wide audiences. OBJECTIVE: We aimed to examine the relationship among subjective measures of user experience, objective measures of popularity, and engagement in mental health apps. METHODS: We conducted a preregistered secondary data analysis in a sample of 56 mental health apps. To measure user experience, we used expert ratings on the Mobile App Rating Scale (MARS) and consumer ratings from the Apple App Store and Google Play. To measure engagement, we acquired estimates of monthly active users (MAU) and user retention. To measure app popularity, we used download count, total app revenue, and MAU again. RESULTS: MARS total score was moderately positively correlated with app-level revenue (Kendall rank [T]=0.30, P=.002), MAU (T=0.39, P<.001), and downloads (T=0.41, P<.001). However, the MARS total score and each of its subscales (Engagement, Functionality, Aesthetics, and Information) showed extremely small correlations with user retention 1, 7, and 30 days after downloading. Furthermore, the total MARS score only correlated with app store rating at T=0.12, which, at P=.20, did not meet our threshold for significance. CONCLUSIONS: More popular mental health apps receive better ratings of user experience than less popular ones. However, user experience does not predict sustained engagement with mental health apps. Thus, mental health app developers and evaluators need to better understand user experience and engagement, as well as to define sustained engagement, what leads to it, and how to create products that achieve it. This understanding might be supported by better collaboration between industry and academic teams to advance a science of engagement.

## `summarise()` has grouped output by 'name'. You can override using the `.groups` argument. summary.parameters_skewness datawizard ## Install package "strengejacke" from GitHub (`devtools::install_github("strengejacke/strengejacke")`) to load a ll sj-packages at once! d <-df_app %>% select(`MARS Mean`,Engagement:Information,credibility,`Average Rating`, `Retention D7`:avg_session_seconds) d_smaller <-d %>% select(-c(credibility,DAU,`Total Time Spent In App`,`Retention D30`, avg_session_seconds,Sessions,ARPU,`Retention D14`)) %>% rename('Monthly Active Users'='MAU','App Store Rating'='Average Rating') sjp.corr(d_smaller,title = 'Kendall Tau Correlations (n=54-56)', decimals=2,corr.method = "kendall",na.deletion = 'pairwise')+ theme(axis.text.x = element_text(angle=15,hjust=0.5)) ## Warning: 'sjp.corr' is deprecated. Please use 'correlation::correlation()' and ## its related plot()-method.

## Computing correlation using kendall-method with pairwise-deletion... Checking for normality of daily values and re-running analyses with median instead of mean 

cor.test(df_app_median$`MARS Mean`,df_app_median$Downloads

## ## Kendall's rank correlation tau ## ## data: df_app_median$`MARS Mean` and df_app_median$Downloads ## z = 4.4331, p-value = 0.00000929 ## alternative hypothesis: true tau is not equal to 0 ## sample estimates: ## tau ## 0.4088712 df_app_median %>% ggplot(aes(`MARS Mean`,log(Downloads)))+ geom_point()+geom_smooth(method = 'lm',se=F)+ ylab('Downloads (log adjusted)')+ ggtitle('MARS Mean Predicting Downloads',"Kendall's Tau= 0.41, p< 0.001,n=56") ## `geom_smooth()` using formula 'y ~ x'