Effectiveness of the CoronaCheck Mobile Health Application: An Analytical Cross-sectional Study in Lower–Middle-Income Countries

Introduction

The COVID-19 pandemic has given a new impetus to the adoption of digital health solutions by transforming the landscape of healthcare delivery across the globe [1]. The 21st century has witnessed successive waves of crises, encompassing natural disasters, conflicts, and widespread infectious diseases. These occurrences underscore the critical imperative to strengthen health system strategies, with a particular emphasis on preparedness and response efforts, both at local and global levels. This becomes especially relevant, considering the global spread of the novel coronavirus (COVID-19) in 2020, which has led to over 6,057,853 confirmed cases and more than 371,166 deaths [2] and has undoubtedly resulted in widespread, devastating social and economic impacts. The World Health Organization (WHO) has emphasized that the response to fighting COVID-19 should be underpinned by a comprehensive approach, which prioritizes basic health services, trained staff, and appropriate contact tracing and testing mechanisms, regardless of the virus transmission phase [3]. Beyond this pandemic, according to a United Nations (UN) estimate, up to 5 billion people will not have access to basic health services by 2030, including the ability to see a health worker, which necessitates the importance of alternative service delivery models, particularly for the most vulnerable [4].

It is essential to build established mechanisms for ensuring access to care for vulnerable groups, whose needs are further exacerbated in times of crisis, including displaced populations, and women and girls [5]. Among those who are the most underserviced in crisis and noncrisis settings include refugees and internally displaced populations (IDPs). Living in constrained spaces, including camps and urban slums [6], internally displaced and refugee populations tend to suffer poorer health outcomes and experience a higher risk of mortality [7]. Individuals residing in refugee camps face heightened barriers to care as they tend to be located remotely with poor access and limited power supply [7]. The design of these camps with high mobility and unclear health provision systems further complicates the delivery of long-term sustained care [8]. The annual report of the Internal Displacement Monitoring Centre indicated that COVID-19 could add further risks to refugee and internally displaced populations, as overcrowded conditions make it difficult to implement physical distancing and hygiene measures required to prevent the spread of the virus. A broad review of evidence in humanitarian crisis settings indicates there are several gaps in health interventions in IDP settings. As a result, prevention approaches, such as screening, surveillance, education, and immunization are most effective in managing the health and well-being of refugees and IDPs [9]. Achieving optimal intervention delivery requires innovative strategies, which include a combination of programming, the use of pre-existing structures, and a focus on strengthening of health systems [9]. This applies to the current crisis, whereby practical recommendations to address COVID-19 concerns in refugee camps discuss the importance of individual health screening to identify high-risk individuals requiring immediate referral, quarantine, or self-isolation measures [10].

In addition to IDP and refugee populations, it is well known that women and girls in low- and middle-income countries (LMICs) face challenges in accessing care due to security constraints, cultural norms, and patriarchal systems in which they are not permitted or comfortable traveling without a male attendant. Given their limited control over household finances, lower levels of mobility, and decision-making due to socio-cultural norms, women and girls are at greater risk of experiencing health inequities. Evidence suggests that these pre-existing gender and intersectional inequalities often worsen during a crisis, including public health emergencies such as the COVID-19 pandemic, placing women and girls particularly at risk. In fact, the current pandemic has amplified the risk of intimate partner violence experienced predominantly by women and girls [11]. According to UN Women, reports of domestic violence have increased, suggesting an intensified threat for women and girls in countries practicing physical isolation or lock-down procedures [12].

In the context of LMICs, digital health has repeatedly proven to play a promising role in optimizing the level of care available and accessible to vulnerable populations in a timely and cost-effective manner [13, 14]. It has the potential to promote behavior change among communities while also enhancing the capacity of healthcare providers to offer services and an opportunity to receive education and training [13–15]. Despite financial constraints, digital technologies have demonstrated unique potential in reaching typically inaccessible groups, including refugee and internally displaced populations. Research has indicated that refugee populations spend up to a third of their disposable income on connectivity and that connectivity is often their primary request upon reaching a camp, in contrast to food, water, or other supplies [16]. Although connectivity challenges continue to pose a problem in rural areas, 90% of refugees in urban areas are covered by 3G/LTE networks. Though historically there has been a consistent gender-based gap in access to mobile phones and the internet, the number of women with mobile phones in LMICs has risen by over 250 million, with over 80% of women now having access to mobile phones and 48% women using mobile internet [17]. Digital health technology has also proven to serve as an effective tool to provide adolescents and young people with health-related information and to support self-management of illness. Worldwide, people aged 16–24 are almost twice as likely as other age groups to have access to the internet. The features of digital health technologies that particularly appeal to adolescents include the ability to receive information confidentially and anonymously, ongoing availability of information and support, and convenience [18]. The main obstacles associated with using mobile applications more broadly relate to language barriers and low digital literacy [19], which were addressed in the study by developing select applications in local languages, with audio features.

To address the concerns of the spread of COVID-19 within vulnerable populations, CoronaCheck, an artificial intelligence (AI)-based mobile application, has been developed by the Aga Khan University for COVID-19 self-assessment, raising awareness, behavior change, and contact tracing among women, men, girls, and boys as shown in Figure 1. The application leveraged AI in clinical decision support systems and recognized the vital role of timely contact tracing in mitigating the spread of infection. In response to the increased risk of domestic violence incidents during the COVID-19 pandemic, the app also extended support to survivors of gender-based violence (GBV), encompassing the needs of all through the inclusion of relevant resources and providing referrals to service providers and women’s organizations for further support. This support included the dissemination of essential messaging, relevant resources, and referrals to service providers and women’s organizations, ensuring survivors had access to the necessary assistance and support they required. To offset language barriers, the application features risk assessments, awareness videos, and resources in the local language. The risk assessments feature an audio function that can be activated for those who may experience challenges with literacy. Given the potential of digital health to bridge gaps in healthcare coverage, enhance quality, and improve affordability, this study aimed to assess the effectiveness of the CoronaCheck mHealth application among people in LMICs, specifically within Pakistan, Afghanistan, Kenya, and Tajikistan.

Methodology

Results

A total of 1507 participants responded to the survey of which 737 (48.9%) were females, 770 (51.1%) were males, and 506 (33.6%) identified themselves as refugees or migrants. Table 2 shows the characteristics of the study participants.

Figure 2 illustrates the distribution of refugees or migrants and nonrefugees or migrants across four LMICs expressed as n (%). It shows that the proportion of refugees/migrants varied widely by country, with Afghanistan representing the largest refugee population in the sample.

Table 3 shows the characteristics of the study participants by change in knowledge. Notably, a statistically significant difference in knowledge among countries was observed, with a p-value of <0.001. Specifically, Kenya exhibits the highest increase in knowledge (95.6%) followed by Afghanistan (94.2%), Pakistan (82.4%), and Tajikistan (75.8%). Additionally, a substantial increase in knowledge was observed among individuals classified as refugees/migrants (89.9%) with a statistically significant p-value of <0.001. Furthermore, an increase in knowledge was identified among those residing in informal settlements (96.1%) followed by those living in cities (84.4%) and villages (80.1%), all of which were supported by a highly significant p-value of <0.001.

Table 4 presents the characteristics of the study participants categorized by self-reported behavior change. Primarily, it indicates a statistically significant difference between males and females as evidenced by t statistic 3.35 and a highly significant p-value of <0.001. Furthermore, when considering the countries included in our study, there is a statistically significant difference, as demonstrated by an f statistic of 14.12 and a highly significant p-value of <0.001. Notably, Kenya stands out with the highest mean ± SD (12.41 ± 2.20), aligning with the same order observed in the knowledge change sequence. Similarly, a substantial self-reported behavior change was identified among those residing in informal settlements characterized by a mean ± SD of 11.88 ± 2.31, following the same order as the change in knowledge. The finding is supported by f statistic 13.62 and a highly significant p-value of <0.001.

Table 5 provides insights into both the crude and adjusted odds ratios (ORs), along with their corresponding 95% confidence intervals (CIs), for each predictor variable concerning knowledge change. In the univariate analysis, each predictor variable was individually regressed against knowledge change using simple logistic regression. Significantly, there was a noteworthy knowledge change observed in LMICs as evidenced by a likelihood ratio (LR) 𝜒2 of 86.31 and a highly significant p-value of <0.001. Additionally, when comparing specific countries, Afghanistan stands out with knowledge change odds of 5.13 times than that of Tajikistan. Similarly, Pakistan exhibits 1.48 times odds of knowledge change than Tajikistan, and Kenya odds of 6.89 times than Tajikistan.

Moreover, a substantial change in knowledge was seen concerning the participant’s place of residence as supported by an LR 𝜒2 of 41.76 and a highly significant p-value of <0.001. The odds of knowledge change among those residing in informal settlements is 4.59 times than those living in cities. Conversely, the odds of knowledge change among those living in villages is 26% less as compared to those residing in cities.

Furthermore, a significant change in knowledge was observed among refugees/migrants as indicated by an LR 𝜒2 of 14.12 and a p-value of <0.001. The odds of knowledge change among refugees/migrants is 1.85 times as compared to those who were not categorized as such. However, no significant change in knowledge was observed between males and females (LR 𝜒2 of 0.20 and a p-value of 0.6571).

No multicollinearity was detected between predictor variables, and those with p-value <0.25 were considered for the multivariable analysis using multiple logistic regression. The model displayed an LR 𝜒2 of 118.93, signifying a p-value of <0.001. A significant effect modification, also known as interaction, was observed between gender and refugees/migrants with a p-value of <0.1. The odds of knowledge change among male refugees/migrants is 42% less as compared to male nonrefugees/migrants when adjusted for other predictors in the model. The odds of knowledge change among female refugees/migrants is 1.77 times than that of male nonrefugees/migrants and the odds of knowledge change among female nonrefugees/migrants is 1.02 times than that of male nonrefugees/migrants when adjusted for other predictors in the model.

Furthermore, after adjusting for the other variables and interaction in the model, the odds of a change in knowledge in Afghanistan is 6.26 times compared to Tajikistan. Likewise, the odds of a change in knowledge in Pakistan is 1.57 times than in Tajikistan while in Kenya, it is 4.50 times than in Tajikistan. Besides that, the odds of a change in knowledge among those living in informal settlements is 3.18 times than those living in a city and the odds of a change in knowledge among those living in a village is 32% less than those living in a city when adjusted for the other variables and interaction in the model.

Table 6 presents both the crude and adjusted beta coefficients with 95% CI for the self-reported behavior change associated with each predictor variable. In the univariate analysis, each predictor variable was individually assessed for its association with self-reported behavior change using simple linear regression. All predictor variables were statistically significant except for refugees/migrants with f statistic 1.02 and p-value 0.312. Specifically, the predictors that exhibited statistical significance included country ( f statistic 14.12 with p-value <0.001), gender ( f statistic 11.29 with p-value <0.01), and residence ( f statistic 13.62 with p-value <0.001).

As mentioned earlier, there was no indication of multicollinearity among the predictors, and those with p-value <0.25 were deemed eligible for inclusion in the multivariable analysis employing multiple linear regression. The model f statistic was 13.48 with a highly significant p-value of <0.001. With regard to the differences in the mean scores of self-reported behavior change, 1.10 is the difference in mean scores of self-reported behavior change between Afghanistan and Tajikistan whereas 1.76 is the difference in mean scores of self-reported behavior change between Kenya and Tajikistan keeping all other predictors constant. When all other predictors are held constant, 0.63 is the difference in mean scores of self-reported behavior change between males and females. Keeping other predictors constant, −0.86 is the difference in mean scores of self-reported behavior change between those residing in the village and those living in the city while −0.30 is the difference in mean scores of self-reported behavior change between those residing in the informal settlements and those living in the city.

Figure 3 presents a comparison of the future use of the CoronaCheck application by LMICs expressed as n (%). It shows that the users would consider using this application in the future. Specifically, in Kenya, 175 (96.7) of the participants were willing to use this application in the future followed by Afghanistan with 402 (93.9), Tajikistan with 434 (89.7), and Pakistan with 340 (82.1).

Figure 4 illustrates user satisfaction among LMICs expressed as n (%). The data reveals that the users expressed a high level of satisfaction with the application. Notably, Pakistan recorded 246 (91.45) user satisfaction, followed closely by Afghanistan with 240 (88.2), Kenya with 112 (86.8), and Tajikistan with 269 (86.8).

Discussion

The study included a total of 1507 participants who had used the CoronaCheck application over a specified duration of 2 years. Among 1507 participants, 737 (48.91) were females and 770 (51.09) were males. Notably, no significant difference in knowledge change was observed between males and females about COVID-19. These findings are consistent with a multicenter study conducted in India, which similarly reported a lack of statistically significant difference between males and females in terms of their COVID-19-related knowledge (p-value 0.063) [20]. Additionally, our findings also align with a study conducted in Bangladesh that assessed knowledge related to the symptoms and transmission of COVID-19, and similarly found no statistically significant gender differences in knowledge [21]. These consistent findings across similar geographic regions (India and Bangladesh), as well as within our study, collectively suggest that gender may not be a significant determinant when it comes to COVID-19-related knowledge acquisition.

However, a significant mean difference was observed in the scores of self-reported behavior change, encompassing practices such as wearing a mask, maintaining physical distancing, and avoiding public spaces. Our findings are consistent with those from a study conducted among public and private bank workers in Ethiopia, which similarly revealed a substantial and statistically significant difference between males and females in terms of their adherence to face-mask-wearing practices (p-value <0.001) [22]. Moreover, a study on US adults assessing the impact of COVID-19-related knowledge on protective behaviors also showed a significant difference between males and females in the behavior scores (p-value <0.001) [23]. Females were consistently and significantly associated with all these findings, and this concurrence in results underscores the significance of gender as a determining factor in the adoption of preventive measures and highlights the need for tailored public health interventions to address these gender-based disparities in behavior during the COVID-19 pandemic.

Apart from that, 815 (54.08) participants reside in cities, 433 (28.73) in villages, and 259 (17.18) in informal settlements. Primarily, a significant change in knowledge was noted considering the participant’s place of residence. Our findings are in line with the study conducted among undergraduate university students in Bangladesh, which indicated that urban students tend to have higher overall knowledge scores related to COVID-19 compared to their rural counterparts [24]. Furthermore, a statistically significant association was found between self-reported behavior changes and the participant’s place of residence. The results of our study are consistent with the study conducted in China, where the urban setting (compared to rural: OR 0.586, p-value <0.05) demonstrated a significant association with an individual’s willingness to visit a fever clinic for suspected infection [25]. Additionally, as part of assessing attitudes, preventive measures such as reducing outdoor activities and wearing masks were also considered.

Moreover, 506 (33.57) participants were classified as migrants/refugees, whereas 1001 (66.42) were nonmigrants/nonrefugees. A significant difference in change of knowledge related to COVID-19 was observed between those classified as refugees/migrants and those classified as nonrefugees/nonmigrants. The findings of our study are concurrent with the study conducted on Farsi- and Arabic-speaking refugees, where knowledge about COVID-19 was significantly different between refugees and controls (p-value <0.001) [26]. However, no significant difference was observed in self-reported behavior change between refugees/migrants and nonrefugees/nonmigrants in our study. In 2020, the WHO conducted a survey called “ApartTogether survey” to explore refugees and migrants’ self-reported impact of COVID-19. In that survey, participants were asked about their ability and inclination to adhere to various precautions and preventive measures related to COVID-19 alongside their preferred sources of information regarding the pandemic [27]. The survey revealed that a significant portion of participants showed widespread compliance with self-reported behaviors such as increased handwashing (73%), maintaining physical distance (62.8%), covering the nose and mouth (67.9%), and avoiding public transport (50.9%) [27].

An effect modification was discovered between gender and refugees/migrants with a statistically significant p-value of <0.1. This suggests that the effect of gender on knowledge change differs significantly depending on whether individuals are refugees/migrants or not. In other words, gender’s influence on knowledge change varies depending on one’s refugee/migrant status, indicating a complex interplay between these predictors in our study.

This multicountry study had a specific focus on LMICs to evaluate the impact of the CoronaCheck application on knowledge change and self-reported behavior change. However, it is important to acknowledge the potential for biases as the study relies on self-reported data for behavior change and knowledge assessment. This introduces the possibility of bias, where participants may respond to what they believe are socially acceptable rather than their true behaviors or knowledge levels. In recognition of this concern, the confidentiality of study participants was considered the utmost priority to create a comfortable environment for them to engage with the application. Moreover, the assurance of response anonymity played a pivotal role in establishing trust among participants, encouraging them to candidly complete the survey. Despite that, users who declined to provide consent for the study were excluded from participation. This could introduce a bias as those who declined may have different views or experiences related to the application and its impact. Nevertheless, it is important to emphasize that we made efforts to reach out to these participants by sending reminders. The aim was not only to understand how the application benefited them but also to gather valuable insights into how to enhance the overall user experience of the application.

Conclusion

The CoronaCheck application has demonstrated its effectiveness in promoting both knowledge change and self-reported behavior change among people in LMICs. Although numerous mHealth and telehealth applications existed, a notable deficiency was observed in the establishment of robust monitoring and evaluation procedures for most of these implemented projects. Our study, therefore, plays a pivotal role in advancing the field by contributing to the body of evidence necessary for the design and implementation of effective digital health solutions. The user-friendly nature of the application, coupled with its accessibility at no cost, represents an asset in promoting health education and awareness. The findings of this study will not only help us to bridge the existing gap in the digital healthcare literature within Pakistan but also in other LMICs. Furthermore, these findings can serve as a foundation for policy recommendations, guiding the future deployment of mHealth initiatives aimed at improving public health outcomes. In essence, this study not only accentuates the impact of the CoronaCheck application but also underscores its potential to shape the landscape of digital health interventions in LMICs and beyond.

Authors’ contributions

Shaikh, Tehniat: writing-reviewing & editing; Samnani, Saira: conceptualization, methodology, analysis, writing- review and editing; Muqeet, Abdul: project management, supervision, technical writing; Khan, Amna: project management, validation;  Narejo, Usama: conceptualization, writing-original draft;  Sayani, Saleem: conceptualization, methodology, writing-review and editing. All authors participated in manuscript writing and reviewing and gave their approval for the final version.

Funding

This research project was funded by the Aga Khan Foundation Canada (AKFC) and the International Development Research Centre (IDRC), Canada (Grant No. CON000000000949).

Ethics approval and consent to participate

This is briefly mentioned in Section 2.11.

Data availability statement

Source data sharing is not applicable.

Conflict of interests

The authors declare that they have no conflict of interests.

Acknowledgments

The authors of this manuscript would like to express their gratitude for the support received from Aga Khan University (AKU) during the implementation of the project. They greatly appreciate AKU’s efforts in facilitating this research endeavor.

The authors acknowledge the dedication of the implementing agency in ensuring the completion of this study.