key: cord-0949609-3z592r2q authors: Min Lim, Hooi; Abdullah, Adina; Jenn Ng, Chirk; Hai Teo, Chin; Gayatri Valliyappan, Indra; Abdul Hadi, Haireen; Leik Ng, Wei; Muhaimin Noor Azhar, Abdul; Kian Chiew, Thiam; Sun Liew, Chee; Seng Chan, Chee title: Title: Utility and Usability of An Automated COVID-19 Symptom Monitoring System (CoSMoS) in Primary Care During COVID-19 Pandemic: A Qualitative Feasibility Study date: 2021-09-06 journal: Int J Med Inform DOI: 10.1016/j.ijmedinf.2021.104567 sha: ff72ef123bb4f4d081de95a34817f3d65a0eb311 doc_id: 949609 cord_uid: 3z592r2q BACKGROUND: COVID-19 telemonitoring applications have been developed and used in primary care to monitor patients quarantined at home. There is a lack of evidence on the utility and usability of telemonitoring applications from end-users’ perspective. OBJECTIVES: This study aimed to evaluate the feasibility of a COVID-19 symptom monitoring system (CoSMoS) by exploring its utility and usability with end-users. METHODS: This was a qualitative study using in-depth interviews. Patients with suspected COVID-19 infection who used CoSMoS Telegram bot to monitor their COVID-19 symptoms and doctors who conducted the telemonitoring via CoSMoS dashboard were recruited. Universal sampling was used in this study. We stopped the recruitment when data saturation was reached. Patients and doctors shared their experiences using CoSMoS, its utility and usability for COVID-19 symptoms monitoring. Data were coded and analysed using thematic analysis. RESULTS: A total of 11 patients and 4 doctors were recruited into this study. For utility, CoSMoS was useful in providing close monitoring and continuity of care, supporting patients’ decision making, ensuring adherence to reporting, and reducing healthcare workers’ burden during the pandemic. In terms of usability, patients expressed that CoSMoS was convenient and easy to use. The use of the existing social media application for symptom monitoring was acceptable for the patients. The content in the Telegram bot was easy to understand, although revision was needed to keep the content updated. Doctors preferred to integrate CoSMoS into the electronic medical record. CONCLUSION: CoSMoS is feasible and useful to patients and doctors in providing remote monitoring and teleconsultation during the COVID-19 pandemic. The utility and usability evaluation enables the refinement of CoSMoS to be a patient-centred monitoring system. (271 words) Telemonitoring has been used in outpatient settings to improve self-care in patients 55 with chronic diseases (1) . Telemonitoring refers to the transmission of patients' symptoms 56 and physiological data directly to healthcare providers via automated electronic means, web-57 based or phone-based data entry (2). During the COVID-19 pandemic, telemonitoring 58 systems were deployed in several countries to monitor patients with suspected or confirmed 59 COVID-19 infection (3-5). Besides that, teleconsultation, which is defined as synchronous or 60 asynchronous consultation using information and communication technology (6), was also 61 incorporated into the COVID-19 monitoring system to assess patients who required medical 62 consultations. Telemonitoring with teleconsultation has emerged as a valuable tool for remote 63 monitoring and reducing healthcare resources and hospitalisation (7, 8). 64 COVID-19 telemonitoring applications can enhance patient safety at home while 65 reducing the risk of transmission to healthcare workers (9). However, patient engagement and 66 sustainability were reported barriers to implementing COVID-19 monitoring systems (10, 67 11). Normally, digital applications for non-communicable diseases will undergo beta-testing 68 at the initial phase to evaluate the utility and usability and identify changes before 69 implementation (12, 13) . Unfortunately, due to the urgency of a pandemic, digital COVID-19 70 monitoring apps were developed and launched hastily without evaluating the utility and 71 usability of these apps. Digital applications have been proven effective in COVID-19 72 surveillance, but some were reported to have usability issues (14) . 73 Utility is defined as the functionality of the application and how useful it is to the 74 users, while usability is defined as how easy to use and pleasant are functions to the users 75 (15). Experts recommend that patients' acceptance of telemonitoring technology should be 76 assessed before widespread implementation (16, 17) . In addition, end-users involvement in 77 beta testing improves the utility and usability of an app (17). Hence, engaging patients and healthcare providers in the utility and usability evaluation process of a COVID-19 79 telemonitoring system is important before its widespread implementation during a pandemic 80 situation. As the evidence of COVID-19 evolves rapidly, recommendations and guidelines are 82 frequently revised following the latest evidence and pandemic situation. This makes the 83 development of a telemonitoring system challenging; therefore, a rapid assessment of the 84 utility and usability during the development process is critical to ensure the system can be 85 updated and enhanced quickly. Many studies have reported their development and outcomes 86 of the COVID- 19 recruiting and monitoring the patients using CoSMoS were recruited. We stopped the 105 recruitment when data saturation was reached during data collection and analysis (20), when 106 no additional data was found in both patient and doctor interviews. Telegram bot, provides close monitoring to patients using a dashboard and timely 120 teleconsultation. Once enrolled in CoSMoS, patients will be asked to report their symptoms 121 using an interactive Telegram bot daily for 14 days. A 'ping' will be sent at 9 am every day to Figure 1 shows the CoSMoS Telegram bot for patients, while Figure 127 2 shows the CoSMoS dashboard for doctors to monitor the patients. A total of 15 participants (11 patients and 4 doctors) participated in the beta-testing and in-165 depth interviews. All 11 patients completed the 14-day monitoring using CoSMoS. The 166 details of the participants are shown in Table 1 . Table 3 shows the themes and quotes regarding the usability of CoSMoS. The doctors found that the CoSMoS dashboard was easy to use and navigate. The 241 information in the CoSMoS dashboard was adequate to provide a remote monitoring service. However, there was a suggestion to integrate patient data from the electronic medical record 243 (EMR) into the CoSMoS Dashboard to have more comprehensive patient data. The CoSMoS system, which includes an algorithm-driven chatbot application, a 246 dashboard for monitoring and teleconsultation, was found to be feasible and useful by both The CoSMoS system was developed within a short period due to the urgent need, and 299 it was designed as a system independent from the hospital electronic medical record (EMR). 306 The strength of this study was the involvement of both patients and doctors. By exploring the 307 views and experiences from both parties, richer data was captured and allowed for 308 triangulation. The limitations of this study were that we did not interview those patients who 309 refused to use CoSMoS. Their refusal may point to usability and utility issues that could not 310 be explored. The IDIs were conducted via phone calls, which tends to be shorter than a face- We would like to thank the students and ex-graduates from the Faculty of Computer Science 333 and Information Technology, the University of Malaya for developing the CoSMoS system. The impact 355 of telemedicine on patient self-management processes and clinical outcomes for patients with 356 Types I or II Diabetes Mellitus in the United States: A scoping review Chapter 58 -Outpatient Monitoring and Self-359 Heart Failure 360 in the Child and Young Adult Telesurveillance Solution for Home Monitoring Patients With COVID-19 Monitoring and Management of 365 Home-Quarantined Patients With COVID-19 Using a WeChat-Based Telemedicine System: 366 Retrospective Cohort Study Home 368 monitoring for COVID-19 Teleconsultation and Clinical Decision Making: 370 a Systematic Review Telemonitoring for Patients With COVID-19: Recommendations for Design and 373 Implementation The Value of Remote Monitoring for the COVID-375 Use of Telemedicine and Virtual Care for Remote Treatment in 377 Response to COVID-19 Pandemic Rapid 379 Implementation of an Outpatient Covid-19 Monitoring Program. Nejm Catalyst Innovations in 380 Care Delivery A Mobile Web App to Improve Health Screening Uptake in Men (ScreenMen): Utility and Usability Evaluation Study Usability and Utility of a Mobile App to Improve Medication 388 Adherence Among Ambulatory Care Patients in Malaysia: Qualitative Study Adoption 391 of Digital Technologies in Health Care During the COVID-19 Pandemic: Systematic Review 392 of Early Scientific Literature Home telemonitoring in COPD: A systematic review of 397 methodologies and patients' adherence An exploration of usability issues in telecare 399 monitoring systems and possible solutions: a systematic literature review A 402 Web-and App-Based Connected Care Solution for COVID-19 In-and Outpatient Care Qualitative Study and Application Development COVID-19 Outcome Prediction 406 and Monitoring Solution for Military Hospitals in South Korea: Development and Evaluation 407 of an Application Saturation in 409 qualitative research: exploring its conceptualization and operationalization Ministry of Health Malaysia 412 Self-Monitoring System to Reduce Health Care System Burden During the COVID-19 Pandemic in Malaysia: Development and Implementation Study Usability Engineering Efficacy of telemedical interventional management in patients with heart failure (TIM-HF2): a 436 randomised, controlled, parallel-group, unmasked trial Telemonitoring to Manage 438 Chronic Obstructive Pulmonary Disease: Systematic Literature Review Systematic review of home telemonitoring for chronic 441 diseases: the evidence base Long-term follow-up of recovered patients with 443 COVID-19. The Lancet Qualitative study 445 to elicit patients' and primary care physicians' perspectives on the use of a self-management 446 mobile health application for knee osteoarthritis A New Method 448 for Structured Integration of User Needs in Two Health Technology Development Projects: 449 Action Sheets Collaboration and Co-Production of The electronic 454 medical record and COVID-19: Is it up to the challenge? Internet Hospitals Help Prevent and Control 457 the Epidemic of COVID-19 in China: Multicenter User Profiling Study Clinical characteristics and risk factors for severe COVID-19 infections in Malaysia: A 461 nationwide observational study  An automated COVID-19 symptom monitoring application and teleconsultation 346 improve primary care delivery and provide support to patients who quarantined at 347 home.  Utility and usability testing with the end-users would identify the required changes to 349 the COVID-19 monitoring system to ensure patient safety.