key: cord-0059778-y1q1o2o2
authors: Almaiah, Mohammed Amin; Almomani, Omar; Al-Khasawneh, Ahmad; Althunibat, Ahmad
title: Predicting the Acceptance of Mobile Learning Applications During COVID-19 Using Machine Learning Prediction Algorithms
date: 2021-03-21
journal: Emerging Technologies During the Era of COVID-19 Pandemic
DOI: 10.1007/978-3-030-67716-9_20
sha: 347f6864b7aebf89d7ad9c4153b26fa245a6c66d
doc_id: 59778
cord_uid: y1q1o2o2

The global spread of COVID-19 has motivated many universities to adopt online distance learning systems. Mobile learning applications could play a crucial role during this pandemic. Mobile learning applications are increasing popularity among learners due to their benefits and effectiveness. However, the acceptance of mobile learning system among university students is limited. Therefore, this study seeks to understand the main factors influencing the acceptance of mobile learning applications by proposing a hybrid model by combining the TAM with new constructs of TUT model. Machine learning algorithms were employed to analyze the hypothesized relationships among the constructs in the proposed model. The research findings found that RandomForest and IBK algorithms are the best two algorithms in predicting the main determinants of mobile learning acceptance as comparison with other machine learning algorithms with an accuracy of 81.3%. The results of machine learning predictive algorithms showed that constructs of perceived enjoyment, perceived ease of use, perceived usefulness, effectiveness, efficiency, behavioural intention to use and utilization could predict the acceptance of mobile learning within accuracy rate of 87%. The results of this paper will offer valuable directions for mobile learning designers and developers to better promote mobile learning application utilization in universities.

During the COVID-19 pandemic, several universities across the world have started to resume their lectures and exams through online learning tools such as e-learning and mobile learning applications [1] [2] [3] . In fact, mobile learning has been increasingly regarded as a promising tool to improve students' learning motivation. It provides learning environment in which students acquire information and knowledge from mobile devices [4] [5] [6] [7] . Mobile learning not only offer students online learning space, but also enable them to quick access to learning activities and materials anytime, anywhere and anyhow and create pioneering opportunities for innovative learning [8, 9] . This kind of technology enables students to reach to the knowledge not only from teachers during the classroom, but also through their mobile device, this can develop their learning capability, and thereby achieve meaningful learning [10] [11] [12] [13] . Hence, mobile learning has attracted many researchers' attention and have been introduced into many fields [14] .

The use of mobile learning systems will help in reducing the transmission of COVID-19 between students with ensuring continuation the learning process during this lockdown. Mobile learning applications could play a crucial role during this pandemic because it has several features such as portability, where mobile learning applications can be taken in different locations at home, office and others by using mobile devices [15] [16] [17] instant connectivity, where mobile learning applications can be used to access a variety of information and learning activities anytime and any-where with instant connectivity facility between students and instructors [18, 19] , context sensitivity, where mobile devices can be used to find and gather real or simulated data [2] , interactivity and mobility [20] .

Several studies have further indicated that mobile learning technology will improve students' learning effectiveness and performance [21] [22] [23] [24] . Different from other educational technologies, mobile learning need to be designed carefully, so it is important to identify students' requirements and perceptions before implementing [25] [26] [27] ; otherwise, they will fail. Accordingly, integrating students' perceptions into mobile learning development has become a crucial issue, for such integration can provide universities, designers and developers proper guidance to ensure the successful usage and acceptance of mobile learning in future [28] [29] [30] . Researchers have thus incorporated students' requirements and priorities into the development of mobile learning and examined their influences on acceptance of mobile learning.

However, the question as to whether the embedding of students' requirements' and perceptions will influence students' acceptance of mobile learning at different stages of mobile learning usage that has not been specifically addressed. Several studies have clearly indicated that a successful mobile learning technology should be accepted by students wholeheartedly, otherwise it will fail [31] [32] [33] . Accordingly, investigation into students' acceptance of mobile learning technology has been considered as a critical step for ensuring the success of mobile learning technology in educational environment [34, 35] . More interestingly, this kind of investigation will help designers and developers to optimise the mobile learning system in a more effective manner as well as enables students to the full potential of the mobile learning technology.

Although prior studies have highlighted the importance of mobile learning applications in university settings [10] [11] [12] , there is a limited of knowledge regarding the understanding of the factors affecting the use of mobile learning applications in educational and learning activities among university students. To verify the actual usage of mobile learning applications by university students, this research extends the technology acceptance model (TAM) to examine the students' acceptance of mobile learning applications. This study selected the TAM model for several reasons are: (1) prior studies have confirmed that TAM has been successfully adopted to study the usage and acceptance of many types of educational technologies, (2) the integration of TAM model with TUT model have not been used to study the actual use of mobile learning applications among university students. In accordance with these reasons, this study aims to fill this research gap by integrating the TAM with TUT constructs to determine the main factors affecting the students' acceptance of mobile learning applications at the university level.

TAM model developed by Davis [36] , which it has been widely used to study students' acceptance of educational technologies [37] [38] [39] [40] . In fact, investigating the critical factors behind users' choices of mobile educational technologies has been proven helpful in providing users with more acceptable mobile learning applications, and therefore has been widely regarded as a vital issue [10] [11] [12] . Davis [36] developed the TAM based on five main constructs namely perceived usefulness (PU), perceived ease of use (PEU), attitude toward to use (ATU), behavioural intention (BI) and actual use (AU). PU means that "the degree to which a person believes that using a particular system would enhance his or her job performance", and PEU signifies "the degree to which a person believes that using a particular system would be free from effort," [36] . ATU defined as the degree to which a user holds positive or negative feelings about using a particular technology, and BI defined as the degree to which a user is willing to use a particular technology [36] .

Due to the success TAM model in exploring the user acceptance of technology, many researchers have used the TAM model extensively to clarify the factors that affect students' acceptance of mobile learning. For example, Almaiah et al. [11] used the TAM to identify the factors that influence intention to use mobile learning system among students. Their results showed that the users' acceptance of mobile learning was positively influenced by perceived ease of use, perceived usefulness and seven quality factors. They also revealed that TAM model is a useful instrument for exploring the attitude of learners to accepting mobile learning system. Al-Emran et al. [8] also employed the TAM, aiming to examine students' acceptance of mobile learning. Their results suggested that the students' preference for using mobile learning was most significantly affected by their perception of usefulness and ease of use. Overall, these studies not only signified the extensive use of TAM in examining students' acceptance of mobile learning, but also indicated the lack of scholarly attention to the critical role of important other factors such as perceived enjoyment, predictive effectiveness and predictive efficiency behind students' acceptance of mobile learning applications.

TUT is a new model developed by Ghapanchi and Talaei-Khoei [41] to study the technology acceptance among users by using three main constructs (effectiveness, efficiency and utilization). In this model, utilization of new technology is measured by two main constructs are predictive effectiveness and predictive efficiency. Effectiveness defined as getting the right things done [41] . Predictive effectiveness means the expected effect or impact of the specific technology. On the other hand, efficiency means doing things in the most economical way. Predictive efficiency means the expected output created out of particular amount of input (e.g. cost, time) for the specific technology.

To understand the acceptance of mobile learning applications, it is important to determine whether students are willing to adopt mobile learning apps and which factors influence their decision to use the mobile learning apps. This research proposes a comprehensive model, which incorporates TAM model with adding new constructs related to enjoyment and construct related to TUT model, which have not been examined previously. According to 

To clarify the main determinants of mobile learning acceptance among students, online questionnaires were distributed for both undergraduate and postgraduate students, who they play a key role in actual use of mobile learning systems at five universities in Jordan. The employ of online questionnaire in this study, specifically in Corona virus time is considered the best method to collect the data. In addition, previous studies pointed out that it is an effective method to measure the hypotheses in the proposed model [4] . These universities have already developed mobile learning systems in their settings. Using online survey questionnaire, students were invited to participate in this study through online classes, during the second semester 2020. In total, 487 online questionnaires were distributed, with 397 questionnaires being returned, indicating an 81.52% response rate. Most of responses had incomplete or invalid answers and therefore were excluded. Hence, 

The items and scales for testing the constructs in the developed model were adopted from current research in the literature. A 5 point scale similar to Likert model was utilized for testing every item, ranging from "strongly disagree = 1" to "strongly agree = 5". We invited six university lectures, each of them holding significant expertise in the mobile learning felid, to examine the appropriateness and clarify of the questionnaire. After that, pre-tested was carried out with 25 post-graduate students from University of Jordan, with the results indicating that the instructions and questions were completely understood. The survey questionnaire consists of seven constructs (perceived ease of use, perceived usefulness, perceived enjoyment, behavioural intention to use, effectiveness, efficiency and utilization) and includes demographic information (e.g., gender and age). The items for measuring perceived ease of use, perceived usefulness, behavioural intention to use, were developed from the measurements used by Almaiah and Al-Khasawneh [35] . The measurement items for perceived enjoyment was drawn from Al-Shihi et al. [42] . Effectiveness, efficiency and utilization were adapted from Ghapanchi and Talaei-Khoei [41] .

Machine learning prediction is an analysis technique that is used to predict future events based on current and historical data. For the context of this study, machine learning prediction algorithms could predict the main determinants of mobile learning applications acceptance. Machine learning techniques have become popular among researchers and analysts as it helps them to understand more about any system and it is easy to develop the prediction model [43] . Among the most popular machine learning techniques used in prediction are rule-learner (PART), meta-classifier (Bagging), decision-tree (RandomForest), lazy-classifier (IBk), logistic regression classifier (SMO) and bayesian classifier (NaiveBayes) [44, 45] . In this research, machine learning classification approaches were used to analyze the proposed model based on the distribution of the class scales with regard to predictor features. Thus, five machine-learning classifiers were employed to predict the acceptance of mobile learning applications based on seven constructs of perceived ease of use, perceived usefulness, perceived enjoyment, effectiveness, efficiency, behavioural intention to use and utilization. The methodology used in this research was adopted from a study conducted by Al-Maroof et al. [44] for studying the acceptance of WhatsApp stickers among students.

In this paper, we have used two primary methods to analyze the data and evaluate the developed model of this research. The first method is the confirmatory factor analysis (CFA) in order to evaluate the measurement model in terms of reliability, convergent validity, and discriminant validity. In the second method, machine learning classification techniques were applied to test the hypotheses in the proposed model. In this research, the methodology used to analyze the data was adopted from a study conducted by Al-Maroof et al. [44] for analysing the acceptance of WhatsApp stickers among students.

(1) Reliability Analysis

The Cronbach's alpha coefficient was applied to determine the reliability of measures for each construct in the proposed research model. As presented in Table 1 , the value of this coefficient ranged between 0.795 and 0.934, exceeding the critical value of 0.7 as suggested by Christmann and Van Aelst [46] , and indicating satisfactory reliability for all constructs in the proposed research model.

For the current study, each construct was assessed in terms of its convergent and discriminant validity. For convergent validity analysis, Table 1 shows that the average variance extracted (AVE) was above (0.5). According to Hair et al. [47] , specify that a variance greater than 0.5 is acceptable. Therefore, the convergent 

This research employed machine learning classification techniques to test the relationships among the constructs in the proposed research model. This study used Weka (version 3.8.3) to analyze the collected data by applying the percentage split (66%) test mode based on using five machine learning techniques including, a rule-learner (PART), a meta-classifier (Bagging), a decision-tree (RandomForest), a lazy-classifier (IBk), a logistic regression classifier (SMO) and a bayesian classifier (NaiveBayes) [44, 45] . Based on Table 3 , the findings showed that IBK algorithm have a highest score in predicting the perceived enjoyment (PEJ) by the construct of perceived ease of use (PEU) in accuracy rate of 83.75% as comparison with other machine learning algorithms. The IBK algorithm also have a high performance score in terms of True Positive (TP = 0.836), precision (0.825) and ROC area (0.942). Thus, this result support H1.

According to the results in Table 4 , which indicated that both IBK and RamdomForest algorithms had the best results in predicting behavioral intention to use (BI) by the constructs of perceived ease of use (PEU), perceived usefulness (PU) and perceived enjoyment (PEJ). This means that both classifiers predict the perceived enjoyment at the highest accuracy scores as comparison with other classifiers (IBK = 86.72% and RamdomForest = 85.17%). In addition, Further, both algorithms have a better performance in precision (IBK = 0.827 and RamdomForest = 0.809) and TP rate (IBK = 0.867 and RamdomForest = 0.851). These results imply that hypotheses H3, H4 and H5 were supported. The results in Table 5 revealed that RandomForest algorithm had the best performance in predicting the perceived usefulness by the construct of perceived ease of use than other algorithms at 84.51% of accuracy. In addition, the results showed that RandomForest algorithm had a better performance in terms of precision (0.831) and TP rate (0.845). Thus, hypothesis H2 was supported.

The findings in Table 6 presented that NaiveBayes algorithm had the highest score of accuracy (85.72%) in predicting the effectiveness by the attribute of perceived enjoyment as comparison with other algorithms. In addition, it had a better performance in precision (0.874) and TP rate (0.875). Thus, hypothesis H6 was supported. Furthermore, the results indicated that NaiveBayes algorithm also had a better performance in predicting the efficiency by the attribute of perceived enjoyment with an accuracy rate (85.66%) as shown in Table 7 . Finally, the findings in Table 8 showed that both algorithms of IBK and RamdomForest had the best performance in predicting the utilization by the constructs of behavioural intention to use, effectiveness and efficiency with an accuracy rate of 87.06% as comparison with other machine learning algorithms. The IBK and RamdomForest algorithms also had a high performance score in terms of True Positive (TP = 0.870), precision (0.863) and ROC area (0.964). Thus, these results support H8, H9 and H10. In fact, COVID-19 pandemic has affected many universities over the world. Specifically, this pandemic has changed the form of education process form face to face to online distance learning. Online distance learning is aimed to minimize the community transmission of COVID-19, which can rapidly spread in densely populated places such as universities and schools. This transition has motivated many universities to adopt many types of online learning systems in order to ensure continuation the learning process during COVID-19 pandemic. One of these tools, which known as mobile learning applications. Mobile learning applications can play a significant role during this pandemic, it aims to help instructors, and universities facilitate student learning during periods of universities closure [48] . Besides, most of these applications in mobile devices are free which can help ensure continuous learning during Coronavirus pandemic [1] . Therefore, this study was empirically applied with the purpose of identifying and understanding the main factors influencing the students' acceptance of mobile learning applications. To achieve this objective, this study proposes a predictive model by integrating the TAM model with the TUT constructs to understand the main determinants influencing the students' decisions to accept mobile learning applications. In order to test the proposed research model, machine learning algorithms were applied using five classifiers are rule-learner (PART), a meta-classifier (Bagging), a decision-tree (RandomForest), a lazy-classifier (IBk), a logistic regression classifier (SMO) and a bayesian classifier (NaiveBayes) by using Weka version 3.8.3. The results of machine learning predictive algorithms showed that constructs of perceived ease of use, perceived usefulness, perceived enjoyment, effectiveness, efficiency, behavioural intention to use and utilization could predict the acceptance of mobile learning applications within accuracy rate of 87%. The results also found that RandomForest and IBK algorithms are the best two algorithms in predicting the main determinants of mobile learning acceptance as comparison with other machine learning algorithms with an accuracy of 81.3%.

The findings of this research offer both theoretical and practical implications. First, this study is among of the few studies that seek to understand the acceptance of mobile learning applications among university students in one of the developing countries like Jordan during COVID-19 pandemic. Second, this study proposes a predictive model by combining TAM with constructs of TUT model for predicting the essential determinants of mobile learning acceptance among students. Third, this research employs a powerful method for testing the proposed research model and hypotheses by using machine learning algorithms. This novel method was rarely employed in mobile learning acceptance literature, and thereby, it is confirmed that this technique will add an important contribution to the literature of mobile learning. Finally, the findings of this research can offer for both designers and developers of mobile learning applications important recommendations to better promote mobile learning application utilization in universities during COVID-19 pandemic.

Exploring the critical challenges and factors influencing the E-learning system usage during COVID-19 pandemic

Students' attitudes towards the use of mobile technologies in e-Evaluation

Analysis of the essential factors affecting of intention to use of mobile learning applications: a comparison between universities adopters and non-adopters

Development of M-learning application based on knowledge management processes

Acceptance and usage of a mobile information system services in University of

Analysis of Twitter data using evolutionary clustering during the COVID-19 pandemic

The development and initial tests for the psychometric properties of the COVID-19 Phobia Scale (C19P-S)

An empirical examination of continuous intention to use m-learning: an integrated model

Investigating students' perceptions on mobile learning services

Investigating attitudes towards the use of mobile learning in higher education

Extending the TAM to examine the effects of quality features on mobile learning acceptance

Determining the factors influencing students' intention to use m-learning in Jordan higher education

Empirical investigation to explore factors that achieve high quality of mobile learning system based on students' perspectives

A examination of factors influencing the use of mobile learning system: an empirical study

An analysis of mobile learning acceptance by college students

Preliminary study for exploring the major problems and activities of mobile learning system

Academics' awareness towards mobile learning in Oman

Applying the UTAUT model to explain the students' acceptance of Mobile learning system in higher education

Examining the effect of knowledge management factors on mobile learning adoption through the use of Importance-Performance Map Analysis (IPMA)

A conceptual framework for determining the success factors of e-learning system implementation using Delphi technique

Impact of mobile technology on student attitudes, engagement, and learning

Testing the novelty effect of an m-learning tool on internalization and achievement: a self-determination theory approach

Technology acceptance model in m-learning context: a systematic review

Analysis the effect of different factors on the development of Mobile learning applications at different stages of usage

Essential factors for the application of education information system using mobile learning: a case study of students of the university of technology

Proposing a new technical quality requirements for mobile learning applications

Malay language mobile learning system (MLMLS) using NFC technology

Advantages and disadvantages of M-learning in current education

Towards a conceptual model for examining the impact of knowledge management factors on mobile learning acceptance

Mobile government adoption model based on combining GAM and UTAUT to explain factors according to adoption of mobile government services

A review of experimental mobile learning research in 2010-2016 based on the activity theory framework

Analysis of the effect of course design, course content support, course assessment and instructor characteristics on the actual use of E-learning system

Social media applications affecting students' academic performance: a model developed for sustainability in higher education

Habit and hedonic motivation are the strongest influences in mobile learning behaviours among higher education students in Malaysia

Investigating the main determinants of mobile cloud computing adoption in university campus

Perceived usefulness, perceived ease of use, and user acceptance of information technology

An empirical investigation of computer simulation technology acceptance to explore the factors that affect user intention

Factors affecting the E-learning acceptance: a case study from UAE

Towards a model of quality features for Mobile social networks apps in learning environments: an extended information system success model

Factors influencing the adoption of e-government services among Jordanian citizens

Rethinking technology acceptance: towards a theory of technology utilization

Neural network approach to predict mobile learning acceptance

A hybrid modeling approach for predicting the educational use of mobile cloud computing services in higher education

Examining the acceptance of whatsapp stickers through machine learning algorithms

A novel approach for predicting the adoption of smartwatches using machine learning algorithms

Robust estimation of Cronbach's alpha

Multivariate data analysis, 7th edn

Mobile government adoption model based on combining GAM and UTAUT to explain factors according to adoption of mobile government services