key: cord-0058151-wzsdayln
authors: Kato, Takumi
title: Should e-Learning Providers Encourage Users to Set a Target Score?
date: 2020-11-20
journal: Comprehensible Science
DOI: 10.1007/978-3-030-66093-2_27
sha: 2d9ffed15e435ad2e799ee2eac213f29b22247d0
doc_id: 58151
cord_uid: wzsdayln

The need for e-learning continues to increase due to technological innovation and social changes. As service providers, it is extremely important to understand the factors that increase user satisfaction and continue to improve services. It is recognized that it is important to achieve targets to increase the satisfaction of education. However, the negative effect of not having a target has not been sufficiently discussed. Therefore, in this study, based on the results of an online survey of Japanese business persons who take the e-learning Test of English for International Communication (TOEIC), the positive effects of achieving targets and the negative effects of the non-existence of targets were evaluated. The target was to set the TOEIC score and make a judgment in relation to the achievement of the actual test. As a result of an evaluation by the logistic regression model, significant effects were confirmed. Therefore, it is considered that service providers not only improve the quality of education and the environment, but they also encourage users to set concrete targets.

Advances in technology have raised expectations for e-learning, which provides education/training online. It is attractive that users can enjoy high-quality content at a low cost without being bound by time or place. There are service platforms not only for students but also for employees who work in various industries. For example, Coursera provides academic content provided by Stanford University and Google [1] , and Skillshare provides creative content such as animation and photos [2] . In the past, because of the immaturity of technology and content, there was concern about the poor quality of e-learning. However, many technical problems have been resolved, and as the number of e-learning providers has increased, the concern has diminished. A previous study reported that students who used e-learning were more satisfied than students who used the conventional face-to-face learning method [3] .

As a result, of this background, the global e-learning market size is expected to exceed US$200 billion in 2019 and exceed US$375 billion by 2026 [4] . In addition, the occurrence of the coronavirus disease 2019 has forced schools, universities, and companies to conduct remote meetings and work, so the need for e-learning is becoming stronger [5] . In fact, Harvard University is scheduled to reopen in the fall of 2020. However, it is expected that much of the learning will be done remotely [6] .

It can easily be imagined that competition in the e-learning market will become fiercer in the future. Therefore, it is extremely important for e-learning providers to understand the factors that help increase user satisfaction and continue to improve their services. Not only the quality of education and equipment but also the usability of the system peculiar to e-learning has been clarified as a factor that enhances the satisfaction of education [7, 8] . Furthermore, it has been pointed out that the degree of achievement of the target is also an important factor [9] . However, in existing studies, there were many subjective criteria, and it was unclear whether the achievement of the above was possible. Furthermore, the negative effect of not having a target has been lacking in the discussion. Therefore, in this study, not only the positive effects of achieving targets but also the negative effects of the non-existence of targets were evaluated based on the satisfaction levels of Japanese business persons who took up e-learning. The target elearning test was the Test of English for International Communication (TOEIC). The TOEIC is an English language proficiency test developed by the Educational Testing Service for people who speak English as a second or foreign language and has been implemented in 160 countries [10] . In Japan, it is used for university entrance examinations and company personnel evaluations, and 2,456 thousand persons took the test in the 2018 fiscal year [11] . If this study confirms the negative effects of the nonexistence of targets, it is important that service providers not only improve the quality of education and the environment, but they should also encourage users to set concrete targets.

Educational satisfaction is a long-standing academic study theme. Higher education, which is the source of economic and cultural development, is required to meet the needs of ever-changing environmental conditions, and so many related studies have been conducted [12] . Today, the increasing number of new higher education providers is intensifying the competition to attract and retain students. Therefore, it is important to identify the factors that contribute to student satisfaction and improve the educational program and environment [13] . It has been pointed out that satisfaction increases loyalty to service providers [14] , significantly contributes to positive reviews, and is more tolerant of changes in tuition [15] .

A study of British universities reported that the quality of education contributed more than physical facilities [16] . In Malaysia, knowledgeable and responsive faculties, textbooks and tuition fees, student support facilities, and relationships with teaching staff have been extracted [17] . In Turkey, the technical, functional, infrastructure, interaction, and atmosphere of higher education institutions are shown [18] . Others include a positive experience at university [19] , the freedom to influence school policies and procedures, and the intellectual stimulation of peers [20] .

Although it is almost the same in e-learning, perceived usability [7, 8] , interaction [21] , and the prevention of system failure [22] are reported as characteristic factors. In e-learning at American universities, it has been pointed out that not only instructor-student but also student-student dialogue interaction is important [23] . From another perspective, the degree of achievement of learning is considered to influence satisfaction [24] . A study of Australian universities concluded that learning achievement is extremely important [9] . It has been proven that a Korean university also increases students' satisfaction by providing a dashboard that allows them to intuitively check the degree of learning achievement [25] .

We focus on the achievement of targets. In this study [9] , the criteria were subjective because the degree of achievement was evaluated by self-evaluation, and so objective evaluation was emphasized. This is because there is a concern that people who tend to respond positively to the survey will highly evaluate both achievement and satisfaction. Naturally, the reverse is also considered to occur. In addition, the effect of not only achieving the target but also not having targets is insufficiently discussed. Therefore, the hypotheses of this study are as follows:

H1: The probability of satisfaction increases when one achieves the target score in e-learning H2: The probability of satisfaction decreases when there is no specific target score in e-learning To objectively judge whether the target score was achieved, H1 was judged based on the target score and the actual results of the TOEIC in this study. The TOEIC is mainly divided into listening and reading tests and speaking and writing tests. The former is widely spread in Japan, so it was adopted. The test has a maximum score of 990 and is not a pass/fail judgment. Note that the average scores of examinees in each country are 871 for Canada, 798 for Germany, 684 for Russia, 673 for Korea, 520 for Japan, and 478 for Thailand [26] . It is speculated that H2 lacks targets, which makes the purpose of the lecture ambiguous and makes it difficult to recognize the value of the lecture, resulting in low satisfaction.

In this study, the "Oricon Dataset" [27] , provided by Oricon ME Inc. via the IDR Dataset Service of the National Institute of Informatics, was used. As a third-party organization, Oricon regularly conducts satisfaction surveys objectively for people who use the product/service of various industries. Among them, the results of the online survey conducted from May 15th to June 5th, 2017, for those who took the e-learning TOEIC and the TOEIC within the past five years from the time of the survey were analyzed. The providers of e-learning are not specific to one company, but many major Japanese companies. The total sample size was 652.

However, infrequent attributes are difficult to analyze. Hence, respondents were screened on the condition that they were in their 20s to 50s and that their occupation was a company employee. Furthermore, to equalize the number of TOEIC target scores, 60 respondents were randomly sampled in each group: target score 400, target score 600, target score 800, and without a target score. Therefore, 240 people were the subjects of this study. Table 1 shows the distribution of the survey attributes.

The survey items used in this study were (1) satisfaction with e-learning, (2) target achievement, (3) target score, (4) gender, (5) age, (6) marriage, (7) residential area, (8) annual household income, (9) emphasized points in e-learning, and (10) situation of word of mouth (WoM).

The satisfaction score had ten levels, as shown in Table 2 . In total, the mean was 6.871, and the median was seven. Looking at each target score, the group with no target score is the lowest, and the mean of satisfaction increases as the target score increases. In the analysis, those who chose 8-10 were defined as "satisfied." In total, 37.9% of cases are applicable. The target achievement of (2) was judged by the actual TOEIC results asked in the survey. As shown in Table 3 , when you look at the distribution of achievements by target score, the achievement rate is higher for people with a score of between 600 and 800 than for people with a low target score of 400. Those who did not have a target score were set as unachieved.

The reason for using (4)- (10) was to control the effects other than the achievement and not having a target. The emphasized points of e-learning (9) were asked with a multi-answer from a list of ten features in e-learning. As shown in Nos. 22-31 of Table 4 , the features include items shown in previous studies, such as inquiry correspondence, the quality of educational materials, and appropriate tuition fees. In (10) , regarding the situation of WoM concerning the e-learning that respondents took, questions related to eight patterns were asked with multiple answers, as shown in Nos. 32-39 of Table 4 . WoM has a great influence on consumer behavior. Since consumers who use the products/services communicate their impressions frankly, it is easy to gain trust. Therefore, we must perform a precise evaluation.

Then, all the asked data were converted into dummy variables. The representative values of each variable are shown in Table 4 . A logistic regression model was used for the evaluation. No. 1 (satisfaction) in Table 4 was the objective variable, No. 2 (target achievement), and Nos. 3-6 (target score) were explanatory variables. The attribute variables, Nos. 7-39, were input to control for other effects. The variables marked with * were not used in the analysis because they were set as the criteria for the dummy variables. Since there are many variables, the variables were selected by using the stepwise method, and the contribution was confirmed by using the odds ratio of the variables that became significant at the 5% level. In addition, 10-fold cross-validation was performed to confirm the validity of the evaluation. The statistical analysis software R was used for the analysis, and the stepAIC function of the MASS package was applied to the stepwise method. Table 3 . Target score and achievement status.

First, as shown in Table 5 , when the correlation matrix is confirmed, the achievement is 0.419, which is the highest for Satisfactionflg. As a result of the test of no correlation, p-value < 0.000, and there is a correlation between satisfaction and achievement. Conversely, Target_None had the smallest value of -0.253. As a result of the test of no correlation, p-value < 0.000, and a negative correlation was confirmed. Table 6 shows the estimation results of the logistic regression model. Among the significant variables, WoM05 had the largest odds ratio of 5.328. WoM01 also contributed 2.549, but the effect of WoM05 was greater. In other words, it can be understood that people who talk positively in the digital sphere tend to be more satisfied than people who talk in real-time. The variable showing the next highest odds ratio of WoM05 is the achievement of 4.325. Therefore, "H1: the probability of satisfaction increases when one achieves the target score in e-learning" was supported.

The variable with the lowest odds ratio is a WoM08 of 0.130. As for the negative effects, it can be understood that viewing negative digital reviews has a greater impact than conveying negative content. The next lowest variable is 0.236, Feature02. In other words, people who attach importance to the straightforward enrollment procedure rather than the content of e-learning have little imagination after enrollment, so they are unlikely to be satisfied. The odds ratio of Target_None was 0.310, which is a negative effect. Therefore, "H2: the probability of satisfaction decreases when there is no specific target score in e-learning" was supported.

Finally, as shown in Table 7 , as a result of cross-validation, the mean prediction accuracy was 81.3%, and the validity of the model was confirmed.

The limitations of this study are as follows. The first is that we judge the targets and achievements by using an online survey to ask questions only after the TOEIC has been taken. Ideally, it is desirable to ask the target score in advance and check the results, including the evidence after the test. The second is the result of the fact that only elearning exists in relation to the TOEIC in Japan, so there is a limit on the generalization of the conclusion. In the future, it will be necessary to expand target services and countries. Table 6 . Result of the logistic regression model. Table 7 . Result of cross-validation

As the market size of e-learning expands, it is expected that competition will become more intense in the future. As an e-learning provider, it is important to increase user satisfaction, for the purpose of maintaining user retention rates and continuing to acquire new users by spreading positive WoM. To that end, it is extremely important to understand the factors that contribute to satisfaction and continue to improve the service. In academic studies, the improvement of educational satisfaction has been a theme that has been emphasized for a long time. So far, many factors have been shown, such as the quality of education and equipment, the user support system, and the usability of e-learning systems. Among them, it is recognized that achieving the target is very important. However, in existing studies, there were many subjective criteria, and it was unclear whether the achievement of the above was possible. Furthermore, the negative effects of not having a target were lacking in the literature. Therefore, in this study, the effects on satisfaction were evaluated for Japanese business people who took the e-learning TOEIC. As a result, significant effects were confirmed for both the positive effect of achieving the target and the negative effects of not having a target. Therefore, it is considered that service providers not only improve the quality of education and the environment, but they also encourage users to set concrete targets.

The comparison of learning effectiveness between traditional face-to-face learning and e-learning among goal-oriented users

Global Market Insights, eLearning market size worth over $375bn by 2026

The rise of online learning

Harvard will open this fall, but it may be online

Modeling key drivers of e-learning satisfaction among student teachers

Usability, quality, value and elearning continuance decisions

Students' expectations of, and experiences in elearning: Their relation to learning achievements and course satisfaction

The Institute for International Business Communication, About the TOEIC® Program

Satisfaction with higher education: Critical data for student development

Implementation of student satisfaction index model in higher education institutions

Images, satisfaction and antecedents: drivers of student loyalty? A case study of a Norwegian university college

Student satisfaction with Canadian music programmes: the application of the American Customer Satisfaction Model in higher education

Measuring student satisfaction at a UK university

Dimensions driving business student satisfaction in higher education

Assessing quality in higher education: new criteria for evaluating students' satisfaction

Determinants of business student satisfaction and retention in higher education: applying Herzberg's two-factor theory

Student satisfaction with graduate education: dimensionality and assessment in a college education

A study of student satisfaction in a blended e-learning system environment

Kinshuk: Analysing users' satisfaction with e-learning using a negative critical incidents approach

The determinants of students' perceived learning outcomes and satisfaction in university online education: an update

Student perceptions and achievement in a university blended learning strategic initiative

Effects of learning analytics dashboard: analyzing the relations among dashboard utilization, satisfaction, and learning achievement

Informatics Research Data Repository