key: cord-0875967-dicwavje authors: Lu, Chang; Cutumisu, Maria title: Online engagement and performance on formative assessments mediate the relationship between attendance and course performance date: 2022-01-17 journal: Int J Educ Technol High Educ DOI: 10.1186/s41239-021-00307-5 sha: 46f226146f9970fb9537d188944560af3be60a64 doc_id: 875967 cord_uid: dicwavje In traditional school-based learning, attendance was regarded as a proxy for engagement and key indicator for performance. However, few studies have explored the effect of in-class attendance in technology-enhanced courses that are increasingly provided by secondary institutions. This study collected n = 367 undergraduate students’ log files from Moodle and applied learning analytics methods to measure their lecture attendance, online learning activities, and performance on online formative assessments. A baseline and an alternative structural equation models were used to investigate whether online learning engagement and formative assessment mediated the relationship between lecture attendance and course academic outcomes. Results show that lecture attendance does not have a direct effect on academic outcomes, but it promotes performance by leveraging online learning engagement and formative assessment performance. Findings contribute to understanding the impact of in-class attendance on course academic performance and the interplay of in-class and online-learning engagement factors in the context of technology-enhanced courses. This study recommends using a variety of educational technologies to pave multiple pathways to academic success. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s41239-021-00307-5. 1. Does in-class attendance predict academic performance? 2. Does LMS engagement mediate the relationship between attendance and academic performance? 3. Does performance on formative assessments mediate the relationship between attendance and academic performance? Among studies on technology-enhanced learning (TEL), engagement and motivation have been regarded as two essential components (Bedenlier et al., 2020; Lan & Hew, 2020) . Several theories were employed to address the relationship between motivation and learning, such as expectancy-value theory (Eccles & Wigfield, 2002; Wigfield & Eccles, 2000) , goal-orientation theory (Dweck, 1986 (Dweck, , 1989 (Dweck, , 1992 , and self-determination theory (Deci & Ryan, 1985 , 2004 . Expectancy-value theory defines motivation by two main factors: expectancy, which refers to individuals' expectations of success (i.e., the degree to which they believe they will be successful) and value, which refers to individuals' perceived value or interests in completing tasks (Cook & Artino Jr, 2016) . Goalorientation theory is a socio-cognitive theory of motivation that explicates motivation via three types of goals: mastery-oriented goals (i.e., how to master skills), performanceoriented goals (i.e., how to perform better than others and receive positive judgments), and performance-avoidance goals (i.e., how to avoid failure; Cook & Artino Jr, 2016). Self-determination Theory (SDT) is a motivation theory proposed by Deci and Ryan (1985 Ryan ( , 2004 Ryan ( , 2012 . SDT conceptualizes three universal and basic needs of humans for outcomes: autonomy, competency, and relatedness (Deci & Ryan, 1985) . Autonomy refers to a sense of control and agency. Competency shows that one is feeling confident to complete a task. Relatedness is the feeling of included and connected with others. SDT hypothesized that students will be more self-determined when their psychological needs are satisfied, and therefore will be more likely to be motivated and yield positive outcomes among various contexts. In contrast with other motivation theories, SDT treats motivation as a continuum that falls into three categories: intrinsic motivation, extrinsic motivation, and amotivation. Intrinsic motivation is the state of performing tasks out of enjoyment, satisfaction, and self-fulfillment (i.e., self-determined), extrinsic motivation refers to the state of performing tasks out of compliance, external rewards, or punishment (i.e., self-control), and amotivation is the state of lacking motivation (i.e., non-self-determined). According to SDT, more autonomous forms of motivation will lead to higher levels of students' engagement and learning across various educational and cultural contexts (Ryan & Deci, 2020) . Previous studies mainly tested the SDT among traditional school contexts and concluded that students engage more in learning activities and achieve higher performance if the pedagogical practices adequately address the basic psychological needs of students and promote autonomous motivation (Cerasoli & Ford, 2014; Cerasoli et al., 2014; Jang et al., 2012; Reeve, 2002 Reeve, , 2013 Vasconcellos et al., 2020) . Compared with studies situated in traditional school-based contexts, few studies explored SDT and its applications in the technology-enhanced learning (Chiu, 2021a; Ryan & Deci, 2020) . Chen and Jang (2010) used structural equation modeling to examine Deci and Ryan's SDT model among n = 262 learners from two online teaching certificate programs. Findings of their study confirmed the dynamics among contextual support, psychological needs, motivation, and learning outcomes theorized by SDT. More specifically, the effect of need satisfaction mediates the relationship between contextual support and motivation (i.e., self-determination). However, the self-determined motivation failed to predict learning outcome. On the other hand, Hsu et al. (2019) modified Chen and Jang's model by adopting alternative conceptualizations and operationalization of the key variables and re-examined the impact of SDT among n = 330 undergraduate students' academic performance. Results of their study supported that enhanced self-determination motivation facilitated the satisfaction of the SDT basic psychological needs and higher levels of student engagement, which were positively associated with higher perceived knowledge transfer and better course performance. Ryan and Deci (2020) highlighted the promises and importance to study SDT in technology-enhanced education. The main challenges for future SDT research include how to retain students' attention and creating more engagement for learning tasks to enhance motivation through various learning technologies. As a key component of SDT, engagement refers to individuals' levels of endeavor and involvement in their own learning (Fredricks et al., 2004; Wang et al., 2017) . Engagement can be categorized into four dimensions: emotional, behavioral, cognitive, and social dimensions Wang et al., 2016 Wang et al., , 2019 . The emotional dimension refers to individuals' positive or negative mental states when confronted with peers, teachers, learning, and feedback. The cognitive dimension includes individuals' cognitive skills exerted on learning including thinking, applying, connecting, understanding, and reflecting. The behavioral dimension refers to taking actions such as participating in class, concentrating, and making efforts to learn . Lastly, the social dimension focuses on the interaction or collaboration with peers. Student engagement is often operationalized by indicators such as school attendance, activity participation, and social interactions in traditional schools , whereas TEL often conceptualizes engagement as time invested on learning environments (Henrie et al., 2018) , online interactions with different modules, peers, or teachers (Hung & Crooks, 2009; Pellas, 2014) , postings and discussions on the forums (Broadbent & Poon, 2015) , or self-assessment (Kibble, 2007; Zacharis, 2015) . Empirical evidence shows that active student engagement substantially promotes academic performance (Fredricks et al., 2004 Wang et al., 2017) . In contrast, disengagement is defined Lu and Cutumisu Int J Educ Technol High Educ (2022) 19:2 by absenteeism, withdrawal, school dropout, and low interactions with the online learning environments, which are often associated with low academic performance (Skinner et al., 2009; Wang et al., 2016) . The present study probes the impact of students' in-class engagement (i.e., class attendance as proxy) on course performance. Further, this study examines whether and how educational technology potentially mediates the effect of traditional in-class engagement on academic performance. The rest of the paper is organized as follows. First, empirical studies are reviewed on the impact of class attendance and online engagement on academic performance, which align with our theoretical framework. Other confounding predictor for performance is also discussed. Second, the research questions and hypotheses are outlined, followed by the proposed learning analytical methods. Then, the results and discussion are presented. Lastly, the conclusion, educational implication, and recommendations are provided. The relationship between attendance and academic performance in higher education has been explored extensively for decades. The debate on whether it is necessary to require mandatory attendance in secondary institutions has been going on concurrently. Most previous studies found a positive relationship between attendance and academic performance (Devadoss & Foltz, 1996; Kirby & McElroy, 2003) . Romer (1993) first found a significant positive correlation between performance and attendance based on an analysis of n = 195 undergraduate students' attendance and course performance and advocated for mandatory attendance in school to promote performance. Durden and Ellis (1995) then defined attendance as a proxy for motivation. They collected n = 346 students' selfreported absence records, examined the relationship between attendance and academic achievement in an economics course, and found that absenteeism led to poor academic performance. Credé et al. (2010) later conducted a systematic review on 69 empirical studies and found that lecture attendance was a significant medium-strong predictor of academic performance, before and after controlling for other potential confounding variables, such as student age, gender, grade, SAT score, IQ, hours of employment, and motivation levels. More recently, similar findings were also presented by studies across different subjects with varying effect sizes (Hollett et al., 2020; Louis et al., 2016) . For example, Landin and Pérez (2015) recruited four cohorts of university students from a pharmacy course and correlated their attendance with performance separately. Positive correlations were observed across all four cohorts, suggesting a positive effect of attendance on performance. Andrietti (2014) also analyzed longitudinal data from undergraduate students enrolled in an introductory macroeconomic course across the academic year to evaluate the relationship between lecture attendance and academic performance using proxy variable regressions. Findings revealed that attendance had a moderate positive impact on performance, although the effect disappeared after introducing time-invariant variables. This suggests that unobservable mechanisms such as students' characteristics or motivation may interact with the relationship between attendance and performance. Similarly, Krohn and O'Connor (2005) observed students in three undergraduate macroeconomics courses and found a positive significant effect of attendance. However, the relationship became non-significant when instrumental variable techniques were applied to analyze the data collected during the term. No relationship as well as minimal or conditional relationships between attendance and performance have increasingly been found in recent studies (Andrietti & Velasco, 2015; Büchele, 2020; Choi-Lundberg et al., 2020; van Walbeek, 2004) and were attributed to two main reasons. First, with the world-wide digitalization of education, students no longer must attend classes to gain access to course materials, so attendance is not vital for achievement (Büchele, 2020). Second, unlike previous studies that only correlate performance with attendance, more studies seek to address the endogenous bias of attendance by controlling confounding variables, such as student characteristics and motivations (Choi-Lundberg et al., 2020) , or introducing mediators that are related to engagement, such as task engagement, tutorial engagement, and metacognition regulation (Büchele, 2020). On the other hand, Schneider and Preckel (2017) argued that the effect of attendance on learning outcomes has remained significant and withstood the great advance of learning technologies over the years. They conducted a systematic review of 38 metaanalyses to investigate the variables associated with achievement in higher education. Class attendance (d = .98, ranked 6) ranked the sixth most significant predictor for academic achievement among all the 105 variables examined, and ranked the most significant predictor within student variable category. In addition, their study revealed that online courses and blended courses does not seem to mitigate the importance of class attendance for academic achievement. However, they argued that it is still too early to draw conclusions on mandatory attendance policies before the mechanism underlying class attendance has been fully understood when information and educational technology overtake the field of education. Undoubtedly, attendance has been proved to impact performance. However, there are still some unresolved issues that remain to be further studied on this topic. First, most previous studies adopted self-reported attendance records as predictors of academic achievement, in which researchers requested participants to recall their attendance rate at the end of the semester. The self-reported attendance rate introduces measurement bias. Second, attendance is an endogenous factor for learning, with highly motivated and high-achieving students being more likely to attend lectures regularly and engage in the class contents, and thus, achieve higher course performance (Andrietti, 2014) . Although some studies attempted to control student-level variables to mitigate the upward endogeneity error of attendance, few incorporated the instruction-level variables, such as in-class activity engagement, peer or teacher interaction, or performance on formative assessments. The potential measurement error and endogeneity bias may severely attenuate the validity of the conclusions presented in the related research. Based on the TEL engagement, in-class attendance serves as an indicator of traditional school engagement. Concomitantly, online engagement is indicated by self-regulated online learning activities and performance on online formative assessments. Both traditional and online engagement may be essential determinants of academic success. In the era of TEL-based education, more research needs to be done to understand the dynamics among in-class engagement, online engagement, and academic performance. Moreover, the potential mediating effects of TEL engagement indicators upon the relationship between in-class engagement and academic performance are also underexplored. Technology-enhanced learning has become a major trend in education, especially in today's climate of the COVID-10 pandemic. TEL transforms the conditions of engagement learning from traditional classroom-based to blended and, currently, to fully online through various digital technologies (López-Pérez et al., 2011; Nouri et al., 2016) . A substantial body of literature has investigated the relationships between student online self-regulated learning and self-assessment with academic performance using data extracted from LMSs (e.g., Hung & Crooks, 2009; Shi et al., 2015) . Most previous studies have concluded consistent results with traditional schooling contexts that higher levels of TEL engagement could facilitate academic success (Hung & Crooks, 2009; Kibble, 2007; Zacharis, 2015) . However, few have investigated the associations among traditional engagement indicators such as attendance, TEL engagement indicators such as self-regulated learning and self-assessment, and academic performance. As an essential TEL engagement indicator, students' online self-regulated learning plays an increasingly important role in the formal contexts of higher education, for LMSs such as Canvas, D2LBrightspace, Moodle, and Sakai have been regarded as critical digital tools that assists faculty members in delivering poly-synchronous materials, lectures, and assessments (Gautreau, 2011; Washington, 2019) . Some of the studies have been done to evaluate the relationship between attendance, online learning engagement, and performance in online learning in higher education (Bekkering & Ward, 2020; Doggrell, 2020; Nieuwoudt, 2020) . Doggrell (2020) inspected the associations between lecture attendance, lecture recordings access, and academic achievements on n = 117 medical students sampled from two sessions of medical laboratory science courses. They found that, with the availability of lecture recording, there is no significant correlation between lecture attendance and academic achievement. They suggested that using a mixture of multimedia educational technologies is likely to ensure higher academic success. Online formative assessment is another important indicator of TEL engagement that predicts performance (Gikandi et al., 2011; Spector et al., 2016) . Educators need to consider formative practices and optimally integrate them into their teaching and assessments. Online formative assessment also provides learners with self-evaluation and feedback to help them orient and adapt their own self-regulated learning (Zimmerman, 2002) . Gikandi et al. (2011) conducted a review of literature on 19 empirical studies about online formative assessment in the context of online learning in higher education. They found that online formative assessment effectively promoted learner engagement and learner community development. Other studies also confirm the constructive, beneficial effect of formative assessment on learning outcomes (Rakoczy et al., 2019; Robinson & Udall, 2006) . With the fast development of the areas of educational data mining (EDM) and learning analytics (LA), a great number of studies emerged using EDM and LA to measure online engagement and learning by analyzing web-based log event data generated during the LMS usage recording the users' activities, IP address, date, and time sequence (Aldowah et al., 2019; Dutt et al., 2017; Papamitsiou & Economides, 2014; Romero & Ventura, 2010; Romero et al., 2008) . Common practices of EDM and LA include applying feature-engineering techniques to extract engagement indicators, such as analyzing the text posted on online forums (Larsen et al., 2008) or counting individuals' click frequencies and total time spent in different LMS sessions throughout a course (Geigle & Zhai, 2017; Zacharis, 2015) . Students' online learning engagement can be objectively reflected by their actual web usage on the LMS. Most studies conducted using EDM/ LA approaches reported that higher levels of self-regulated learning are positively correlated with academic performance (Geigle & Zhai, 2017; Hung & Crooks, 2009; Zacharis, 2015) . However, few studies have explored the impact of online learning engagement as captured by features extracted from log data on the relationship between in-class lecture attendance and academic performance. Prior knowledge is constantly regarded as a significant student characteristic to predict performance in TEL education (Asarta & Schmidt, 2017; Kinsella et al., 2017; Schneider & Preckel, 2017; Song et al., 2016; Spires & Donley, 1998; Tobias, 1994) . Prior knowledge is defined as the information or experiences that a learner already established regarding a new topic either taught from learning or drawn from experiences (Tobias, 1994) . Previous studies commonly found that prior knowledge is positively related with academic performance through the facilitation of higher levels of motivation, engagement, and self-regulation (e.g., Schneider & Preckel, 2017; Song et al., 2016) . Song et al. (2016) conducted a study to examine the effects of prior knowledge, self-regulation, and motivation on performance via structural equation modeling. They assessed 386 medical clerk students' prior knowledge through multiple choice items and measured their selfreported self-regulation and motivation. A knowledge post-test and a clinical reasoning test were administered as performance measures. Findings revealed both direct and indirect positive correlations of prior knowledge with learning outcome and self-efficacy. Conversely, students with little or no prior knowledge will be disadvantaged when they process and memorize entirely new information. In the worst case, students with false prior knowledge will have to correct and update the false information and reconstruct their knowledge system (Kowalski & Taylor, 2009) . From a systematic review of metaanalyses of variables associated with achievement in higher education, Schneider and Preckel (2017) also found that prior intelligence or prior knowledge is an important predictor for achievement (d = .90, ranked 7 out of 105). Given the fact that prior knowledge is reported to account for a large proportion of variances of learning outcomes (Schneider & Preckel, 2017; Song et al., 2016; Tobias, 1994) , the present study controlled the effect of prior knowledge when examining the relationship among attendance, self-regulated learning, performance of formative assessments, and academic performance to exclude the confounding bias. With the rapid digitalization of education around the world, online learning and formative assessment have become essential components of both formal and informal learning in higher education and the key to academic success. The findings on impact of attendance on Page 9 of 23 Lu and Cutumisu Int J Educ Technol High Educ (2022) 19:2 performance are no longer valid if online learning and online formative assessment are not considered and evaluated. Moreover, most previous studies adopted an instrumental approach, such as the National Survey of Student Engagement (NSSE: Ewell, 2010; Kuh, 2009) , the Australian Survey of Student Engagement (AUSSE: Coates, 2010), or the Utrecht Work Engagement Scale for Students (UWES-S: Carmona-Halty et al., 2019; Seppälä et al., 2009) to measure engagement and other student characteristics. The self-reported scales exhibit inherent measurement errors and may not reflect students' real level of engagement. The advancement of educational data mining (EDM) and learning analytics (LA) methods could provide more insights in TEL contexts. The LMS can be used to record attendance more accurately, compared with the self-reported attendance rate recalled by the students. In addition, researchers can extract students' online learning activities from the automated generated log files in the LMS. The use of LMS also enables instructors to examine the students' prior knowledge, to organize in-class online activities, to administer online formative assessments in and outside the classroom, and to revise their instruction because of the way students interact with the materials. The collection of all the information above through LMS and its inclusion into the analysis of attendance and performance can help minimize the endogeneity and measurement bias mentioned in previous studies. Thus, we propose a learning analytics approach to measure students' lecture attendance and online learning engagement through information extracted from the log file generated by LMS. Additionally, we regard TEL engagement indicators-self-regulated online learning and online formative assessment administered on LMS-as important indicators of academic performance in addition to traditional engagement indicator attendance. We propose a novel method that uses a learning analytic approach to mine the LMS log data to extract event-based variables that record students' in-class attendance, measure their online learning engagement, and collect their performance on online formative assessments. With the measures extracted from the LMS log data, we investigate the associations among attendance, online learning engagement, performance on formative assessment, and course academic performance. This study makes the following hypotheses: • Hypothesis 1: In-class attendance positively predicts the final course score. • Hypothesis 2: LMS engagement positively mediates the relationship between in-class attendance and the final course score. • Hypothesis 3: Performance on formative assessments positively mediates the relationship between in-class attendance and the final course score. Participants were n = 367 Elementary and Secondary Education undergraduate students at a large university from Western Canada enrolled in three sections of an undergraduate mandatory Educational Assessment course in Winter 2019. Participants were recruited using convenience sampling, as one of the authors was the instructor of the course. Students were required to bring a digital device to class to participate in the interactive classroom activities, keep up with class readings, complete tasks, and participate in class discussions. Class attendance was not mandatory, but highly encouraged to engage participants in classroom discussions, involve them in class activities to facilitate their understanding of complex concepts and intricacies of classroom assessment. There were 13 lectures during the term, and each lecture started with a formative quiz administered on Moodle. On the first day of class, a prior knowledge quiz was administered on Moodle. Then, each lecture started with a quiz testing the material taught in the previous lecture on Moodle. Each quiz was timed, and it was opened at the beginning of each lecture. Students who physically attended were instructed to complete the in-class quiz at the beginning of each class. Attendance was operationalized as the timestamp of the start of the quiz on Moodle. Asynchronous course materials including the syllabus, assignments, lecture notes, handouts, external links, formative quizzes, and a discussion forum were available on Moodle. The instructor also provided weekly in-class lectures interspersed with hands-on individual and group activities. Log data of the three course sections was downloaded by a third party and anonymized before the analyses commenced. After pre-processing the log file, the dataset included the following variables: Student ID (i.e., corresponding to each student), Activity Name (i.e., corresponding to different Moodle modules), Activity Context (i.e., corresponding to specific actions within the Moodle), IP address, and Timestamp in the format of Year-Month-Date and time. The study was approved by the University of Alberta's Human Research Ethics Board (Pro00095249). Students' lecture attendance was measured by the timestamp of the quiz attempt at the beginning of each lecture. More specifically, students were counted as absent if their IP address did not match the IP of the classroom; otherwise, they were counted as present. Students' knowledge of the course material was measured using the scores on the Prior Knowledge quiz administered in the first day of the course, at the beginning of the first lecture. The prior quiz consisted of 11 questions, with each question testing the material of a corresponding lecture during the term. The latent variable students' levels of engagement in the LMS were measured by total click frequencies of different LMS modules accessed, including File (i.e., accessing, viewing, and downloading lecture notes and other course materials), Forum (i.e., posting or viewing the content of the discussion forum), URL (i.e., clicking on an external link posted on the Moodle), and Assignment (i.e., accessing, viewing, or submitting an assignment or viewing feedback on an assignment). Students' online quiz scores were calculated for every lecture. A total of twelve formative quizzes were administered in class, in addition to the Prior Knowledge Quiz. The quizzes were reopened by the instructor after each lecture, so that students could access and review the formative assessments in preparation for the midterm and final exams. The online formative quizzes were only designed for students to practice the material learned in class and they did not count towards their final grade. Students' final scores in the class were collected, ranging from 0 to 100 and included two assignment scores, a midterm exam score, and a final exam score. Data normalizations were performed prior to statistical analyses because the scales of the variables included in the study varied significantly (e.g., the click frequencies of behavior-based variables, attendance frequencies, and the performance measures). All selected variables were normalized to the scale of 0-1. Then, two Structural Equation Model (SEM) were fitted to the dataset. The Baseline Model (M0) or null model testifies the direct effect of Attendance on Academic Performance, whereas all the other structural path coefficients from were fixed to zeros. The Mediation Model (M1) examined the mediating effect of Online Learning Engagement and Formative Assessment Performance on the relationship between Attendance, and Course Academic Performance. The measurement model examined the factor loadings of online learning activities extracted from different LMS modules on the latent factor Online Learning Engagement. The structural model examined whether: (1) Attendance directly predicted Course Academic Performance; and (2) Learning Engagement and Formative Assessment Performance mediated the relationship between Attendance and Course Academic Performance. For both SEMs, we controlled for the Prior Knowledge covariate in the model. The SEM analysis was conducted using the sem function within the lavaan package (Rosseel, 2012) in R. The model was estimated using the Robust Maximum Likelihood. The model fitness was evaluated by the Chi-Square fit index test, Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Comparative Fit Index (CFI), Tucker Lewis index (TLI), Root Mean Square Error of Approximation (RMSEA), and Root Mean Square Residual (RMSR; Hu & Bentler, 1998 , 1999 . Model comparisons between the two SEMs were performed using Akaike Information Criterion (AIC; Akaike, 1974) and Bayesian Information Criterion (BIC; Schwarz, 1978) . The top of Table 1 presents the descriptive statistics of participants' raw course attendance frequency and click frequencies of different LMS modules. Out of the 13 lectures in the Winter 2019 term, participants attended 10.57 lectures on average with an SD of 3.09. Among all the modules, participants were most active in accessing, A summary of the prior knowledge quiz, the twelve formative quizzes, and the course final scores are shown at the bottom of Table 1 . The prior knowledge quiz, the 12 formative quizzes, and the course final score were measured on a scale of 0-100 before variable normalization. The prior knowledge quiz had the lowest average score, whereas the average scores of the 12 quizzes ranged from Mean = 60 (SD = 28.89) to Mean = 84.62 (SD = 24.49). In general, there were large individual variations on the formative assessment scores. Participants' course final score had a mean of 82.02 and a SD of 7.73 (Table 2) . Table 3 presents the bivariate Pearson correlations among the observed variables. Results show that the observed variables File, Forum, URL, and Assignment underlying the latent construct Online Learning Engagement are significantly correlated with each other with small to medium effect sizes, which suggests that the observed LMS variables can load on the single latent variable without multicollinearity. In addition, participants' Formative Assessment Performance is only significantly correlated with URL (r = .18, p < .001), but not significantly correlated with other LMS variables including File (r = .05, p > .05), Forum (r = .10, p > .05), and Assignment (r = .03, p > .05), indicating that the two variables Online Learning Engagement and Formative Assessment Performance represent two distinct constructs relevant to TEL engagement. The in-class engagement indicator Attendance is significantly correlated with the outcome variable Course Academic Performance (r = 0.21, p < .001) and all the TEL engagement indicators except for Page 13 of 23 Lu and Cutumisu Int J Educ Technol High Educ (2022) 19:2 Assignment (r = .007, p > .05). Thus, there are potential interactions among class attendance, online engagement, performance on formative assessment, and course final score. Lastly, the outcome variable Course Academic Performance has significant positive correlations with all the selected predicting variables, which shows that both in-class engagement indicator and online learning/TEL indicators could have positive impacts on academic performance. The results from the Pearson correlations lay the foundations for the following SEM analyses. The model fit indices and the error terms of the two SEMs are presented in Table 3 . When RMSEA < .08, GFI, AGFI, CFI and TLI > .90, and SRMR < .08, the model is regarded as fitting well (Marsh et al., 1988) . Hypothesis 1 In-class attendance positively correlates with the final course score. Therefore, Hypothesis 1 is confirmed that in-class attendance positively correlates with the final course score, when TEL indicators are excluded. The path diagram of the Baseline model is plotted in Fig. 1 in standardized coefficients. LMS engagement and performance on formative assessments positively mediates the relationship between in-class attendance and the final course score. The Mediation model addresses the mediating effect of LMS engagement and formative assessment performance on the relationship between in-class attendance and final course score. Compared with the Baseline model, the total variance explained of Course Final Score by the Mediation model increased to 32% ( R 2 = 0.32, Δ R 2 = 0.26, p < .001). Table 5 presents that after controlling for the effects of Prior Knowledge, Online Learning Engagement, and Formative Assessment Performance, in-class attendance has no significant direct impact on Course Final Score (β = − 0.06, p = .437) in the Mediation model. The non-significant path coefficient indicates that the effect of in-class attendance on final course score is fully mediated by Online Learning Engagement and Formative Assessment Performance. More specifically, the structural model shows that 14% of the variance of Online Learning Engagement ( R 2 = 0.14, p < .001) can be explained by the Attendance, whereas 7% of the variance of Formative Assessment Performance ( R 2 = 0.07, p < .001) can be explained by Attendance. In addition, the structural coefficients revealed that the indirect effect of Attendance on Course Final Course is mediated by both Online Page 16 of 23 Lu and Cutumisu Int J Educ Technol High Educ (2022) 19:2 Learning Engagement (β = (.38)*(.29), p < .001) and Performance on Formative Assessments (β = (.27)*(.46), p < .001). Thus, both Hypotheses 2 & 3 were confirmed. The controlled variable Prior Knowledge also positively predicted Course Final Score (β = 0.14, p < .01). Figure 2 plotted the standardized coefficients of the Mediation model. In terms of the measurement model, the top of Table 5 revealed that all selected LMS modular variables contributed significantly to the variation of the latent factor Online Learning Engagement. More specifically, URL (β = 0.74, p < .001) and File (β = 0.60, p < .001) accounted for the largest variation, suggesting that accessing and viewing external extra-curriculum materials and course lecture notes are the most salient behaviors for online learning engagement levels. Assignment (β = 0.39, p < .001) and Forum (β = 0.29, p < .01) also explained at least 10% of the variance of Online Learning Engagement. The SEM model fit statistics and the factor loadings of the measurement model showed that variables extracted from LMS log data can effectively represent students' levels of online learning engagement. Moreover, behaviors related to accessing intra-and extra-curricular course resources were the most significant indicators of online learning engagement. Attendance and performance are underexplored in the current online education landscape, although their relationship has been discussed extensively for decades. Even fewer studies have examined how learning technology may mediate this relationship by adopting learning analytics tools to collect objective evidence of in-class attendance and online engagement. When online courses and blended courses are ubiquitous across Lu and Cutumisu Int J Educ Technol High Educ (2022) 19:2 formal and informal educational contexts, it is especially important to understand the impact of in-class attendance on learning outcome, and how learning technologies could facilitate efficient learning by creating more engagement opportunities and mitigate the absence of in-class interactions. In the relevant studies, self-reported attendance and engagement measured by various instruments still constitute the most common practices. The present study used log data generated from an LMS to explore the relationship between in-class attendance and academic achievement, and further examine how online learning engagement and formative assessment mediate this relationship in the context of a TEL-based course via learning analytics approach and structural equation modeling. We proposed three hypotheses regarding the potential associations among the selected variables. The findings show that all three hypotheses were confirmed. The Baseline model shows that Attendance is still significantly correlated with Course Final Score, if TEL predictors are excluded. The Mediation model reveals that online learning engagement and formative assessment performance fully mediate the relationship between in-class attendance and academic achievement. The non-significant, negative direct effect of Attendance on Course Final Score implies that in-class attendance fails to positively predict performance. The result is contradictory to many previous studies that examined the relationship between attendance and performance in traditional classroom-based courses (Hidayat et al., 2012; Hollett et al., 2020) . However, we found consistent results in some of the recent studies conducted in TEL settings (Doggrell, 2020; van Walbeek, 2004) . For example, Doggrell (2020) found that there is no direct association between course attendance and academic achievement in medical laboratory science courses when lecture recordings were available. In the 1990s, Durden and Ellis (1995) proposed to use attendance as the proxy of engagement and viewed it as one of the most important indicators to performance in addition to student characteristics, such as aptitude and motivation. More recent research studies provided more supporting evidence from various disciplines (Kirby & McElroy, 2003; Louis et al., 2016; Stegers-Jager et al., 2012) . However, blended education has gained momentum in recent years and online education has become omnipresent in higher education around the world. The present study suggests that it is no longer valid to simply use attendance to represent motivation or engagement. The SDT argues that higher levels of autonomous forms of motivation rather than external punishment and awards are more effective in promoting students' engagement and learning (Ryan & Deci, 2020) . Previous SDT studies situated in traditional classroom-based courses confirmed the importance of autonomous motivation (Cerasoli & Ford, 2014; Jang et al., 2012; Reeve, 2002 Reeve, , 2013 . Results from the current study show that in-class attendance itself does not influence course performance, but that online learning engagement and performance on formative assessments, respectively, fully mediate the relationship between attendance and course performance, after controlling for prior knowledge. Findings of the present study suggest that the new educational technologies reduce the importance of attendance on learning outcomes compared with traditional school contexts. More importantly, educational technologies have been creating more opportunities for engagement. Nonetheless, we found significant positive mediating effects of both Online Learning Engagement and Performance on Online Formative Assessments on the relationship between Attendance and Course Final Score. The results of the bivariate correlations Page 18 of 23 Lu and Cutumisu Int J Educ Technol High Educ (2022) 19:2 indicate that Online Learning Engagement and Performance on Online Formative Assessments are two distinct constructs, which can be regarded as two forms of TEL. All the path coefficients on the two mediation pathways are positive, that is, in-class attendance positively impacts the levels of self-regulated online learning and performance on formative assessments, and the two factors subsequently positively influence students' course final scores. Therefore, students who are more likely to attend the lectures are also more active on learning and self-assessment in LMS. The endogenous nature of attendance is addressed by the mediation effects. The findings reveal similar implications as in Büchele's (2020) study, where the author used the MSLQ (Pintrich et al., 1991) and the tutorial engagement scale (Handelsman et al., 2005) to evaluate the link between lecture attendance and performance mediated by metacognitive regulation, task value, and tutorial engagement in higher education. Büchele (2020) concluded that it does not matter whether students attend the class with respect to their course success. Rather, levels of cognitive and behavioral engagement mediate the relationship between attendance and performance. In the present study, attendance also does not directly correlate with performance, but has a positive effect on performance through online self-regulated learning and formative assessment. Thus, the two mediation paths further confirm that attendance does not serve as the sole important engagement indicator for predicting academic performance. TEL engagement indicators including self-regulated learning and formative assessment fully mediate the relationship between attendance and performance. Researchers and teachers may re-evaluate the behavioral dimensions when online courses take over in higher education. In the current climate of online education, where multimedia technologies are widely developed and used, attendance should not be treated as the only proxy for engagement, as it hardly determines academic performance on its own and its impact on learning outcomes can be compensated by various learning technologies. In the present study, we identified two pathways that bridge attendance to performance: one through online engagement and the other through formative assessment. The two engagement indicators incorporate students' self-regulated learning and assessment in and outside the classroom, which could provide more insights into understanding the interplay of inclass attendance, online learning engagement, and formative assessment in affecting academic performance. The present study extends SDT and engagement theory from traditional school-based contexts to a TEL-based course by adding the elements of Online Learning Engagement and Online Formative Assessment to the behavioral dimension. The mediation effects of the two behavioral engagement indicators on the relationship between attendance and performance are scrutinized to understand how in-class engagement interacts with online engagement and how the two aspects work jointly on performance. Previous studies generally focused only on in-class engagement or only on online engagement to explain individual differences on performance. Findings of the study help fill in the gaps by merging the boundaries of traditional in-class Page 19 of 23 Lu and Cutumisu Int J Educ Technol High Educ (2022) 19:2 engagement and online engagement to entangle the mechanism of engagement in contemporary education, enhanced by various digital technologies. Methodologically, previous studies on attendance and performance generally used students' self-recalled information to collect attendance records and adopted selfreported instruments to measure engagement, such as questionnaires or validated tests. However, self-reported measures tend to yield higher measurement errors. This study applied learning analytics methods to analyze the log file data automatically generated in the LMS. Students' attendance, online learning activities, and performance on formative assessments are well documented in sequences of web-usage log events. Thus, measurement error is greatly reduced. In addition, we performed an SEM analysis, where both observed and latent variables were included, and the interactive effects among variables were estimated simultaneously with minimized error to examine the construct validity of Online Learning Engagement. In online courses, the classroom is not the only medium that involves engagement. Specifically, online learning platforms are also essential media that deliver course materials and provide digital tools for communication and self-evaluation. Thus, SDT and engagement research should dedicate more attention to engagement in online learning contexts. This study highlights the significance and role of online learning engagement and performance on formative assessments in positively influencing academic performance. Moreover, the SEM results suggest that the availability of diversified digital technologies in education provides more ways to foster learning and engagement in and outside the classroom. We provide instructors with the following instructional suggestions to promote student engagement in blended and online courses. First, it is important to enhance efficient student-faculty interaction and foster positive relationships between students and teachers in technology-enhanced learning (Chiu, 2021a; Paulsen & McCormick, 2020; Robinson & Hullinger, 2008) . More specifically, teachers could improve online communication skills so that students could receive high-quality feedback and be emotionally engaged (Chiu, 2021a; Robinson & Hullinger, 2008) . Second, instructors are encouraged to design online activities or tasks that facilitate peer discussions and collaborations in class and in digital learning environments (Dumford & Miller, 2018) . Peer interactions could connect students online and create opportunities to provide and receive feedback from classmates to improve their learning outcomes. Third, active learning activities are beneficial for facilitating student perceived learning and self-directed learning (Gray & DiLoreto, 2016; Robinson & Hullinger, 2008) . Instructors could create a learning community where students can contribute to and be engaged in. In sum, a mixture of educational resources and learning activities is recommended for achieving better course performance in higher education through higher levels of engagement in online environments. Page 20 of 23 Lu and Cutumisu Int J Educ Technol High Educ (2022) 19:2 Participants were recruited using a convenience sampling method, so they were all undergraduate students in the Faculty of Education. Further research will be conducted among various participants (i.e., students from different grade levels and programs of study) and in different contexts (i.e., classroom-based courses, online courses, and blended courses) to better understand the impact of attendance and engagement on academic performance in different types of teaching modes. Also, the present study only controlled Prior Knowledge as the confounding variable in the two proposed models. Other variables such as gender and affective factors may also potentially influence students' learning pathways and outcomes. Future studies could collect more demographic information and affective factors to mitigate the extraneous effect of confounding variables. This study examines the relationship between attendance and performance mediated by online self-regulated learning and formative assessment in a TEL-based course. The SEM results show that online self-regulated learning and performance formative assessment fully mediate the relationship between attendance and performance. Attendance alone is not a vital determinant of performance, but it positively impacts performance by inducing more active online self-regulated learning and better performance on formative assessments. Findings suggest that mandatory attendance is not a panacea for improving poor academic performance. Instead, more autonomous motivation and engagement are the key to academic success. Further, different forms of educational technologies may mitigate the negative effect of academic absenteeism in school by creating more learning and engagement opportunities online. Therefore, leveraging engagement through self-regulated learning and self-assessment by using a variety of technologies is recommended to promote performance in higher education. Future research should be conducted to extend SDT and engagement theory to TEL education. Specifically, more empirical studies need to be conducted in various domains to further confirm the importance of in-class and online behavioral engagement on performance (Additional file 1). Using qualitative methods to develop a survey measure of math and science engagement Motivational factors affecting the integration of a learning management system by faculty Modeling MOOC student behavior with two-layer hidden Markov models Online formative assessment in higher education: A review of the literature The effects of student engagement, student satisfaction, and perceived learning in online learning environments The effect of attendance on academic performance A measure of college student course engagement Exploring the potential of LMS log data as a proxy measure of student engagement Where is the "theory" within the field of educational technology research? Pharmacy student absenteeism and academic performance Explaining lecture attendance behavior via structural equation modeling: Self-determination theory and the theory of planned behavior Reexamining the impact of self-determination theory on learning outcomes in the online learning environment Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives Examining online learning patterns with data mining techniques in peer-moderated and teacher-moderated courses Longitudinal test of self-determination theory's motivation mediation model in a naturally occurring classroom context Use of unsupervised online quizzes as formative assessment in a medical physiology course: Effects of incentives on student participation and performance Using pre-lecture activities to enhance learner engagement in a large group setting Does attendance affect grade? An analysis of first year economics students in Ireland The effect of refuting misconceptions in the introductory psychology class Student effort and performance over the semester The national survey of student engagement: Conceptual and empirical foundations Examining learning engagement in MOOCs: A self-determination theoretical perspective using mixed method Class attendance and academic achievement of pharmacy students in a European University Analyzing unstructured text data: Using latent categorization to identify intellectual communities in information systems Exploring the differences in Taiwanese university students' online learning task value, goal orientation, and selfefficacy before and after the COVID-19 outbreak. The Asia-Pacific Education Researcher Blended learning in higher education: Students' perceptions and their relation to outcomes Teaching psychology in Australia: Does class attendance matter for performance? Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size Investigating synchronous and asynchronous class attendance as predictors of academic success in online education The learning activity design framework to support mobile learning in primary school. Designs for Learning Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence Reassessing disparities in online learner student engagement in higher education The influence of computer self-efficacy, metacognitive self-regulation and self-esteem on student engagement in online learning programs: Evidence from the virtual world of Second Life Key variables for academic performance in university accounting studies. A mediation models A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ) Formative assessment in mathematics: Mediated by feedback's perceived usefulness and students' self-efficacy. Learning and Instruction Self-determination theory applied to educational settings. Handbook of Self-Determination Research A self-determination theory perspective on student engagement How students create motivationally supportive learning environments for themselves: The concept of agentic engagement Using formative assessment to improve student learning through critical reflection New benchmarks in higher education: Student engagement in online learning Do students go to class? Should they Educational data mining: A review of the state of the art Data mining in course management systems: Moodle case study and tutorial Lavaan: An R package for structural equation modeling and more. Version 0.5-12 (BETA) Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions Variables associated with achievement in higher education: A systematic review of metaanalyses Estimating the dimension of a model The construct validity of the Utrecht Work Engagement Scale: Multisample and longitudinal evidence VisMOOC: Visualizing video clickstream data from massive open online courses A motivational perspective on engagement and disaffection: Conceptualization and assessment of children's behavioral and emotional participation in academic activities in the classroom Interplay of prior knowledge, self-regulation, and motivation in complex multimedia learning environments Technology enhanced formative assessment for 21st century learning Prior knowledge activation: Inducing engagement with informational texts Motivation, learning strategies, participation and medical school performance Understanding students' engagement in MOOCs: An integration of selfdetermination theory and theory of relationship quality The relationship between lecture attendance and academic performance in an undergraduate psychology class Does lecture attendance matter? Some observations from a first-year economics course at the University of Cape Town Self-determination theory applied to physical education: A systematic review and meta-analysis Conceptualization and assessment of adolescents' engagement and disengagement in school: A Multidimensional School Engagement Scale Conceptualization and assessment of adolescents' engagement and disengagement in school The math and science engagement scales: Scale development, validation, and psychometric properties The learning management system matters in face-to-face higher education courses Expectancy-value theory of achievement motivation A multivariate approach to predicting student outcomes in web-enabled blended learning courses. The Internet and Higher Education Becoming a self-regulated learner: An overview. Theory into Practice We would like to thank the students who participated in this study. We would also like to thank the reviewers and editors for their constructive feedback and suggestions that improved our manuscript. We are also grateful to the following granting agencies that supported this research: Page 21 of 23 Lu and Cutumisu Int J Educ Technol High Educ (2022) 19:2 The online version contains supplementary material available at https:// doi. org/ 10. 1186/ s41239-021-00307-5.Additional file 1. Supplementary Material.