key: cord-0257668-patlx2wr authors: Vu, G. T.; Vu, T. M. T.; van Kessel, R.; Wong, B. L. H.; Nguyen, T. T.; Nguyen, T. P. T.; Nguyen, S. H.; Do, H. N.; Tran, B. X.; Latkin, C. A.; Ho, C. S. H.; Ho, R. C. M. title: Factorial Structure of the Transactional eHealth Literacy Among Vietnamese Youth: An Instrument Validation Study date: 2021-08-07 journal: nan DOI: 10.1101/2021.08.06.21261688 sha: f5fd1fbf59606bcd9d2f2c6d57c1ca84754739ca doc_id: 257668 cord_uid: patlx2wr Abstract The progression into the Digital Age has brought an array of novel skill requirements. Unlike traditional literacy, there are currently few measures that can reliably measure eHealth literacy. The Transactional Model of eHealth Literacy and subsequent Transactional eHealth Literacy Instrument may provide a feasible option for measuring eHealth literacy. However, this instrument has yet to be validated, which is the aim of this study. We conducted an online cross-sectional study among 236 Vietnamese young people. Using exploratory factor analysis, we ensured that a model consisting of four factors had the best fit (RMSEA = 0.116; CFI = 0.907) and the highest internal consistency (Cronbach alpha = 0.96). A confirmatory factor analysis tested measurement invariance at four levels: configural, metric, scalar, and strict invariance. Only metric invariance was partially invariant, while the rest tested fully invariant. Even with partial metric invariance, there is reason to assume that functional, communicative, critical, and translational eHealth literacy (the four levels according to the transactional model) are consistently measured when deploying the Transactional eHealth Literacy Instrument across groups. In other words, this study suggests the instrument can be used for comparisons across groups and has the potential to generate high-quality data usable for informing change agents as to whether a particular population is proficient enough to adopt novel eHealth innovations. Introduction eHealth literacy is a skill set that is becoming increasingly relevant as the Digital Age progresses (Sonnier, 2017) . When healthcare and public health services slowly moved to the digital sphere (Odone et al., 2019) , the skills required to search for, identify, and access these services started changing incrementally. However, much like most of the world, these services transitioned to digital environments at the beginning of the COVID-19 pandemic (Almeida et al., 2020) . While innovations are typically taken up gradually by a population (Rogers, 1995) , society adapted acutely to the pandemic -resulting in digital and technological advances neither equally nor equitably permeating all layers of society across the globe and potentially widening existing inequalities (Honeyman et al., 2020) . eHealth literacy is a specific branch in the field of digital literacy. To this day, it remains unclear what digital literacy is, how it can be assessed, and the benefits and risks from (not) being digitally literate (Chetty et al., 2018) . UNESCO frames digital literacy as a set of foundational skills needed to work in the digital world and is considered a catalyst for individuals to achieve health and social outcomes (UNESCO, 2011) . Current training programs tend to focus heavily on technical skills required to use digital technologies, while cognitive and ethical considerations are disregarded entirely (Chetty et al., 2018) . It takes approximately five years for the average individual to become proficient in using digital technologies (Goldstuck, 2010) . However, this five-year gap is not equal across all societal domains. For example, the medical sector has been slower to adapt to the digital world (Bartlett et al., 2021) . Recent work by Paige and colleagues created a tool with which to measure eHealth literacy (Paige et al., 2019) . The Transactional eHealth Literacy Instrument (TeHLI) is a multidimensional instrument to measure increasing levels of eHealth literacy, based on the Transactional Model of eHealth Literacy (TMeHL) (Paige et al., 2018) : functional (the ability to successfully read and write about health using technological devices), communicative (the ability to control, adapt, and collaborate communication about health with others in online social environments), critical (the ability to evaluate the relevance, trustworthiness, and risks of sharing and receiving health-related information on the Internet), and translational eHealth literacy (the ability to apply health-related information from the Internet in different contexts). However, All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 7, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 research is needed to test the TeHLI with younger adults (Paige et al., 2019) . This article aims to validate the TeHLI to see which components of the tool (how many components and which components included) would be the best fit statistically and whether the tool can be applied to groups of different characteristics (i.e. whether the different characteristics in terms of gender, education level, age group and other sociodemographic factors impact the effectiveness of the tool in any way). An online cross-sectional study was conducted among Vietnamese young people from April to June 2020 in Vietnam. The eligibility criteria for participating in this survey were: (1) ages between 16 and 35 years; (2) currently living in Vietnam; and (3) agreed to join this study by providing online informed consent. We recruited participants from all provinces of Vietnam using the snowball sampling technique. First, we developed a core group of participants (including leaders of the Youth Union in different public institutions, companies, and organisations) and invited them to complete the survey about the quality of life among youths. After completing the survey, these participants were requested to invite their peers in their respective networks to complete the online survey. At the end of the data collection period, 236 youths aged 16-35 living in Vietnam agreed to participate and completed the survey. In this study, we built an online survey on the Survey Monkey platform. This approach is low cost, consumes little time, is user-friendly for youth, and is highly accessible to reach the samples nationwide. Each participant spent 5-10 minutes completing the survey. A structured questionnaire was used to collect information, consisting of two components: (1) sociodemographic characteristics and health status, and (2) the TeHLI. The survey was first piloted on five youths to ensure the cross-cultural validity of translated instruments in Vietnamese. After that, the revised questionnaire was uploaded into the online survey portal. The data collection began once the online survey system was tested to assure accurate question contents and no technical issues. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.06.21261688 doi: medRxiv preprint Participants reported their sociodemographic information, including age, sex (male/female), educational attainment (below high school and high school/college/tertiary and higher), marital status (single/other), and living areas (urban/town/rural or mountain area) and acute symptoms in the last four weeks, any chronic conditions that they were diagnosed in the past. The TeHLI consisted of 18 items to reflect four aspects of the TMeHL such as functional (4 items), communicative (5 items), critical (5 items), translational (4 items). On a 5-point scale from 1 (strongly disagree) to 5 (strongly agree) was used for each item (Paige et al., 2019) . The Cronbach's of four domains of TeHLI were 0.91; 0.92; 0.88; 0.92, respectively. Statistical analysis was performed using STATA version 16 and R. Standard descriptive statistical analysis was conducted with mean and standard deviation (SD) for quantitative (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.06.21261688 doi: medRxiv preprint Measurement invariance: To determine the measurement invariance of the TeHLI, we performed a CFA by four increasingly constrained nested models. First, we evaluated configural invariance to assess what model is least stringent. This step aimed to test whether the constructs have the same pattern of free and fixed loadings across groups. Next, when configural invariance was supported, we evaluated metric invariance (also called pattern or weak invariance) or equivalence of the item loadings on the factors. Metric invariance was established by constraining factor loadings to be equivalent across groups. Next, we investigated scalar invariance by constraining equal intercepts and factor loadings across groups. This step assures that participants in different groups, on average, rate items similarly. Finally, we analysed strict invariance, which is supported when equal error variances are constrained in addition to equal intercepts and factor loadings. Testing residual error establishes that the same amount of error, or variance not accounted by the factor, is consistent for each item across groups. Measurement invariance is tested by some goodness of fit indexes, such as ΔCFI, ΔRMSEA, Δχ2. When we used Δχ 2 as a sole measure of exact fit, the results showed that it was associated with lower levels of scalar invariance. By contrast, using the ΔCFI (with or without other criteria like ΔRMSEA) was associated with higher levels metrics, scalar, and strict model. Therefore, in this study, change in the CFI (ΔCFI) was used as the primary criterion for comparing models, and ΔCFI < .01 between successively more restricted models provides evidence for measurement invariance (Hu & Bentler, 1999; Putnick & Bornstein, 2016; Villarreal-Zegarra et al., 2019) . Table 1 showed the sociodemographic and health characteristics of the 236 participants in this study. The mean age of respondents was 21.3 ± 4.8, with the most common age group about above 24 years old (78.8%). The majority of those were female (71.2%). There was about 56.8% of participants had the education tertiary and upper. Regarding health status, about 44.5% and 16.5% of observations suffered from acute symptoms within the last four weeks and chronic conditions within the previous three months, respectively. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Table 2 showed the results of the descriptive analysis for 18 items of the TeHLI. All of the 18 items had a range of scores from 1 to 5. Skewness and Kurtosis coefficients were reported with ranging from -1.0 to -0.2 and 2.8 to 4.3, respectively. Table 3 also presented the reliability of the modified TeHLI. Correlation coefficients with other items in respective factors ranged between 0.7 and 0.8. Internal consistency of factor 1 "Functional", factor 2 "Communicative", factor 3 "Critical", and factor 4 "Translational" were good at 0.82, 0.92, 0.92, and 0.90, respectively. Table 3 further showed the results of Confirmatory Factor Analysis on four different models. The model with 1 factor of the TeHLI showed an RMSEA and CFI score of 0.157 and 0.806. In contrast, the model with four factors showed an RMSEA and CFI score of 0.116 and 0.907, respectively. As such, this model was used to conduct measurement invariance. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 7, 2021. ; The results obtained from the Confirmatory Factor Analysis for the TeHLI were estimated in Figure 2 . The standardised coefficients range from 0.87 to 1.1. The highest coefficient was found in 4 items (Q4, Q11, Q13, and Q17), while the lowest was Q8. The values of ΔCFI were <0.01 when all models, with progressive restrictions, were compared across age groups, education, household economic (Table 4 ). These groups reported strict invariance. Besides, the configural model of gender, marital status, and location had fit statistics of RMSEA <0.08, CFI>0.9. However, in the metric model, the value of ΔCFI was higher than 0.01 at all three variables (gender, marital status, and location). Metric invariance was therefore not supported by the data. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.06.21261688 doi: medRxiv preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 7, 2021. ; this data (Pirralha, 2020) , which suggests that the constructs measured have different meanings across some of the participant groups. Finding partial support for metric invariance is reported to have minimal effects on the mean differences of a latent factor (Steinmetz, 2013) . Since two factor loadings reported metric invariance, there are grounds to assume the TeHLI can be used for comparisons across groups (Byrne et al., 1989; Steenkamp & Baumgartner, 1998) . Simultaneously, there is also the question of whether partial invariance is sufficient as significant bias can still be introduced into study findings if the partial invariance is ignored (Guenole & Brown, 2014; Steinmetz, 2018) . As such, deploying a partially (including invariant items only) and a fully invariant model (including invariant and noninvariant items) and compare the results of interest arises as a viable solution to use the TeHLI across different population groups (Chen, 2008) . The risk of this solution is thatif the results differ substantially -there is no clear path forward. Some limitations apply to this study. The survey findings had a mild-to-moderate negative skewness. However, as extrapolation to the population is impossible given the lack of participants, this limitation does not significantly affect the study findings. The survey was also distributed only among the Vietnamese population. Therefore, the influence of culture could not be determined and needs to be examined in a future study that includes multiple countries or regions. Ultimately, the TeHLI can be considered a valuable tool to measure different competency levels of eHealth literacy. As such, the TeHLI has the potential (if deployed correctly) to generate highquality data that can be used to inform governments and change agents whether the general or a target population is capable of understanding, appraising, distinguishing, and acting appropriately on high-and low-quality health information on the Internet. As a result, it can make vital contributions in informing and guiding policy and practice sustainably into the Digital Age. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.06.21261688 doi: medRxiv preprint The Challenges and Opportunities in the Digitalization of Companies in a Post-COVID-19 World How the medtech industry can capture value from digital health Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research Bridging the digital divide: Measuring digital literacy The tech revolution and strategic planning for digital content The consequences of ignoring measurement invariance for path coefficients in structural equation models Digital technology and health All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. inequalities: A scoping review Structural Equation Modelling: Guidelines for Determining Model Fit Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives Individual-patient monitoring in clinical practice: Are available health status surveys adequate? Quality of Life Research Public health digitalisation in Europe Proposing a Transactional Model of eHealth Literacy: Concept Analysis Transactional eHealth Literacy: Developing and Testing a Multi-Dimensional Instrument Measurement Invariance. In Testing for Measurement Invariance with Many Groups Measurement invariance conventions and reporting: The state of the art and future directions for psychological research Attributes of Innovations and their Rate of Adoption The fourth wave: Digital health -a new era of human progress Assessing Measurement Invariance in Cross-National Consumer Research Analysing Observed Composite Differences Across Groups Estimation and Comparison of Latent Means Across Cultures Cross-Cultural Analysis: Methods and Applications United Nations Educational Scientific and Cultural Organization) Valid group comparisons can be made with the Patient Health Questionnaire (PHQ-9): A measurement invariance study across groups by demographic characteristics No conflicts to report. No funding was acquired for this article.