key: cord-0058229-vtcslwli
authors: Wu, Ting-Fang; Sher, Yung-ji; Tai, Kai-Hsin; Hong, Jon-Chao
title: Usability of Virtual Reality Vocational Skills Training System for Students with Intellectual Disabilities
date: 2020-08-10
journal: Computers Helping People with Special Needs
DOI: 10.1007/978-3-030-58796-3_16
sha: 7c3540fc1fbd07b51b52de2af8a4dbf660f6ead0
doc_id: 58229
cord_uid: vtcslwli

Virtual reality has been applied to education widely since technology developed quickly. In order to apply the “Virtual Reality Vocational Skills Training System” to vocational high school students with intellectual disabilities, this study simplify the operation of the original system and develop an easy-to-use version to meet the learning needs of students with intellectual disabilities. Therefore, the purpose of this study is to test the usability of the easy-to-use version through the questionnaire, and to compare the operating efficiency between the easy-to-use version and the original one. Eight students with intellectual disabilities participated in the study. The results indicated that most students expressed that the easy-to-use version had good usability, and reduced the operation time and the number of wrong actions, as well as enhanced the accuracy. Overall, this designed “Virtual Reality Vocational Skills Training System” can be applied to the training of vocational skills for students with intellectual disabilities.

Virtual reality (VR) is a scene generated by a computer that simulates the real environment, and has been gradually applied in education. VR can simulate the real life situation and may enhance learning and transfer skills to everyday circumstances [1] . Tam et al. used a 2-D VR system to train persons with intellectual disabilities (ID) to shop. The result indicated that the VR program appears effective in training persons with ID in learning a community living skill [1] . Tsang and Man investigated the efficacy and effectiveness of VR as a cognitive intervention for enhancing vocational outcome. The results indicated that the participants in the VR-based vocational training group performed better in cognitive function compared with the therapist-administered group and a conventional group. Both the VR-based vocational training and therapist-administered groups showed the better work performance after training [2] . Smith et al. used the virtual reality system to train the interview skills of patients with post-traumatic stress syndrome and got the positive results [3] . VR provides the opportunities for learners to repeat as many times as they need, and to adjust their learning speeds based on their abilities. Therefore, VR seems to be suitable for training persons with ID [1] . In addition, persons with disabilities do not need to consider the consequences of errors even if they make mistakes in performing tasks in a VR environment. Persons with disabilities can learn in a safe and motivating environment [1] . Da Cunha, Neiva, and da Silva reviewed the literature in applying VR for training persons with cognitive impairments, and found that 61% of the studies indicating VR is effective for training persons with autism, intellectual disabilities, and other cognitive disorders [4] . Among those reviewed articles, only 2 out of 28 studies focused on the population of persons with ID. In the literature, VR has a positive effect towards learning skills for persons with disabilities. However, whether it is applicable to students with ID still needs more empirical data to support. We have already developed the Virtual Reality Vocational Skill Training (VRVST) System, which was designed for training individuals to learn new vocational skills via VR system. In order to provide training for students with ID, we developed an easy-to-use version for fulfilling this purpose. Usually a new system is developed, the first step is to perform the usability test of this system. Therefore, the purpose of this study is to test the usability of the easy-to-use version of the VRVST System.

Usability usually refers to the elegance and clarity of computer design or website human-machine interface [5] . Usability test is a necessary process in human-machine interface design. Designers can enhance usability through usability testing and improving the current interface. In this study, a questionnaire survey, where users fill out the subjective feeling after operating the easy-to-use version of VR system, was conducted as a usability test [6] . In addition, in order to understand whether students with intellectual disabilities will improve their performance using the easy-to-use version of VR system, the operational performance of the original version and the easyto-use version were compared.

Eight students with ID participated in this study with their parental consents. They were recruited from vocational high schools in Metropolitan Taipei area. Students with ID were diagnosed and identified by the local education authority based on the following criteria: (a) significant limitations in intellectual functions (reasoning, learning, and problem solving), (b) significant limitations in adaptive behaviors, which includes a range of everyday social and practical skills, and (c) this disability originates before the age of 18 [7] . All the participants met the above criteria. The following table (Table 1) 

Virtual Reality Vocational Skills Training System (VRVST System) The VRVST System was developed by the research team of Professor Jon-Chao Hong at National Taiwan Normal University, in Taipei, Taiwan. It was designed to train students to learn a new vocational skill. For the module of learning to be a kitchen assistant, the VRVST System consists of 8 common kitchen preparation tasks, such as: cutting cucumber, cutting cabbage, and peeling radish. Through the comprehensive work analysis, each kitchen preparation task is analyzed into 8 to 18 steps based on the complexity of the task, and the students can learn the task step by step by following the cues embedded in the VRVST System. There are several different kinds of cues in this system, for example, the VRVST system provides the order of cutting cucumber steps and tips on where to cut (Fig. 1a ). In addition, there are blue flashes to indicate where items should be placed (Fig. 1b) . 

The VRVST System was originally intended to design for vocational high school students to learn vocational skills. In order to make this original system applicable to students with ID, this study simplified the operation of the original VRVST system and developed an easy-to-use version to meet the learning needs of students with ID. There were two differences between the original and easy-to-use versions: (a) Added voice prompts: Since the students with ID usually have poor literacy, voice prompts may provide more direct cues for persons with ID, and (b) Simplified the control buttons: Due to the poor dexterity of the students with ID, they often fail to manipulate the selection action in the original system which require the coordination of the thumb and the index. Therefore, in the easy-to-use version, the operator only needs to put the handle to the target and press the key to select.

The experimental hardware equipment used in this study included Acer Windows Mix Reality headset (AH101 model), which has a built-in gyroscope, acceleration sensor, magnetometer (compass) and proximity sensor, as well as Acer AR/VR remote controller.

The questionnaire, System Usability Scale (SUS), was used to test the usability of the easy-to-use version of VRVST System. It was developed by Brooke, and is a widely used standardized questionnaire in testing the perceived usability of computer programs, systems and website pages [8] . The SUS is a Likert's five-point scale, which the higher the score, the better the user perceive the usability of the system is. The products with SUS scores above 70 indicate the acceptable usability, and if the products' SUS scores less than 70 means should be considered candidates for increased scrutiny and continued improvement, and be marginal at best. Products with scores less than 50 should be considered for significant concern and judged to be unacceptable [5, 9] .

Familiar with the VR Hardware Equipment In order to avoid unfamiliar operation affecting the experimental results, the participants were asked to be familiar with the experimental equipment, including how to operate the VR controller, and the entire experimental process before entering the experiment. The participants will not enter the experiment until they were familiar with the operation of the experimental equipment.

The participants were required to complete the task of cutting cucumber in the VRVST system. This task consists of 8 steps; they are wearing gloves, picking the chopping board, washing the cucumber, putting the cucumber on the cutting board, cutting out the ends of cucumber, cutting the cucumber into 5 pieces, placing the chopped cucumbers in the tray, and placing the tray in the specified area.

In order to eliminate the effect of repeat practice on score improvement, the counterbalance order of operating two versions of the VRVST system was used. For example, if the first student operated the original version first and followed by the easyto-use version, then the second student will operate the easy-to-use first and followed by the original version. Four of the eight students used the easy-to-use version first, and the other four used the original version first.

After completing the experiment, the students were asked to fill in the SUS questionnaire to report their perceived usability about the VRVST system. If the student cannot read the text due to poor literacy, the researcher will read the question and the student answered the question orally.

The students' operational efficacy in the original version and in the easy-to-use version were compared to indicate which version has better operating efficiency. Three parameters, time spent, accuracy, and movement errors, in completing the task of cutting cucumber were used as indicators of operational efficacy. All data was calculated and recorded by the VRVST system automatically. The shorter the time spent, the higher the accuracy, and the fewer movement errors, indicate the better the operating efficiency. The followings are the definitions of those parameters.

• Time spent: Time required to complete all steps in the task • Accuracy: Correct steps divided by total steps • Movement errors: The number of errors in various actions, including not following the instruction line when cutting vegetables, gesture errors of assisting hand when cutting vegetables, the size of the cut vegetables is not even, and so on.

In addition, the Wilcoxon related sample test in the SPSS 23 was used to analyze the differences in the operating performance of students between the original version and the easy-to-use version.

The results indicated that students performed better in the easy-to-use version. The total time spent in completing the task was significantly less, the accuracy was significantly higher, and the number of movement errors was significantly lower in the easy-to-use version when compared to those in the original version (Table 2 ).

The SUS scores filled out by the participants are presented in Table 3 . The students' SUS scores ranged from 47.5 to 90, showing considerable variation. If divide those scores into groups, we found that five out of eight have their SUS scores over 70, which indicating they perceive the VRVST System having good usability. Two participants have their SUS scores between 50 and 70 indicating the marginal usability. Only one student's SUS score is less than 50, which means the system is considered unacceptable usability. However, according to the qualitative statements of students after using the VRVST System, students expressed that this system was easy to operate, and the voice prompts in easy-to-use version was helpful for them performing the tasks more intuitively, without spending time to look at the text prompts.

This study explored the usability of the VRVST System for students with ID from two aspects: students' subjective feelings and objective operational performance. The results showed that most students' SUS scores were high, which indicating the acceptance of this system. Among the 8 students, 5 students had SUS scores above 70, supporting a considered acceptable system; 2 students reported scores between 50-70 showing a "marginal" system; only one student (Students number 4) scored less than 50, which means the system is considered unacceptable. The reason why the students number 4 scored lower in the SUS may due to some items have relatively low scores. For example, his/her score of the item "I will often use this VR game" was relatively low than others. Further inquire about the reason for the low score of this question, this student answered that since the VR system is not widely available, he would not use it frequently. In addition, some students scored low in another item "I need help from others when playing this VR game". Actually, all the participants were able to operate the VR system independently, they only need some assistance in setting up the system before entering the game. Meanwhile, students with ID also provided qualitative feedback about the system. They all responded positively .016* *p < .05, **p < .01 Table 3 . The SUS Scores of the 8 students to this system, such as: easy to operate, the well-integrated overall function of the system, and easy to learn quickly in using this system. The clarity of the questionnaire presentation may need to be defined more clearly for further study.

From the objective view, the students performed significantly better in the easy-touse version than in the original one. According to the qualitative statements, the participants expressed that the voice prompts in the easy-to-use version provides more intuitive cues than text prompts. That might be the reason for their better performance in the easy-to-use version. Whether objective data or subjective expression demonstrated that the easy-to-use version VR system seems to be suitable for students with ID to learn a new daily task. In the future, we will continue to use this system to collect more empirical data to understand the learning curve or leaning path of students with ID. In addition, factors affecting learning in the VR environment for students with ID will also be investigated.

Evaluation of a computerassisted, 2-D virtual reality system for training people with intellectual disabilities on how to shop

A virtual reality-based vocational training system (VRVTS) for people with schizophrenia in vocational rehabilitation

Virtual reality job interview training for veterans with posttraumatic stress disorder

Virtual reality as a support tool for the treatment of people with intellectual and multiple disabilities: a systematic literature review

Determining what individual SUS scores mean: Adding an adjective rating scale

A new view at usability test methods of interfaces for human computer interaction

Intellectual disability: definition, classification, and system of supports

SUS: a quick and dirty usability scale

The System Usability Scale (SUS): an empirical evaluation