key: cord-0891801-7s9yvn72 authors: Aslan, Seda Akti; Turgut, Yigit Emrah; Aslan, Alper title: Teachers' views related the middle school curriculum for distance education during the COVID-19 pandemic date: 2021-06-02 journal: Educ Inf Technol (Dordr) DOI: 10.1007/s10639-021-10587-z sha: 43cdd2944dbff0322d86808c3c94c728c5871683 doc_id: 891801 cord_uid: 7s9yvn72 This qualitative study aims to examine teachers’ opinions concerning the middle school curricula that have been in use for years, in order to determine its suitability for distance education during the COVID-19 pandemic. The participants consisted of eighteen middle school teachers from different subject areas in Turkey. A maximum variation sampling method was employed for participant selection. The data was collected through semi-structured interviews. Due to the pandemic, the interviews were conducted using video conferencing applications. The data was analyzed through content analysis. According to the findings, the teachers report that while they are effective in achieving cognitive objectives, they have difficulties in achieving affective and psychomotor objectives. In addition, the teachers do not prefer to design their own materials, rather they mainly benefit from ready-to-use materials during emergency distance education. The teachers also reveal that direct instruction and questioning were the teaching methods and techniques they preferred in online courses and that they were unable to adapt other methods and techniques to distance education, which resulted in students experiencing teacher-centered activities. Moreover, the teachers complain about students not attending the online courses on time or even at all, their disinterest, a lack of instant feedback, limited communication with students, and insufficient course lengths. Another critical finding was that the teachers only use homework, end-of-unit quizzes, and participation to online courses as measurement and evaluation tools and agree that they are unable to administer reliable and valid evaluation tools. The limitations of the study and implications for future research are later discussed. The whole world has been experiencing a major epidemic for more than a year. The World Health Organization (WHO) declared COVID-19 to be a global public health emergency on 30 January, 2020, and a pandemic on 11 March, 2020 (WHO, 2020a . According to the data of the WHO (2020b), COVID-19, which had infected more than 79 million people in 213 countries, was the cause of death of over 1.7 million people up to 29 December, 2020. Extraordinary prevention arrangements were employed, such as flexible work-hours, home-based work, and online education in order to reduce the spread of the epidemic (Zhao, 2020) , which affected daily life all over the world, as well as the education sector (Agnoletto & Queiroz, 2020) . In the spring of 2020, educational institutions around the world had to suspend face-to-face education until further notice (Bozkurt & Sharma, 2020; Daniel, 2020; Doghonadze et al., 2020) . According to a UNESCO report (2020), schools ceased to operate in 188 countries and 92% of students throughout the world were affected due to this shutdown. In addition, the dropout rate of disadvantaged students increased due to the long-lasting effects of the epidemic in education (UNICEF, 2020) . The pandemic caused all experience and habits within the educational sector to become dysfunctional. In order to continue teaching and learning activities, distance education was the only alternative for educational institutions (Blankenberger & Williams, 2020) . However, the rapid shift to distance education brought many problems with it (Seymour-Walsh et al., 2020) . Efforts to integrate new technologies into learning environments or to design online learning environments are not new processes in education (Cuban, 1986; Hodges et al., 2020) . However, the transformation in the educational sector due to the pandemic took place at an unprecedented pace and emergency distance education (EDD) applications were put into practice all over the world . The critical transformation that occurred due to the pandemic makes it necessary to discuss the effectiveness of online learning and the problems that have arisen (Adarkwah, 2020; Almaiah et al., 2020) . In addition, the fact that teachers and students had to switch to distance education suddenly became a popular topic for research during the pandemic (Hodges et al., 2020) . Studies mainly focus on the success of distance education (Shire et al., 2020) , the problems experienced (Adarkwah, 2020; Kruszewska et al., 2020) , the factors affecting the use of the e-learning systems (Almaiah et al., 2020) , measurement and evaluation processes , digital skills for distance education (Deshmukh, 2020) , students' acceptance of online learning (Aguilera-Hermida, 2020), communication problems (Burke & Dempsey, 2020) , and privacy issues in online settings (Gong, 2020) . Although there are studies emphasizing the need to develop online education programs, in addition to the existing curricula due to the pandemic (Metchik et al., 2020; Wlodarczyk et al., 2020) , none of the studies focus on the implementation of existing curricula in online learning environments. The rapid transformation in the educational sector with the pandemic may accelerate curriculum development. In this context, it is important to investigate the applicability of existing curricula in online learning environments. In addition, studies focusing on distance education during the pandemic have been mostly carried out in higher education (Adarkwah, 2020; Almaiah, 2020; Aguilera-Hermida, 2020; Rapanta et al., 2020; Cutri, 2020; Johnson et al., 2020; Yates et al., 2020; Giavrimis & Nikolaou, 2020) . In this context, it is considered that the results of the study will significantly contribute to the development process of middle school curricula in accordance with distance education. When considered that studies on online learning environments are mostly carried out in higher education during the pandemic, it is also evaluated that the results of the study will have significant contributions to the literature in the context of middle school education. Considering the shift to distance education and the use of already existing curricula in distance education, there is a need to examine how suitable the existing curricula is for distance education. The purpose of this particular study is to examine teachers' opinions about the curriculum they have been teaching during the pandemic, based on their experiences. The term curriculum, dating back centuries (Ornstein & Hunkins, 1993) , was used by Julius Caesar and his soldiers to refer to the oval-shaped running track used by race vehicles (Oliva, 2009) . After this, the term was transferred into English to represent a path or route. Today, the term curriculum is used in education to mean a followed path (Ellis, 2004) . While Saylor et al. (1981) define the word curriculum as a plan for learners to gain learning experiences, while Taba (1962) defines it as a learning plan. In a more detailed explanation, Hewitt (2006) defines curriculum as a series of decision-making processes that focus on the preparation, implementation, and evaluation of plans to influence students' behavior and perspectives and the product that is put forward at the end of this process. A curriculum has four elements, as can be seen in Fig. 1 . According to the figure, these four elements have a dynamic association, which refers to a systematic process of curriculum development (Demirel, 2010) . Specifically, the main purpose and scope of each element should be carefully identified. The element of goals and objectives aims to answer the question, 'Why educate individuals?'. Goals and objectives are the desired traits that learners are expected to gain. Ertürk (1972) states that goals and objectives lead students to gain desired behavior. The content or subject matter element aims to identify what to teach in order to reach the goals and objectives. The teaching and learning processes deal with how to teach the determined content in order to reach the goals and objectives (Yüksel & Sağlam, 2010) . In order to keep students at the center of learning, teaching and learning practices should be determined in line with the goals and objectives. The last element of a curriculum, measurement and evaluation, deals with the issue to what extent students learn the content considering the goals and objectives (Demirel, 2010) . While developing a curriculum, a certain process is followed by taking these four elements into consideration. This process includes the analysis of needs, design, development, implementation, and evaluation steps, respectively. According to Fer (2019) , the curriculum development process includes the development, implementation, and evaluation steps and revisions based on problems arising during the implementation. Specifically, the needs of the audience, society, and subject matter should be examined during the process (Erden, 1995) , since the quality of education depends on the curricula meeting current realities and needs. In Turkey, due to globalization and the changes in education across the world, the curricula were updated with the adoption of a constructivist approach in 2004. The latest revision was completed in 2018, and the renewed curricula were put into practice at all grade levels, starting in the 2018-2019 academic year (MEB, 2020a). In Turkey, due to globalization and the changes in education across the world, the curricula were updated with the adoption of a constructivist approach in 2004. The latest revision was completed in 2018, and the renewed curricula were put into practice at all grade levels, starting in the 2018-2019 academic year (MEB, 2020b) . In 2012, the compulsory education was set to 12 years as 4 + 4 + 4 with a law in Turkey. After the optional kindergarten education, students complete four-year elementary education, and then, continue their education in secondary schools. In secondary schools, there are 40-min compulsory and elective courses for students to take. The compulsory courses are Turkish language and literacy, Mathematics, science, social studies, foreign language, religious and moral education, visual arts, music, physical education and sports, and information technologies and software. The elective courses are the life of the Prophet, basic religious knowledge, science practices, drama, intelligence games, mathematics practices, and reading skills. Teachers use the Education and Information Network (EIN) and other various digital resources through smart boards in the classrooms. They also use textbooks distributed to students free of charge by the Ministry of National Education. Curricula may change over time in order to keep up with the changes experienced over time. The program evaluation process is as important as the program development process. More specifically, the evaluation process is critical to determine to what extent the existing curricula meet social and individual needs. There exist different curriculum evaluation approaches in the literature, one of which is the Educational Connoisseurship and Educational Criticism Model developed by Eisner (1998) . According to this qualitative analysis-oriented model, a curriculum is implemented and qualitative results of the curricula are collected. The model consists of "description", "interpretation" and "evaluation" dimensions (Eisner, 1998) . In the description dimension, characteristics related to the quality of education are defined. For instance, answers to the following questions are sought in this dimension: "What kind of changes has happened to the school as a result of the new program?" and "How are teachers and students affected by the new curriculum?" In the interpretation dimension, events occurring as a result of the curriculum are taken into consideration. In the evaluation dimension, experts first describe the observed situation, interpret the data, and finally make a judgment about the curriculum. In this particular study, teachers' opinions were obtained within the framework of the Educational Connoisseurship and Educational Criticism Model in order to determine how they think about using the curricula which were designed for traditional face-to-face education in emergency remote education during the pandemic. This enabled researchers to evaluate the curricula from different perspectives and to reveal the difficulties encountered in the emergency remote education process. Distance education is a corporate education where students, teachers, and instructional materials from different locations are combined together through information communication technologies (Moore & Kearsley, 1996) . The development of distance education depends on these information and communication technologies. Distance education started with postal services in the eighteenth century and was then carried out, first through radio and television, and later through teleconferencing which provides two-way audio and video communication, and finally through the Internet and web-based technologies (Simonson et al., 2015) . Today, through distance education, students have access to course materials online, are able to complete their homework and take exams, and to communicate with their instructors and classmates. In this way, students can meet their learning needs, share their thoughts in online discussions, and gain new skills and knowledge through collaboration (Moore, 2013) . Distance education not only eliminates time and location limitations, but also encourages students with disabilities to seek their education in the fastest and most practical way (Karal et al., 2011) . For instance, with Massive Open Online Courses (MOOC), which are considered to be the future of distance education, students may improve themselves by participating in an online and free course designed by an instructor, with a specific plan and pedagogical approach anywhere in the world. Since March 2020, students' educational needs have been met through distance education. Since the beginning of the pandemic, schools have been closed locally in China. In Italy, which was the most affected country by the pandemic after China, schools were closed locally at the beginning of the pandemic and then on March 5 2020 all schools were closed down. After Italy, schools in Pakistan, Afghanistan, and Iran were closed. Then, Vietnam, Iraq, Northern Ireland, England and the United States of America decided to close schools locally. Germany and France also closed schools locally. By the beginning of March 2020, the effects of the pandemic were reflected in the education process in India, Ukraine, Greece, and Saudi Arabia, and decisions were taken to close schools. In Turkey, after the first case on March 11, 2020, all schools were closed nationwide and although it was stated that this closure would be short-term, it was reported that the schools would be closed in the spring semester of 2019-2020 academic year (MEB, 2020b). Towards the end of March, Russia, Greenland, Canada, Brazil, Yemen, Indonesia, Malaysia, and Australia employed the same decision nationwide. By April 13 2020, the number of countries closing schools across nationwide was 192 (UNESCO, 2020). Also, the number of students affected by this process is over 18 million (UNESCO, 2020) . During the pandemic, the continuation of education was only possible through distance education. In order to meet the individuals' right to get education and to minimize the inequality in terms of educational opportunities, all countries attempted to find urgent solutions (Eken et al., 2020) . While some countries decided to organize online lectures, some benefited from television broadcasts and developed online educational materials (Chang & Satako, 2020; OECD, 2020) . In China, due to their advanced internet infrastructure, there was a quick shift to distance education through live lectures and TV channels. Teachers, students and parents were able to communicate with each other in online environments including QQWeChat and DingTalk (Durak et al., 2020) . In the United States of America, different practices were observed in terms of distance education due to the inequality in infrastructure among schools. However, due to their stronger economic power compared with the other countries, America and Canada were able to took precautions against the pandemic. For example, in Colombia, where distance education started in March, digital materials were uploaded to the Aprender Digital platform by the Ministry of Education and broadcasts were made on TV and radio channels (The World Bank, 2020). A similar situation was observed in Europe. With the closure of schools in England, the education process continued through various e-learning platforms and TV programs or social media platforms by using the books schools provided. In the Asian continent, inequalities in education were experienced due to the problematic access to the internet and electronic devices. As in many countries, Asian countries do not have a policy on distance education (Jalli, 2020) . Also, TV channels, Facebook and YouTube were used for education in those countries. According to UNESCO (2020), many Arab countries actively used their own learning platforms from the beginning of the pandemic. Teachers and students communicated and conducted their lessons through platforms including Microsoft Teams, Edmodo, and Newton. In addition to these, radio channels and YouTube were also used for educational purposes. Australia has experience the epidemic after the other continents. In Australia, schools provided online education when possible (Wilson & Mude, 2020) . On the other hand, educational process was about to cut off in Africa due to very limited resources. Only a few private schools and public schools offered online courses in line with their technological infrastructure (Veriava, 2020) . In those schools, teachers met with their students through platforms including Google Classroom, Zoom, and Edmodo and gave them homework. As in many other countries, the Turkish government has taken many measures in order to prevent the spread of the pandemic. One of these measures was the closure of schools until further notice. As a result, face-to-face education was suspended starting on March 16 2020. After a week, the educational activities started to be held online through Educational Information Network (EIN) platform and TV for all grade levels from elementary to high schools. In this context, hundreds of teachers from 112 different majors worked together to create professional course videos, which have been broadcasted on TRT EBA kindergarten, primary school, middle school and high school channels. The videos were displayed on TV three times a day in order to allow students to make up the missed videos (MEB, 2020b). Also, many informative programs related to "crisis management", "reducing fear", and "communicating with games" were provided for parents. For teachers, many in-service training programs were provided to adapt them to the distance education process more easily and to eliminate their deficiencies in this regard. In addition, 60 thousand tablet computers sent through public institutions and organizations, local administrations, private sector organizations and civil initiatives were delivered to students in need. These distributions are still in process with support campaigns. With a governmental decision, the hybrid education model started to be implemented. Students were asked to attend schools two days a week and continue their online education one day a week. Also, students were divided into two groups for each classroom. While the first group went to school on Mondays and Tuesdays, the other group went to school on Thursdays and Fridays. Online lessons have been held on Wednesdays. Also, schools were disinfected on Wednesdays. However, this practice recently ended with the joint agreement of the Ministry of Health and the Ministry of National Education due to the increase in the number of cases. For middle schools, only certain courses were held online during the 2019-2020 spring semester. As of the fall semester of 2020-2021 academic year, all courses were held online. In addition, for students who did not have an opportunity to attend online courses at home, they were asked to go to support points schools organized or to watch asynchronous lecture videos on TV. Students attended online courses through EIN platform. The platform has a feature for teachers to schedule synchronous lessons and to upload course materials. In addition, it allows teachers to communicate with their students, to upload homework, and to manage all other classroom activities. In this study, a case study, one of the qualitative research methods, is used. Case studies investigate a current phenomenon in a real-life context, and are used when the boundaries between the phenomenon and the context are not clear (Yin, 2017) . Specifically, it focuses on individuals, communities, organizations, programs, processes, and events in the real world (Glesne, 2016; Yin, 2017) . A direct shift to EDD due to the pandemic is an up-to-date situation, which needs attention in order to better understand teachers' experiences as they implement the curricula in online settings. The participants of the study consist of eighteen teachers who work in middle schools. A maximum variation sampling method was employed for participant selection by considering gender, teaching experience, and subject area in order to obtain more detailed information about the phenomenon. This method enables the determination of common patterns from the differences by dealing with a wide range of differences related to the situation (Patton, 2014) . In this context, the researchers aimed to obtain the opinions of teachers from all majors since a complete shift to distance education was employed for all grade levels and all courses during the pandemic. While recruiting participants, the researchers ensured that there were two teachers from each major with different teaching experiences. Demographic information about the participants included in the study is presented in Table 1 . Data was collected using semi-structured interview forms developed by the researchers. The interviews allowed researchers to determine teachers' perceptions, reactions, thoughts, and interpretations, as suggested by Patton (2014) . A list of questions was prepared based on the four elements of the curriculum; goals and objectives, content, teaching and learning process, and evaluation. In addition to the main questions, prompt questions were also prepared in order to obtain more detailed information about the implementation of a curriculum in online settings during the pandemic. After the interview form was reviewed by two experts from the field, it was revised. A pilot study was conducted with four teachers from different subject areas and the interview form was finalized (see Appendix 1). Participation to the study was voluntary. After providing information to the participants about the study, meeting times were scheduled. The interviews were held via video-conferencing and each took about twenty minutes. The interviews were recorded with the approval of the participants. The questions in the form were asked in the same order to all participants, and the prompt questions were used when necessary. The data was transferred into a written format and then analyzed via MaxQDA using the content analysis method, which allows researchers to examine largescale qualitative data, such as interviews, diaries and documents, by arranging and interpreting it within the framework of certain concepts and themes (Patton, 2014; Yıldırım & Şimşek, 2018) . In the analysis process, categorization and coding were completed in terms of a curriculum's four elements. In order to ensure consistency, data from five randomly selected interviews was coded individually by the researchers, as suggested by Miles and Huberman (1994) . After this, the codes were compared to find any differences and to reach consensus. Next, the data was analyzed using computer software. Analiz sürecinde Strauss ve Corbin'in (1990) belirttiği kodlama türlerinden biri olan önceden belirlenmiş kavramlara göre kodlama yapılması tercih edilmiştir. Bu sayede toplanan verilerin önceden belirlenen temalar çerçevesinde kodlanması daha kolay olmaktadır. Bu doğrultuda, öğretmenlerin görüşleri öğretim programlarının dört temel öğesinin -hedef, içerik, öğretme-öğrenme süreci ve değerlendirme-çerçevesinde kodlanmıştır. During the analysis, the codes representing a similar phenomenon/thought were gathered together under the same theme. The final code-theme structure was reached through continuous comparisons in codes and themes. The findings were transformed into figures containing frequencies in line with the research questions. These figures were also supported through direct quotations from the participants. This section provides the findings regarding the perspectives of the teachers towards the implementation of a curriculum in middle schools. The findings are organized based on the four elements of a curriculum. The themes and codes are provided in Fig. 2 . In line with the research question, the first findings are related to the goals and objectives element. According to the findings, three themes were identified; learning domain efficiency, difficulties encountered, and suggestions. The themes and codes are provided in Fig. 3 . The learning domain efficiency theme consists of three categories related to the affective, cognitive, and psychomotor objectives. The teachers report that while they are effective in achieving cognitive objectives, they are not so effective in achieving the other types of objectives. One of the teachers (1SS) states that, "During this time, we were only able to achieve goals related to the cognitive area; we did not do anything about the affective and psychomotor areas." According to the results, teachers mention two specific reasons for not being effective in achieving all types of objectives; students not attending the online courses and not caring about the course content. Examples quotations are provided below: 'Not all students attend the online courses, and those who do attend are regularly absent. This is a big problem'. [11SO] . 'In face-to-face education, there was discipline, but when it comes to online education, students do not believe that they are being educated. They are not fully integrated into the concept of school. [12IN] . In addition, a mathematics teacher [16MA] places great emphasis on instant feedback saying that, "In math lessons, we have to follow up students to see where they get stuck when solving a problem and we need to give them instant feedback. However, this does not happen in online education." Another teacher [17SS] complains about the same difficulty in science education saying, "We get used to conducting scientific experiments in face-to-face education in science lessons. Compared to other fields, we have so many hands-on activities. However, in online education, we cannot do this." A number of teachers also complain about the method of communication. One teacher [9RE] says, "I mean the cameras are turned off, and I cannot see what my students are doing over there behind the cameras. I feel like I am talking to myself." While half of the teachers mention insufficient course length, some of them discuss other issues, including lack of eye-contact, late participation to the course, and insufficient course hours per week. The participants also provide suggestions in order to solve the problems they encounter. One teacher [12IN] mentions technical issues and states that, "If we get rid of physical deficiencies, I mean if everyone can participate under equal conditions, a critical milestone can be achieved in distance education and we can make progress." Another critical suggestion concerns revision of the current curriculum by reducing the number of goals and objectives. A science teacher [3SC] states that, "This science curriculum is designed considering the 40-min course in face-to-face education. Plus, it is designed for face-to-face education. We are trying to adapt it to distance education. We will never cover all the topics in it. I think we need to drop two or three units." Some teachers mention development of ready-to-use materials. For instance, one teacher [8TU] states that, "There is a need for materials in order to develop students' writing skills for the Turkish literacy course, because it is not just about reading." Another teacher [12IN] says that, "Teachers need to arrange the content per course hour; I mean teachers need to be able to choose activities." According to the teachers' statements, there were two themes related to the content element; resources and suggestions. Fig. 4 provides the themes and codes. The teachers report that they employ ready-to-use materials obtained from the Educational Information Network, educational websites, and/or social media or develop their own materials during the EDD process. Example quotations are provided below. "There is very cool stuff in Educational Information Network and I used them." [2EN] . "I generally benefited from websites that are related to my field." [9RE] . In addition, while one of the teachers [16MA] states that, "The number of highquality materials needs to be increased, because we do not have enough time to develop such materials," another teacher [4EN] states the opposite view saying, "Actually, we can develop our own materials, different materials; I think we have a little deficiency in this regard." According to the findings, five themes are identified; teaching methods and techniques, student-centered activities, appropriateness of materials, difficulties encountered, and suggestions. Fig. 5 provides the themes and codes. According to the findings, teachers report that they generally use direct instruction and questioning as teaching methods and techniques while teaching. One Teachers also mention that they encounter many difficulties during the EDD. One of these problems is the inability of teachers to adopt activities for distance education. One teacher [9RE] says, "Some activities can only be conducted in face-toface learning environments. We cannot do all of the activities that we can do in classrooms in online settings." According to the findings, one of the reasons why the teachers are unable to adapt the methods and techniques they use in face-to-face education to the online environment is insufficient course length. Another teacher [16MA] complains about the lack of video communication saying, "Turning cameras off was a big problem for me because I was unable to follow up my students." One music teacher [6MU] talks about a problem which was not experienced by the other teachers. She says, "While I am playing piano, I wanted all my students to accompany me. But they did not hear me synchronously and I did not hear them synchronously. This is a big problem." A number of the teachers also mention problems regarding limited course length, student disinterest, and the language of the program interface. "We were unable to interact with all students due to limited course hours." [15IN] . "Due to students' disinterest only a few students actively participated in online courses." [8TU] . "We did not get to meet new students in person. So, we don't know their personalities. If we knew them better, we would organize the activities to be more effective." [11SO] . The teachers report that the educational materials they use in EDD are insufficient. One teacher [15IN] states, "I do not think we used materials effectively, because most of the students used their smartphones to connect to the online courses. So, I was unable to use all materials effectively through smartphones." On the other hand, there are other teachers who claim that the educational materials they use in their courses are sufficient or are somewhat sufficient. In addition, a number of the teachers report that since they are not able to transfer certain materials to the online setting, they encounter problems. As a result, although they conduct student-centered activities in real classrooms, they cannot achieve this in EDD. One teacher [16MA] says, "Although my classroom activities were student-centered in face-to-face learning environments, I have teacher-centered activities in distance education." Teachers also provide suggestions. One of the suggestions concerns designing activities that are suitable for distance education in order to increase the quality of distance education. One teacher [8TU] says, "Methods and techniques should be integrated and adapted to distance education." Another teacher [5MA] emphasizes the importance of students' participation saying, "If students' participation in online courses increases, then we can administer methods and techniques more effectively." A number of the teachers also mention the necessity of reducing class size in order to conduct student-centered activities. One teacher [4EN] states, "If the class size is reduced, then we can conduct student-centered activities." In addition, the teachers mention the need to develop tools that enable co-operation, provide in-service training programs to improve teachers' knowledge regarding teaching methods and techniques suitable for distance education, and to resolve technical problems for better distance education. According to the results, three categories are identified; evaluation methods, difficulties encountered, and suggestions. Fig. 6 provides the themes and codes related to the evaluation element of a curriculum. The teachers report that they use different evaluation techniques to measure students' learning during the EDD. More than half of the teachers state that they ask students to do homework. One teacher [4EN] says, "I uploaded tests on EIN [the Educational Information Network] and checked out whether my students completed the tests or not." A number of teachers, on the other hand, conduct exams at the end of each unit they cover. An example quotation is, "There are questions at the end of each unit in the book. I opened these questions on my computer and shared my screen with my students. This allowed me to determine whether my students had learned the concepts or not" [10RE]. Some teachers use students' participation levels for evaluation. One teacher [3SC] states, "I wrote down the names of students who participated in my courses and used their Fig. 6 Themes and codes related to the evaluation element participation for evaluation." Many of the teachers raise concerns about inequality in students' conditions with regard to attending classes regularly. However, other teachers state that although they are aware of this inequality, they believe that students' participation must be considered as one method of evaluation. One teacher [12IN] states that, "The first criterion for me is the students' participation. I know this is not fair, because some students do not have Internet access, and some used their smartphones. This may not be a criterion, but it is for me." In the interviews, most of the teachers believed that evaluation in EDD did not produce accurate results with regard to their students' learning. Their explanations about this belief were related to the unreliability of online exams, inequality among students, lack of individual evaluation, and age group the teachers taught. Example quotations are provided below: "There was an online exam in a high school. I saw one of the students was getting help from some other students and sharing the answers with his classmates through a social media site during the exam. So, I do not believe online exams are reliable." [7TU] . "We cannot give grades to students' performances in distance education because not everyone attended the online courses under equal conditions. If you give them a score you will be unfair. I don't believe it is the right thing to do in this situation." [5MA] . "Considering my field, it is critical to observe each student's development individually in order to conduct evaluation in an appropriate way. This is impossible in distance education." [13PH] . "Of course, university students could do it, they took online exams despite some problems, because they are older than my students. These are so young so how can I ask them to take an online exam. It is a big problem to prepare reliable and valid exams." [1SO] . Teachers also offered solutions regarding the difficulties they encountered. One suggestion concerned standardized exams across Turkey. One of the teachers [4EN] states, "I think we may have online standardized exams for all students in Turkey. I believe we got behind on this." In addition, individual evaluation Following the interviews, the participants were asked to evaluate the suitability of the curriculum they used for the EDD. While twelve teachers thought that it was not suitable, only two teachers considered it as somewhat suitable. Only four teachers agreed that the curriculum they used was suitable. Example quotations are provided below: "In the science curriculum, we are expected to conduct activities and experiments to advance students' higher-order thinking skills. However, distance education makes it very difficult. So, the science curriculum is not suitable." [17SC] . "I do not think teachers have any issue in courses like social studies in terms of their curriculum. But the others who teach, for example, science and mathematics may experience some issues." [1SS] . "Our curriculum is somehow suitable for distance education. The book that was sent to us by the Ministry of Education has some activities that can be conducted in online settings. But it is not like face-to-face education. Students did not take it seriously although they were aware that their education is continuing." [12IN]. Due to the sudden emergence of the COVID-19 pandemic, critical changes were observed in the field of education as well as in other fields. Specifically, there was a direct shift to EDD from face-to-face education in Turkey. Although many people were familiar with distance education, they found themselves immersed in distance education in an unplanned way. Indeed, teachers had to carry out their curriculum, which had been designed for face-to-face education, in online settings. This study aims to gather secondary school teachers' opinions regarding their experiences related to their use of curriculum in EDD during the COVID-19 pandemic. The results are discussed below based on the four elements of a curriculum; aims and goals, content, teaching and learning process, and evaluation. According the findings, teachers report that they focused mainly on cognitive objectives, and less on affective and psychomotor objectives. Similar results are found in other studies as well. Code et al. (2020) report that distance education restricts teachers' and students' hands-on activities, and that the objectives taught remain only at a knowledge level that involves merely recognizing and recalling facts. Furthermore, the fact that distance education was only administered for theoretical-based courses and computer-based courses, which mainly have cognitive objectives, up to the time of the pandemic might be another reason for this situation (Kahraman, 2020) . Therefore, it is expected that teachers may have difficulties in helping students gain affective and psychomotor objectives. The participants also state that many students do not participate in or do not care much about online courses, which causes more problems for teachers while teaching. Specifically, a number of studies reveal an increase in dropout rates during the pandemic epidemic (Saavedra, 2020) , inequality in students' access to online courses due to technical issues and limited Internet access (Burke & Dempsey, 2020; Saran, 2020; Sintema, 2020) , and a gradual decrease in the participation rate in online courses over time (Tsai et al., 2020) . Therefore, it is critical to provide a minimum technical infrastructure for students in order to eliminate the inequality that some experience. In addition, teachers should be provided with inservice training programs concerning alternative approaches that encourage students to collaborate with their classmates. Considering the difficulties encountered, teachers mention that technical issues need to be reduced to a minimum. Queiros and de Villiers (2016) state that it is impossible to continue distance education with students who do not have sufficient access to instructional materials. Limited access to instructional materials is identified as one of the obstacles that diminishes the effectiveness of distance education, since many students may not have access to a technological tool, such as a computer, a tablet PC or a smart phone (Ferraro et al., 2020; Jena, 2020; Thomas & Rogers, 2020) . In this study, a number of teachers complain about the number of objectives in the curriculum and suggest that certain objectives need to be taken out of the curriculum during the EDD process and that course durations need to be adjusted depending on the course content. Students were obliged to be a part of distance education for the first time and, suddenly, course lengths in online settings were less than those in face-to-face education, which negatively affected the quality of distance education (Kruszewska et al., 2020; Kufi et al., 2020) . In this context, it may be concluded that the curricula used in face-to-face education were not suitable for the EDD. Therefore, alternative curricula applicable to distance education should be designed for such emergency situations. Turkey is one of the countries that shifted to distance education in a short time at the beginning of the pandemic. As a result of this shift, teachers, students, and parents found themselves in a situation with which they were not familiar. In addition, teachers were asked to teach the content in their current curricula in their online courses. This rapid shift caused teachers to ask 'what to teach'. During this process, teachers initially focused on keeping their students' motivation high and then on teaching content (Daniel, 2020; Lockee, 2020) . According to the findings, teachers did not develop materials to use in their courses, and they mainly benefited from ready-touse materials from the Educational Information Network, educational websites, and/ or social media sites. In similar studies, it was found that teachers generally used digital documents (slides, books, and quizzes and so on), web applications (for example, z-books), and various videos as instructional materials (Burke & Dempsey, 2020; Roy, 2020) . There may be two critical obstacles for teachers to develop their own materials; limited technological pedagogical knowledge and technical difficulties they experience (Huber & Helm, 2020; Vlachopoulos, 2020) . In order to prepare teachers for similar situations, it is critical to develop holistic teacher education programs and to provide in-service training programs for teachers in terms of technical issues that they may experience in online settings (Jandric, 2020; van der Spoel et al., 2020; Zhu, 2020) . In addition, a number of teachers in this study place emphasis on the need for more digital materials in order for more effective distance education. Likewise, there are studies emphasizing that textbooks should be prepared and restructured in accordance with the digital environment (Bayburtlu, 2020; Di Biase, 2021) . Another view is that since the COVID-19 pandemic caused people to change their view of distance education, students may be allowed to design their own learning, rather than forcing them to learn the same content (Istenič, 2021; Zhao, 2020) . Considering the awareness created by the pandemic, alternative curricula should be designed by considering the aspects of distance education in case of any emergency situations (Mbiydzenyuy & Silungwe, 2020) . Moreover, interactive platforms and materials should be developed in order to encourage students to embrace their own learning responsibilities in accordance with their individual learning pace. The participants report that they generally used direct instruction and questioning methods while teaching during the EDD. Other teaching methods and techniques, including brainstorming, dramatization, exhibits and displays, as well as problembased learning, were also used by the teachers, but at a minimal level. In the literature, questioning, direct instruction, and problem-based teaching were three teaching methods and techniques teachers used during distance education (König et al., 2020; Mortazavi et al., 2021) . The limited number of methods and techniques causes a decrease in students' motivation (Doğan & Koçak, 2020) . The inability of teachers to adapt activities that they easily employ in classrooms to the EDD process, and the inadequacy of course length, were also stated as problems in these studies. Moreover, the participants of the current study reveal that a lack of video communication, limited one-to-one communication, and issues with sound synchronization were also other problems they encountered during synchronous courses. Butler-Henderson and Crawford (2020) also mention similar issues in their study. These issues may cause learning deficiencies and a decrease in students' motivation (Bolliger & Halupa, 2012) . Therefore, with governmental support, uninterrupted audio and video communication should be ensured in the distance education process, which may enable teachers to employ various teaching methods and techniques. Although the learning process is teachers' responsibility in face-to-face education, this responsibility belongs to students in distance education (Simonson et al., 2019) . However, this study reveals that teachers designed only a few studentcentered activities in distance education, which may be related to the fact that teachers were dealing with distance education for the first time, did not have sufficient technical skills, and were not able to benefit from digital presentation techniques. In order to increase the effectiveness of distance education, the participants suggest that the number of in-service training programs should be increased to advance their technical skills, the participation of more students in online courses should be ensured, and that class sizes should be reduced. Considering all these suggestions, it may be concluded that the COVID-19 pandemic reveals that traditional teaching and learning approaches are insufficient and that alternative approaches should be planned for distance education in case of any future emergency situations. Therefore, from pre-school to higher education, the existing curricula should be revised and any infrastructure deficiencies of the distance education system should be eliminated. Due to the interruption of face-to-face education with the closure of schools, teachers have not been able to use many measurement and evaluation techniques. Therefore, a valid and reliable evaluation process could not be conducted. In institutions providing distance education, measurement and evaluation techniques are identified before online courses begin (Daniel, 2020) . However, this preliminary preparation did not occur due to the sharp shift to distance education because of the pandemic, which created many measurement and evaluation problems. In this study, the teachers report that they only use homework, end-of-unit quizzes, and students' participation and activity level in online courses as evaluation tools. In addition, they agree that these tools are not valid or reliable enough, due to an inequality among students in terms of their conditions for attending online courses. In order to have reliable and valid evaluation, they also place strong emphasis on standardized exams, exams through video conferencing, exams with open-ended questions, and the use of course-specific criteria. In this context, the pandemic epidemic may be considered as an opportunity for governments and leaders in education to re-structure measurement and evaluation techniques to be used in emergency situations (Daniel, 2020; Emler et al., 2019; Toquero, 2020) . Therefore, measurement and evaluation processes should also be revised to be applicable in distance education. Governments should also act to eliminate inequalities among students and to improve the technical infrastructure of schools. This study provides critical findings, based on middle school teachers' opinions, regarding the administration of curriculum they use during the EDD. However, this study has certain limitations. The first is related to the sample size and the data collection tool. This study was conducted with the participation of eighteen teachers and the data was collected through certain questions asked in interviews. Therefore, future studies must replicate the study with a larger sample size using different data collection tools. For this reason, there is a need for similar studies with different study groups using qualitative methods or studies involving larger samples with quantitative research methods. The second limitation concerns the grade level that the teachers taught. The participants were teachers who worked in middle schools. Future studies must consider obtaining elementary school and high school teachers' opinions about the administration of their curricula, which may allow stakeholders to identify deficiencies in the implementation of existing curricula in online learning environments. 7 Appendix 1 8 Interview questions 1. Do you think you can achieve the same curriculum objectives that you follow in the distance education process as in face-to-face education. Can you explain the reasons why you think? • What are your suggestions on this issue? 2. What did you do for cognitive, affective and psychomotor objectives? What did you experience in the process of during the acquisition of these objectives? • What are your suggestions on this issue? 3. What did you experience while using the course contents in the distance education process? (What kind of content did you use in the process? Which platform did you use? Was EIN sufficient in this process? • What are your suggestions on this issue? 4. Which methods and techniques did you use in the teaching-learning process? Why? • Was the course duration sufficient for you? • Were you able to include student-centred activities for your course? • Were you able to use the necessary materials for the course efficiently? • What did you experience while doing activities? Can you explain? • Do you think distance education imposes a restriction on the methods and techniques you use? Why? • What are your suggestions on this issue? 5. Which assessment and evaluation techniques did you use in the distance education process? What do you think about the techniques you used in the process? • What are your suggestions on this issue? 6. Do you think your curriculum is suitable for distance education? Why? I'm not against online teaching, but what about us? College students' use and acceptance of emergency online learning due to Covid-19 Exploring the critical challenges and factors influencing the E-learning system usage during COVID-19 pandemic. Education and Information Technologies COVID-19 and the challenges in education Covid-19 Pandemi Dönemi Uzaktan Eğitim Sürecinde Öğretmen Görüşlerine Göre Türkçe Eğitimi COVID and the impact on higher education: The essential role of integrity and accountability Administrative Theory & Praxis Student perceptions of satisfaction and anxiety in an online doctoral program Emergency remote teaching in a time of global crisis due to Cor-onaVirus pandemic COVID-19 Practice in primary schools in Ireland report A systematic review of online examinations: A pedagogical innovation for scalable authentication and integrity How are countries addressing the Covid-19 challenges in education? A snapshot of policy measures Pandemic designs for the future: perspectives of technology education teachers during COVID-19 COVID-19: 20 countries' higher education intra-period digital pedagogy responses Teachers and machines: The classroom use of technology since 1920 Faculty readiness for online crisis teaching: transitioning to online teaching during the COVID-19 pandemic Education and the COVID-19 pandemic Eğitimde program geliştirme. Pegem Yayıncılık Social Realities of Higher Education in the Age of Uncertainties Student engagement in distance learning environment: The experience of language certification preparation courses during the Coronavirus pandemic The Degree of Readiness to Total Distance Learning in the Face of COVID-19-Teachers' View (Case of Azerbaijan EBA sistemi bağlamında uzaktan eğitim faaliyetleri üzerine bir inceleme Examining the Turkish Universities' Distance Education Systems During the COVID-19 Pandemic The enlightened eye: Qualitative inquiry and the enhancement of educational practice Covid-19 Salgini ile Acil ve Side effects of large-scale assessments in education Eğitimde program değerlendirme Eğitimde program geliştirme Eğitimde program geliştirme kuramsal temellere bakış Distance Learning in the COVID-19 Era: Perceptions in Southern Italy The Greek University Student's Social Capital during the COVID-19 Pandemic Becoming qualitative researchers: An introduction Coping with Covid-19: Distance Learning and the Digital Divide The difference between emergency remote teaching and online learning COVID-19 and schooling: evaluation, assessment and accountability in times of crises-reacting quickly to explore key issues for policy, practice and research with the school barometer Shifting to digital during COVID-19: are teachers empowered to give voice to students? Educational Technology Research and Development Lack of internet access in Southeast Asia poses challenges for students to study online amid COVID-19 pandemic Postdigital research in the time of Covid-19 Impact of pandemic COVID-19 on education in India US Faculty and Administrators' Experiences and Approaches in the Early Weeks of the COVID-19 Pandemic COVID-19 salgınının uygulamalı derslere etkisi ve bu derslerin uzaktan eğitimle yürütülmesi: Temel tasarım dersi örneği Synchronic distance education from the perspective of a handicapped person COVID-19 Salgını Ve Beraberinde Getirdiği Eğitim Süreci Adapting to online teaching during COVID-19 school closure: teacher education and teacher competence effects among early career teachers in Germany Polish teachers of early education in the face of distance learning during the COVID-19 pandemic-the difficulties experienced and suggestions for the future Impact of corona pandemic on educational undertakings and possible breakthrough mechanisms Shifting digital, shifting context: (re)considering teacher professional development for online and blended learning in the COVID-19 era. Educational Technology Research and Development Teaching and Learning in resource-limited settings in the face of the COVID-19 pandemic Öğretim Programlarını Değerlendirme Raporu How We Do It: Implementing a Virtual, Multi-Institutional Collaborative Education Model for the COVID-19 Pandemic and Beyond Qualitative data analysis: An expanded sourcebook Handbook of distance education Distance education: A system view Students' perspectives on the virtual teaching challenges in the COVID-19 pandemic: A qualitative study A framework to guide an education response to the COVID-19 Pandemic of 2020 Devoloping the Curriculum (seventhedition) Curriculum: Foundatıons. Prıncıples and Issues Qualitative research & evaluation methods: Integrating theory and practice Online learning in a south African higher education institution: Determining the right connections for the student. The International Review of Research in Open and Distance Learning Online university teaching during and after the Covid-19 crisis: Refocusing teacher presence and learning activity Trying to home school because of coronavirus? Here are 5 tips to help your childlearn Educational challenges and opportunities of the Coronavirus (COVID-19) pandemic Technology: Digital Epiphany? COVID-19 and Our Tech Futures. Insight Report Challenges and Opportunities in the Post-COVID-19 World Curriculum planning for better teaching and learning Basic of qualitative research: Grounded theory procedures and techniques Adapting to a new reality: COVID-19 coronavirus and online education in the health professions. Rural and Remote Health Comparison of face-to-face and remote support for interventionists learning to deliver JASPER intervention with children who have autism Teaching and learning at a distance: Foundations of distance education Teaching and Learning at a Distance: Foundations of Distance Education Effect of COVID-19 on the performance of grade 12 students: implications for STEM education Curriculum development: Theory and practice. Harcourt, Brace and World How countries are using edtech (including online learning, radio, television, texting) to support access to remote learning during the COVID-19 pandemic Education, the science of learning, and the COVID-19 crisis Challenges and Opportunities for Higher Education Amid the COVID-19 Pandemic: The Philippine Context Experiencing the Transition to Remote Teaching and Learning during the COVID-19 Pandemic Teachers' online teaching expectations and experiences during the Covid19-pandemic in the Netherlands Education must continue throughout the pandemic COVID-19: Threat or opportunity for online education? WHO Director-General's opening remarks at the media briefing on COVID-19 COVID-19 Weekly Epidemiological Update Australian schools are closing because of coronavirus Development and emergency implementation of an online surgical education curriculum for a General Surgery program during a global pandemic: The University of Southern California experience High school students' experience of online learning during Covid-19: the influence of technology and pedagogy Technology, Pedagogy and Education Sosyal bilimlerde nitel araştırma yöntemleri Case study research and applications: Design and methods Eğitimde program değerlendirme COVID-19 as a catalyst for educational change Building up National Online Teacher Education System Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations