Creating the aspired intelligent assessment systems for teaching materials Expert Systems with Applications 38 (2011) 12168–12179 Contents lists available at ScienceDirect Expert Systems with Applications j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / e s w a Creating the aspired intelligent assessment systems for teaching materials Cheng-Hsiung Chen a, Gwo-Hshiung Tzeng b,c,⇑ a Department of Information Management, Kainan University, No. 1, Kainan Road, Shinshing Tsuen, Luchu Shiang, Taoyuan 338, Taiwan b Distinguished Chair Professor, Kainan University, No. 1, Kainan Road, Shinshing Tsuen, Luchu Shiang, Taoyuan 338, Taiwan c Institute of Management of Technology, National Chiao-Tung University, 1001, Ta-Hsueh Road, Hsin-Chu 300, Taiwan a r t i c l e i n f o a b s t r a c t Keywords: Intelligent assessment system Teaching materials MCDM (Multiple Criteria Decision Making) DEMATEL ANP (Analysis Network Process) VIKOR Core competence (CC) 0957-4174/$ - see front matter � 2011 Elsevier Ltd. A doi:10.1016/j.eswa.2011.03.050 ⇑ Corresponding author at: Distinguished Chair Pro 1, Kainan Road, Shinshing Tsuen, Luchu Shiang, Taoy 937 541 184. E-mail addresses: hsiung.chen@msa.hinet.net (C. edu.tw, ghtzeng@mail.knu.edu.tw, arthur@mail.knu.e The core objective of the Nine-Year Integrated Curriculum for primary schools is to ‘‘enable students to demonstrate their talents instead of just scoring high on exams’’ around the world. The determining fac- tor in education reform is to declare the competence indicators of necessary educational behavior in pri- mary and junior high school. In the reform process, for all domains, the enriched rate of competence indicators for educational materials and methods is very meaningful. Because educational materials and methods in different domains have their own style, we should evaluate these teaching materials sep- arately. Thus, in this research we propose a novel MCDM (Multiple Criteria Decision Making) framework for evaluating, comparing, and improving the effectiveness of competence indicators in the various pub- lications for teaching materials in primary school based on different viewpoints. The ANP (Analysis Net- work Process) weights are based on the DEMATEL technique with the MCDM method for resolving the problems of dependence and feedback among criteria. Then, a VIKOR technique with ANP weights is proposed for addressing and reducing the performance gaps for each criterion, thus hopefully improving, re-configuring and selecting the aspired Intelligent Assessment Systems (IAS) for teaching materials. An empirical study of Mandarin Chinese based on this system design of three publishers is illustrated to verify the effectiveness of the proposed method. The results can improve the efficiency and quality of the authored Mandarin Chinese teaching materials and may extend to other Learning Areas. � 2011 Elsevier Ltd. All rights reserved. 1. Introduction Every country has made the cultivation of human talent a priority in the 21st century. As other advanced countries propose education reform in different forms throughout the world, Taiwan also recognizes education as the bedrock of national development, implementing various education reforms such as pre-school edu- cation reform, grade 1–9 curriculum reform, the restructuring of secondary education, the enhancement of higher education, and lifelong learning projects (MOE, 2008). Therefore, the purpose of this research is to propose a novel technique and evaluation method that can improve, re-configure and select the aspired Intel- ligent Assessment Systems (IAS) for teaching materials to promote education levels. Since 1990, the four British educational reform contexts have been: (a) the right to national centralization reform; (b) the priority of compulsory education; (c) vocational education mainstreaming; and (d) the ability to pursue educational evaluation (DfE, 1991). ll rights reserved. fessor, Kainan University, No. uan 338, Taiwan. Tel.: +886 -H. Chen), ghtzeng@cc.nctu. du.tw (G.-H. Tzeng). The report of the Mayer Committee proposed to advise the Austra- lian Education Council (AEC) and the Ministers for Vocational Educa- tion, Employment and Training (MOVEET) on employment-related key competencies for post-compulsory education and training (Mayer, 1992). The Education Commission of Hong Kong proposed the following: ‘‘Learning is the key to one’s future, and education is the gateway to our society’s tomorrow’’. Education enables indi- viduals to develop their potential, construct knowledge and enhance the quality of their personal lives (EC, 2000). In general, no matter what style of education reform is used, the key competencies are the major concern for national education re- form at the beginning of the 21st century. The core objective of Nine-Year Integrated Curriculum for pri- mary schools in Taiwan is to ‘‘enable students to demonstrate their talents instead of just scoring high on exams’’. The determining factor in education reform is to declare the competence indicators for necessary educational behavior in primary school (MOE, 2002). In the reform process, for all domains, the enriched rate of compe- tence indicators for educational materials and methods is very meaningful. Because educational materials and methods in differ- ent domains have their own style, we should evaluate these teach- ing materials separately. Thus, in this research we propose a novel MCDM (Multiple Criteria Decision Making) framework based on the DEMATEL http://dx.doi.org/10.1016/j.eswa.2011.03.050 mailto:hsiung.chen@msa.hinet.net mailto:ghtzeng@cc.nctu.edu.tw mailto:ghtzeng@cc.nctu.edu.tw mailto:ghtzeng@mail.knu.edu.tw mailto:arthur@mail.knu.edu.tw http://dx.doi.org/10.1016/j.eswa.2011.03.050 http://www.sciencedirect.com/science/journal/09574174 http://www.elsevier.com/locate/eswa C.-H. Chen, G.-H. Tzeng / Expert Systems with Applications 38 (2011) 12168–12179 12169 (Decision Making Trial and Evaluation Laboratory) technique for evaluating, comparing, and improving the effectiveness of compe- tence indicators in the various publications for teaching materials in primary school based on different viewpoints. The ANP (Analysis Network Process) weights are based on the DEMATEL technique with the novel MCDM method for resolving the problems of depen- dence and feedback among criteria. Then, a VIKOR technique with ANP weights is proposed for addressing and reducing the perfor- mance gaps for each criterion, thus hopefully improving, re-config- uring and selecting the aspired Intelligent Assessment Systems for teaching materials. An empirical study of Mandarin Chinese teach- ing materials in Grade 1 of primary school based on this system de- sign of three publishers is illustrated to verify the effectiveness of the proposed method. The results can improve the efficiency and quality of the authored Mandarin Chinese teaching materials and may extend to other Learning Areas. The remainder of this paper is organized as follows. In Section 2, the aspired intelligent assessment systems for teaching materials with MCDM are introduced. In Section 3, a novel MCDM method based on the DEMATEL technique is proposed. In Section 4, an empirical study for the aspired IAS for Mandarin Chinese teaching materials is presented to show the process that our proposed method entails, and discussions are conducted. Finally, in Section 5, concluding remarks are presented. 2. Intelligent assessment systems for teaching materials with MCDM method In recent decades, competence-based education has become the mega trend impacting the education reform strategies of the majority number of governments throughout the world. In the fol- lowing section, the literatures related to core competence (CC), the intertwined effects of an assessment system for teaching materials, will be reviewed as a foundation for the development of the theo- retical framework of this paper. We also give an example of Taiwan to explain the basic concepts of Educational Reform for teaching materials. 2.1. Educational reform of Taiwan (MOE, 2002) In keeping with the progress of the 21st century and the global trends of educational reform, Taiwan must engage in educational reform in order to foster national competitiveness and boost the overall quality of our citizen’s lives. The Ministry of Education (MOE) of Taiwan, therefore, has initi- ated curricular and instructional reforms in primary and junior high school education. These reforms have been based on the Table 1 Numbers for competence indicators for the mandarin Chinese curriculum for ea Criterion of CC Stag D1: Physical, mental, and spiritual mold C1: Self-understanding and exploration of potential 24 C2: Appreciation, representation, and creativity 20 C3: Career planning and lifelong learning 9 D2: Interpersonal and social relations C4: Expression, communication, and sharing 8 C5: Respect, care and teamwork 9 C6: Cultural learning and international understanding 6 C7: Planning, organizing and putting plans into practice 7 D3: The use of Life Science and Technology C8: Utilization of technology and information 4 D4: Logical thinking and reasoning C9: Active exploration and study 8 C10: Independent thinking and problem-solving 9 Sum 104 Sources: (MOE, 2002) Action Plan for Educational Reform approved by the Executive Yuan of Taiwan. Because the curriculum is not only the core of schooling but also the foundation on which teachers plan learning activities, the MOE places top priority on the development and implementation of the Grade 1–9 Curriculum. Curricular reforms are necessary and will be timely for the following reasons: (a) meeting national development needs; and (b) meeting public expectations. 2.2. Assessment system for mandarin chinese teaching materials Mandarin Chinese is one of the curricula included in the Lan- guage Arts. It is divided into 3 stages: Grades 1–3, Grades 4–6, and Grades 7–9. 2.2.1. Mandarin Chinese’s curriculum goal Based on the Curriculum Goals and CCs of the Grade 1–9 Curric- ulum, a more detailed description is given in sub Sections 4.1.2 and 4.1.3, with the Mandarin Chinese Curriculum Goal defined as fol- lows: (1) to utilize language to inspire individual potential and to cultivate learning; (2) to cultivate interest in writing and enhance ability to appreciate literature; (3) to equip with the self-learning ability for language learning and lay the foundation for lifelong learning; (4) to utilize language and words for expressions of emotion, experience-sharing and communications; (5) to utilize language expression to adapt to one’s environment and to demon- strate appropriate behavior; (6) to understand and recognize the Chinese, Taiwanese and foreign cultures and rituals through lan- guage learning; (7) to utilize the power of language to develop and implement plans effectively; (8) to combine the information of lan- guage and technology to enhance learning and to expand fields of study; (9) to cultivate an interest in language exploration and a pro- active attitude towards language-learning; (10) to utilize language for independent thinking and problem-solving. 2.2.2. Competence indicators of Mandarin Chinese’s curriculum of grade 1–9 curriculum Based on the requirements for children’s intellectual develop- ment, the numbers for the competence indicators of the Mandarin Chinese Curriculum for each stage are defined (see Table 1). In general, this includes listening, speaking, reading and writing of languages, and developing basic communication competencies. From the academic point of view, six terms were developed as fol- lows: (1) phonetic symbol applications; (2) listening skills; (3) the ability to speak; (4) literacy and writing ability; (5) reading skills; and (6) writing skills. ch stage. e1 Stage2 Stage3 Sum 19 18 61 17 17 54 13 10 32 9 12 29 9 13 31 8 6 20 7 6 20 11 8 23 7 9 24 8 7 24 108 106 318 Fig. 1. An analytical framework for aspired assessment system for teaching materials. s2 2 3 1 3 s1 s4 s3 s2 2 3 1 3 s1 s4 s3 Fig. 2. An example of the directed graph. 12170 C.-H. Chen, G.-H. Tzeng / Expert Systems with Applications 38 (2011) 12168–12179 3. A novel MCDM method based on the DEMATEL technique with ANP The structure of the MCDM problem will be derived using the DEMATEL method. The priorities of every determinant are based on the structure derived by using the ANP. The VIKOR technique will be leveraged to calculate the compromise ranking of the alter- natives. Finally, the assessment system for Mandarin Chinese teaching materials will be derived. In summary, this evaluation framework consists of five main phases: (1) establishing determi- nants using the questionnaire survey method; (2) building the structure of network relation map (NRM) for the determinants by using DEMATEL; (3) calculating the priorities of every determinant using the ANP based on the structure of NRM derived by using DEMATEL in (2); (4) ranking the priorities of the assessment sys- tem for teaching materials using the VIKOR technique; and finally (5) establishing the assessment system for teaching materials and achieving the aspired levels (see Fig. 1). 3.1. DEMATEL Technique with ANP The DEMATEL technique was developed by the Battelle Geneva Institute: (1) to analyze complex ‘world problems’ dealing mainly with interactive man-model techniques; and (2) to evaluate qual- itative and factor-linked aspects of societal problems (Gabus & Fontela, 1972). The applicability of the method is widespread, with applications ranging from industrial planning and decision-making to urban planning and design, regional environmental assessment, the analysis of world problems, and so forth. It has also been suc- cessfully applied in many situations and fields, such as those of marketing strategies, control systems, safety problems, developing the competencies of global managers, group decision-making and so on (Chen, Lien, & Tzeng, 2010; Chiu, Chen, Tzeng, & Shyu, 2006; Huang, Shyu, & Tzeng, 2007; Lee, Tzeng, Hsu, & Huang, 2009; Li & Tzeng, 2009a, 2009b; Lin & Tzeng, 2009; Lin & Wu, 2008; Liou, Tzeng, & Chang, 2007; Ou Yang, Leu, & Tzeng, 2009; Ou Yang, Shieh, Leu, & Tzeng, 2008; Tzeng, Chen, Yu, & Shih, 2010; Wu & Lee, 2007). Furthermore, a hybrid model combining the two methods has been widely used in various fields – for example, e-learning evaluation (Tzeng, Chiang, & Li, 2007), teach- ing materials assessment (Chen and Tzeng, 2009), airline safety measurement (Liou et al., 2007), and innovation policy portfolios for Taiwan’s SIP Mall (Huang et al., 2007). Therefore, in this paper we use DEMATEL not only to detect complex relationships and build a NRM for the criteria but also to obtain the influence levels of each element over others; we then adopt these influence level values as the basis of the normalization supermatrix for determin- ing ANP weights to obtain the information about relative impor- tance. To apply the DEMATEL method smoothly, the authors refined the definitions based on the above authors and produced the essential definitions indicated below. The DEMATEL method is based upon graph theory, enabling us to plan and solve problems visually, so that we may divide multiple criteria into a relationship of cause and effect to better understand causal relationships. Direc- ted graphs (also called digraphs) are more useful than directionless graphs because digraphs will demonstrate the directed relation- ships of sub-systems. A digraph typically represents a communica- tion network or a domination relationship between individuals. Suppose a system contains a set of elements, S = {s1, s2, . . . , sn}, and particular pair-wise relationships are determined for modeling with respect to a mathematical relationship, MR. Next, portray the relationship MR as a direct-relation matrix that is indexed equally in both dimensions by elements from the set S. Then, extract the case for which the number 0 appears in the cell (i, j) if the entry is a positive integral that has the following meaning: the ordered pair (si, sj) is in the relationship MR; and has the kind of relation- ship regarding that element such that si causes element sj. The di- graph portrays a contextual relationship between the elements of the system, in which a numeral represents the strength of influ- ence (Fig. 2). The number between factors is the influence or influ- enced degree. For example, an arrow from s1 to s2 represents the fact that s1 influences s2 and that its influenced degree is two. The DEMATEL method can convert the relationship between the causes and effects of criteria into an intelligible structural model of the system (Chiu et al., 2006).. Definition 1. The pair-wise comparison scale may be designated as five levels, where the scores 0, 1, 2, . . . , 4 represent the range from ‘no influence’ to ‘very high influence’. Definition 2. The initial direct relation/influence matrix A is an n � n matrix obtained by pair-wise comparisons in terms of influ- ences and directions of influence between the determinants, in which aij is denoted as the degree to which the ith determinant affects the jth determinant. A ¼ a11 a12 � � � a1n a21 a22 � � � a2n .. . .. . . . . .. . an1 an2 � � � ann 2 66664 3 77775 Definition 3. The normalized direct relation/influence matrix N can be obtained through Eqs. (1) and (2), in which all principal diagonal elements are equal to zero. N ¼ zA ð1Þ Goal Criteria Subcriteria Control Criteria Goal Criteria Subcriteria Control Criteria C.-H. Chen, G.-H. Tzeng / Expert Systems with Applications 38 (2011) 12168–12179 12171 where z ¼ 1 max max 16i6n Xn j¼1 aij; max 16j6n Xn i¼1 aij !, : ð2Þ In this case, N is called the normalized matrix. Because limq!1N q ¼ ½0�n�n A possible different network under each subcriterion of the control hierarchyA possible different network under each subcriterionof the control hierarchy Source: (Saaty, 1996) Fig. 3. The control hierarchy. Definition 4. Then, the total relationship matrix T can be obtained using Eq. (3), where I stands for the identity matrix. T ¼ N þ N 2 þ�� �þ N q ¼ NðI � NÞ�1 ð3Þ ½Explain� T ¼ N þ N 2 þ�� �þ N q ¼ NðI þ N þ N2 þ���þ N q�1ÞðI � NÞðI � NÞ�1 ¼ NðI � N qÞðI � NÞ�1 ¼ NðI � NÞ�1; when limq!1N q ¼ ½0�n�n where q ? 1 and T is a total influence-related matrix; N is a direct influence matrix and N = [xij]n�n;limq?1(N 2 + � � � + Nq) stands for a indirect influence matrix, 0 6 Pn j¼1xij < 1 and 0 6 Pn i¼1 xij < 1, and only one Pn j¼1 xij or Pn i¼1xij is equal to 1 for "i, j, but not all. So, limq?1 N q = [0]n�n. The (i, j) element tij of matrix T denotes the di- rect and indirect influences of factor i on factor j. Definition 5. The row and column sums are separately denoted as r and c within the total-relation matrix T through Eqs. (4) and (5). T ¼ ½tij� i; j 2f1; 2; . . . ; ng ð4Þ r ¼ ½ri�n�1 ¼ Xn j¼1 tij " # n�1 and c ¼ ½cj�n�1 ¼ Xn i¼1 tij " #0 1�n ð5Þ where the vectors r and c denote the sums of the rows and columns, respectively. Definition 6. Suppose that ri denotes the row sum of the ith row of matrix T. Then, ri is the sum of the influences of factor i on the other factors, both directly and indirectly. Suppose that cj denotes the column sum of the jth column of matrix T. Then, cj is the sum of the influences that factor j is receiving from the other factors. Fur- thermore, when i = j (i.e., the sum of the row sum and the column sum (ri + ci) represents the index indicating the strength of the influence, both dispatching and receiving), (ri � ci) is the degree of the central role that factor i plays in the problem. If (ri � ci) is positive, then factor i primarily is influencing the strength of the other factors; and if (ri � ci) is negative, then factor i primarily is receiving influence from other factors (Huang et al., 2007; Liou et al., 2007; Tamura, Nagata, & Akazawa, 2002). 3.2. The ANP method The ANP method, a multi-criteria theory of measurement devel- oped by Saaty (1996), provides a general framework for dealing with decisions without making assumptions about the indepen- dence of higher-level elements from lower-level elements and about the independence of the elements within a level as in a hier- archy. Compared with traditional MCDM methods (Saaty, 2005) – e.g. AHP (Analytic Hierarchy Process), TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution), ELECTRE (ELimina- tion Et Choix Traduisant la REalité), etc. (Lee et al., 2007) – which usually assume independence between criteria, ANP, a new theory that extends AHP to deal with dependence in feedback and utilizes the supermatrix approach (Saaty, 2003), is a more reasonable tool for dealing with complex MCDM problems in the real world. In this section, the concepts of the ANP are summarized based on Saaty’s earlier works (Saaty, 1996, 1999, 2005). The ANP is a coupling of two parts. The first consists of a control hierarchy or network of criteria and subcriteria that control the interactions. The second is a network of influences among the ele- ments and clusters. The network varies from criterion to criterion, and a different supermatrix of limiting influence is computed for each control criterion. Finally, each of these supermatrices is weighted based on the priority of its control criterion, and the re- sults are synthesized through addition for all the control criteria (Saaty, 2004). A control hierarchy is a hierarchy of criteria and sub- criteria for which priorities are derived in the usual way with re- spect to the goal of the system being considered. The criteria are used to compare the components of a system, and the subcriteria are used to compare the elements. The criteria with respect to which influence is presented in individual super- matrices are called control criteria. Because all such influences ob- tained from the limits of the several supermatrices will be combined to obtain a measure of the priority of overall influence, the control criteria should be grouped in a structure to be used to derive priorities for them. These priorities will be used to weight the corresponding individual supermatrix limits and sum up. The analysis of priorities in a system can be thought of in terms of a control hierarchy with dependence among its bottom-level alter- natives arranged as a network as shown in Fig. 3. Dependence can occur within the components and between them. A control hierarchy at the top may be replaced by a control net- work with dependence among its components, which are collec- tions of elements whose functions derive from the synergy of their interaction and that hence have a higher-order function not found in any single element. The criteria in the control hierarchy that are used for comparing the components are usually the major parent criteria whose subcriteria used to compare the elements need to be more general than those of the elements because of the greater complexity of the components. A network connects the components of a decision system. According to size, there will be a system made up of subsystems, with each subsystem made up of components and each component made up of elements. The elements in each component interact or have an influence on some or all of the elements of another com- ponent with respect to a property governing the interactions of the entire system, such as energy, capital, or political influence. Fig. 4 demonstrates a typical network. Those components that no Source Component Source Component Source Component (Feedback loop) Source Component (Feedback loop) Intermediate Component (Transient State) Intermediate Component (Transient State) Sink Component (Absorbing State) Sink Component (Absorbing State) Intermediate Component (Recurrent State) Intermediate Component (Recurrent State) Outerdependence Innerdependence loop C1 C3 C4 C5 C2 Source Component Source Component Source Component (Feedback loop) Source Component (Feedback loop) Intermediate Component (Transient State) Intermediate Component (Transient State) Sink Component (Absorbing State) Sink Component (Absorbing State) Intermediate Component (Recurrent State) Intermediate Component (Recurrent State) Source Component Source Component Source Component (Feedback loop) Source Component (Feedback loop) Intermediate Component (Transient State) Intermediate Component (Transient State) Sink Component (Absorbing State) Sink Component (Absorbing State) Intermediate Component (Recurrent State) Intermediate Component (Recurrent State) Outerdependence Innerdependence loop C1 C3 C4 C5 C2 Source: (Satty, 1996) Fig. 4. Connections in a network. 12172 C.-H. Chen, G.-H. Tzeng / Expert Systems with Applications 38 (2011) 12168–12179 arrow enters are known as source components, such as C1 and C2. Those from which no arrow leaves are known as sink components, such as C5. Those components which arrows both enter and exit leave are known as transient components such as C3 and C4. In addition, C3 and C4 form a cycle of two components because they feed back and forth into each other. C2 and C4 have loops that con- nect them to themselves and are inner-dependent. All other con- nections represent dependence between components, which are thus known to be outer-dependent. A component (dimension) of a decision network that was de- rived by the DEMATEL method in sub Section 3.1 will be denoted by Ch, h = 1, . . . , m; assume that it has nh elements (determinants), which we denote by eh1; eh2; . . . ; ehnh . The influence of a given set of elements (determinants) in a component (dimension) on any element in the decision system is represented by a ratio scale pri- ority vector derived from paired comparisons of the comparative importance of one criterion and another criterion with respect to the interests or preferences of the decision-makers. This relative importance value can be determined using a scale of 1–9 to repre- sent equal importance to extreme importance (Saaty, 1996). The influence of elements (determinants) in the network on other ele- ments (determinants) in that network can be represented using the following supermatrix: A typical entry Wij in the supermatrix is called a block of the supermatrix in the following form, where each column of Wij is a principal eigenvector of the influence of the elements (determi- nants) of the ith component of the network on an element (determi- nant) in the jth component. Some of its entries may be zero corresponding to those elements (determinants) that have no influence. W ij ¼ wi1 j1 wi1 j2 � � � wi1 jnj wi2 j2 wi2 j2 � � � wi2 jnj .. . .. . . . . .. . wini j1 wini j2 � � � wini jnj 2 666664 3 777775 After forming the supermatrix, the weighted supermatrix is derived by transforming all columns sum to unity exactly. This step is very much similar to the concept of the Markov chain in terms of ensur- ing that the sum of these probabilities of all states equals 1. Next, the weighted supermatrix is raised to limiting powers, such as limh?1W h to get the global priority vector or called weights (Huang, Tzeng, & Ong, 2005). 3.3. VIKOR The compromise ranking method (VIKOR) was proposed by Opricovic (Opricovic, 1998) as one applicable technique to imple- ment within MCDM. We assume that the alternatives are denoted as A1, A2, . . . , Ak, . . . , Am. The rating (performance score) of the jth criterion is denoted by fkj for alternative Ak, wj is the weight of the jth criterion, expressing the relative importance of the criteria, where j = 1, 2, . . . , n, and n is the number of criteria. The VIKOR method began with the following form of theLp � metric: Lpk ¼ Xn j¼1 wj f � j � fkj ��� ���� �h . f �j � f �j ��� ���� � #p( )1=p where 1 6 p 61, k = 1, 2, . . . , m; weight wj is derived using the ANP according to the NRM based on the DEMATEL method. The VIKOR method also uses Lp¼1k (as Sk) and L p¼1 k (as Qk) to formulate the rank- ing measure (Opricovic, 1998; Opricovic & Tzeng, 2002, 2004, 2007; Tzeng, Teng, Chen, & Opricovic, 2002, 2005). Sk ¼ Lp¼1k ¼ Pn j¼1 wj f �j � fkj ��� ���� �= f �j � f �j ��� ���� �h i Q k ¼ L p¼1 k ¼ maxj wj f � j � fkj ��� ���� �= f �j � f �j ��� ���� �jj ¼ 1; 2; . . . ; nn o The compromise solution minkL p k will be chosen because its value is closest to the ideal/aspired level. In addition, when pis small, group utility is emphasized (such as p = 1), and as pincreases to p = 1, the individual maximal regrets/gaps receive more importance, as shown by Yu (Freimer & Yu, 1976; Yu, 1973). Therefore, miniSi emphasizes the maximum group utility, whereas miniQi empha- sizes selecting the minimum of the maximum individual regrets. Based on the above concepts, the compromise ranking algorithm VI- KOR has the following steps. Step 1. Normalize the original rating matrix. In this step, we deter- mine the best f �j and the worst f � j values of all criterion functions, j = 1, 2, . . . , n. If we assume that the jth function represents a benefit, f �j ¼ maxifkj (or setting an aspired level) and f �j ¼ minifkj (or setting a tolerable level). Alter- natively, if we assume that the jth function represents a cost, f �j ¼ minifkj (or setting an aspired level) and f �j ¼ maxi fkj (or setting a tolerable level). Moreover, an ori- ginal rating matrix is transformed into a normalized weight-rating matrix with the following formula: rkj ¼ f �j � fkj ��� ���� �= f �j � f �j ��� ���� � Step 2. Compute the values Sk, Qk, when k = 1, 2, . . . , m, using the rela- tions (Ou Yang et al., 2008, Ou Yang, Leu, & Tzeng, 2009) C.-H. Chen, G.-H. Tzeng / Expert Systems with Applications 38 (2011) 12168–12179 12173 Sk ¼ Xn j¼1 wjrkj and Q k ¼ maxjfrkjj ¼ 1; 2; . . . ; ng where Sk and Qk show the mean of group utility and maximal regret, respectively. Using the traditional VIKOR method, Qk is represented as maxj{wjrkjjj = 1, 2, . . . , n}, which implies that group utility is more important than maximal regret. Because Qk is only a part of Sk, Sk is unquestionably more than Qk. Therefore, Sk is emphasized more than Qk using the traditional VIKOR method. However, maximal re- gret is also very important in practice and is usually taken into ac- count to improve it. Therefore, to balance Sk and Qk, Qk = maxj{rkjjj = 1, 2, . . . , n} is used instead of the traditional VIKOR Qk. Step 3. Compute the index values Rk, i = 1, 2, . . . , m,using the relation Rk ¼ vðSk � S �Þ=ðS� � S�Þþð1 � vÞðQ k � Q �Þ=ðQ� � Q�Þ where S� ¼ minkSk; S � ¼ maxkSk; Q � ¼ mink Q k; Q � ¼ maxkQ k; (here, we can also set the best value to 0 and the worst value to 1) and 0 6 v 6 1, where v is introduced as a weight for the strategy of maximum group utility, whereas 1 � v is the weight of the individ- ual regret. Step 4. Rank the alternatives, sorting by the value of Sk, Qk, and Rk, for k = 1, 2, . . . , m, in decreasing order. Propose as a compro- mise the alternative (A(1)), which is ranked first by the measure min{Rkjk = 1, 2, . . . , m} if the following two condi- tions are satisfied: C1. Acceptable advantage: R(A(2)) � R(A(1)) P 1/(m � 1), where A(2) is the alternative with the second position on the ranking list by R; m is the number of alternatives. C2. Acceptable stability in decision making: Alternative A(1) must also be the best ranked by Sk or/and Qk, k = 1, 2, . . . , m. A set of compromise solutions is proposed if one of the condi- tions is not satisfied. The set of compromise solutions consists of the following: (1) alternatives A(1) and A(2) if only condition C2 is not satisfied; (2) alternatives A(1), A(2), . . . , A(M) if condition C1 is not satisfied. A(M) is determined by the relation R(A(M)) � R(A(1)) < 1/(m � 1) for maximum M (the positions of these alternatives are close). The compromise-ranking method (VIKOR method) determines the compromise solution; the obtained compromise solution is acceptable to the decision-makers because it provides the maxi- mum group utility of the majority (represented by minS) and a minimum of individual maximal regret for the opponent (repre- sented by minQ). The model uses the DEMATEL and ANP proce- dures in sub Sections 3.1 and 3.2 to obtain the weights of the criteria with dependence and feedback; it uses the VIKOR method to obtain the compromise solution. 4. An empirical study of the aspired assessment systems for mandarin chinese teaching materials In this section, an example being modified from a real case will be presented to demonstrate the effectiveness of the proposed novel MCDM framework using the DEMATEL technique. One empirical study example will be based on an example modified from three leading publishers. In this case study, Mandarin Chinese teaching materials (six books) for Grade 1 in primary school in Taiwan are selected; these materials were edited by those publishers. 4.1. Background description In the twenty-first century, major changes have taken place around the world in social, political, economic and cultural arenas. These changes are not only global but also national. In a drastic change, the most countries have become aware of the importance of education and culture. Education reform in these countries has been carried out to stimulate personal potential, to overtake the fine culture, and to promote social progress. After six decades of postwar development, Taiwan has transi- tioned from a traditional agricultural society into a modern indus- trial society. Political, economic, and cultural environments are faced with modernization, industrialization and the technological influences of structural adjustment and reconstruction. The impact of education reform is one of the most far-reaching and extensive implications of a major reform. It affects national self-positioning, slushing the social consciousness, establishing a new culture and developing national competitiveness for the new century. The effectiveness of economic development and of the develop- ment of democracy in Taiwan have aroused praise from more developed countries. It is a fact that the main contributing factor is the spread of education and enhancement of people’s quality of life. Taiwan is becoming an educational community. However, in the development of education over the years, many problems have emerged, and delays in solving the problems have made them even more complicated. In light of this, the Council on Education Reform was established in September, 1994. It was responsible for educational reform and the educational development of research and consideration, the Commission ‘‘Education overall reform Consultation Report’’ was presented in December of 1996. Taiwan should actively engage in reform. Issues that need to be addressed include the following: establishing a life-long learning society; enhancing equal opportunities in education; guiding examination biased towards intellectual culture; improving course materials and assessments; improve the multi-teacher education system; and improving the efficiency of the use of educational re- sources. The influence of education reform on the social and per- sonal is significant. We must ponder social change movements and carefully consider how the value of benchmarks of socio-cul- tural development is determined, when they occur in the context of education reform, rather than acting rashly. The Council has developed a comprehensive education reform proposal. It is divided into five dimensions as follows: (1) educa- tion deregulation, (2) good educational care for every student, (3) smoother avenues to higher education, (4) improved education quality, and (5) developing a life-long learning society. Speaking of the need to provide proper educational care for every student, it is clear that schools have not given adequate attention to students, especially disadvantaged students. This has been mainly due to the rigidity of primary school and junior high school education (compulsory education) in Taiwan, with a uni- form system and curriculum, coupled with a lack of long-term investment in resources, the unusual effect of teaching, and a sin- gle mode for entrance writing examinations at senior high schools and universities. Early on in the education of those disadvantaged students, they were not able to develop a good foundation for learning, and they were then exposed to large class size and a lack of timely and adequate care in general, so that their performance was different from that of the other students and were relatively vulnerable at their schools. 12174 C.-H. Chen, G.-H. Tzeng / Expert Systems with Applications 38 (2011) 12168–12179 The Council proposed the following specific recommendations: (1) to reform the curriculum and instruction; (2) to reduce school size and class size; (3) to open autonomous schools; (4) to stimu- late schools to use the inner strength of their own power; (5) to as- sist each student with basic competence; (6) to establish a remedial teaching system; (7) to strengthen career counseling and provide multiple approaches; (8) to renew a system of student behavioral counseling; (9) to weaken the physical and psychologi- cal barriers to education; (10) to attach importance to aboriginal education; (11) to implement gender equality in education; and (12) to protect the quality of early childhood education. In the meantime, the Education Reform Committee, Taiwan, MOE proposed the curricular reforms for the Grades 1–9 (compul- sory education) Curricula. The current Curriculum Frameworks for Primary Schools and Junior High Schools were revised and promulgated in 1993 and 1994, respectively. Although the current Curriculum Frameworks have been gradually and properly implemented, the MOE believes that innovative thinking and practice in education are the prereq- uisites for success in the new century. Therefore, the MOE has launched plans for another curricular reform to build up consensus and integrate efforts at education reform to create a new and bet- ter environment for school education. The development of a new curriculum was divided into three stages. The duration and major tasks for each stage are shown in detail as follows: (1) Stage 1 (from April 1997 to September 1998): Establishing the Special Panel on the Development of Primary and Junior High Schools’ Cur- ricula; (2) Stage 2 (from October 1998 to November 1999): Estab- lishing the Panel on Researching and Formulating the Guidelines of Each Learning Area in the Grade 1–9 Curriculum, which covered the primary school and junior high school levels of education; and (3) Stage 3 (from December 1999 to August 2002): Establish- ing the Review Committee on the Revision and Formulation of Ele- mentary and Junior High School Curricula. 4.1.1. Core rationale The aim of education is to foster sound minds and character in students. Students should be taught democratic values, the Rule of Law, and humanitarian ideals; they should develop strong and healthy physiques, learn how to think for themselves and be crea- tive. Every government hopes that the school system will produce outstanding citizens with a sense of patriotism and the ability to adopt a global perspective. In essence, education is a learning pro- cess to help students explore their potential and develop their capacity to adapt and make the necessary efforts to improve their living environment. Given that, the following five basic aspects are emphasized and included in Grade 1–9 Curricula designed for the new century: developing humanitarian attitudes, enhancing the ability to integrate, cultivating democratic literacy, fostering both indigenous awareness and a global perspective, and building up the capacity for lifelong learning. The core components of each as- pect are as follows: (A) humanitarian attitude; (B) integration abil- ity; (C) democratic literacy; (D) native awareness and a global perspective; and (E) capacity for lifelong learning. 4.1.2. Curriculum goals The curricula for primary and junior high schools will adopt the following principles: (A) to involve all aspects of daily life that are related to students’ mental and physical development; (B) to encourage the development of individuality and the exploration of one’s potential; (C) to foster democratic literacy and respect for different cultures; and (D) to develop scientific understanding and competencies to meet the needs of modern life. The aim of national education is to teach students to obtain ba- sic knowledge and to develop the capacity for lifelong learning, thus cultivating able citizens who are mentally and physically healthy, vigorous and optimistic, gregarious and helpful to the community, intellectually curious and reflective, tolerant, and cre- ative, with a positive attitude and a global perspective. Schools will achieve such ideals through the promotion of educational learning activities that emphasize humanity, practicality, individuality, comprehensiveness, and modernity. Such activities include inter- actions between oneself and others, individuals and the commu- nity, and humans and nature. Regarding this aspect of national education, we must guide our students to achieve the following curriculum goals: (A) to enhance their self-understanding and ex- plore their individual potential; (B) to develop creativity and the ability to appreciate beauty and present their own talents; (C) to promote abilities related to career planning and lifelong learning; (D) to cultivate knowledge and skills related to expression, com- munication, and sharing; (E) to learn to respect others, care for the community, and facilitate teamwork; (F) to promote further cultural learning and international understanding; (G) to strength- en knowledge and skills related to planning, organizing, and imple- mentation; (H) to acquire the ability to utilize technology and information; (I) to encourage an attitude of active learning and studying; and (J) to develop abilities related to independent think- ing and problem-solving. 4.1.3. Core competencies To achieve the aforementioned goals, the curricular design of primary and junior high school education shall focus on the needs and experiences of students and aim to develop the core compe- tencies that a modern citizen should possess. Such CCs (core com- petencies) refer to the key competencies, which defined by the Mayer Committee are: (A) collect, analyze and organize informa- tion; (B) communicate ideas and information; (C) plan and orga- nize activities; (D) co-operate with others and help sustain the group’s ability to work; (E) use mathematical concepts and tech- nologies; (F) solve problems; (G) use technology (Mayer, 1992). Thus, the CCs for the Grade 1–9 curriculum reform in Taiwan may be categorized as follows (MOE, 2002): (A) self-understanding and exploration of potential; (B) appreciation, representation, and creativity; (C) career planning and lifelong learning; (D) expres- sion, communication, and sharing; (E) respect, care and teamwork; (F) cultural learning and international understanding; (G) plan- ning, organizing and putting plans into practice; (H) utilization of technology and information; (I) active exploration and study; and (J) independent thinking and problem-solving. With reference to curricular principles (see Section 4.1.2), the CCs may be divided into 4 dimensions: (1) the physical, mental, and spiritual mold (A-C); (2) interpersonal and social relations (D-G); (3) the use of Life Sciences and Technology (H); and (4) log- ical thinking and reasoning (I-J). 4.1.4. Learning areas To foster the CCs in citizens, the curricula for primary and junior high school education should emphasize three dimensions: indi- vidual development, community and culture, and natural environ- ment. Thus, the Grade 1–9 Curriculum encompasses seven major learning areas: (A) Language Arts, including Mandarin and English; (B) Health and Physical Education; (C) Social Studies, including his- tory and culture, geography, social institutions, morals and norms, etc., and the incorporation of the aforementioned subjects into one’s daily life; (D) Arts and Humanities, including music instruc- tion and instruction in the visual and performing arts; (E) Science and Technology, including learning about substances and energy, nature, the environment, ecological conservation, and information technology; (F) Mathematics, including acquiring the basic con- cepts of figures, shapes, and quantity; the ability to calculate and organize; and the ability to apply such knowledge and skills to dai- ly life; and (G) Integrative Activities, referring to activities that may Table 2 Rating the CCs relationships/influences for grades 1–3 mandarin curricula. D1 D2 D3 D4 C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 D1 C1 0.000 3.500 3.375 3.625 3.250 2.250 2.750 2.375 2.875 3.000 C2 3.250 0.000 2.875 3.000 2.625 2.875 2.750 2.250 2.500 2.375 C3 3.750 2.375 0.000 2.875 2.625 2.500 3.000 2.750 3.000 3.125 D2 C4 3.750 3.250 3.000 0.000 3.250 3.125 3.250 2.750 3.125 3.250 C5 2.375 3.000 2.125 3.500 0.000 3.500 3.375 2.375 2.750 3.000 C6 2.625 3.000 2.500 3.250 3.125 0.000 2.875 2.375 2.625 2.625 C7 2.625 2.625 3.375 3.375 3.375 2.875 0.000 2.750 3.000 3.125 D3 C8 2.375 2.500 2.875 3.125 2.375 2.500 3.000 0.000 3.250 3.250 D4 C9 3.000 3.000 3.375 3.250 3.000 2.875 3.500 3.125 0.000 3.750 C10 3.375 3.375 3.125 3.000 2.875 2.625 3.250 2.875 3.875 0.000 Remark: Surveyed based on the opinions of teacher(s) who are editing Mandarin Chinese teaching materials or have been teaching Mandarin Chinese teaching in primary school, Taiwan. Core Competencies (CCs) D1: Physical, mental, and spiritual mold D2: Interpersonal and social relations D3: The use of Life Science and Technology D4: Logical thinking and reasoning C1: Self-understanding and exploration of potential; C2: Appreciation, representation, and creativity; C3: Career planning and lifelong learning; C4: Expression, communication, and sharing; C5: Respect, care and teamwork; C6: Cultural learning and international understanding; C7: Planning, organizing and putting plans into practice; C8: Utilization of technology and information; C9: Active exploration and study; C10: Independent thinking and problem-solving. C.-H. Chen, G.-H. Tzeng / Expert Systems with Applications 38 (2011) 12168–12179 12175 guide learners to practice, experience, and reflect upon the learning process as well as to testify to and apply what they have learned in real situations. Based on the analytical frame for expanding them, CCs were first selected as determinants. Then, the structure of the assessment system for the Mandarin Chinese teaching materials definition problem was established using DEMATEL. After that, the weight of each determinant for the decision structure would be decided by using the ANP. The determinants are CCs. The crite- ria were confirmed as Competence Indicators and as determinants for editing Mandarin Chinese teaching material. Meanwhile, the relationships between the determinants and the ANP derivations of the weights of each determinant would also be derived for the case study. The relationships between the determinants for the assessment system for Mandarin Chinese teaching materials were surveyed based on the opinions of teacher(s) who are editing Mandarin Chi- nese teaching materials or have been teaching Mandarin Chinese in primary school, Taiwan. The teachers are familiar with the assessment of Mandarin Chinese teaching materials. With the understanding of the determinants of the assessment system for Mandarin Chinese teaching materials, appropriate assessment strategies will also be proposed to shorten the gap be- tween the level of the selected current teaching material and the Table 3 Result of the DEMATEL analysis of CC Relationships/Influences for Grades 1–3 Mandarin C D1 D2 C1 C2 C3 C4 C5 D1 C1 1.1989 1.2839 1.2798 1.3786 1.2 C2 1.1962 1.0744 1.1645 1.2523 1.1 C3 1.2752 1.2146 1.1384 1.3175 1.2 D2 C4 1.3819 1.3446 1.3369 1.3402 1.3 C5 1.2329 1.2298 1.2038 1.3322 1.1 C6 1.1988 1.1890 1.1737 1.2813 1.1 C7 1.2867 1.2638 1.2847 1.3773 1.2 D3 C8 1.2067 1.1886 1.1998 1.2927 1.1 D4 C9 1.3678 1.3436 1.3544 1.4480 1.3 C10 1.3587 1.3351 1.3284 1.4204 1.3 Remark: Core Competencies (CCs) D1: Physical, mental, and spiritual mold D2: Interpe thinking and reasoning C1: Self-understanding and exploration of potential; C2: Appreci Respect, care and teamwork; C6: Cultural learning and international understanding; C7: P information; C9: Active exploration and study; C10: Independent thinking and problem- aspired levels of the determinants. Meanwhile, the proposed assessment system for Mandarin Chinese teaching materials will assist the publishers in surpassing the publisher leaders. Detailed procedures and results are illustrated below. 4.2. Decision problem network relation map structuring by DEMATEL The inter-relationships between the ten determinants will be deduced using the DEMATEL method introduced in sub Section 3.1. First, the direct relation/influence matrix A is introduced (see Ta- ble 2). After that, the direct relation/influence matrix A is normal- ized based on Eq. (1). Then, the total relationship matrix is deduced based on Eq. (3) (see Table 3). Finally, the strength of the influence for each determinant is deduced based on Eq. (5) (see Table 4) (see Fig. 6). The sequence for the strength of the influence of the determi- nants is as follows: (1) the use of Life Science and Technology (D3), which includes the utilization of technology and information (C8); (2) logical thinking and reasoning (D4), which is sequent as (a) inde- pendent thinking and problem-solving (C10) and (b) active explora- tion and study (C9); (3) physical, mental, and spiritual mold (D1), which is sequent as (a) career planning and lifelong learning (C3). (b) self-understanding and exploration of potential (C1) and (c) appreciation, representation, and creativity (C2); (4) interpersonal urricula. D3 D4 C6 C7 C8 C9 C10 721 1.1868 1.3055 1.1287 1.2787 1.3022 539 1.1083 1.2002 1.0342 1.1654 1.1802 173 1.1574 1.2736 1.1063 1.2450 1.2678 398 1.2754 1.3896 1.1999 1.3542 1.3787 317 1.1851 1.2818 1.0928 1.2346 1.2607 885 1.0379 1.2253 1.0562 1.1897 1.2083 810 1.2097 1.2242 1.1445 1.2880 1.3116 809 1.1305 1.2441 0.9939 1.2232 1.2419 390 1.2745 1.4038 1.2172 1.2641 1.4001 162 1.2490 1.3767 1.1927 1.3628 1.2651 rsonal and social relations D3: The use of Life Science and Technology D4: Logical ation, representation, and creativity; C3: Career planning and lifelong learning; C5: lanning, organizing and putting plans into practice; C8: Utilization of technology and solving. Table 4 ri + ci and ri � ci of the DEMATEL analysis of CC Relationships/Influences for Grades 1–3 Mandarin Curricula. CC ri + ci ri � ci D1: Physical, mental, and spiritual mold C1: Self-understanding and exploration of potential 25.3189 (5) �0.0886 (5) C2: Appreciation, representation, and creativity 23.9970 (8) �0.9377 (10) C3: Career planning and lifelong learning 24.6777 (6) �0.2515 (8) D2: Interpersonal and social relations C4: Expression, communication, and sharing 26.7817 (1) �0.0992 (6) C5: Respect, care and teamwork 24.6058 (7) �0.2351 (7) C6: Cultural learning and international understanding 23.5633 (9) �0.0659 (4) C7: Planning, organizing and putting plans into practice 25.5963 (4) �0.2534 (9) D3: The use of Life Science and Technology C8: Utilization of technology and information 23.0685 (10) 0.7357 (2) D4: Logical thinking and reasoning C9: Active exploration and study 26.0180 (3) 0.8069 (1) C10: Independent thinking and problem-solving 26.0219 (2) 0.3886 (3) Fig. 5. The analytic network based on the casual diagram of total relationships. 12176 C.-H. Chen, G.-H. Tzeng / Expert Systems with Applications 38 (2011) 12168–12179 and social relations (D2), which are sequent as (a) respect, caring and teamwork (C5), (b) cultural learning and international understand- ing (C6), (c) expression, communication, and sharing (C4) and (d) planning, organizing and putting plans into practice (C7). 4.3. Calculating the weights of determinants by ANP Setting an appropriate assessment system as the goal, pairwise comparisons of the determinants were executed based on the total relationship matrix as deduced by DEMATEL. The inter-relation- ships between the goal and the dimensions of determinants of an assessment system are illustrated in Fig. 5 where the direction of the arrows indicates the direction of the influences. Finally, the to- tal relationship matrix serves as inputs for the ANP. By implement- ing the ANP, the limit super matrix W is calculated. Weights corresponding to each determinant (Table 5) are derived accord- ingly that will be used for the calculation of weighted averages and VIKOR scores. 4.4. Compromise ranking by VIKOR The VIKOR technique being introduced for compromise ranking was applied after the determinants’ weight calculations by ANP in sub Section 4.3. Meanwhile, the weighted averages for the Mandarin Chinese teaching materials (six books) for Grade 1 in primary school, which were edited by three leading publishers, are also calculated for comparison. In general, the calculation results (Table 6–8) demonstrate that both global and local reached the same conclusions: publisher A � publisher C � publisher B. 4.5. Discussions and implications Authoring teaching material is not an easy task. Meanwhile, there are no straightforward answers to the question of how teach- ing material should be authored to meet particular criteria. Not only criteria but also determinant, how teaching material should be authoring by considering factors being related to the authored curriculum. Also, from the perspective of MCDM, very little re- search has addressed the assessment system for teaching material, not to mention Mandarin Chinese teaching materials. In this research, a novel MCDM framework combining the DEMATEL tech- nique, ANP and VIKOR was proposed to address the above- mentioned problems; using this technique yielded satisfactory results. The novel MCDM model consisting of DEMATEL, ANP and VI- KOR was formed as follows: (1) overcome the issue of defining the assessment system for teaching material; (2) use innovative traditional MCDM approaches to resolve the problem of defining the assessment system for teaching material; (3) clarify the vague correlations between the determinants of teaching material; and (4) create the aspired priorities for the authored teaching material. For the assessment system for teaching materials, CCs were se- lected as determinants of the assessment, and categorized into 4 groups as the criteria of the assessment. These groups were as follows; D1: Physical, mental, and spiritual mold C1: Self-understanding and exploration of potential; C2: Appreciation, representation, and creativity; C3: Career planning and lifelong learning; D2: Interpersonal and social relations C4: Expression, communication, and sharing; C5: Respect, care and teamwork; C6: Cultural learning and international understanding; C7: Planning, organizing and putting plans into practice; D3: The use of Life Science and Technology C8: Utilization of technology and information; D4: Logical thinking and reasoning C9: Active exploration and study; C10: In- dependent thinking and problem-solving. Through application studies, we found the novel MCDM model to be applicable. DEMATEL establishes a reasonable assessment structure for dealing with criteria influence. Basically, the influence relationships of the results (see Fig. 6) were quite reasonable: Table 5 Weights of the determinants being derived by ANP. CC Local weights Globe weights (ANP) D1: Physical, mental, and spiritual mold 0.3014 C1: Self-understanding and exploration of potential 0.3377 0.1018 C2: Appreciation, representation, and creativity 0.3312 0.0998 C3: Career planning and lifelong learning 0.3312 0.0998 D2: Interpersonal and social relations 0.4054 C4: Expression, communication, and sharing 0.2656 0.1077 C5: Respect, care and teamwork 0.2455 0.0995 C6: Cultural learning and international understanding 0.2335 0.0947 C7: Planning, organizing and putting plans into practice 0.2554 0.1035 D3: The use of Life Science and Technology 0.0895 C8: Utilization of technology and information 1.0000 0.0895 D4: Logical thinking and reasoning 0.2037 C9: Active exploration and study 0.4959 0.1010 C10: Independent thinking and problem-solving 0.5041 0.1027 Table 6 Satisfaction of Grade 1 Curriculum. CC A Publisher B Publisher C Publisher 11 12 11 12 11 12 D1: Physical, mental, and spiritual mold C1: Self-understanding and exploration of potential 6.0000 5.7500 5.5000 5.2500 6.2500 6.2500 C2: Appreciation, representation, and creativity 6.5000 6.7500 5.5000 7.0000 5.7500 7.2500 C3: Career planning and lifelong learning 5.7500 4.7500 4.2500 4.5000 6.2500 4.7500 D2: Interpersonal and social relations C4: Expression, communication, and sharing 6.7500 7.5000 6.0000 6.5000 6.2500 7.2500 C5: Respect, care and teamwork 6.7500 6.2500 6.2500 6.5000 6.5000 6.5000 C6: Cultural learning and international understanding 5.2500 4.0000 4.2500 4.2500 4.0000 4.0000 C7: Planning, organizing and putting plans into practice 5.0000 5.0000 4.5000 4.7500 4.5000 4.2500 D3: The use of Life Science and Technology C8: Utilization of technology and information 3.7500 3.7500 4.0000 4.7500 4.5000 4.5000 D4: Logical thinking and reasoning C9: Active exploration and study 5.0000 5.2500 4.2500 4.2500 4.7500 4.5000 C10: Independent thinking and problem-solving 5.2500 4.7500 4.7500 4.5000 5.0000 6.0000 Remark: 11: the 1st Semester of Grade 1; 12: the 2nd Semester of Grade 1. Table 7 Aspired level for Grade 1 Curriculum. Criterion of CC Local weights Global weights (ANP) A Publisher B Publisher C Publisher 11 12 11 12 11 12 D1: Physical, mental, and spiritual mold 0.3014 Si 0.3917 0.4250 0.4914 0.4419 0.3916 0.3916 Qi 0.4250 0.5250 0.5750 0.5500 0.4250 0.5250 C1: Self-understanding and exploration of potential 0.3377 0.1018 0.4000 0.4250 0.4500 0.4750 0.3750 0.3750 C2: Appreciation, representation, and creativity 0.3312 0.0998 0.3500 0.3250 0.4500 0.3000 0.4250 0.2750 C3: Career planning and lifelong learning 0.3312 0.0998 0.4250 0.5250 0.5750 0.5500 0.3750 0.5250 D2: Interpersonal and social relations 0.4054 Si 0.4047 0.4263 0.4730 0.4472 0.4661 0.4459 Qi 0.5000 0.6000 0.5750 0.5750 0.6000 0.6000 C4: Expression, communication, and sharing 0.2656 0.1077 0.3250 0.2500 0.4000 0.3500 0.3750 0.2750 C5: Respect, care and teamwork 0.2455 0.0995 0.3250 0.3750 0.3750 0.3500 0.3500 0.3500 C6: Cultural learning and international understanding 0.2335 0.0947 0.4750 0.6000 0.5750 0.5750 0.6000 0.6000 C7: Planning, organizing and putting plans into practice 0.2554 0.1035 0.5000 0.5000 0.5500 0.5250 0.5500 0.5750 D3: The use of Life Science and Technology 0.0895 Si 0.6250 0.6250 0.6000 0.5250 0.5500 0.5500 Qi 0.6250 0.6250 0.6000 0.5250 0.5500 0.5500 C8: Utilization of technology and information 1.0000 0.0895 0.6250 0.6250 0.6000 0.5250 0.5500 0.5500 D4: Logical thinking and reasoning 0.2037 Si 0.4874 0.5002 0.5498 0.5624 0.5124 0.4744 Qi 0.5000 0.5250 0.5750 0.5750 0.5250 0.5500 C9: Active exploration and study 0.4959 0.1010 0.5000 0.4750 0.5750 0.5750 0.5250 0.5500 C10: Independent thinking and problem-solving 0.5041 0.1027 0.4750 0.5250 0.5250 0.5500 0.5000 0.4000 Integration 1.0000 Si 0.4374 0.4587 0.5056 0.4760 0.4606 0.4446 Qi 0.6250 0.6250 0.6000 0.5750 0.6000 0.6000 Remark: 11: the 1st Semester of Grade 1; 12: the 2nd Semester of Grade 1. C.-H. Chen, G.-H. Tzeng / Expert Systems with Applications 38 (2011) 12168–12179 12177 (a) The aim of education is to foster students’ sound mind and char- acter – first, enhancing ‘‘D3: The use of Life Science and Technol- ogy’’ may strengthen ‘‘D4: Logical thinking and reasoning’’ for students; then, improving ‘‘D2: Interpersonal and social rela- tions’’ becomes the goal; finally, the curriculum goal gives stu- dents a ‘‘D1: Physical, mental, and spiritual mold’’. Table 8 VIKOR versus weighted average results. m A Publisher B Publisher C Publisher 11 12 11 12 11 12 D1 Physical, mental, and spiritual mold (Si) m = 1 0.0016(4) 0.3350(3) 1.0000(1) 0.5041(2) 0.0000(6) 0.0001(5) m = 0.5 0.0008(5) 0.5008(3) 1.0000(1) 0.6687(2) 0.0000(6) 0.3334(4) (Qi) m = 0 0.0000(5) 0.6667(3) 1.0000(1) 0.8333(2) 0.0000(5) 0.6667(3) D2 Interpersonal and social relations (Si) m = 1 0.0000(6) 0.3153(5) 1.0000(1) 0.6223(3) 0.8984(2) 0.6031(4) m = 0.5 0.0000(6) 0.6577(5) 0.8750(2) 0.6862(4) 0.9492(1) 0.8015(3) (Qi) m = 0 0.0000(6) 1.0000(1) 0.7500(4) 0.7500(4) 1.0000(1) 1.0000(1) D3The use of Life Science and Technology (Si) m = 1 m = 0.5 1.0000(1) 1.0000(1) 0.7500(3) 0.0000(6) 0.2500(4) 0.2500(4) (Qi) m = 0 D4 Logical thinking and reasoning (Si) m = 1 0.1478(5) 0.2933(4) 0.8568(2) 1.000(1) 0.4319(3) 0.0000(6) m = 0.5 0.0739(6) 0.3133(5) 0.9284(2) 1.0000(1) 0.3826(3) 0.3333(4) (Qi) m = 0 0.0000(6) 0.3333(4) 1.0000(1) 1.0000(1) 0.3333(4) 0.6667(3) Integration (Si) m = 1 0.0000(6) 0.3133(4) 1.0000(1) 0.5671(2) 0.3403(3) 0.1069(5) m = 0.5 0.5000(3) 0.6567(2) 0.7500(1) 0.2836(6) 0.4201(4) 0.3034(5) (Qi) m = 0 1.0000(1) 1.0000(1) 0.5000(3) 0.0000(6) 0.5000(3) 0.5000(3) Remark: 11: the 1st Semester of Grade 1; 12: the 2nd Semester of Grade 1. Fig. 6. The causal diagram of total relationship. 12178 C.-H. Chen, G.-H. Tzeng / Expert Systems with Applications 38 (2011) 12168–12179 (b) For ‘‘D4: Logical thinking and reasoning’’, ‘‘C9: Active explo- ration and study’’ is better than ‘‘C10: Independent thinking and problem-solving’’ from the well-known Knowledge Management point of view. (c) For ‘‘D2: Interpersonal and social relations’’, starting from ‘‘C6: Cultural learning and international understanding’’ may shorten the spiritual distance of the individual; then, there will be improvement in ‘‘C4: Expression, communica- tion, and sharing’’; naturally, ‘‘C5: Respect, care and team- work’’ will be consolidated; and ‘‘C7: Planning, organizing and putting plans into practice’’ will be thoroughly considered. (d) For ‘‘D1: Physical, mental, and spiritual mold’’, ‘‘C2: Appreci- ation, representation, and creativity’’ are the benchmark of ’’C1: Self-understanding and exploration of potential’’; in particular, ’’C3: Career planning and lifelong learning’’ is bright and healthy. ANP provides a general framework for dealing with decisions without making assumptions about the independence of higher-le- vel elements from lower-level elements and about the indepen- dence of the elements within a level as in a typical hierarchy (Saaty, 2005). Thus, in defining the assessment system for teaching material illustrated in the paper, ANP is apparently a more reason- able tool for analysing the network structure with feedback. In- deed, the weighting sequence for the CCs (see Table 5) is seemingly the priority in the practice of CCs. It is contracted with the influence relationships mentioned above using the DEMATEL technique. For long-term concerns, who is responsible for assess- ing teaching materials is a serious concern. The VIKOR method uses an aggregating function R representing ‘‘closeness to the ideal’’. In comparison with the TOPSIS, which determines a solution with the shortest distance from the ideal solution and the farthest distance from the negative-ideal solution (Chen & Hwang, 1992; Tzeng, Shiau, & Teng, 1994), VIKOR can se- C.-H. Chen, G.-H. Tzeng / Expert Systems with Applications 38 (2011) 12168–12179 12179 lect the real ‘‘closest to the ideal’’ solution. On the other hand, a solution by TOPSIS is not always the closest to the ideal. A detailed comparison of TOPSIS and VIKOR has already been presented in the article by Opricovic and Tzeng (Opricovic & Tzeng, 2004, 2007). Furthermore, the Lp-metric of VIKOR represents the grouping le- vel in power p; in other words, p = 1 is the most emphasized group utility; p = 1 represents the most emphasized individuals. For example (see Table 8), regarding Book 11 from A Publisher, one can say that when m ¼ 1; RD1 ¼ 0:0016; RD2 ¼ 0:0000; RD3 ¼ 1:0000; RD4 ¼ 0:1478; RA11 ¼ 0:0000, and the sequences are, respectively, 4, 6, 1, 5, and 6. However, when m = 0, RD1 ¼ 0:0000; RD2 ¼ 0:0000; RD3 ¼ 1:0000; RD4 ¼ 0:0000; RA11 ¼ 1:0000, and the sequences are, respectively, 5, 6, 1, 6, and 1. In this case, Book 11 from A Publisher has good behavior on average, but D3 is the worst. Thus, when m = 0.5, the integration sequence is 3. This means that improving D3 will achieve higher performance. In contrast, Book 12 from B Pub- lisher may indicate the reverse conclusion. 5. Concluding remarks Based on effective concern, begin from Mandarin Chinese. This paper would have mainly advanced work in the field of the assess- ment system for teaching materials. First, a novel MCDM framework was proposed to define the determinants of teaching materials (not exclusively for Mandarin Chinese). Second, the traditional problem of the difficulty of defining the assessment system for teaching materials was resolved based on the novel MCDM approach being proposed. An important reason- ing conclusion was obtained: the more important determinant, the less influent determinant. The influent sequence of determi- nants is as follows: The use of Life Science and Technology (D3) � Logical thinking and reasoning (D4) � Interpersonal and so- cial relations (D2) � Physical, mental, and spiritual mold (D1) (see Fig. 5). However, the weighting sequence of determinants is as fol- lows: Interpersonal and social relations (D2, 0.4054) � Physical, mental, and spiritual mold (D1, 0.3014) � Logical thinking and rea- soning (D4, 0.2037) � The use of Life Science and Technology (D3, 0.0895) (see Table 5). Finally, the difficulty of the traditional MCDM approach in selecting ‘‘rotten apple(s)’’ was also resolved based on the concep- tual advance in achieving the aspired level of criteria. Acknowledgment Over 20 teachers provided feedback on the questionnaire. Although I do not know who are you, I sincerely appreciate your kindness. References Chen, C.H., & Tzeng, G.H. (2009a). Combined DEMATEL technique with a novel MCDM method for creating the aspired intelligent assessment systems for mandarin Chinese teaching materials. In The 10th Asia Pacific industrial engineering and management systems conference (APIEMS2009), Kitakyushu, Japan (pp. 2050–2061). Chen, C. H., & Tzeng, G.H. (2009b). Creating the aspired intelligent fuzzy assessment systems for mandarin Chinese teaching materials. In The 17th national conference on fuzzy theory and its application, Kaohsiung, Taiwan (pp. 787–792). Chen, Y. C., Lien, H. P., & Tzeng, G. H. (2010). Measures and evaluation for environment watershed plan using a novel hybrid MCDM model. Expert Systems with Applications, 37(2), 926–938. Chen, S. J., & Hwang, C. L. (1992). Fuzzy multiple attribute decision making: Methods and applications. Berlin: Springer. Chiu, Y. J., Chen, H. C., Tzeng, G. H., & Shyu, J. Z. (2006). Marketing strategy based on customer behavior for the LCD-TV. International Journal of Management and Decision Making, 7(2/3), 143–165. DfE (1991). Education and training for the 21st century. London: HMSO. Education Commission (2000). Reform proposal for the education system in Hong Kong. Hong Kong: Education Bureau. Freimer, M., & Yu, P. L. (1976). Some new results on compromise solutions for group decision problems. Management Science, 22(6), 688–693. Gabus, A., & Fontela, E. (1972). World Problems, an invitation to further thought within the framework of DEMATEL. Geneva, Switzerland: Battelle Geneva Research Centre. Huang, C. Y., Shyu, J. Z., & Tzeng, G. H. (2007). Reconfiguring the innovation policy portfolios for Taiwan’s SIP mall industry. Technovation, 27(12), 744–765. Huang, J. J., Tzeng, G. H., & Ong, C. S. (2005). Multidimensional data in multidimensional scaling using the analytic network process. Pattern Recognition Letters, 26(6), 755–767. Lee, W. S., Tzeng, G. H., Hsu, C. Y., & Huang, J. M. (2009). Combined MCDM techniques for exploring stock selection based on Gordon model. Expert Systems with Applications, 36(3/2), 6421–6430. Li, C. W., & Tzeng, G. H. (2009a). Identification of a threshold value for the DEMATEL method using the maximum mean de-entropy algorithm to find critical services provided by a semiconductor intellectual property mall. Expert Systems with Applications, 36(6), 9891–9898. Li, C. W., & Tzeng, G. H. (2009b). Identification of interrelationship of key customers’ needs based on structural model for services/capabilities provided by a semiconductor-intellectual property mall. Applied Mathematics and Computation, 215(6), 2001–2010. Lin, C. L., & Tzeng, G. H. (2009). A value-created system of science (technology) park by using DEMATEL. Expert Systems with Applications, 36(6), 9683–9697. Lin, C. J., & Wu, W. W. (2008). A causal analytical method for group decision-making under fuzzy environment. Expert Systems with Applications, 34(1), 205–213. Liou, J. J. H., Tzeng, G. H., & Chang, H. C. (2007). Airline safety measurement using a hybrid model. Air Transport Management, 13(4), 243–249. Mayer, E. (1992). Putting general education to work: The key competencies report. Canberra: Australian Government Publishing Service. Ministry of Education (2002). General guidelines of grade 1-9 curriculum of elementary and junior high school education. Taipei: Ministry of education. Ministry of Education (2008). Education in Taiwan 2008. Taipei: Ministry of Education. Opricovic, S. (1998). Multicriteria optimization of civil engineering systems. Belgrade: Faculty of Civil Engineering. Opricovic, S., & Tzeng, G. H. (2002). Multicriteria planning of post-earthquake sustainable reconstruction. Computer-Aided Civil and Infrastructure Engineering, 17(3), 211–220. Opricovic, S., & Tzeng, G. H. (2004). Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. European Journal of Operational Research, 156(2), 445–455. Opricovic, S., & Tzeng, G. H. (2007). Extended VIKOR method in comparison with outranking methods. European Journal of Operational Research, 178(2), 514–529. Ou Yang, Y. P., Leu, J. D., & Tzeng, G. H. (2009). A VIKOR-based multiple criteria decision method for improving information security risk. International Journal of Information Technology and Decision Making, 8(2), 1–21. Ou Yang, Y. P., Shieh, H. M., Leu, J. D., & Tzeng, G. H. (2008). A novel hybrid MCDM model combined with DEMATEL and ANP with applications. International Journal of Operations Research, 5(3), 1–9. Saaty, T. L. (1996). Decision making with dependence and feedback: The analytic network process. Pittsburgh: RWS Publication. Saaty, T.L. (1999). Fundamentals of the analytic network process. In Proceedings of international symposium on analytical hierarchy process, Kobe, Japan. Saaty, R.W. (2003). The analytic hierarchy process (AHP) for decision making and the analytic network process (ANP) for decision making with dependence and feedback. Pittsburgh, PA: Creative Decisions Foundation. Saaty, T. L. (2004). Fundamentals of the analytic network process – Dependence and feedback in decision-making with a single network. Journal of Systems Science and Systems Engineering, 13(2), 71–91. Saaty, T. L. (2005). Theory and applications of the analytic network process. Pittsburg, PA: RWS Publications. Tamura, M., Nagata, H., & Akazawa, K. (2002). Extraction and systems analysis of factors that prevent safety and security by structural models. In Proceedings of the 41st SICE annual conference, 3, Osaka, Japan (pp. 1752–1759). Tzeng, G. H., Chiang, C. H., & Li, C. W. (2007). Evaluating intertwined effects in e- learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL. Expert Systems with Applications, 32(4), 1028–1044. Tzeng, G. H., Chen, W. H., Yu, R., & Shih, M. L. (2010). Fuzzy Decision Maps–A Generalization of the DEMATEL Methods. Soft Computing, 14(11), 1141–1150. Tzeng, G. H., Lin, C. W., & Opricovic, S. (2005). Multi-criteria analysis of alternative- fuel buses for public transportation. Energy Policy, 33(1), 1373–1383. Tzeng, G. H., Shiau, T. A., & Teng, J. J. (1994). Multiobjective decision-making approach to energy supply mix decisions in Taiwan. Energy Sources, 16(3), 301–316. Tzeng, G. H., Teng, M. H., Chen, J. J., & Opricovic, S. (2002). Multicriteria selection for a restaurant location in Taipei. International Journal of Hospitality Management, 21(2), 171–187. Wu, W. W., & Lee, Y. T. (2007). Developing global managers’ competencies using the fuzzy DEMATEL method. Expert Systems with Applications, 32(2), 499–507. Yu, P. L. (1973). A class of solutions for group decision problems. Management Science, 19(8), 936–946. Creating the aspired intelligent assessment systems for teaching materials 1 Introduction 2 Intelligent assessment systems for teaching materials with MCDM method 2.1 Educational reform of Taiwan (MOE, 2002) 2.2 Assessment system for mandarin chinese teaching materials 2.2.1 Mandarin Chinese’s curriculum goal 2.2.2 Competence indicators of Mandarin Chinese’s curriculum of grade 1–9 curriculum 3 A novel MCDM method based on the DEMATEL technique with ANP 3.1 DEMATEL Technique with ANP 3.2 The ANP method 3.3 VIKOR 4 An empirical study of the aspired assessment systems for mandarin chinese teaching materials 4.1 Background description 4.1.1 Core rationale 4.1.2 Curriculum goals 4.1.3 Core competencies 4.1.4 Learning areas 4.2 Decision problem network relation map structuring by DEMATEL 4.3 Calculating the weights of determinants by ANP 4.4 Compromise ranking by VIKOR 4.5 Discussions and implications 5 Concluding remarks Acknowledgment References