A scenario-based modeling method for controlling ECM performance Cristina Lópeza* and Alessio Ishizakab aUniversity Pablo of Olavide, Department of Business Administration and Marketing. Pedro R. Campomanes Building, Road Utrera, km. 1 – 41013 Seville, Spain bUniversity of Portsmouth, Portsmouth Business School. Richmond Building, Portland Street, Portsmouth PO1 3DE, United Kingdom *Corresponding author Cristina López. Telephone number: + 00 34 95 4977324. Fax number: + 34 95 4348353. Email address: clopvar@upo.es. Postal address: Ctra. Utrera, km. 1 – 41013 Seville, Spain ABSTRACT Firms are increasingly more integrating Enterprise Content Management (ECM) into their IT infrastructure in order to manage a growing volume of complex structured and unstructured data. These packages allow capturing, generating and delivering accurate information and contents in real time to support decision making in the whole business process. However, it is not easy to achieve the full benefits of an ECM package as this relies on a variety of indicators from the organizational and technical perspectives. This paper proposes a new expert system to monitor ECM performance. It is based on the innovative hybrid technique incorporating Fuzzy Cognitive Maps (FCM) and the Analytic Hierarchy Process (AHP). As these indicators are interrelated, experts’ knowledge is extracted, modeled, combined and processed in the new proposed expert system. This enables managers to precisely forecast the impact of changes in control indicators on system performance through the simulation of different scenarios over time. This approach has practical impacts as managers can make informed decisions based on the analysis of the expert system and thus prevent ECM malfunctions or misuses. Keywords: Enterprise Content Management; Decision Support Systems; System performance; FCM; AHP. 2 1. Introduction In our digital era, companies face the challenge of handling vast volumes of complex information and contents arising from their own worldwide daily activities. Therefore, executive managers require help from operational support systems to identify, generate and evaluate relevant information. This is a crucial step to get a more adequate acknowledgement and treatment of uncertainty in decision support endeavors. Hence, many firms have adopted diverse decision support tools in their computing infrastructure: Enterprise Information Systems (EIS) (Leidner & Elam, 1993), Expert Systems (ES) (Luconi, Malone & Morton, 1986), Decision Support Systems (DSS) (Sprague, 1980), and Group Decision Support Systems (GDSS) (DeSanctis & Gallupe, 1987). As technology evolves, new business analytic tools, such as Enterprise Content Management (ECM) systems, have also emerged to support the integrated wide management of all types of information and contents (Smith & McKeen, 2003; Tyrväinen, Päivärinta, Salminen, & Iivari, 2006). ECMs are computer-based information systems that employ web technologies for capturing, processing, storing, protecting, maintaining and delivering structured and unstructured contents related to business processes (AIIM, 2010; Jan Vom Brocke, Simons, & Cleven, 2011). These solutions have the capacity to efficiently handle the whole content’s lifecycle, as well as the possibility of integrating it with other sources of data. Likewise, they provide different means for analyzing the information over a wide range of managerial decisions. Thus, ECM can enhance the decision-support capabilities of adopter firms (Alalwan, 2013), as well as complying with legal requirements (Blair, 2004). 3 The research on ECM adoption has increased in the last years (Alalwan & Roland, 2012; Grahlmann, Helms, Hilhorst, Brinkkemper, & Van Amerongen, 2012; Tyrväinen et al., 2006). Recent studies have explored the key challenges in ECM adoption (Andersen, 2008; Hullavarad, O’Hare, & Roy, 2015; Vom Brocke, Simons, Herbst, Derungs, & Novotny, 2006). Researchers have also proposed a step-by-step development process to successfully implement ECM systems (Nordheim & Päivärinta, 2006; O’Callaghan & Smits, 2005). Once the ECM implementation is accomplished and the application is operative, the normal ECM activity can provide a wide range of benefits at an operational, tactical and strategic level (Alalwan, Thomas, & Weistroffer, 2014; Paivarinta & Munkvold, 2005; Smith & McKeen, 2003; Tyrväinen et al., 2006). Yet to achieve them, organizations have to control the performance of their enterprise systems until all bugs, misuse, etc., are corrected (López & Salmeron, 2014). Despite its significance, this issue has received little attention. The goal of this paper is to propose a scenario-based approach for controlling ECM performance. Our approach is based on a new knowledge acquisition technique for expert systems based on the hybrid method combining Fuzzy Cognitive Maps (FCM) and the Analytic Hierarchy Process (AHP). Many different industries have implemented expert systems (Wagner, 2017) . One of the most cited obstacles to a successful development of an expert system is the problem of acquiring specific domain knowledge and representing it (Holsapple, Raj, & Wagner, 2008). In our paper, we propose FCM, which are capable of modeling a target real-world dynamic system, which may not be well-defined (Özesmi & Özesmi, 2004). Therefore, the connections between them can be represented by fuzzy weights 4 on a linguistic scale (Kosko, 1986). Linguistic expressions are more intuitive than quantitative scales to represent knowledge. Nevertheless, the problem is how to transform this scale into a quantitative one. For this purpose, we use AHP (Saaty, 1977). Furthermore, FCM provide mechanics to study the evolution of a scenario at successive times. This soft computing technique also enables developing “what-if” analysis to investigate alternative scenarios. For all these reasons, FCM have been successfully applied in diverse areas such as marketing (Lee, Lee, Lee, & Lim, 2013), tourism (Kardaras, Karakostas, & Mamakou, 2013), agriculture (Papageorgiou, Markinos, & Gemptos, 2009), medicine (Lee, Yang, & Han, 2012), and energy (Espinosa-Paredes, Nuñez-Carrera, Laureano-Cruces, Vázquez-Rodríguez, & Espinosa-Martinez, 2008; Kyriakarakos, Dounis, Arvanitis, & Papadakis, 2012), among many others. In the present study, the new proposed expert system will enable the ECM performance to be forecasted by allowing relevant control indicators to interact with each other. That is, for example, how an increase in the numbers of business tasks supported by the ECM would affect system complexity. In this way, three scenarios related to technological, decision- making and organizational indicators were simulated. The results reveal that depending on managers’ actions, ECM performance can be improved or damaged to differing degrees. As a result of this analysis, practitioners will be able to apply adequate response actions aimed at maintaining a proper ECM performance. This research is organized into five sections. Section 2 presents a brief review of the ECM literature. Section 3 sets out the theoretical background of the new scenario-based method 5 proposed. Section 4 describes a real case study where the new approach is applied. Section 5 discusses contributions from both theoretical and practical perspectives. Finally, Section 6 offers the concluding remarks and the future research perspectives. 2. Enterprise Content Management Based on the convergence of the two previous mechanics - Document Management (DM) and Content Management (CM) - for managing unstructured information, ECM packages first appeared in the late 1990s (Boiko, 2001). This term was specifically coined in 2001 by the Association for Information and Image Management (AIIM), a global forum of information professionals (http://www.aiim.org/). Since then, it has been widely extended among key actors involved in managing organizations’ digital information assets. The ECM term is related to the integrated management of all information assets closely linked to the day-to-day conduct of business activity. These systems have been defined on many occasions from both the technical and the process perspectives. Smith and McKeen (2003) proposed one of the most referenced ECM definitions in the literature, which is: “the strategies, tools, processes and skills an organization needs to manage all its information assets (regardless of type) over their lifecycle”. This definition stresses the following aspects, which characterize ECM as regards other enterprise systems:  ECM can be considered not only as a set of isolated technologies (Jan Brocke, Seidel, & Simons, 2010), since it also implies people and processes (Blair, 2004). In fact, these solutions require the combination of strategies, processes, tools and even skills to 6 successfully manage the content of assets in the adopter company as a single solution. Over the past few years, ECM systems have rapidly evolved in line with business technology trends such as cloud computing (Alalwan & Roland, 2012; Hullavarad et al., 2015). This has enabled content to be made internally and externally accessible (i.e., between the adopter company and its suppliers and/or customers).  ECM packages are capable of supporting from well-structured data (i.e., reports, word processing documents, spreadsheets) to less-structured data (i.e., emails, webpages) and even non-informational assets (i.e., videos and music files) (Tyrväinen et al., 2006).  These enterprise-wide applications manage the entire information resource lifecycle by means of web technologies and a repository (on-site or in the cloud), which is accessible via internet through a central interface. That is, ECM systems allow the capturing, management, storage, preservation, and delivery of contents and information profoundly related to the organizational processes in the most efficient way (AIIM, 2010). ECM implementation can yield many benefits to adopter companies (Alalwan et al., 2014; Paivarinta & Munkvold, 2005; Salamntu & Seymour, 2015; Smith & McKeen, 2003; Tyrväinen et al., 2006). In order to achieve them, capabilities embedded within the system package have to fit the functionalities of the business processes in the implementing companies (Seddon & Calvert, 2010). Notwithstanding, processes are in constant change because they need to adapt to the environment’s requirements. ECM should be therefore 7 adjusted to business activities throughout its lifecycle. This encompasses the following main stages described hereafter (Alalwan & Roland, 2012). During the adoption stage, managers have to prepare a comprehensive project feasibility study. This should detect the effects of ECM adoption on the firm performance. Iverson and Burkart (2007) present a framework for evaluating these impacts. The feasibility study should explore a wide range of issues related to ECM implementation, such as project risks, legal issues, benefits, costs, barriers and challenges of change management required. It must stress the active user support in the change management required for the successful ECM adoption (Munkvold, Paivarinta, Hodne, & Stangeland, 2003). In particular, Van Rooij (2013) describes how legacy issues need to be taken into account in the ECM adoption. Allen (2007) explains how to develop a return on investment (ROI) model when investing in an ECM system. Based on the viability analysis, decision makers will or will not implement the ECM solution. An affirmative decision marks the beginning of the acquisition stage. This consists in looking for the ECM solution that best suits business requirements. Choosing the most suitable content management system from among the large number of options on the market is a complex and difficult process. The decision makers have to consider multiple functional and non-functional criteria of different levels of importance (Lin, Hsu, & Sheen, 2007) under the constraint of a limited budget, time and human resources. In order to support this task, several studies have proposed a framework to compare ECM tools under specific criteria (Escalona, Domínguez-Mayo, García-García, Sánchez, & Ponce, 2015; Vitari, Ravarini, & Rodhain, 2006). In the same line, Oztaysi (2014) builds a hybrid AHP-grey TOPSIS model to evaluate 8 several ECM alternatives. Moreover, Votsch (2001) provides a taxonomy to clearly identify different kinds of content management systems. The evolution stage covers all the activities needed to install the ECM package chosen in the IT infrastructure, to integrate the system within the existing information sources and to test it. Nordheim and Päivärinta (2006) describe in detail how they implement the ECM in a large Norwegian oil firm. Simons, Brocke, Lässer, & Herbst(2014) distinguish important lessons to bear in mind during ECM implementation, while Herbst, Simons, Brocke, & Derungs (2014) explore critical success factors for ECM readiness assessment. Vom Brocke et al. (2006) present an ECM-blueprinting framework to guide professionals in the system implantation and at the same time to re-design affected business processes. Hullavarad et al. (2015) identify the main challenges a company faces during this stage. They also provide a graphical analysis of how each implantation phase gradually improves the business efficiency. In addition, other studies describe critical factors for successfully implementing ECM systems (Haug, 2012; Horne & Hawamdeh, 2015). Although the ECM implementation can finish successfully, it does not guarantee the long- term performance of the ECM. Accordingly, during the evaluation stage, managers assess the system performance and its effect on business activity. In this regard, Scott (2011) explores how users perceived ECM performance and what factors lead to the system’s acceptance. Arshad et al. (2014; 2015) studied how ECM usage supports business processes and Alalwan et al. (2014) the decision making. Subsequently, managers compare evaluation results with expected benefits and impacts. They can thus plan response actions to correct misalignment, 9 underperformance and/or even system bugs. A previous work underlined the need to develop new ECM evaluation practices (Paivarinta & Munkvold, 2005). Nonetheless, little attention has been paid to addressing this issue. A recent work provides a specific ECM research agenda based on an extensive literature review (Alalwan & Roland, 2012). This encourages further research that determines how ECM performance should be assessed and what control indicators ought to be used to evaluate ECM performance from different perspectives. Hence, this study provides a novel technique to bridge this gap between the literature and the practice. The next section presents the method proposed. 3. Research method In this paper, we will use a hybrid method combining Fuzzy Cognitive Maps and the Analytic Hierarchy Process to capture the importance of elements. FCM comprise theoretical foundations of Cognitive Maps (CM). This method (also called mental maps) was introduced in psychology by Tolman (1948) to represent the mental representation of physical locations. Irrelevant or unimportant information is omitted from the mental map. Thus, CM can be very different from an actual place. The differences between the mental representation and the physical characteristics of a location may reveal what humans or animals consider important. Later, Axerold (1976) advanced the idea of CM for supporting decision making. In addition to the important points (called nodes), he added causal connections between them (called edges). CM have been thereafter useful in problem solving (Eden, 2004) when many decisional variables are causally interrelated (Kim & Lee, 1998) because this can help decision 10 makers to highlight and analyze hidden relationships that contribute the most to attaining relevant and significant solutions. CM are signed digraphs where only the direction of the change between two nodes is modeled. Fig. 1 presents a CM example. A positive edge (noted with a positive sign +) indicates that the causal node casually increases or decreases the effect node in the same direction. A negative edge means that the causal node increases or decreases the effect node in the opposite direction. FCM were introduced to model the intensity of the change when an event occurs to some degree (Kosko, 1986). This is a graph-based knowledge representation technique (Dickerson & Kosko, 1993) which models a static or dynamic system using causal dependencies between a set of n nodes 𝑉 = (𝑣1,𝑣2,…,𝑣𝑛). FCM with fuzzy weights have the ability to deal with uncertain and imprecise data. Fig. 2 includes an FCM example, where numbers in brackets represent negative connections. The entire relationships can be represented in an adjacency matrix with their sign and intensity (1). If there is no relationship, then the entry is empty. 𝑊 = ( … … … 𝑤1→𝑛 … … … … … … 𝑤𝑖→𝑗 … … … … … … … … 𝑤𝑛→𝑛) (1) where 𝑤𝑖→𝑗 is a directed edge which indicates the influence of the causal node (𝑣𝑖) on the effect node (𝑣𝑗). 11 Fig. 1 CM example Fig. 2 FCM example FCM can be based on interviews, a Delphi process, and group discussions, among others (Jetter & Kok, 2014; Kardaras et al., 2013; Özesmi & Özesmi, 2004). They can be easily modified or extended by adding new nodes or causal links, or changing the weight assigned to the causal link. The difficult part is to assign the weight to the causal links. Some researchers have used a linguistic evaluation (Mourhir, Rachidi, & Karim, 2015; Obiedat & Samarasinghe, 2016). However, the difficulty is only shifted because the question remains on how to translate a linguistic evaluation into a quantitative evaluation. In this paper, we propose a new method based on AHP (Ishizaka & Labib, 2011; Saaty, 1980) for this transformation. The n linguistic terms are pairwise compared in a square matrix A (2) on a 1-9 evaluation scale, where 1 indicates equal importance and 9 extreme importance. A = (2) where aij is the comparison between the linguistic term i and j The matrix is reciprocal aij = 1/aii with the diagonal being equal at the unity because the              1...... ....../1... ...... 1 1 21 112 n ijji ij n a aa aa aa 12 linguistic term is compared with itself. Therefore, only the upper part of the matrix is required from the decision maker (Ishizaka, 2012). If a matrix is sufficiently consistent, the weights are calculated as shown in formula (3) (Ishizaka & Lusti, 2006): AW=λmaxW (3) where A is the comparison matrix, λmax is the principal eigenvalue W is the vector of the weights. As A has a redundancy of information, the consistency of the judgments entered by the decision maker can be tested with the consistency ratio (CR): CR=CI/RI (4) where CI=(λmax−n)/(n−1) is the consistency index n is the dimension of the comparison matrix λmax is the principal eigenvalue RI is the ratio index. The ratio index (RI) is the average of the consistency index of 500 randomly filled matrices. Saaty (1977) considers that a consistency ratio exceeding 10% may indicate a set of judgments that is too inconsistent to be reliable and therefore recommends revising the evaluations. FCM can be used not only for a simple causal reasoning of the phenomena represented. They also predict ECM performance systems by means of simulations of scenarios. In doing so, FCM incorporate the concept of neurons in the sense that they can be “on” (+1) or “off” (- 1), but also states in-between and therefore fuzzy states. When a node changes its state, it 13 affects all other connected nodes. If the threshold level of the effect node is reached, it will also change state and by consequence may also change further nodes within the network. Nodes already activated may be even activated again due to a feedback loop. By consequence, the activation spreads in a non-linear manner until the system reaches its stability or shows chaotic behavior. The inference process begins by assigning an input value [0, 1] to each FCM node, which corresponds to the initial state vector: 𝑉𝑆𝑖 0 = (𝑣1 0 𝑣2 0 … 𝑣𝑛−1 0 𝑣𝑛 0) (5) where 𝑣𝑛 0 points out the value of the node at the instant 0. During the simulation process, inputs are computed through a finite number of interactions in chain according to the following formula (Stylios & Groumpos, 2004) 𝑣𝑖 𝑡+1 = 𝑓(𝑣𝑖 𝑡 +∑ 𝑣𝑗 𝑡 ∙𝜔𝑗→𝑖 𝑛 𝑖≠𝑗 ) (6) where 𝑣𝑖 𝑡+1 is the value of node 𝑣𝑖 at the instant 𝑡 +1, 𝜔𝑖→𝑗 is the fuzzy weight between nodes 𝑣𝑗 and 𝑣𝑖, and 𝑓(𝑥) is the transformation function. A new value is calculated with (6) for each node at each time step. The most commonly applied transformation functions are the sigmoid function, the hyperbolic tangent function, the step function and the threshold linear function (Bueno & Salmeron, 2009; Yaman & Polat, 2009). Researchers should assess them in order to select the one that is most suitable to the requirements of the study. 14 FCM inference finishes when the limit vector is reached. This happens when either 𝑉𝑡 = 𝑉𝑡+1 or 𝑉𝑡+1 −𝑉𝑡 ≤ 𝜀; where 𝜀 is a residua (Espinosa-Paredes et al., 2008). The inference process can also result in a limit cycle. This implies that the vector state continues changing around several fixed states. When a continuous function is used, chaotic behaviour is possible (Papageorgiou, 2011). This happens when the inference process finds different outputs for each time step and, therefore, the FCM does not attain stability. The next section presents an application validating the method proposed. 4. Case Study It is well known that companies must control the normal operation of their installed enterprise systems in order to achieve the expected benefits and this is also the case for an ECM adoption. Therefore, firms need an appropriate method to detect whether these solutions work as expected. Hence, the present study proposes using the hybrid FCM and AHP technique to monitor ECM performance. This approach is explained through a real case study. Unlike empirical studies which pursue generalized findings, case studies have proved to be an excellent vehicle to test the applicability of the technique in a concrete and real-life situation (Yin, 2013). 4.1 Company description The case study was carried out in a European company which is a pioneer in providing services for Business Process Outsourcing (BPO) and digital technology. It had over 18 years of experience and more than 70,000 employees in 2015 and generated a turnover of more than 15 4,800 million euros. Today, the case study company operates services in over 40 countries around the world. The case study was carried out in one of their platforms located in Spain. The platform activity mainly focuses on providing a set of services related to customer relationship management, data intelligence and a wide range of business solutions. This branch implemented an ECM solution several years ago. Since then, the computer-based system centralizes in a single electronic solution the management of information assets (including structured and unstructured) generated and used in their own business processes. The tool provides relevant, precise and up-to-date information to help users in decision making and business process development. For this purpose, the enterprise system and the business process have to be completely aligned (Seddon & Calvert, 2010). If this were not the case, ECM underperformance would not occur. To control if this is really happening, the participants of the case study shared with us their ECM experience from a technological and a business perspective. The following subsection describes their active participation in building the FCM model. 4.2 Building the FCM model Different methods can be used to build FCM (Kang, Lee, & Choi, 2004; Sangjae Lee & Ahn, 2009; Papageorgiou et al., 2009; Schneider, Shnaider, Kandel, & Chew, 1998). Often each expert builds his/her own FCM. They represent their knowledge of their particular expertise. As there may be differences between their FCM, it may be necessary to use other methodologies to reach a consensus between participants, such as Delphi or the augmented FCM method. 16 In the augmented FCM method, the adjacency matrix of each participant is added to obtain the final diagraph-based FCM model (Dickerson & Kosko, 1993). This approach does not need participants to change their former opinions to obtain a consensus as in the Delphi methodology (Salmeron, 2009). In addition, experts’ models are not constrained by a closed list of concepts in such a way as to ensure that the final FCM represent all the insights. For these reasons, we decided to use this approach to build the FCM. Fig. 3 summarizes the main activities during the FCM building process. In order to build the FCM, which helps managers to control the performance of their ECM solutions, we held individual face-to-face interviews with the participants during the months of September and October 2016. They then occupied the IT manager and the operation manager positions. Each one built his/her own FCM model during the interview itself. We therefore designed a questionnaire for the purpose of guiding participants during the FCM building process. The questionnaire was organized into four parts. In the first part, the participants answered general open questions about their company and the ECM implemented. The goal was to elicit a better understanding about the firm’s activities and the system in use. In the second part, the participants identified the most relevant indicators that control ECM performance in an effective way. To make this work easier, we prepared an initial list of indicators based on the literature (Alalwan et al., 2014; Brocke et al., 2010; López & Salmeron, 2014). The participants could check it and add new elements. They could also remove elements that according to their experience were irrelevant. 17 Fig. 3 FCM building process Table 1 summarizes the final FCM nodes and their descriptions. Each indicator (I) node represents one specific measure required to monitor or control the ECM performance. The ECM node corresponds to the system performance. The nodes I1-I5 are related to business tasks supported, the nodes I6-I11 are related to decision making and the nodes I12-I14 are related to the technological perspective. 18 Table 1 FCM nodes, in italics, are new nodes proposed by the participants Code Nodes Description I1 Business tasks supported Number of business tasks supported by the ECM system I2 Efficiency of business tasks supported Users can carry out their business task in a more efficient way I3 Automatizing business task Procedures are properly defined and therefore can be automated in the ECM system I4 Velocity of business task development Users develop more easily and therefore more quickly business tasks when supported by ECM I5 Compliance with standards, certifications and rules Adopter company complies with standards, certifications and rules supported by ECM I6 Decision-making speed Users make quicker decisions I7 Decision-making quality Outcomes of the decision are usually accurate (without errors) and precise I8 Relevant information available Users can access diverse sources of relevant information during business tasks development I9 Creativity and critical thinking ECM system encourages creativity and critical thinking of users I10 Problem identification speed ECM system helps users to shorten the time frame to identify key factors related to the problem I11 Alternatives evaluation Users can evaluate more alternatives I12 System complexity Interactions of the system with existing architectural layers as well as between components and subsystems. I13 System volatility System ability to keep operating despite continuous changes I14 System adaptability System ability to work in new environments ECM ECM performance Indicators describing ECM performance In the third part, we defined the five linguistic terms used in the research to reflect the influence or strength of the causality (𝑤𝑖→𝑗) between a pair of nodes < 𝑣𝑖,𝑣𝑗 >. These are depicted in Table 2. Yet, fuzzy weights need be translated into real numbers in the space [0, 19 1] to obtain the final digraph-based FCM model. With this in mind, the participants carried out a pairwise comparison between linguistic terms using a 9-point scale proposed in (Saaty, 1977). Fig 4 shows an extract of the questionnaire. Later, the comparisons were imported into Expert Choice (http://expertchoice.com/). This software enabled us to calculate real numbers with the AHP method (Section 3). Table 2 also shows the real number derived from it. Table 2 Linguistic variables and the associated quantitative value. Abbreviation Description Quantitative value VW Very Weak 0.025 W Weak 0.049 M Moderate 0.113 H High 0.243 VH Very high 0.571 Fig. 4 Extract of the questionnaire Circle one number per row below using the scale: 1 = Equal 3 = Moderate 5 = Strong 7 = Very strong 9 = Extreme 2, 4, 6, 8 are intermediate values Compare the relative performance of one linguistic term with all other linguistic terms to determine the strength of relationship between indicators of ECM performance. Very weak 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Weak Very weak 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Moderate Very weak 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Strong Very weak 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Very strong Weak 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Moderate Weak 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Strong Weak 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Very strong Moderate 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Strong Moderate 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Very strong Strong 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Very strong http://expertchoice.com/ 20 Finally, in the fourth part, the participants drew their FCM model. To do so, they defined the causal relationships between nodes. This can be expressed in the form of IF-THEN rules, where the sender or influencing node follows a binary code (ON or OFF) and the receiver or influenced node increases (+) or decreases (-) by a level evaluated using linguistic terms. With this in mind, the participants assigned the 𝑤𝑖→𝑗 values and generated their adjacency matrices (𝑊𝑃𝑖). When a participant describes them, three points exist and must be kept in mind. In the first place, they should be aware that the 𝑤𝑖→𝑗 weight indicates how strongly the 𝑣𝑖 node influences the 𝑣𝑗 node. Secondly, the strength of the relationship is given by a fuzzy weight preceded by a positive or negative sign. This shows whether the connection is direct (+) or inverse (-). Lastly, they also define the causal relation between variables. That is, if the 𝑣𝑖 node is a cause of the 𝑣𝑗 node or vice versa. We generated the final adjacency matrix (𝑊𝐴𝑢𝑔) on the outputs achieved in the previous activities. To do this, we added the 𝑊𝑃𝑖 of each participant When more than one participant assigns the 𝑤𝑖→𝑗 value, then 𝑤𝑖→𝑗 𝐴𝑢𝑔 = ∑ 𝑤𝑖→𝑗 𝑃𝑘𝑛 𝑘=1 𝑛⁄ where k is the identifier of each participant and 𝑛 is the number of participants. The FCM model also incorporates connections indicated by one participant. These do not require any additional transformation. The 𝑊𝐴𝑢𝑔 obtained is the following: 21 𝑊𝐴𝑢𝑔 = ( .000 .000 .243 .000 .000 −.24 .178 .000 .000 .000 .000 .178 .243 .000 .178 .000 .000 .000 .000 .000 .000 .113 .000 .000 .000 .000 .000 .000 .000 .000 .000 .243 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .113 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .243 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .113 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .113 .000 .000 .000 .000 .000 −.11 .178 .000 .000 .000 .342 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .113 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .049 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .243 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 −.24 .000 .000 .000 .000 .000 .000 .113 .000 −.407 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 −.113 −.113 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .113 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 ) Finally, we drew a graph-based model in line with the adjacency matrix obtained. Fig.5 presents the FCM proposed. The model consists of the fourteen representative nodes of control indicators (I) as well as the representative node of system performance (ECM). It also shows the static connections existing between each of them. The numbers in brackets represent negative relationships. For example, 𝑤𝐼12→𝐸𝐶𝑀 = (.407) means that an increase of “System complexity” (I12) causes a moderate decrease of ECM performance. Fig. 5 FCM for controlling ECM performance 22 4.3 Simulating scenarios to control ECM performance ECM systems performance should be monitored from a technical and also an organizational perspective (Arshad et al., 2014; Bianco & Michelino, 2010). The FCM model incorporates indicators related to both perspectives. If we focus our analysis on the organizational perspective, we can observe nodes or indicators related to the business task supported and the decision making with ECM. If we focus our analysis on the technical perspective, we can supervise nodes related to the ECM technological issues. Accordingly, we defined the following “what-if” scenarios at the instant 0:  Scenario 1a activates indicators related to business task supported by ECM.  Scenario 1b activates indicators related to business task supported by ECM, with the exception of I1.  Scenario 2 activates indicators related to decision making using ECM.  Scenario 3 activates indicators related to the ECM technological perspective. Table 3 shows the inputs and outputs of the scenarios simulated. In this case, the fuzzy weights are within the range [-1, 1]. These values can only be transformed by the hyperbolic tangent function (7) (Feyzioglu, Buyukozkan, & Ersoy, 2007; Stylios & Groumpos, 2000). This was chosen for the simulation of the model proposed. tanh(𝑥) = 𝑒𝜆∙𝑥−𝑒−𝜆∙𝑥 𝑒𝜆∙𝑥+𝑒−𝜆∙𝑥 (7) The hyperbolic tangent function uses lambda (𝜆) as a constant for the function slope. Although 𝜆 = 5 has shown to get a good degree of normalization with the sigmoid function 23 (Bueno & Salmeron, 2009), it might tend to approximate outputs to extreme values, i.e., one and zero. On the contrary, for smaller values of 𝜆, the hyperbolic function approximates a linear function. Earlier studies researched the best 𝜆 (Mago, Mehta, Woolrych, & Papageorgiou, 2012; Miao & Liu, 2000). Based on their results, we used a 𝜆 value equal to 1 in the inference process. Table 3 presents the findings obtained by simulating Scenarios S1a, S1b, S2 and S3. They reached stability after 629, 536, 581 and 534 time steps respectively. The results express how an activated indicator may affect other indicators and the ECM’s performance. Table 3 Inputs and outputs indexed for scenarios. Scenarios S1a S1b S2 S3 Nodes Input Output Input Output Input Output Input Output I1 1 0.049 0 0.000 0 0.000 0 0.000 I2 1 0.617 1 0.588 0 0.000 0 0.000 I3 1 0.401 1 0.331 0 0.000 0 0.000 I4 1 0.049 1 0.053 0 0.000 0 0.000 I5 1 0.049 1 0.053 0 0.000 0 0.000 I6 0 -0.579 0 0.000 1 -0.209 0 -0.331 I7 0 0.571 0 0.546 1 0.617 0 0.000 I8 0 0.000 0 0.000 1 0.051 0 0.000 I9 0 0.000 0 0.000 1 0.051 0 0.000 I10 0 0.000 0 0.000 1 0.051 0 0.000 I11 0 0.000 0 0.000 1 0.364 0 0.000 I12 0 0.292 0 0.000 0 0.000 1 0.053 I13 0 0.487 0 0.000 0 0.000 1 0.259 I14 0 -0.516 0 0.000 0 0.000 1 -0.427 ECM 0 -0.751 0 0.534 0 0.489 0 -0.664 Interactions 629 536 581 534 25 The simulation of Scenario 1a shows how the changes of indicators related to business task supported may affect other indicators and the ECM performance. We see that the nodes are in the range of -0.752 to 0.572. This scenario produces remarkable influences on both the decision-making indicators group and the technological indicators group. As regards the first group, only “Decision-making speed” (I6) and “Decision-making quality” (I7) were altered in the inference process. Increases in indicators related to business task supported cause a high reduction (-0.579) in “Decision-making speed” (I6) with ECM. This is due to the fact that the negative impact of “Relevant information available” (I8) in “Decision-making speed” (I6) is higher than the positive impact of the “Problem-identification speed” (I10) effect. Indeed, if ECM users have to access diverse sources of relevant information during business tasks development, they will make slow decisions. By contrast, indicators related to business task supported have a slightly positive impact on “Decision-making quality” (0.572). In this case, the effect of “Relevant information available” (I8) is positive, the same as the “Alternatives evaluation” (I11). Therefore, if managers aim to make more accurate and precise decisions, they should increase the source of “Relevant information available” (I8) and improve “Alternatives evaluation with ECM” (I11), although these actions might slow down the decision- making processes. On the technical side, Scenario 1a leads to modifications throughout all its indicators. It exerts a positive influence on both the “System complexity” (I12) and the “System volatility” (I13) but with different intensities. “Systems volatility” receives a moderate impact (0.487), while “System complexity” grows at a low rate (0.292). “System adaptability” (I14) suffered a negative and moderate influence (-0.516). Moreover, the results reveal that Scenario 1a highly negatively affects the ECM performance (-0.751). It would be reasonable to believe that increases in indicators related to business task 26 supported causes a positive effect in system performance. However, as we can observe in the FCM model, the node number of “Business tasks supported” by the ECM system (I1) impacts positively on “System complexity” (I12) and “System volatility” (I13). Hence, we defined Scenario 1b. This scenario reveals the influence of the indicators related to business tasks supported when the number of “Business tasks supported” remains stable (I1). Contrary to Scenario 1a, this does not have any influence on any of the technological indicators. It does not slow down “Decision-making speed” with ECM (I6) as in Scenario1a. The influence on “Decision-making quality” (I7) is similar to Scenario 1a with (0.546). As before, Scenario 1b impacts moderately positively on ECM performance (.534). Therefore, the managers can carry out actions to improve indicators related to business tasks supported, although preserving “System complexity” (I12) and “System volatility” (I13). The simulation of Scenario 2 indicates how the changes of the indicators related to decision-making modify only the ECM performance. This affects moderately and positively the system performance (0.489). Nevertheless, Scenario 2 does not have any influence on the indicators related to business tasks supported and those related to technological issues. The simulation of Scenario 3 clarifies how the modification of the indicators related to technological issues may impact on other indicators and ECM performance. This reveals that the combined increase of “System complexity” (I12), “System volatility” (I13) and “System adaptability” (I14) causes a slightly moderate slowdown of the decision-making speed when using ECM (-0.331). As in Scenario 1a, Scenario 3 negatively affects ECM performance (-0.665). The comparison between the impacts of the four scenarios on the ECM performance highlights relevant issues. Fig. 6 represents the scenarios’ influence on the system 27 performance. We observe that these range from to -.752 to .534. Depending on the scenario simulated, ECM performance can be improved or damaged. Scenario 1b and Scenario 2 improved ECM performance, while Scenario 1a and Scenario 3 damaged it. It is interesting to note that Scenario 1a and Scenario 1b are very similar. The unique difference is that the node number of “Business tasks supported” remains stable (I1) in Scenario1b. Notwithstanding, there is a considerable disparity in their impacts on ECM performance. If we observe the static representation of the FCM, only I1 is directly connected to I12 and I13. This fact is reflected in the results of Scenario 1a. As mentioned before, the simulation of Scenario 3 impacts negatively on the ECM performance. In this way, the influence of node I1 would explain the negative effect of Scenario 1a on system performance. Therefore, if the users require adding new tasks supported by the systems, professionals should carefully control the system complexity and the system volatility as the ECM performance could be seriously affected. Fig. 6 Simulations effects on ECM performance. 5. Discussion ECM packages have proven to be an excellent enterprise-wide approach for managing all information assets. Nonetheless, as the above results show, system performance would 28 be altered in many circumstances. Hence, this study presents a novel expert system for controlling ECM performance since it is operative until its withdrawal. The findings highlight new advances in both the theoretical and the practical field. 5.1 Theoretical contribution Nowadays, numerous firms have already adopted ECM solutions in their IT infrastructures. In order to get a proper ECM performance, they have to continuously overcome challenges derived from the technology itself and the deployment of new work routines from both business process and decision-making perspectives. Laumer, Beimborn, Maier, & Weinert (2013) already characterized ECM packages on three levels. (1) The enterprise level makes possible the supply of required information for carrying out business-related procedures. Our model brings together indicators related to business tasks supported by system, which make it possible to control that level of ECM performance. (2) The system level represents technological artefacts on which ECM solutions run. Accordingly, ECM performance should be also controlled by technological indicators, and these are covered in the model. (3) The information level determines the data required by users. Other studies also considered the people dimension or the relationship between users and contents in their view of ECM system (Alalwan & Roland, 2012; Tyrväinen, Päivärinta, Salminen, & Iivari, 2006). This one is closely related to the information level and therefore our research proposes to combine them under the same level named decision making. Our case study enriches what we already know about ECM because it is the first time that an investigation identifies indicators used to control ECM performance from the above-mentioned levels. Nordheim & Päivärinta (2006) identified critical issues that had arisen during the ECM implementation stage, which cannot be extrapolated to ECM post- implementation stage. Brocke, Simons, Sonnenberg, Agostini, & Zardini, (2010) 29 presented a framework to measure the financial performance of business processes, while our article focuses on ECM performance. Wiltzius, Simons, Seidel, & Vom Brocke (2014) explored which acceptance factors can affect users’ perception of the usefulness and ease of use of ECM. Yet, a good level of ECM users’ acceptance does not necessarily imply a good system performance level. Hence, those factors cannot be applied to control ECM performance. Laumer, Maier, & Weitzel (2017) have recently provided indicators related to system quality, service quality and information quality for understanding users’ satisfaction and the manifestations of workarounds. Given that the dimensions under study are not the same, it is normal that most of the indicators do not match. This fact highlights the gap filled, so that there is a complementarity of our research with the existent investigations. Our research also provides a new advance in the ECM body of knowledge since the findings highlight how indicators of ECM performance are closely related. A change in one indicator can be without a direct effect on the system performance, but it may cause a cascading effect and thus impact on the package performance. With this in mind, we developed a specific expert system to predict these effects. This is based on a novel approach based on the hybridation of FCM and AHP. FCM is an artificial intelligence technique that incorporates advances from fuzzy logic and neural networks into CM approach (Ferreira, Ferreira, Fernandes, Meidutė-Kavaliauskienė, & Jalali, 2017). Table 4 summarizes the pros and cons of FCM-AHP with regard to previous modeling techniques (Baldi & Rosen-Zvi, 2005; Bañuls, López, Turoff, & Tejedor, 2017; Kitchenham, Pickard, Linkman, & Jones, 2003; López & Salmeron, 2014). 30 Table 4 Comparison of modeling methods in terms of pros and cons. Modelling techniques Pros Cons Systems Dynamics  Capable of modeling all possible relationships.  Capable of modeling directed graph with cycles.  The propagation does not follow an established pattern.  Cannot model uncertainty.  Cannot assume scarcity of information. Bayesian Networks  Capable of modeling all possible relationships.  Can incorporate uncertainty.  Cannot model directed graph with cycles.  Does not quantify the uncertainty in causal connections in a precise way.  The propagation follows an established pattern.  Cannot assume scarcity of information. Neural Networks  Capable of modeling all possible relationships.  Can incorporate uncertainty.  Assumes information is scarce.  Cannot model directed graph with cycles.  Does not quantify the uncertainty in causal connections in a precise way.  The propagation follows an established pattern. FCM  Capable of modeling all possible relationships.  Can model uncertainty.  Can model directed graph with cycles.  The propagation does not follow an established pattern.  Assumes information is scarce.  Does not quantify the uncertainty in causal connections in a precise way. FCM-AHP  Capable of modeling all possible relationships.  Can incorporate uncertainty.  Capable of modeling directed graph with cycles. 31  The propagation does not follow an established pattern.  Assumes information is scarce.  Capable of quantifying the uncertainty in causal connections in a precise way. FCM is not exempt of drawbacks. In fact, it lacks a clear instrument to transform linguistic evaluations of existing interactions into quantitative evaluations. This transformation is normally performed by using the centroid method, the max aggregation method or the mamdani inference mechanism (Mago et al., 2012), although these methods do not consider possible imprecisions in experts’ evaluations. Earlier studies bring to light that AHP can compute quantitative evaluations from the linguistic evaluation of the experts (Ishizaka & Nguyen, 2013; Meesariganda & Ishizaka, 2017). Additionally, this method provides a consistency measure, which strengthens the robustness of the final FCM model. 5.2 Practical contribution Expert systems have been long recognized to be of invaluable resources because if experts leave an organisation their experience and knowledge also goes (Dehghani & Akhavan, 2017). Our developed system is much more than an expert system. It incorporates expert knowledge but it also allows simulating scenarios that are too complex for a human mind to process. Results of simulations help practitioners to gain a better understanding on how to control and improve ECM performance. In this way, the application of our developed systems has provided important insights.  Increases in indicators related to business task supported may cause a high and negative effect on ECM performance, as well as “Decision-making speed” (I6) 32 and “System adaptability” (I14). Results highlight this effect is explained by the effect of “Business tasks supported” (I1) on “System complexity” (I12) and “System adaptability” (I13), which in turn may cause a damage in ECM performance. This consequence is very difficult to predict without the proposed system, since an increase of “Business tasks supported” (I1) directly prompts an improvement in ECM performance. Therefore, when a new business task is supported by the ECM, practitioners should monitor its effects on both system complexity and system adaptability.  Increases in indicators related to decision making may result in a better ECM performance. However, simulations reveal “Decision-making speed” (I6) can be damaged by increases in “Business tasks supported” (I1) and “System complexity” (I12). Practitioners should control them to avoid a slowdown in the final users’ decisions, and thus the possible manifestation of workarounds as explained in (Laumer, Maier, & Weitzel, 2017).  Increases in indicators related to technological issues may negatively impact on ECM performance. This happens because the “System complexity” (I12) and “System volatility” (I13) nodes impact negatively on system performance. During the post-implementation stage, IT staff identify, assess and carry out the modifications required for improving or maintaining system usability and performance. The maintenance operation can thus generate additional interactions of the system with architectural layers, as well as further components, artefacts and subsystems. Through a ripple effect, the modifications may affect other modules of the system, increasing system complexity. This might increasingly lead to costly maintenance and damages in the solution stability (Pereira, 2001). Concerning an uncontrollable volatility, it may even cause the system not to meet 33 user requirements and to need to be completely revised. Hence, increasing “System complexity” (I12) and “System volatility” (I13) should be avoided. 6. Conclusions and directions for future research The present study provides a coupled FCM-AHP method to control ECM performance during its post-implementation stage. The application of an FCM technique is proposed to represent a real-world dynamic system target. The findings provide a graphical description of the ECM performance and facilitate an increased understanding of this problem. In fact, the model enables the representation of the most relevant indicators related to ECM performance. These can be brought together into categories. The first includes the indicators related to business tasks supported by ECM. The second contains the indicators related to decision making using ECM. The third encompasses indicators related to ECM technological issues. The diagraph-based FCM shows the interaction between them. Furthermore, this illustrates their direct effects on the ECM performance. The usefulness of such a model is also explored through the FCM’s dynamic behavior. Moreover, implications for its use in ECM supervision decision making are described. FCM specifically predict ECM performance by allowing relevant indicators to interact with each other. Practitioners will thus be able to apply measures aimed at maintaining an adequate ECM exploitation. In this way, FCM simulations reveal how three categories of indicators affect ECM performance in different ways. While indicators related to decision making prompt better system performance, indicators related with technological issues cause underperformance. This is due to the fact that the negative effects of increased “System complexity” (I12) and “System volatility” (I13) outweigh the positive effects of “System adaptability” (I14) on ECM performance. This would be enhanced if managers took steps aimed at improving indicators related to decision making (I6-I11) 34 and “System adaptability” (I13). At the same time, managers should control an unavoidable rise in “System complexity” (I12) and “System volatility” (I13). The influence of the indicators related to business tasks supported is to a great extent conditioned by the effect of “Business tasks supported” (I1). If the number of business tasks is stable, advances in the other indicators may improve ECM performance. On the contrary, if the number of business tasks increases, the ECM performance will be negatively affected. This can be explained by the increased complexity and volatility of the system caused by I1. Hence, to avoid this negative chain effect on ECM performance, when new business tasks are automatized, managers ought to carry out actions to control complexity and system volatility. This study illustrates how a hybrid FCM-AHP method was applied to control ECM performance in a concrete case. It was able to quantify the uncertainty in causal connections in a precise way. Nonetheless, it would be relevant to validate the proposed model with Exploratory Factor Analysis (EFA) and/or Confirmatory Factor Analysis (CFA). This would help the development of a unique model generalizable to other cases. As the developed method is generic, flexible and easily adaptable, it can be adapted without difficulties to other studies. Concerning the artificial intelligence perspective of the developed expert system, the more data is available to represent the system, the better FCM becomes at reaching a more precise solution. However, the incompleteness and uncertainties in experts’ perceptions might hinder this and therefore a new method to overcome this issue should be developed. Furthermore, a widely recognized method that measures the accuracy of the FCM model does not in this case exist. Likewise, FCM enable what-if scenarios to be forecasted over time. Yet they lack a measure of time (e.g., when it will happen, tomorrow, in one year, etc.). These would be very relevant advances in the field. 35 Finally, it would be pertinent to study the importance of each indicator. The idea is to rank them according to the level of the importance perceived by the key actors (i.e., users, managers, analysts, and vendors). The findings from doing so would help decision makers to manage their ECM systems in a more effective way. References AIIM. (2010). What is enterprise content management (ECM)? Retrieved January 1, 2016, from http://www.aiim.org/ Alalwan, J. A. (2013). A taxonomy for decision support capabilities of enterprise content management systems. Journal of High Technology Management Research, 24(1), 10–17. 10.1016/j.hitech.2013.02.001 Alalwan, J. A., & Roland, W. H. (2012). Enterprise content management research: a comprehensive review. Journal of Enterprise Information Management, 25(5), 441–461. 10.1108/09564230910978511 Alalwan, J. A., Thomas, M. A., & Weistroffer, H. R. (2014). Decision support capabilities of enterprise content management systems: An empirical investigation. Decision Support Systems, 68, 39–48. 10.1016/j.dss.2014.09.002 Allen, D. (2007). Cost / Benefit analysis for implementing ECM, BPM systems. The Information Management Journal, (May-June), 34–41. Andersen, R. (2008). The rhetoric of Enterprise Content Management (ECM): confronting the assumptions driving ECM adoption and ransforming Technical Communication. Communication, Technical Communication Quarterly, 17, 61–87. Arshad, N. I., Bosua, R., & Milton, S. K. (2015). Towards a model to understand ECMS-use in supporting business processes. Procedia Computer Science, 72, 194–200. 10.1016/j.procs.2015.12.121 Arshad, N. I., Milton, S. K., & Bosua, R. (2014). Enterprise Content Management systems-use supports standardized business processes. International Journal of Engineering & Technology, 14(2), 81–87. Axelrod, R. (1976). Structure of decision :The cognitive maps of political elites. Princeton, New Jersey: Princeton University Press. Baldi, P., & Rosen-Zvi, M. (2005). On the relationship between deterministic and probabilistic directed Graphical models: from Bayesian networks to recursive neural networks. Neural Networks : The Official Journal of the International Neural Network Society, 18(8), 1080– 1086. 10.1016/j.neunet.2005.07.007 Bañuls, V. A., López, C., Turoff, M., & Tejedor, F. (2017). Predicting the impact of multiple risks on project performance: a scenario-based approach. Project Management Journal, 48(November), 1–20. Bianco, F., & Michelino, F. (2010). The role of content management systems in publishing firms. International Journal of Information Management, 30(2), 117–124. 10.1016/j.ijinfomgt.2009.11.001 Blair, B. T. (2004). An enterprise content management primer. Information Management Journal, 38(October), 64–66. Boiko, B. (2001). Understanding Content Management. Bulletin of the American Society for Information Science and Technology, 28(1), 8–13. 10.1002/bult.221 Brocke, J., Seidel, S., & Simons, A. (2010). Bridging the gap between Enterprise Content Management and creativity: a research framework. In Proceedings of the 43rd Hawaii International Conference on System Sciences (pp. 1–10). Honolulu, HI, USA. Brocke, J., Simons, A., Sonnenberg, C., Agostini, P. L., & Zardini, A. (2010). Value Assessment of Enterprise Content Management Systems: A Process-oriented Approach. In 36 Information Systems: People, Organizations, Institutions, and Technologies (1st ed., pp. 131–138). Berlin: Heidelberg: Physica-Verlag. 10.1007/978-3-7908-2148-2 Bueno, S., & Salmeron, J. L. (2009). Benchmarking main activation functions in fuzzy cognitive maps. Expert Systems with Applications, 36(3), 5221–5229. 10.1016/j.eswa.2008.06.072 Dehghani, M., & Akhavan, P. (2017). An experimental investigation of knowledge acquisition techniques. Journal of Management Development, 36(4), 493–514. 10.1108/JMD-07- 2016-0132 DeSanctis, G., & Gallupe, B. (1987). A foundation for the study of group decision support systems. Management Science, 33(5), 589–609. 10.1287/mnsc.33.5.589 Dickerson, J. A., & Kosko, B. (1993). Virtual worlds as fuzzy cognitive maps. In IEEE Virtual Reality Annual International Symposium (pp. 471–477). Seatle, WA, USA: IEEE. 10.1109/VRAIS.1993.380742 Eden, C. (2004). Analyzing cognitive maps to help structure issues or problems. European Journal of Operational Research, 159(3), 673–686. 10.1016/S0377-2217(03)00431-4 Escalona, M. J., Domínguez-Mayo, F. J., García-García, J. A., Sánchez, N., & Ponce, J. (2015). Evaluating enterprise content management tools in a real context. Journal of Software Engineering and Applications, 8(August), 431–453. 10.4236/jsea.2015.88042 Espinosa-Paredes, G., Nuñez-Carrera, A., Laureano-Cruces, A. L., Vázquez-Rodríguez, A., & Espinosa-Martinez, E. G. (2008). Emergency management for a nuclear power plant using fuzzy cognitive maps. Annals of Nuclear Energy, 35(12), 2387–2396. 10.1016/j.anucene.2008.07.007 Ferreira, F. A. F., Ferreira, J. J. M., Fernandes, C. I. M. A. S., Meidutė-Kavaliauskienė, I., & Jalali, M. S. (2017). Enhancing knowledge and strategic planning of bank customer loyalty using fuzzy cognitive maps. Technological and Economic Development of Economy, 4913(June), 1–17. 10.3846/20294913.2016.1213200 Feyzioglu, O., Buyukozkan, G., & Ersoy, M. S. (2007). Supply chain risk analysis with fuzzy cognitive maps. In IEEE International Conference on Industrial Engineering and Engineering Management (pp. 1447–1451). Singapore. 10.1109/IEEM.2007.4419432 Grahlmann, K. R., Helms, R. W., Hilhorst, C., Brinkkemper, S., & van Amerongen, S. (2012). Reviewing Enterprise Content Management: a functional framework. European Journal of Information Systems, 21(3), 268–286. 10.1057/ejis.2011.41 Haug, A. (2012). The implementation of enterprise content management systems in SMEs. Journal of Enterprise Information Management, 25(4), 349–372. 10.1108/17410391211245838 Herbst, A., Simons, A., Brocke, J. vom, & Derungs, R. (2014). Critical Success Factors in Enterprise Content Management: Toward a Framework for Readiness Assessment. In Enterprise Content Management in Information Systems Research (1st ed., pp. 109–124). Berlin: Heidelberg: Physica-Verlag. 10.1007/978-3-642-39715-8 Holsapple, C. W., Raj, V., & Wagner, W. P. (2008). An experimental investigation of the impact of domain complexity on knowledge acquisition (KA) methods. Expert Systems with Applications, 35(3), 1084–1094. 10.1016/j.eswa.2007.08.004 Horne, S. B., & Hawamdeh, S. (2015). Factors impacting the implementation of enterprise content management systems. Journal of Information & Knowledge Management, 14(1), 1–11. 10.1142/S0219649215500082 Hullavarad, S., O’Hare, R., & Roy, A. K. (2015). Enterprise Content Management solutions - Roadmap strategy and implementation challenges. International Journal of Information Management, 35(2), 260–265. 10.1016/j.ijinfomgt.2014.12.008 Ishizaka, A. (2012). Clusters and pivots for evaluating a large number of alternatives in AHP. Pesquisa Operacional, 32(1), 87–101. Ishizaka, A., & Labib, A. (2011). Review of the main developments in the analytic hierarchy process. Expert Systems with Applications, 38(11), 14336–14345. 10.1016/j.eswa.2011.04.143 Ishizaka, A., & Lusti, M. (2006). How to derive priorities in AHP: A comparative study. Central European Journal of Operations Research, 14(4), 387–400. 10.1007/s10100-006- 37 0012-9 Ishizaka, A., & Nguyen, N. H. (2013). Calibrated fuzzy AHP for current bank account selection. Expert Systems with Applications, 40(9), 3775–3783. 10.1016/j.eswa.2012.12.089 Iverson, J., & Burkart, P. (2007). Managing electronic documents and work flows. Enterprise Content Management at work in nonprofit organizations. Nonprofit Management & Leadership, 17(4), 18–20. 10.1002/nml Jetter, A. J., & Kok, K. (2014). Fuzzy Cognitive Maps for futures studies-A methodological assessment of concepts and methods. Futures, 61, 45–57. 10.1016/j.futures.2014.05.002 Kang, I., Lee, S., & Choi, J. (2004). Using fuzzy cognitive map for the relationship management in airline service. Expert Systems with Applications, 26(4), 545–555. 10.1016/j.eswa.2003.10.012 Kardaras, D. K., Karakostas, B., & Mamakou, X. J. (2013). Content presentation personalisation and media adaptation in tourism web sites using Fuzzy Delphi Method and Fuzzy Cognitive Maps. Expert Systems with Applications, 40(6), 2331–2342. 10.1016/j.eswa.2012.10.031 Kim, H. S., & Lee, K. C. (1998). Fuzzy implications of fuzzy cognitive map with emphasis on fuzzy causal relationship and fuzzy partially causal relationship. Fuzzy Sets and Systems, 97(3), 303–313. 10.1016/S0165-0114(96)00349-1 Kitchenham, B., Pickard, L. M., Linkman, S. G., & Jones, P. W. (2003). Modeling software bidding risks. IEEE Transactions on Software Engineering, 29(6), 542–554. 10.1109/TSE.2003.1205181 Kosko, B. (1986). Fuzzy Cognitive Maps. Int. Jornal of Man-Machine Studies, 1(April 1985), 65–75. 10.1016/S0020-7373(86)80040-2 Kyriakarakos, G., Dounis, A. I., Arvanitis, K. G., & Papadakis, G. (2012). A fuzzy cognitive maps-petri nets energy management system for autonomous polygeneration microgrids. Applied Soft Computing Journal, 12(12), 3785–3797. 10.1016/j.asoc.2012.01.024 Laumer, S., Beimborn, D., Maier, C., & Weinert, C. (2013). Enterprise Content Management. Business & Information Systems Engineering, 5(6), 449–452. 10.1007/s12599-013-0291-3 Laumer, S., Maier, C., & Weitzel, T. (2017). Information quality, user satisfaction, and the manifestation of workarounds: a qualitative and quantitative study of enterprise content management system users. European Journal of Information Systems, 26(4), 333–360. 10.1057/s41303-016-0029-7 Lee, K. C., Lee, H., Lee, N., & Lim, J. (2013). An agent-based fuzzy cognitive map approach to the strategic marketing planning for industrial firms. Industrial Marketing Management, 42(4), 552–563. 10.1016/j.indmarman.2013.03.007 Lee, S., & Ahn, H. (2009). Fuzzy cognitive map based on structural equation modeling for the design of controls in business-to-consumer e-commerce web-based systems. Expert Systems with Applications, 36(7), 10447–10460. 10.1016/j.eswa.2009.01.070 Lee, S., Yang, J., & Han, J. (2012). Development of a decision making system for selection of dental implant abutments based on the fuzzy cognitive map. Expert Systems with Applications, 39(14), 11564–11575. 10.1016/j.eswa.2012.04.032 Leidner, D. E., & Elam, J. J. (1993). Executive information systems: their impact on executive decision making. Journal of Management Information Systems, 10(3), 139–155. 10.1109/HICSS.1993.284314 Lin, H. Y., Hsu, P. Y., & Sheen, G. J. (2007). A fuzzy-based decision-making procedure for data warehouse system selection. Expert Systems with Applications, 32(3), 939–953. 10.1016/j.eswa.2006.01.031 Lopez, C., & Salmeron, J. L. (2014). Dynamic risks modelling in ERP maintenance projects with FCM. Information Sciences, 256, 25–45. 10.1016/j.ins.2012.05.026 López, C., & Salmeron, J. L. (2014). Modeling maintenance projects risk effects on ERP performance. Computer Standards & Interfaces, 36(3), 545–553. 10.1016/j.csi.2013.11.002 Luconi, F. L., Malone, T. W., & Morton, M. S. S. (1986). Expert systems: the next challenge for managers. Sloan Management Review, 27(4), 3–14. Mago, V. K., Mehta, R., Woolrych, R., & Papageorgiou, E. I. (2012). Supporting meningitis 38 diagnosis amongst infants and children through the use of fuzzy cognitive mapping. BMC Medical Informatics and Decision Making, 12(98), 1–12. 10.1186/1472-6947-12-98 Meesariganda, B. R., & Ishizaka, A. (2017). Mapping verbal AHP scale to numerical scale for cloud computing strategy selection. Applied Soft Computing Journal, 53, 111–118. 10.1016/j.asoc.2016.12.040 Miao, Y., & Liu, Z.-Q. (2000). On causal inference in fuzzy cognitive maps. IEEE Transactions on Fuzzy Systems, 8(1), 107–119. 10.1109/91.824780 Mourhir, A., Rachidi, T., & Karim, M. (2015). Employing fuzzy cognitive maps to support environmental policy development. In IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (pp. 1–8). Istambul, Turkey: IEEE. 10.1109/FUZZ-IEEE.2015.7337969 Munkvold, B. E., Paivarinta, T., Hodne, A. K., & Stangeland, E. (2003). Contemporary issues of Enterprise Content Management: the case of Statoil. In ECIS 2003 Proceedings (pp. 1– 23). Naples, Italy: AISeL. Retrieved from http://aisel.aisnet.org/ecis2003/11 Nordheim, S., & Päivärinta, T. (2006). Implementing enterprise content management: from evolution through strategy to contradictions out-of-the-box. European Journal of Information Systems, 15(6), 648–662. 10.1057/palgrave.ejis.3000647 O’Callaghan, R., & Smits, M. (2005). A strategy development process for enterprise content management. In Proceedings of the 13th european conference on information systems (ECIS) (pp. 1271–1282). Regensburg, Germany. Retrieved from http://aisel.aisnet.org/ecis2005%5Cnhttp://aisel.aisnet.org/ecis2005/148 Obiedat, M., & Samarasinghe, S. (2016). A novel semi-quantitative Fuzzy Cognitive Map model for complex systems for addressing challenging participatory real life problems. Applied Soft Computing, 48, 91–110. 10.1016/j.asoc.2016.06.001 Özesmi, U., & Özesmi, S. L. (2004). Ecological models based on people’s knowledge: A multi- step fuzzy cognitive mapping approach. Ecological Modelling, 176(1–2), 43–64. 10.1016/j.ecolmodel.2003.10.027 Oztaysi, B. (2014). A decision model for information technology selection using AHP integrated TOPSIS-Grey: The case of content management systems. Knowledge-Based Systems, 70, 44–54. 10.1016/j.knosys.2014.02.010 Paivarinta, T., & Munkvold, B. E. (2005). Enterprise Content Management: an integrated perspective on information management. In Proceedings of the 38th Annual Hawaii International Conference on System Sciences (Vol. 0, pp. 1–10). Big Island, HI, USA: IEEE. 10.1109/HICSS.2005.244 Papageorgiou, E. I. (2011). A new methodology for decisions in medical informatics using fuzzy cognitive maps based on fuzzy rule-extraction techniques. Applied Soft Computing Journal, 11(1), 500–513. 10.1016/j.asoc.2009.12.010 Papageorgiou, E. I., Markinos, A., & Gemptos, T. (2009). Application of fuzzy cognitive maps for cotton yield management in precision farming. Expert Systems with Applications, 36(10), 12399–12413. 10.1016/j.eswa.2009.04.046 Pereira, R. E. (2001). Computing ripple effect for software maintenance. Journal of Software Maintenance and Evolution, 13(4), 263–279. 10.1002/smr.233 Saaty, T. L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15(3), 234–281. 10.1016/0022-2496(77)90033-5 Saaty, T. L. (1980). The analytic hierarchy process: planning, priority setting, resource allocation (Decision making series). New York, NY: McGraw-Hill International Book Co. Retrieved from http://www.amazon.com/The-Analytic-Hierarchy-Process- Allocation/dp/0070543712 Salamntu, L. T. P., & Seymour, L. (2015). Growth and maturation of ECM from the year 2001 to 2011. In Fifth International Conference on Digital Information Processing and Communications (pp. 31–37). Sierre/Siders, Switzerland: IEEE. 10.1109/ICDIPC.2015.7323002 Salmeron, J. L. (2009). Augmented fuzzy cognitive maps for modelling LMS critical success factors. Knowledge-Based Systems, 22(4), 275–278. 10.1016/j.knosys.2009.01.002 Schneider, M., Shnaider, E., Kandel, a, & Chew, G. (1998). Automatic construction of FCMs. Fuzzy Sets and Systems, 93(2), 161–172. 10.1016/S0165-0114(96)00218-7 39 Scott, J. E. (2011). User perceptions of an Enterprise Content Management system. In 44th Hawaii International Conference on System Sciences (pp. 1–9). Kauai, HI, USA: IEEE. 10.1109/HICSS.2011.473 Seddon, B. P. B., & Calvert, C. (2010). A multi-project model of key factors affecting organizational benefits from enterprise systems. MIS Quarterly, 34(2), 305–328. Simons, A., Brocke, J. vom, Lässer, S., & Herbst, A. (2014). Lessons Learned from Implementing Enterprise Content Management at the National Public Administration in Liechtenstein Alexander. In Enterprise Content Management in Information Systems Research (1st ed., pp. 199–216). Berlin: Heidelberg: Physica-Verlag. 10.1007/978-3-642- 39715-8 Smith, H. A., & McKeen, J. D. (2003). Developments in practice VIII: Enterprise Content Management. Communications of the Association for Information Systems, 11, 647–659. Retrieved from http://aisel.aisnet.org/cais/vol11/iss1/33 Sneed, H. M., & Brössler, P. (2003). Critical success factors in software maintenance - A case study. In International Conference on Software Maintenance (pp. 190–198). Amsterdam, The Netherlands. 10.1109/icsm.2003.1235421 Sprague, R. H. (1980). A Framework for the Development of Decision Support Systems for the Development of Decision Support Systems, 4(4), 1–26. Stylios, C. D., & Groumpos, P. P. (2000). Fuzzy Cognitive Maps in modeling supervisory control systems. Journal of Intelligent and Fuzzy Systems, 8(1), 83–98. 10.1016/S0960- 0779(98)00303-8 Stylios C.D., & Groumpos P.P. (2004). Modeling complex systems using Fuzzy Cognitive Maps. IEEE Transaction on Systems, Man, and Cybernetics - Part A: Systems and Humans, 34(1), 155–162. Tolman, E. C. (1948). Cognitive maps in rats and men. Psychological Review, 55(4), 198–208. Retrieved from http://psychclassics.yorku.ca/Tolman/Maps/maps.htm Tyrväinen, P., Päivärinta, T., Salminen, A., & Iivari, J. (2006). Characterizing the evolving research on enterprise content management. European Journal of Information Systems, 15, 627–634. 10.1057/palgrave.ejis.3000648 Van Rooij, J. C. G. M. (2013). Legacy Issues in the Implementation of Enterprise Content Management. International Journal of Information and Communication Technology Research, 3(3), 120–123. Vitari, C., Ravarini, A., & Rodhain, F. (2006). an Analysis Framework for the Evaluation of Content Management Systems. Communications of the Association for Information Systems, 18(2006), 782–804. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=25591421&site=ehost- live Vom Brocke, J., Simons, A., & Cleven, A. (2011). Towards a business process-oriented approach to enterprise content management: The ECM-blueprinting framework. Information Systems and E-Business Management, 9(4), 475–496. 10.1007/s10257-009- 0124-6 Vom Brocke, J., Simons, A., Herbst, A., Derungs, R., & Novotny, S. (2006). The business drivers behind ECM initiatives: a process perspective. Business Process Management Journal, 17(6), 965–985. http://dx.doi.org/10.1108/09564230910978511 Votsch, V. (2001). A taxonomy for content management systems. The Seybold Report - Analyzing Publishing Technologies, 1(11), 13–19. Wagner, W. P. (2017). Trends in expert system development: A longitudinal content analysis of over thirty years of expert system case studies. Expert Systems with Applications, 76, 85– 96. 10.1016/j.eswa.2017.01.028 Wiltzius, L., Simons, A., Seidel, S., & Vom Brocke, J. (2014). Factors in the Acceptance of Enterprise Content Management Systems. In Enterprise Content Management in Information Systems Research (1st ed., pp. 37–61). Berlin: Heidelberg: Physica-Verlag. 10.1007/978-3-642-39715-8 Yaman, D., & Polat, S. (2009). A fuzzy cognitive map approach for effect-based operations: An illustrative case. Information Sciences, 179(4), 382–403. 10.1016/j.ins.2008.10.013 40 Yin, R. K. (2013). Case study research: design and methods (5th ed.). London, UK: SAGE Publications, In.