key: cord-020193-3oqkdbq0
authors: Bley, Katja; Schön, Hendrik; Strahringer, Susanne
title: Overcoming the Ivory Tower: A Meta Model for Staged Maturity Models
date: 2020-03-06
journal: Responsible Design, Implementation and Use of Information and Communication Technology
DOI: 10.1007/978-3-030-44999-5_28
sha: 
doc_id: 20193
cord_uid: 3oqkdbq0

When it comes to the economic and strategic development of companies, maturity models are regarded as silver bullets. However, the existing discrepancy between the large amount of existing, differently developed models and their rare application remains astonishing. We focus on this phenomenon by analyzing the models’ interpretability and possible structural and conceptual inconsistencies. By analyzing existing, staged maturity models, we develop a meta model for staged maturity models so different maturity models may share common semantics and syntax. Our meta model can therefore contribute to the conceptual rigor of existing and future maturity models in all domains and can be decisive for the success or failure of a maturity measurement in a company.

Economic development, the assumption of growth, and the ongoing transformation of processes increase the competitive pressure on enterprises of all sizes. Hence, they search for tools that can help determine their current benchmarking position or assess their subsequent performance compared to a predefined best-practice performance [1] . One famous example of these benchmarking tools is Maturity Models (MMs). Considering their components, two definitions result: 'maturity' as a "state of being complete, perfect or ready" [2] and 'model' as "an abstraction of a (real or language based) system allowing predictions or inferences to be made" [3] . Thus, an MM can be regarded as an abstraction of a system that allows predictions or inferences about a complete or perfect state. Its aim is a structured, systematic elaboration of best practices and processes that are related to the functioning and structure of an organization [4] . The MM is divided into different levels, which are used as benchmarks for the overall maturity of an organization. By application, an object is able to assess its own position in a predefined best-practice approach. An awareness of the maturity level in a particular domain is necessary in order to recognize improvement potentials and stimulate a continuous improvement process [5] . Especially when it comes to emerging phenomena like digitalization or the Industrial Internet, MMs became a favored instrument especially for SMEs in order to determine their own digitalization status and corresponding guidelines for improvement. But it is not only in this area that there has been an enormous increase in MM publications in recent years. Due to their general applicability, the rather simple development, and the promising competitive advantages many MMs have emerged in both science and practice over the last decade. They differ in the concepts of maturity, the considered domains, the development approach, or the target group [4, 6] .

Although there exist several approaches to the rigorous development (i.e. [7] [8] [9] ), and there are MMs that already follow these guidelines and should be considered as complete and thorough, a plethora of developers still see a need for new and different kinds of MMs. A possible reason for this phenomenon may be that current MMs lack in consistency regarding their semantics, syntax, and concepts. This leads to a situation where many different MMs introduce ontologies that are either conceptually the same but are used for different subjects and in different contexts so researchers do not understand or recognize similar models or they try to differentiate from existing approaches by using different terminology (e.g. level vs. stages, maturity vs. assessment, dimension vs. area). This in turn weakens the benefits of MMs for companies, as they are unable to understand, compare, and apply the models due to different terminology or incomprehensible concepts and relationships. As a consequence, a solution is needed that proposes theoretical and practical solutions for the semantic and syntactical pitfalls and difficulties mentioned.

We address this problem of conceptual divergence by introducing a UML-based meta model of MMsa formal construct that summarizes the various MM concepts, features, and interdependencies and relates them to each other. We introduce this meta model regarding the different MM concepts, where each MM can be an instance of it as it provides a conceptual template for the rigorous development of new and the evaluation of existing maturity models. We thereby focus on staged MMs, as they have a holistic and cross-organizational structure that is dominant in MM research. Based on the most popular MMs as well as several newly developed MMs, we summarize and discuss already existing MM concepts. We are able to show a valid instantiation of our maturity meta model by classifying different staged MMs in their properties and concepts with our meta model.

According to [10] , MMs typically consist of: (a) a number of levels (commonly 3-6), (b) a generic description of each level, (c) a number of dimensions or "process areas", (d) a number of elements or activities for each process area, and (e) a description of the performance of each activity or element as it might perform or might be performed at each maturity level. The purpose of MM assessment can vary. Depending on its intended aim, the MM focuses on an assessment of the current state of development (descriptive), the identification of improvement potential (prescriptive), or its application as a benchmarking tool (comparative) for an object under study [11] .

MM development has faced an evolution of around 40 years. The early approaches of these models reach back to the late 1970s, when Crosby and Nolan developed the first preliminaries of today's MM: the five-staged Quality Management Maturity Grid and the stage theory on electronic data processing [12, 13] . The concept of these models gained more and more attention, and to date, the Capability Maturity Model Integration (CMMI) [14] represents one of the most well-known and applied MMs.

Although there are different types of MMs, we focus on the dominant form, the staged MM. Its basic assumption is that an organization is constantly evolving, interand intra-organizationally, due to learning effects and improvements. This evolution process is represented in a simplified step-by-step approach using a certain number of degrees of maturity (usually 3-6), a combination of key factors that must be given in order to achieve a certain level of maturity. Figure 1 shows a common representation of a staged MM. When using such an MM, the presence of relevant factors is measured via indicators. After conducting this assessment, the results are summarized with corresponding evaluations and a representative maturity level (see Fig. 1 ).

Researchers who develop an MM have to decide on a research design for model development (i.e., empirical qualitative/quantitative, conceptual, design-oriented or other), specific research methods to be applied (i.e., case studies, surveys, literature reviews, interviews, conceptual developments or no method at all), and the research content that has to be elaborated and described (i.e., conceptual, descriptive, assessment, comparative, or others) [4] . Considering the long history of MMs, multiple development guidelines evolved over time that focus on these decisive processes and can be regarded as supportive instruments for MM development. [9] present the most established standard for the development of MMs: a six-step generic phase model comprising scope, design, populate, test, deploy, and maintain. A different procedure model is postulated by [8] , who strongly focus on a design science research approach. They suggest an eight-phase development approach, focusing on problem definition, comparison of existing MMs, determination of the development strategy, iterative MM development, the conception of transfer and evaluation, implementation of transfer media, evaluation, and rejection of MM. [7] set up general design principles (principles of form and function) for the development of useful MMs. A framework for the characterization of existing MMs is presented by [15] . In order to create sound and widely accepted MMs, he proposes two "maturity model cycles". Based on a Summarizing, many approaches can support researchers in creating MMs. However, these guidelines are limited in their interpretability and validity, as they do not provide concrete terminology specifications or structural concept models. Consequently, many models were built following these recommendations, but they still differ in the terminology used, their descriptive principles, the extent of provided information about factors, and the factors' influence on the degree of maturity [7] .

In spite of their salient popularity, there are also critical voices regarding the scientific implications and the practical applicability of MMs. From a scientific point of view, [7] criticize a lack of empirical research foundation during the development process. This in turn, often leads to a simple copy of existing model structures (i.e., five-staged-MMs) without considering the conceptual suitability for the given context. Furthermore, [16] mention the lack of validation when selecting factors or dimensions for the model as well as the linear course of maturation. Another aspect is the missing operationalization of maturity measurements which impedes replication of an evaluation. Similarly, [17] argue that a simple demonstration of a gap (as done in many MMs) is not enough. Only understanding cause and effect can help to guide action. Otherwise these models might convey the impression of "falsified certainty" to their users [18] . A common promise of MMs is an increase in competitiveness, effectiveness and efficiency in a specific domain or a general improvement of constitution. However, MMs lack a clear description and definition of these concepts in their conceptual grounding, making them theoretical constructs but not applicable models for users (i.e., SMEs).

[19] summarize shortcomings of MMs from a practical perspective. They state that MMs are often inflexible, although a high flexibility is required. Similar to [17] , they argue that MMs focus on identifying gaps and raising awareness towards a specific topic, but enterprises are in need of concrete recommendations for action. Furthermore, these models are disciplinary, impractical, and overwhelming for many enterprises, especially for SMEs where a corresponding executive management level is missing.

However, not only the application and general structure of models is a topic of criticism. Inconsistent terminology of the models' concepts plays a major role in MM research. When developing new MMs, developers tend to invent new titles for common MM approaches in order to stand out from the plethora of already existing MM terms. For instance, designations like "Maturity Model", "Assessment Model", "Roadmap", "Maturity Framework", "Maturity Matrix", "Guide to Maturity", and "Excellence Model" are synonyms for the concept of an MM. Inconsistent terminology used within MMs, indirectly results in an inflation of similar MMs, as these models will not be identified as identical or at least similar approaches. This inconsistent terminology may cause, among other problems, two different semantic-syntax errors between MMs: (a) developers use the same terminology for maturity concepts, but each concept has a different interpretation (e.g., homonyms), and (b) developers address the same concept but label it in a different way (e.g., synonyms). For instance, to describe the field of interest an MM is applied to, MMs use different terms: "domain", "area", or "dimension". But not only the terminology, also the relationships between these concepts remain unclear. There rarely is a standardized definition framework of concepts and relationships. Although there exists a meta model for project management competence models, it is only available in German and focuses on the broader application context, not on the maturity model itself [20] . In conclusion, these inconsistencies represent a research gap as they lead to a situation where a comparison of existing MMs is almost impossible due to a lack of understanding what is actually provided.

A recent study revealed that almost all scientific and many consultancy MMs (in the field of Industrial Internet) are still not being used or are even unknown in business practice, although potential applicants stated a need for and interest in these models in general [21] . As this discrepancy comes as a surprise, we assume a weak point either in the development or application of MMs (Fig. 2) ; otherwise, the reluctant use in praxis is not explicable. A possible explanation for this phenomenon may be located in the different interpretations of MMs' concepts on both sides of stakeholders. Developers may use concepts and relationships that are interpreted differently by model applicants and by other developers in the same field of research. As a result, possible applicants as well as researchers misinterpret an MM's structure and concepts. In other words, syntax and semantics vary across MMs, which is why, on the one hand, companies (as applicants) do not use or apply the models, as they simply do not understand their structure and effects. On the other hand, researchers tend to develop even more and new MMs, as they do not recognize existing models as similar or comparable MM approaches (i.e., from 2016 to 2018, 18 new MMs had been developed in the domain of Industry 4.0 [21] ). What is needed is an overarching, holistic conceptual framework that can unify and standardize the individual objectives and goals. By focusing on the developer's perspective in this paper, the consistency of existing models can be validated, and future models can be developed in a rigorous way, which will then be the foundation for a sound assessment of models later on. A Meta Model, as "a model of models" [22] , is an approach that is able to meet the above-mentioned requirements. Originating from software engineering, it can be understood as a model specifying underlying models that are considered instantiations of the meta model. The meta model itself is instantiated from a meta model [23] . In many cases, the meta meta model specifies a subset of the modeling language UML. A meta model typically represents the language (linguistic) and/or structure (ontological) of the underlying (that is, the subordinate) models [24] . It is used as an abstract description of (1) the unification of concepts that are part of the model, (2) the specifications and definition of concepts, and (3) the specifications and definition of concepts. Thus, multiple valid models can be instances from the same meta model. The schematic meta-modeling approach for MMs is shown in Fig. 3 . On the top layer (M2), the developed meta model is located. It specifies the underlying MM on M1. Every concept on M1 has to adhere to a type concept specified on M2. Please note that the MM on M1 is, in principle, generally valid for all enterprises. It specifies the ranges and used concepts for a later assessment. The developer creates an MM type on M1 (according to the meta model on M2), which is "instantiated" on M0. On the M0 level, the "instantiated" and applied MM characterizes a concrete enterprise. This can be done by the user (the person who applies a constructed MM onto an enterprise).

With a detailed meta model for MMs, different MMs can be regarded as instances of it, leading to common semantics and syntaxes between those models. Only by providing a unified meta model ("specification" in Fig. 3) , that reveals concepts, relations and entities, will the developer be able to develop a rigorous and logical MM through instantiation ("structure" in Fig. 3 ). Our research can therefore contribute fundamentally to the development of future staged MMs in all areas, as its structure on the M1 level and its instantiation on the M0 level are directly interdependent and thus decisive for the success or failure of a maturity measurement in a company.

However, compared to classical meta-modeling, where the M1 model concepts serve as types for M0 level instances, here, the instantiation of M0 from M1 must be interpreted differently. During assessment (the process that creates the M0 instances), only some of the concepts from the M1 level are instantiated with "values" that ultimately (typically by calculation) yield the resulting maturity level for the assessed enterprise.

The development of the Meta Model for Maturity Models (4M) was based on a study of the most common and representative staged MMs. In order to elaborate sufficient meta model elements that are valid for a broad class of staged MMs, an analysis of different staged MMs, their development and their structure was conducted to summarize and analyze existing concepts, their relationships as well as their multiplicities and instantiations. As a result, universally applicable meta concepts were found and related to each other. For that, we followed an opportunistic approach: we identified the necessary meta model concepts (and their relations) by reduction and selection via decision criteria based on the available content of the investigated MMs. The resulting meta model in Fig. 4 and Table 1 show the M2 level structure as a conceptual and formal description of an MM, whose instances (on M1 level) reflect the actual concepts and interrelations within various staged MMs investigated in our study (see also Sect. 5). The staged structure of MMs is built on common assumptions, which influenced the meta model's design:

1. A maturity model is typically developed within a domain, as it represents a path of growth for a specific field of interest. 2. An MM consists of several maturity levels (i.e., L 1 -L 5 ), which have an ascending order and are intended to represent the improvement path of an object's ability. 3. The MM can (but does not have to) be divided into dimensions (i.e., D 1 -D 5 ). These dimensions divide the object of study into fields of interest, mostly depending on the domain in which the model is applied. This subdivision serves the possibility of an incremental measurement of maturity in a certain area. 4. Factors are properties of the maturity level and (if available) are grouped in dimensions. There are two possible ways of maturity determination by factors: a. In the first approach, the respective maturity level is specified by checking the threshold of certain (related) factors that are applied. Due to the fact that a factor could be used on several levels (however, with different values per level), the factor specification, which assigns the concrete expected value of a factor per maturity level, is used between both. Thus, e.g., level L 1 could use factor f with the factor specification f L1 while level L 2 is still able to use f as well (then with its own factor specification f L2 ). This is used to express evolving enterprise maturity via growing requirements per level of the same factor. b. Second, there is the possibility that the maturity level can be determined per dimension instead of "requirements per level". For this purpose, the factors need to be assigned to the respective dimensions and calculated by using a dimensionspecific aggregation formula (e.g., sum, average, median). In contrast to the approach (a), the dimensions do not use any threshold as intermediate value.

Thus, a factor can only be assigned to one dimension, and the dimension maturity is calculated by its set of factors (and their current value). A factor cannot be used twice, and the dimension maturity is always a representation of the current state of the enterprise (within this dimension).

The MM can make use of both, several dimensions of maturity and an overall level maturity since maturity levels and dimensions are modeled orthogonally in the meta model. However, the specification of the maturity level is still mandatory.

within an MM, its value is still abstract and not directly transferable into an object under study. This transfer is done by using indicators. Indicators are measurable properties of one or more factors and are directly retrievable (measurable) for an object under study. As a factor can have many indicators, and each indicator can be relevant for multiple factors, the factors retrieve their values from their indicators by using an indicator aggregation. This is a pure technical construct to allow the shared usage of indicators by different factors. However, the aggregation functions (e.g., sum, average, median) are assigned to the factor to yield its value. 6. The indicator measurement is done on the basis of indicator types. This could be, e.g., Likert, ordinal, or cardinal scales, counting or index values.

The final meta model with its concepts can be used to develop the initial-staged MM. The defined elements (concepts) and their relationships are regarded as best practices in MM development and are practically needed concepts in order to build a functioning staged MM. Although one could decide for several variations, the already discovered concepts in the meta model are the sum of (extracted) expert knowledge from previous MMs. Thus, the non-existence of any part of the meta model in an actual MM is neither considered as an issue in the meta model nor is the MM regarded as incorrect. Rather, the interpretation is that the actual MM lacks a feature that is typically used by other MMs, and it could potentially be improved with such. Further, the relatively small amount of meta concepts is not necessarily a drawback, since the meta model only contains the most powerful concepts found. Despite the few concepts, it provides a strong, defined, and universal framework for a whole class of staged MMs. Table 2 summarizes an analysis of currently existing and applicable staged MMs regarding the mapping between the developed meta model and the concepts used. The selected MMs represent a sample from different domains, years, and development approaches. The BPMM and CMMI are representatives of the most famous MMs developed [14, 25] . Leyh et al. [26] , Gökalp et al. [27] , Schumacher et al. [28] , and Luftman [29] are representatives of scientific MM approaches from different years. Property of the organization, which represents the object/area/process of investigation; used by one or more maturity level and can be related to a dimension within an organization Factor specification Technical and foundational requirement construct for determining the maturity level; acts as the maturity level's individual expression of a factor (needed due to multi-usage of factors) Indicator

Measurable property of one or more factors within an organization; uses an indicator type for measurement Indicator type

Measuring method for determining of the value of the indicator Maturity level

Rank of the organizational maturity that results from factor evaluation by using the factor specification (see 4.a) or aggregation of a dimension's related factors (see 4.b); subdivides the maturity model and represents a relative degree of organizational ability/maturity From the analysis of existing MMs, we conclude that very few models have indepth explanations of all the concepts and relationships used. However, almost every examined model fulfills common concepts like the MM's name, the domain in which it is located, as well as a description of its respective maturity levels (except one). The concepts' names "factor" and "indicator" are not used within the models. Although [27] use the term "indicators", the definition matches the 4M-concept of a factor. The 4M-concept of "indicator" is not further described in their model. In general, the lowlevel concepts (like, e.g., indicator) are rarely specified, possibly due to abstraction from the actual calculation of a maturity level. When it comes to the interpretation of the examined concepts and relationships of factors, indicators, and requirements, no MM can be regarded as an exact instantiation of our meta model, as not every concept is present. There is often a lack of information in the available publications about factors and indicators, which are subject to further calculations, or information on which requirements can be used to derive the maturity level. The mere mentioning of concepts allows conclusions to be drawn about the fact that the authors have basically dealt with the respective concepts, but the relationships between them remain undefined.

Possible, general reasons for this phenomenon could be the length of the publications in which the MMs are presented. Conference papers are often too short to describe a comprehensive presentation of all underlying assumptions. Another reason could be that papers do not describe MMs for application purposes, but rather focus on their contribution. However, we do not consider existing staged MMs that do not match the meta model as incorrect. The meta model, as a conceptual orientation, could help other researchers or developers to understand and interpret the respective intention and improves the developer's cycle of MM development. We claim that this will help these models to overcome their ivory tower. The rather few but strong concepts defined in the meta model are a core skeleton of staged MMs that may lead both the development as well as the understanding of MMs.

Although we have built our meta model on a broad analysis of existing models, our research has limitations. Thus, our meta model approach is only valid for staged models and therefore cannot explain other types of MMs (continuous or focus area MMs). In 

addition, the meta model initially only considers the development cycle and must be further developed for the application cycle. However, as [15] proposed, both cycles should not be analyzed concurrently anyway, as they differ in their requirements. To this point, we have been able to show with the current approach that the consistency and concepts of existing MMs are often not entirely described in the development cycle. It will be the focus of our future work to concentrate on the application cycle and to compare concepts of assessment with provided concepts in existing MMs.

In this paper, a meta model for MMs was introduced, which can be used for a standardized and consistent development of MMs. The related concepts, elements, and their relationships were explained and specified in detail. This was done by the analysis of relevant literature that introduces staged MMs and by extracting their core concepts, including their syntax and semantics. Further, the final 4M was evaluated against several MMs from the research literature showing that the majority of MMs lack in the exact specification of their elements and relationships. This uncertainty and divergence in MM specifications often lead to inconsistent applications and implications derived from their application. The presented 4M is therefore beneficial regarding consistent MM development and comparison. The 4M, together with its defined concepts and relationships, is a compact but powerful tool for MM developers for initial development and evaluation of their work, as well as to be consistent with related work and the staged MM semantics in general. However, the 4M only covers the structural part of an instantiated MM. Consideration of the assessment part as an integrated aspect of the 4M would be valuable for the MM user. We therefore intend to develop the assessment aspect in the next research step. Also, we do not claim completeness regarding the concepts in the 4M since we only introduced the concepts that are used by many different MMs. Additional concepts could be introduced; however, we strived for simplicity and usability instead of a complex and over specified meta model. The 4M is only applicable for staged MMs; thus, a meta model for other MM types (e.g., continuous MMs) has to be constructed separately.

A role-based maturity model for digital relevance

The Oxford English Dictionary. Clarendon

Matters of (meta-) modeling

The maturity of maturity model research: a systematic mapping study

Operational excellence driven by process maturity reviews: a case study of the ABB corporation

Business process maturity models: a systematic literature review

What makes a useful maturity model? A framework of general design principles for maturity models and its demonstration in business process management

Developing maturity models for IT management

Understanding the main phases of developing a maturity assessment model

The use of maturity models/grids as a tool in assessing product development capability

Prozessverbesserung mit Reifegradmodellen

Quality is Free: The Art of Making Quality Certain

Software Engineering Institute: CMMI® for Development

Maturity assessment models: a design science research approach

Maturity models development in IS research: a literature review

Knowing "what" to do is not enough: turning knowledge into action

Inductive design of maturity models: applying the rasch algorithm for design science research

Project management maturity models: the silver bullets of competitive advantage?

Kompetenz-und Reifegradmodelle für das Projektmanagement: Grundlagen

Maturity models in the age of Industry 4.0 -do the available models correspond to the needs of business practice?

Object Management Group: MDA Guide Version

Object Management Group: UML Infrastructure Specification

Metamodellierung als Instrument des Methodenvergleichs: eine Evaluierung am Beispiel objektorientierter Analysemethoden. Shaker

Business Process Maturity Model (BPMM)

The application of the maturity model SIMMI 4.0 in selected enterprises

Development of an assessment model for

A maturity model for assessing Industry 4.0 readiness and maturity of manufacturing enterprises

Assessing business-IT alignment maturity