Earthquake Prediction Using Expert Systems: A Systematic Mapping Study sustainability Review Earthquake Prediction Using Expert Systems: A Systematic Mapping Study Rabia Tehseen, Muhammad Shoaib Farooq * and Adnan Abid Department of Computer Science, University of Management and Technology, Lahore 54770, Pakistan; f2017288003@umt.edu.pk (R.T.); adnan.abid@umt.edu.pk (A.A.) * Correspondence: Shoaib.farooq@umt.edu.pk Received: 24 December 2019; Accepted: 15 March 2020; Published: 19 March 2020 ���������� ������� Abstract: Earthquake is one of the most hazardous natural calamity. Many algorithms have been proposed for earthquake prediction using expert systems (ES). We aim to identify and compare methods, models, frameworks, and tools used to forecast earthquakes using different parameters. We have conducted a systematic mapping study based upon 70 systematically selected high quality peer reviewed research articles involving ES for earthquake prediction, published between January 2010 and January 2020.To the best of our knowledge, there is no recent study that provides a comprehensive survey of this research area. The analysis shows that most of the proposed models have attempted long term predictions about time, intensity, and location of future earthquakes. The article discusses different variants of rule-based, fuzzy, and machine learning based expert systems for earthquake prediction. Moreover, the discussion covers regional and global seismic data sets used, tools employed, to predict earth quake for different geographical regions. Bibliometric and meta-information based analysis has been performed by classifying the articles according to research type, empirical type, approach, target area, and system specific parameters. Lastly, it also presents a taxonomy of earthquake prediction approaches, and research evolution during the last decade. Keywords: Expert systems; Systematic Mapping Study (SMS), earthquake prediction; seismic data; Early-warning systems 1. Introduction Earthquakes have been one of the most hazardous but least predictable natural disaster [1,2]. The occurrence of catastrophic earthquakes results in casualties, massive damage to the infrastructure, the vanquishing of societies in a flash and a sudden downfall in the country’s economy [3,4]. There are many geographical factors that may cause an earthquake, including ground motion, heavy rainfall, rock bed material, regional tectonics and altitude [5]. There is a tremendous pressure on geologists and seismologists for the prediction of the time, place and strength of earthquakes [6]. Many researchers have claimed to predict earthquakes by observing multiple precursors such as recording the behavior of animals, observing an increase in temperature, emission of radon gas, and observing the change in seismicity patterns of the region, etc. References [7,8]. However, it is very hard to generalize and standardize these prediction algorithms as the precursors do not necessarily appear before every earthquake [9]. Earthquake prediction is a highly complicated task and many investigators have used different approaches for making forecasts. Among these different approaches the methods and algorithms that are based on a variety of expert systems (ES)have exhibited promising results in this area. The literature survey reveals that different approaches of expert systems including fuzzy, rule-based, neuro-fuzzyand machine/deep learning methods have been used to forecast future earthquake from historic and instrumental data. In practice, ES have also been efficiently used for risk analysis and assessment in Sustainability 2020, 12, 2420; doi:10.3390/su12062420 www.mdpi.com/journal/sustainability http://www.mdpi.com/journal/sustainability http://www.mdpi.com https://orcid.org/0000-0002-4095-8868 http://www.mdpi.com/2071-1050/12/6/2420?type=check_update&version=1 http://dx.doi.org/10.3390/su12062420 http://www.mdpi.com/journal/sustainability Sustainability 2020, 12, 2420 2 of 32 multiple areas such as, information technology (IT) [10], engineering [11], economics, healthcare [12], and civil engineering [13–15]. The motivation behind applying expert system technique for earthquake prediction lies in its noticeable effectiveness and reliability [16] of such approaches in other disciplines. This research presents a systematic mapping study to facilitate the researchers and practitioners in understanding the fundamentals of earthquake prediction systems; evolution of research in this area, and prominent research directions in this area of research. To these ends, it presents an analysis of eighty four articles while presenting a classification scheme showing multiple aspects covered in the literature addressing earthquake prediction. The study presents a summary of various aspects of the expert systems given in multiple articles for earthquake prediction. It also determines the most frequently used variants of ES for the prediction of earthquake while highlighting the accuracy in prediction results claimed in multiple articles. It is pertinent to mention that to the best of our knowledge, no mapping study has been found about using ES for earthquake prediction. Furthermore, this mapping study not only covers the bibliometric aspects of the selected research articles, but also presents a taxonomy of approaches, variants of ES, their strengths and comparative analysis of their effectiveness in prediction accuracy, and the widely used seismic data sets. Lastly, the open research areas and future directions in this area have also been presented to the researchers working in this area. There are certain limitations of this survey. These limitations relate to the selection of primary studies [17]. In order to ensure that as many relevant publications as possible have been included, we have identified search terms in several iterations. Terms related to ES and earthquake prediction were used in the search string. However, the list might not have been complete, and additional or alternative terms might have altered the final list of papers found [18]. The search was performed by using the Elsevier Scopus Digital Library. According to the statistics of the publications retrieved, we believe that most of the research on earthquake can be found in this electronic library. However, certain papers may have been overlooked due to the subscription limitations. Another threat is related to the handling of duplications, which might have slightly changed our results. Kappa measure has been used for making decisions about possible duplications. The data has been extracted from the primary studies and classified to generate the final results. The decision about which data to collect and how to classify the papers therefore depended on the judgement of the authors conducting the systematic mapping study [19]. The Kappa coefficient has resulted in 0.95 which indicated an agreement among the authors about data inclusion. Data extraction from prose could also result in a misclassification, but this problem was addressed by developing a classification scheme on the basis of widely accepted guidelines [17] and terminology proposed for use in [20]. It would, therefore, only have a minor influence on the classification scheme developed in this mapping study. Validity limitation refers to the missing studies, incorrect data extraction [21] and determining the incorrect relationship among multiple facets. To overcome this threat, we have clearly described the activities involved in publication selection and data extraction in multiple sections. The traceability between the data extracted and the conclusion drawn has been presented through bubble plots and frequency plots. The quality assessment problem is concerned with the quality of study selection [5,22,23]. The systematic mapping results were considered in regard to the seismic domain, and the validity of the conclusions drawn concerns the earthquake prediction context only. To focus on state of the art methods, we have applied time restriction in searching for published studies and have included the papers published during January 2010 till January 2020. The search string and the classification scheme presented in this paper may serve as a starting point for researchers working on the problem of earthquake prediction, and they can search for and categorize additional papers accordingly. The rest of the article has been structured in the following manner. Section 2 presents the background of this study, while the research methodology of this study has been presented in Section 3. The analysis of all the selected and reviewed articles has been presented in Section 4. The analysis about the findings of the literature review has been discussed in Section 5. Lastly, the article has been concluded in Section 6. Sustainability 2020, 12, 2420 3 of 32 2. Background In this section, we have presented the general outline of the research work about using expert system for earthquake prediction. Multiple approaches, including rule-based, fuzzy, neuro-fuzzy and machine / deep learning methods have been used for earthquake prediction. 2.1. Fuzzy Expert System (FES) The concepts of fuzzy set and fuzzy logic were introduced in 1965 by [20]. Fuzzy expert system accepts input as crisp variables and converts it into fuzzy variables. Fuzzy inference engine applies the rules suggested by an expert to formulate the knowledge base. Fuzzy variables combined with linguistic variables to generate membership functions. Techniques based on Fuzzy logic have the benefits over multiple procedures due to their ability to combine with linguistic variables. These fuzzy variables would be converted back into the crisp variable to generate output through the process called defuzzification. Fuzzy logic is more suitable in the situations where a greater number of uncertainties have been involved, such as, earthquake prediction and in the scenarios where an approximate but quick solution is required. Fuzzy logic is not a logic that is fuzzy itself, but a logic that can be used to demonstrate fuzziness [21]. The vagueness of fuzzy logic has been highlighted in [10] by examining the events that cannot be recorded statistically such as crack in the undergroundfault, etc. A new attenuation relationship has been proposed in [8] using three fuzzy input sets including epicentral distance, earthquake magnitude and intensity using earthquake data set of Taiwan and United states of America (USA). A normalized fuzzy ground motion model has been demonstrated using a rational design tool through a combination of natural language with seismic data statistics to quantify response frequency. The earthquake pattern in the Zagros range has been examined in [9] using fuzzy rule-based ES model for some earthquakes. The proposed model has been evaluated using the Molchan statistical procedure by comparing complicated reasoning procedure of the forecasting model with knowledge simulation provided by human experts using the datasets of Iran. A rock burst forecasting model has been presented in [13] by studying the seismic features of coal mining in China. In this study, Gaussian shaped membership function has been combined with the exponential distribution function using reliability theory. The comprehensive forecasting result was obtained by integrating the maximum membership degree principle (MMDP) and the variable fuzzy pattern recognition (VFPR) method. The performance of the proposed model has been evaluated using seismic data collected over the period of four months. The proposed model has been able to forecast the rock burst incident in the coal mine of China. Multiple algorithms have been combined for development of the hybrid prediction model [24,25]. Ionospheric disturbance has been examined in [26] and a fuzzy logic-based gradient descent method has been proposed to forecast the ionospheric change parameters. The gradient descent estimated values were used to tune the membership function. The satisfactory performance has been observed during evaluated of the proposed model using data collected from two geomagnetic storms on the low latitude. Reference [1] has claimedearthquake prediction on the bases of the classification of seismicsignals. 2.2. Rule Based Expert System (RBES) In RBES domain knowledge is represented by a set of rules and the current situation is presented with the set of facts stored in the database. An inference engine is responsible to match the rule with the fact. The fired rule may change the set of facts and add new facts. Many researchers have used rule based expert system for earthquake prediction. A belief rule based expert system has been presented in [27] to predict the earthquake under uncertainty. Specific animal behavior in response to environmental and chemical changes has been examined for earthquake prediction. Reference [20] developed rules from historical earthquake data using predicate logic. These rules have been mathematically validated on real time data. Prediction is performed through RBES that takes current earthquake attributes for prediction of future earthquake. Sustainability 2020, 12, 2420 4 of 32 2.3. Neuro Fuzzy Expert System (NFES) Fuzzy logic is combined with neural networks to develop expert systems. Fuzzy logic provided a high level reasoning procedure by including domain information from the domain expert and neural network has been used to develop low level computational structures. The Neuro fuzzy expert system has been used in many articles to analyze multiple aspects of data for earthquake predictions. Reference [28] combined grid partition, subtractive clustering and fuzzy C-means (FCM) for the development of models using NFES structure. Reference [5] applied NFES to compute land sliding susceptibility using statistical index (WI). Reference [22] collected geographical information to pass through six different membership functions for measuring land sliding susceptibility using NFES. Many researchers have analysed combination of artificial neural network and fuzzy inference system [29–32]. Earthquake attribute such as magnitude, depth, longitude and latitude has been studied in [33] to provide input to NFES for computation of the future earthquake. 2.4. Machine Learning (Ml) Machine Learning has been widely used for making earthquake predictions due to their ability to improve over time. With the huge amount of earthquake instrumental data, machine learning approaches are capable enough to improve efficiency and accuracy in earthquake prediction. Multiple machine learning methods including, Artificial Neural Network (ANN), Support Vector machine (SVM), K-nearest neighbour (KNN), Native Bayes (NB) and random forest algorithms have been exercised for earthquake prediction. Reference [34] applied Artificial Neural Networks, Support Vector Machines and Random Forests to perform temporal investigations on earthquake catalogue of Cyprus region and calculated sixty seismic indictors for making short term earthquake prediction. Reference [35] applied different machine learning algorithms namely support vector machine (SVM), K-nearest neighbor (KNN), random forest (RF), and Naïve Bayes (NB) algorithms in R programming language for earthquake prediction using seismic dataset of India. Reference [36] studied the thermal anomalies that happened before the earthquake occurred in Imphal, India, in 2016 and investigated multiple seismic facts through satellite data using machine learning algorithms for an earthquake. Reference [37] collected records of aftershocks of the Kermanshah (Iran) Earthquake and applied different machine learning (ML) algorithms, including Naive Bayes, k-nearest neighbors, a support vector machine, and random forests to predict future earthquakes by observing aftershock patterns. Reference [38] exercised neural networks for earthquake signal detection. Reference [39] listed the detailed description of the monitoring techniques used for earthquake prediction. References [40,41] presented a comprehensive review of machine learning methods used for earthquake prediction. Reference [42] made seismic hazards forecasts by using two different machine learning based methods for both spatial and space-time prediction of strong earthquakes. Reference [43] determined the significance of shallow land slide triggers in making earthquake forecasts using machine learning methods. Reference [44] improved the conventional waveform correlation method and presented a new method for detection of seismic signals for monitoring the false alarms using machine learning. Reference [45] identified, classified and reviewed the prominent machine/deep learning models used in energy systems. Reference [46] discussed multiple artificial intelligent models utilized for hydrologic model prediction in past decade. Reference [47] highlighted the opportunities and challenges presented by big data for informed decision-making. Reference [48] developed a food forecast model using multiple optimization methods. 3. Research Methodology The objective of systematic mapping study is to present an overview of the research area and quantify the results presented by the selected studies. We intend to determine the research trends by mapping the frequency of publications over time. For this purpose, we have adopted the methodologies Sustainability 2020, 12, 2420 5 of 32 of [45,46] and performed a number of activities as shown in the Figure 1. Our main goal is to provide deep inside of expert system based solutions proposed for earthquake prediction in literature. Sustainability 2020, 11, x FOR PEER REVIEW 5 of 33 Figure 1. Research Methodology of Systematic Mapping Study (SMS). 3. Research Methodology The objective of systematic mapping study is to present an overview of the research area and quantify the results presented by the selected studies. We intend to determine the research trends by mapping the frequency of publications over time. For this purpose, we have adopted the methodologies of [45,46] and performed a number of activities as shown in the Figure 1. Our main goal is to provide deep inside of expert system based solutions proposed for earthquake prediction in literature. 3.1. Defining Research Questions We have addressed three research questions in this mapping study. These research questions have served as a guide for the classification of research articles. A set of research questions has been described in Table 1. Table 1. Set of research questions (RQs) and their motivation. Research Question (RQ) RQ Statement Motivation RQ 1 What are the bibliometric key facts of expert systems (ES) based earthquake prediction publications? RQ 1.1 How many studies have been contributed from January 2010 to January 2020? The intentions of this research question is to find out the number of publications that have been contributed in the selected time period and the main venues where the studies have been published. RQ 1.2 What are the venues where these studies have been published? RQ 2 Which research type facets do the identified publications address? RQ 2.1 What is the type of research conducted in the publication? The main intention is to categorize the selected publications through the schema established by [17,49]. Therefore, we use the research type facets given by Zhang et al. [49]. Based on these type facets, we wanted to find out multiple research contexts, including the type of the research, empirical type of the research, approaches used in the research and areas targeted by the researchers for data extraction. RQ 2.2 What is the empirical type of the research conducted in the publication? Figure 1. Research Methodology of Systematic Mapping Study (SMS). 3.1. Defining Research Questions We have addressed three research questions in this mapping study. These research questions have served as a guide for the classification of research articles. A set of research questions has been described in Table 1. Table 1. Set of research questions (RQs) and their motivation. Research Question (RQ) RQ Statement Motivation RQ 1 What are the bibliometric key facts of expert systems (ES) based earthquake prediction publications? RQ 1.1 How many studies have been contributed from January 2010 to January 2020? The intentions of this research question is to find out the number of publications that have been contributed in the selected time period and the main venues where the studies have been published.RQ 1.2 What are the venues where these studies have been published? RQ 2 Which research type facets do the identified publications address? RQ 2.1 What is the type of research conducted in the publication? The main intention is to categorize the selected publications through the schema established by [17,49]. Therefore, we use the research type facets given by Zhang et al. [49]. Based on these type facets, we wanted to find out multiple research contexts, including the type of the research, empirical type of the research, approaches used in the research and areas targeted by the researchers for data extraction. RQ 2.2 What is the empirical type of the research conducted in the publication? RQ 2.3 What approach has been used by the researcher? RQ 2.4 Which area has been targeted by the research for data collection? RQ 3 What is the type and other key aspects of proposed Expert System (ES) in the classified publications? RQ 3.1 What type of expert system has been proposed in the selected studies? The main aim is to determine the types of proposed ES used for earthquake prediction in the articles published during January 2010 till January 2020. This question is helpful in highlighting the other parameters of the proposed ES like input domain, number of input attributes passed, type of the input attributes, prediction logic, the tools and techniques used in the articles have been categorized. RQ 3.2 Which input domain does the proposed ES address? RQ 3.3 How many input attributes are passed to the proposed ES? RQ 3.4 What is the type of the input attributes passed to the proposed ES? RQ 3.5 Which type of prediction logic has been used by the proposed ES? RQ 3.6 Which tool or technique has been used to develop the proposed ES? 3.2. Search and Selection Strategy After defining research questions, next activity was to select the sources from where the articles would be retrieved. For this purpose, we have adopted the searching strategy given in [47,48] and Sustainability 2020, 12, 2420 6 of 32 have developed a comprehensive search string based on the key terms given in Table 2. Articles have been collected from Elsevier Scopus digital library (www.scopus.com).The terms stated in Table 2 have been used to develop the search string for searching articles from the given sources. Table 2. Distinct key terms used in developing the search strings. AND Terms OR Terms Earthquake Rule based, Fuzzy, Frame based Machine Learning, Deep learning, Expert system Seismic, Tremor Indicator Precursor, Feature Prediction Predict* (* means wildcard) (“Rule based” OR “*Fuzzy” OR “Frame based” OR “neural” OR “machine learning” OR “deep learning” OR “Expert system”) AND (“Earthquake” OR “Seismic” OR “Tremor”) AND (“Predict*”) AND (“Indicat*” OR “Precursor” OR “features”) 3.2.1. Identification of Search String The primary keywords were selected as key identifiers of work in the field of earthquake prediction using expert system. A set of distinct keywords is listed in Table 2. This mapping study has been conducted to examine the literature about earthquake prediction using expert systems. We have used multiple key terms like fuzzy, frame based and rule based methods for extracting such articles that describe the use of expert systems for earthquake prediction, but does not necessarily have an expert system explicitly written in their titles. In the same way, earthquake has been presented as a seismic event or a tremor in some studies, so we have also included these keywords. The change in the behaviour of precursors has been studied in many articles for earthquake prediction. Therefore, we have also included a few keywords describing the indicators of future earthquakes. Search string has been developed after defining key terms. The following search string has been used to search articles from Elsevier Scopus digital library. We have included the articles published from January 2010 till January 2020. 3.2.2. Screening and Selection Criteria Screening of the articles has been performed after retrieval. The main purpose is to select most relevant articles. Every paper was retrieved and evaluated by considering title, abstract, keywords, introduction and conclusion. The inclusion and exclusion criteria given in Table 3 have been used for article selection. We have included only those articles that satisfy the inclusion criteria given in Table 3. We have excluded those papers which have been retrieved from multiple sources or representing different stages of the same project. For inclusion of the articles that have identical abstracts we have calculated Kappa coefficient. The papers that are not written in the English language have also not been included. The thesis has also been excluded because they normally cover multiple aspects of the problem. Papers with unclear methodology and not satisfying our quality criteria have also been excluded. Table 3. Inclusion and Exclusion Criteria. Inclusion Criteria (IC) Criteria Description IC1 Articles in which an expert system has been developed for earthquake prediction IC2 Articles in which earthquake precursors have been analyzed IC3 Articles presenting unique and new ideas IC4 Literature published as book chapter and technical reports for earthquake prediction IC5 Articles with identical abstracts (on the basis of Kappa coefficient) Exclusion Criteria (EC) EC1 Duplicates and identical titles EC2 Papers not in English language EC3 Thesis (cover several different aspects) EC4 Papers with unclear methodology EC5 Papers not satisfying quality criteria www.scopus.com Sustainability 2020, 12, 2420 7 of 32 We have taken a good care of article quality selection in the article selection process. To ensure high quality in article selection, criteria given in [19] have been adopted and elaborated in Table 4. Table 4. Quality Criteria. Quality Ranking Sr. Criteria Type Weight a. Study Presents contribution Yes 1 No 0 Partially 0.5 b. Study presents solution Yes 1 No 0 Partially 0.5 c. The study presents empirically validated results Yes 1 No 0 Partially 0.5 Search Process resulted in the retrieval of 2137 articles that passed through multiple phases. In the first phase, our search string has retrieved 2137 research articles. After passing through multiple screening phases of selection criteria seventy articles have been considered original, non-duplicate, with clear methodology and satisfying our quality criteria. The coefficient has been calculated to determine the relevance among the articles. In every screening phase, two authors of this mapping study have been asked to make judgement about the relevance of the article by selecting any choice from “accept”, “reject” or “differ” options. In case of difference in the opinion of both judges, comprehensive discussion has been carried out to a decision point has been reached in the form of acceptance or rejection. 3.3. Data Extraction and Synthesis It is based on providing a set of answers to the research questions. Table 5 represents the data that we intend to extract by asking research questions prescribed in Table 1. Table 5. Data extracted through each research question. Research Questions (RQs) Data Extracted RQ 1 RQ 1.1 Number of publications contributed in the given time period has been determined. RQ 1.2 A main venue where the study has been published has been noted. RQ 2 RQ 2.1 Research type (solution, evaluation, experience) has been determined. RQ 2.2 Empirical type (Experiment, survey, case study) has been determined. RQ 2.3 The approach used (model, method, guideline, framework, tool) has been noted. RQ 2.4 Seismic zone (global, regional) focused by the study has been determined. RQ 3 RQ 3.1 Type of the proposed expert system (Fuzzy expert system, rule based expert system, Neuro fuzzy expert system) has been noted. RQ 3.2 Identification of the input domain i.e., quake or precursive RQ 3.3 Number of input attributes, i.e., single or multiple that have been passed to the proposed ES for earthquake prediction. RQ 3.4 Type of the input attributes (numeric or discrete) has been determined. RQ 3.5 Prediction logic (inductive or deductive) used by the proposed expert system has been noted. RQ 3.6 Tools and techniques used to develop the proposed expert system given in the studies have been categorized. RQ1 extracts bibliometric facts, including a number of publications contributed in the period of ten years from January 2010 till January 2020 and the main venue where the publication has been submitted e.g., journal, conference, book chapter, etc. RQ1 highlights the trend of the researchers regarding article submission in the last ten years. RQ2 deals with various research type facets of the articles, including research type, empirical type, approach and the targeted area. Type of the research determined that the article has presented a novel solution or illustrated an extension of already existing technique. Evaluation research examined the Sustainability 2020, 12, 2420 8 of 32 techniques used in the articles have passed through an evaluation process before its implementation. Experience papers presented the personal experiences of the author explaining how something has been done in practice. Empirical type illustrated that an experiment, survey, or case study has been performed in the selected article. We have also collected the information regarding focused seismic zone through RQ2. Detailed type facets of the proposed ES for prediction of earthquake have been listed in the classification Table 6. We have provided a set of distinct keywords in Figure 2 to explain the contents of Table 6. The relationship between number of publications and the type facets determined through RQ2 has been shown in Figure 3. Sustainability 2020, 11, x FOR PEER REVIEW 9 of 33 RQ 3 RQ 3.1 Type of the proposed expert system (Fuzzy expert system, rule based expert system, Neuro fuzzy expert system) has been noted. RQ 3.2 Identification of the input domain i.e quake or precursive RQ 3.3 Number of input attributes, i.e. single or multiple that have been passed to the proposed ES for earthquake prediction. RQ 3.4 Type of the input attributes (numeric or discrete) has been determined. RQ 3.5 Prediction logic (inductive or deductive) used by the proposed expert system has been noted. RQ 3.6 Tools and techniques used to develop the proposed expert system given in the studies have been categorized. Classification of Articles We have classified the articles to determine multiple parameters summarized in Table 6 presenting the bibliographic values, research type facets, type of the expert system used and other key aspects of the ES proposed in these articles for earthquake prediction. Quality criteria defined in Table 4 have also been applied to categorize the articles accordingly. Classification Table 6 had been developed on the basis of research questions given in Table 1 and the quality criteria prescribed in Table 4. 4. Analysis The results obtained from the research questions (as given in Table 1) have been presented in Table 6. All articles have been selected to illustrate their relevance and contribution in the earthquake prediction process by providing an answer to every research question. 4.1. Basic Analysis After deep investigationeighty four articles have been selected and main approaches for earthquake prediction including machine learning, neuro-fuzzy, fuzzy and rule-based approaches have been reviewed. These articles were thoroughly analyzed to answer the research questions (RQs) given in Table 1. We have summarized the results obtained from RQs in Table 6. A set of distinct keywords to explain the contents of Table 6 has been given in Figure 2. Figure 2. Set of distinct keywords used to populate classification Table 6. Figure 2. Set of distinct keywords used to populate classification Table 6. Sustainability 2020, 12, x FOR PEER REVIEW 5 of 33 Last type facet of this mapping study is the target area that has been classified as regional and global. 75% of the included researches have been focused to a certain specific regional area where as 24% of the researchers have used global data to forecast earthquake. Figure 3 shows the number of publications (n) according to the type facets given in RQ2. Figure 3. Relationship between number of publications (n) and the type facets. 4.4. ES type and System Specific Key Aspects of Proposed ES Three types of the proposed ES have been identified by RQ3 and presented in Figure 4. Fuzzy expert system (FES) has been proposed for making earthquake predictions by 29% of the studies. Neuro-Fuzzy expert system for making an earthquake prediction has been proposed by 11% of the studies. The rule based expert system has been proposed by 7% of the included studies. 36% of the articles have used machine learning for earthquake prediction and 12% of the studies have used other computational approaches like index development or classifier calculation for earthquake prediction. In this mapping study RQ3 also deals with exploring other key aspects of the proposed ESs like input domain, input attributes and their types, data type, prediction logic and the prediction technique covering the use of computational supporting software or algorithm for performing earthquake prediction task. Input domain can be the earthquake parameters like magnitude, b-value, etc. or precursors calculated from real time earthquake data. In this mapping study, 51% researches have directly consumed earthquake parameters (listed as quake variables in Table 6) whereas 38% have studied precursors that serve as indicators for an upcoming devastating earthquake event. Further, we have explored the type of above stated input attributes to analyze that whether predictions have been made by consuming single input attribute or combination of multiple attributes has been used for earthquake predictions. In this mapping study, 27%, researchers have made predictions on the basis of a single attribute where as 68% studies have used a combination of multiple input attributes for earthquake predictions. Through data type we have worked to find that input variables passed to the proposed ES are quantitative type (numeric form like magnitude, slope, etc.) or qualitative type (discrete form like water elevation, emission of radon gas etc.). It has been observed that 57% of researchers have used numeric inputs for earthquake forecast whereas 43% of the studies have performed the earthquake prediction task using a discrete type of input. This mapping study has also captured the data regarding prediction logic and prediction methodology. Through prediction logic, we have determined that the article have used the inductive or deductive Figure 3. Relationship between number of publications (n) and the type facets. Sustainability 2020, 12, 2420 9 of 32 Table 6. Classification Table. Ref. Bibliometric Facts Type Facets System Specific Information Quality Ranking Publication Channel Publication Year Research Type Empirical Type Approach Target Area Proposed ES type Input Domain Input Attribute Input Attribute type Data Type Prediction Logic Tools and Techniques (a) (b) (c) Score [1] Journal 2014 Eva CS Mod RL FES PR ML DV Dis IN SW 1 0.5 0.5 2.0 [2] Journal 2017 Eva Ext Met RL FES QE SL DV Num IN AM 1 0.5 1 2.5 [3] Journal 2017 Eva Ext Mod RL Other QE ML DV Num DD SW 1 0.5 1 2.5 [5] Journal 2016 Eva CS Met RL NFES PR SL DV Dis IN AM 1 0.5 0.5 2.0 [6] Journal 2018 Eva Ext Met RL FES QE ML DV Num IN SW 1 0.5 1 2.5 [7] Journal 2015 Eva Ext Mod RL FES QE ML PE Num IN AM 1 0.5 1 2.5 [8] Confe 2012 Sol Ext Met RL FES PR ML DV Dis IN Oth 0.5 1 1 2.5 [9] Journal 2014 Eva Ext Mod RL FES PR ML DV Dis IN SW 1 0.5 1 2.5 [10] Journal 2017 Eva CS Met RL Other QE ML DV Num IN SW 1 0.5 1 2.5 [11] Journal 2017 Eva CS Mod RL FES PR SL DV Dis IN SW 1 0.5 0.5 2.0 [12] Journal 2017 Eva CS Mod RL FES QE ML DV Num IN SW 1 0.5 0.5 2.0 [13] Journal 2018 Sol Ext Mod RL FES QE SL DV Num IN SW 0.5 1 1 2.5 [14] Book 2017 Eva CS Met RL FES PR SL DV Dis IN SW 1 0.5 0.5 2.0 [15] Journal 2017 Exp Sur Met RL FES QE ML PE Dis IN Oth 0.5 0.5 0 1.0 [16] Journal 2013 Exp Sur Mod GL RBES PR SL PE Dis DD SW 0.5 0.5 0 1.0 [17] Journal 2013 Eva Ext Mod RL RBES PR ML PE Dis IN Oth 1 0.5 1 2.5 [18] Journal 2016 Eva Sur Met RL FES QE ML PE Num DD AM 1 0.5 0 1.5 [19] Confe 2015 Eva Ext Mod GL RBES QE ML DV Dis IN SW 1 0.5 1 2.5 [20] Journal 2014 Eva Ext Mod GL RBES QE ML PE Dis IN SW 1 0.5 1 2.5 [21] Journal 2018 Eva Ext Met GL FES QE ML DV Dis DD AM 1 0.5 1 2.5 [22] Journal 2018 Eva Ext Met RL NFES QE ML DV Dis IN SW 1 0.5 1 2.5 [23] Journal 2015 Exp CS Met RL FES QE SL PE Dis IN Oth 0.5 0.5 0.5 1.5 [24] Confe 2016 Exp CS Met RL FES QE ML DV Dis IN Oth 0.5 0.5 0.5 1.5 [25] Confe 2010 Eva Ext Met RL Other PR SL PE Num DD AM 1 0.5 1 2.5 [26] Journal 2017 Sol CS Mod RL FES PR SL PE Dis IN SW 0.5 1 0.5 2.0 [27] Journal 2018 Eva Ext Mod GL RBES PR SL PE Dis IN AM 1 0.5 1 2.5 [28] Journal 2012 Sol Sur Mod GL FES PR ML PE Num IN AM 0.5 1 0 1.5 [30] Journal 2015 Exp Sur Gle GL NFES PR ML PE Num IN SW 0.5 0.5 0 1.0 [31] Journal 2015 Eva Ext Mod RL FES QE ML PE Num IN AM 1 0.5 1 2.5 [32] Journal 2018 Eva CS Mod RL NFES PR SL DV Dis IN SW 1 0.5 0.5 2.0 [33] Journal 2014 Exp Sur Gle GL NFES QE ML DV Num DD SW 0.5 0.5 0 1.0 Sustainability 2020, 12, 2420 10 of 32 Table 6. Cont. Ref. Bibliometric Facts Type Facets System Specific Information Quality Ranking Publication Channel Publication Year Research Type Empirical Type Approach Target Area Proposed ES type Input Domain Input Attribute Input Attribute type Data Type Prediction Logic Tools and Techniques (a) (b) (c) Score [34] Journal 2020 Eva Ext FW RL Ml QE ML PE Num DD AM 1 0.5 1 2.5 [35] Confe 2020 Eva Ext Mod RL Ml QE ML DV Num DD AM 1 0.5 1 2.5 [36] Journal 2020 Exp CS Met RL Ml PR SL DV Dis DD SW 0.5 0.5 0 1 [37] Journal 2019 Exp Ext Met RL Ml QE ML PE Num IN AM 0.5 0.5 1 2 [38] Confe 2019 Exp Ext Met GL Ml PR SL PE Dis DD SW 0.5 0.5 1 2 [39] Confe 2019 Sol Ext Mod GL Ml QE ML PE Num DD AM 0.5 1 1 2.5 [40] Confe 2019 Exp Sur Gle GL Ml PR ML DV Dis IN AM 0.5 0.5 0 1 [41] Confe 2019 Exp Sur Gle GL Ml PR ML PE Dis DD AM 0.5 0.5 0 1 [42] Journal 2019 Eva Ext Met RL Ml QE ML DV Num DD AM 1 0.5 1 2.5 [43] Confe 2019 Eva Ext Mod GL Ml PR ML DV Dis DD AM 1 0.5 1 2.5 [44] Journal 2018 Sol Ext Met GL Ml QE ML DV Num IN AM 0.5 1 1 2.5 [49] Journal 2019 Eva Ext Met RL NFES PR ML DV Num DD AM 1 0.5 1 2.5 [50] Journal 2013 Exp Ext Met RL FES PR ML PE Num IN Oth 0.5 0.5 1 2.0 [51] Confe 2010 Sol CS Mod RL FES PR SL DV Dis DD AM 0.5 1 0.5 2.0 [52] Journal 2018 Eva Ext Mod RL FES PR ML PE Dis IN Oth 1 0.5 1 2.5 [53] Journal 2011 Sol Sur Mod RL Other QE SL DV Num DD AM 0.5 1 0 1.5 [54] Journal 2019 Exp CS Gle RL Other PR SL PE Num DD AM 0.5 0.5 0.5 1.5 [55] Confe 2010 Sol Ext Met RL FES PR SL DV Dis IN Oth 0.5 1 1 2.5 [56] Confe 2018 Exp Ext FW GL NN QE ML DV Num DD AM 0.5 0.5 1 2 [57] Confe 2018 Exp Sur Met GL Ml PR ML PE Num IN AM 0.5 0.5 0 1 [58] Journal 2018 Exp Ext Met RL NN QE ML PE Num DD AM 0.5 0.5 1 2 [59] Journal 2018 Sol CS Met RL Ml QE ML PE Num DD SW 0.5 1 0.5 2 [60] Journal 2018 Eva Ext Met GL Ml QE ML DV Num IN SW 1 0.5 1 2.5 [61] Confe 2018 Sol Ext Met RL Ml QE ML PE Num DD SW 0.5 1 1 2.5 [62] Journal 2017 Eva CS Mod RL Ml QE ML PE Num DD SW 1 0.5 0.5 2 [63] Confe 2017 Exp Ext Mod GL Ml PR SL PE Dis IN AW 0.5 0.5 1 2 [64] Journal 2017 Sol CS Mod RL NN PR ML DV Num IN SW 0.5 1 0.5 2 [65] Journal 2017 Eva CS Met RL Ml QE ML DV Num DD AW 1 0.5 0.5 2 [66] Journal 2017 Sol CS Met RL Ml PR SL PE Dis IN SW 0.5 1 0.5 2 [67] Journal 2017 Eva CS Met RL NN PR ML DV Num DD AW 1 0.5 0.5 2 Sustainability 2020, 12, 2420 11 of 32 Table 6. Cont. Ref. Bibliometric Facts Type Facets System Specific Information Quality Ranking Publication Channel Publication Year Research Type Empirical Type Approach Target Area Proposed ES type Input Domain Input Attribute Input Attribute type Data Type Prediction Logic Tools and Techniques (a) (b) (c) Score [68] Confe 2017 Eva CS Mod RL Ml QE ML PE Num IN SW 1 0.5 0.5 2 [69] Confe 2015 Eva CS Met RL Ml QE ML DV Num IN SW 1 0.5 0.5 2 [70] Journal 2015 Eva Ext Mod GL Ml QE ML PE Num IN SW 1 0.5 1 2.5 [71] Journal 2013 Eva CS Mod RL Ml PR ML DV Dis DD SW 1 0.5 0.5 2 [72] Journal 2013 Eva CS Met RL Ml QE ML DV Num DD SW 1 0.5 0.5 2 [73] Journal 2012 Exp CS Met RL Ml PR SL DV Dis DD SW 0.5 0.5 0.5 1.5 [74] Journal 2016 Sol Ext Met GL Ml QE ML DV Num IN SW 0.5 1 1 2.5 Notes: Publication Channel: Confe = Conference; Research Type: Eva: Evaluation, Exp: experience, Sol: Solution; Emperical Type: CS: Case study, Ext: Experiment, Sur: Survey; Approach: Mod: Model, Met: Method, Gle: Guideline, FW: Framework; Target Area: RL: Regional, GL: Global; Proposed ES type: Ml: Machine learning, FES: Fuzzy expert system, NFES: Nuero fuzzy expert system, RBES: Rulebased expert system; Input Attribute: ML: Multiple, SL: Single; Input Attribute Type: PE: Primitive, DV:Derived; Data Type: Num: Numeric, Dis: Discrete; Input domain: QE: Quake, PR: Precursive; Prediction Logic: IN: Inductive, DD:Deductive; Tools and Techniques: SW:Software, Alg:Algorithm. Sustainability 2020, 12, 2420 12 of 32 We have collected information about the type of ES such as FES, RBES and NFES proposed for earthquake prediction in the selected articles through RQ3. The key aspects of the proposed ES including input domain, number of input attributes, type of input attributes, data type, prediction logic, tools and techniques used for prediction of earthquake have been discussed in Figure 4. Input domain describes the input variables taken by the system to predict an earthquake. It can be either ‘Quake variables’ like latitude, longitude, magnitude, primary wave (P-Wave) attributes, etc. or ‘Precursors variables’ like ionosphere readings, earth’s thermal variations via satellites etc. Input attributes can be single or multiple. Some systems predict using single quake or precursor variable. Many articles used multiple input variables for earthquake prediction. Input attributes type can be primitive or derived. Some techniques directly consume an input parameter and others manipulate (derive) before use. The proposed ES uses variables in primitive form or transforms it into some other form before consuming it. The data type can be numerical or discrete. Prediction logic can be inductive or deductive. Inductive logic has no absolute proof from premises to conclusions, e.g., fuzzy sets. Deductive prediction logic determined an absolute proof from premises to conclusions Sustainability 2020, 12, x FOR PEER REVIEW 6 of 33 logic for making earthquake predictions. Inductive logic has been used by 62% of the studies, whereas deductive logic has been presented in 32% of the included studies for giving absolute results. Next we have worked to determine the predictive methodology to describethe use of any software or an algorithm for evaluation of the proposed system. 42% of the studies have used multiple algorithms for evaluation of their predictions, 48% have used the MATLAB software for evaluation of the proposed ES and 10% have adopted other evaluation mechanisms. Figure 4. Relationship between number of studies and system specific parameters of ES. 4.5. Quality Assessment The quality assessment score obtained by each paper has been presented in Table 6. These articles have been collected in three distinct categories and presented in Table 11 according to the scores obtained by every article given Table 6. By observing the quality scores, 74% of the papers included in the mapping study had obtained above average score, 11% of the studies had obtained average score and 15% of the studies remained below average. This quality assessment may facilitate researchers in the selection of articles for their work. Table 11. Scores. Reference Score Average =1.5 Total % [1–14,17,19–22,25–27,31,32,34,35,37–39,42–44,49–52,55,56,58– 72,74] Above average 55 79 [18,23,24,28,53,54,73] Average 7 10 [15,16,30,33,36,40,41,57] Below average 8 11 5. Discussion In this mapping study, main techniques used for earthquake prediction including rule-based, fuzzy, neuro-fuzzy and machine learning have been explored. Expert systems based approaches have been used for seismic risk assessment for landslide susceptibility through seismic hazard analysis [50–52] and soil classification [53]. ES have also been applied in earthquake engineering for seismic hazard analysis and assessment of bridges and buildings under multiple hazards [24,54,55]. The expert systems have been used to analyze multiple aspects of earthquake prediction, but due to the non-existence of the time-dependent global earthquake forecasting model, the regional earthquake likelihood models have been popular [87]. On the other hand, some initial research has been conducted to design earthquake early-warning systems that work globally [32,49]. 28 46 27 39 48 51 34 13 38 58 45 36 33 40 13 10 25 5 ty pe FE S/ N FE S/ RB ES / M L/ O th er In pu t D om ai n (Q ua ck / Pr ec ur si ve ) In pu t a tt rib ut e (s in gl e / m ul tip le ) in pu t a tt rib ut e ty pe (P rim iti ve /D er iv e d) D at a ty pe (n um er ic / de sc re te ) Pr ed ic tio nl og ic (in du ct iv e/ de du ct iv e) To ol s & te ch ni qu e (A lg o/ SW /o th er ) RQ3 Figure 4. Relationship between number of studies and system specific parameters of ES. Classification of Articles We have classified the articles to determine multiple parameters summarized in Table 6 presenting the bibliographic values, research type facets, type of the expert system used and other key aspects of the ES proposed in these articles for earthquake prediction. Quality criteria defined in Table 4 have also been applied to categorize the articles accordingly. Classification Table 6 had been developed on the basis of research questions given in Table 1 and the quality criteria prescribed in Table 4. 4. Analysis The results obtained from the research questions (as given in Table 1) have been presented in Table 6. All articles have been selected to illustrate their relevance and contribution in the earthquake prediction process by providing an answer to every research question. 4.1. Basic Analysis After deep investigationeighty four articles have been selected and main approaches for earthquake prediction including machine learning, neuro-fuzzy, fuzzy and rule-based approaches have been reviewed. Sustainability 2020, 12, 2420 13 of 32 These articles were thoroughly analyzed to answer the research questions (RQs) given in Table 1. We have summarized the results obtained from RQs in Table 6. A set of distinct keywords to explain the contents of Table 6 has been given in Figure 2. Table 6 describes the list of the selected articles and classifies them by extracting bibliometric facts, type facets, system specific information and quality ranking. Quality scores obtained by every article have been summed up to find the total score. According to Table 6, the average scores obtained by the articles are 1.5. The publications having scored greater than 1.5 are above average and the publications having scores below 1.5 are considered below average. Detailed description of classified results after application of quality criteria has been given in Table 6. Table 7 illustrates the trend of researcher regarding article submission in multiple sources. It lists down multiple sources, publication channels, and the frequency of articles in each source. Table 7. Article submission trend of the researchers. Source Channel Reference International Conference on Natural Computation (ICNC) Conference [75] Pure and Applied Geophysic Journal [76] Expert Systems with Applications Journal [1,9] IEEEACCESS Journal [49] International Journal of Computer Applications Journal [77] Proceedings of Indian National Science Academy Journal [78] Earth Science Informatics Journal [65,79] Journal of Indian Society of Remote Sensing Journal [80] Bulletin of Engineering Geology and Environment Journal [23,60,81,82] Natural Hazards Journal [2,4,12,22,28,51,83] Knowledge Based Systems Journal [17,19] Journal of Environmental Radioactivity Journal [54] International Journal of Coal Geology Journal [53] Computer-Aided Civil and Infrastructure Engineering Journal [15] Applied Sciences Journal [10] International Journal of Disaster Risk Reduction Journal [11] Tunnelling and Underground Space Technology Journal [13] PLoS ONE Journal [18,58] Environmental Earth Sciences Journal [5,52,84–86] International Journal of Fuzzy Systems Journal [32] Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications Journal [27] Applied Soft Computing Journal [87] Journal of Intelligent Information Systems Journal [20] Geodesy and Geodynamics Journal [26] Environmental Monitoring Assessment Journal [21] Earth Science Informatics Journal [65,79] International Journal of Computer Information Systems and Industrial Management Applications Journal [30] Biostatistics and Biometrics Journal [6] International Journal of Engineering Research & Technology Journal [24] Journal Geological Society of India Journal [50] Journal of Sustainability Science and Management Journal [16] Journal of Chemical and Pharmaceutical Sciences Journal [33] Acta Geophysica Journal [3] International Conference on Fuzzy Systems and Knowledge Discovery (FSKD) Conference [25,55,88] International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems Journal [7,71] International Conference on Information Management, Innovation Management and Industrial Engineering Conference [51] Analysis & Computation Specialty Conference Conference [8] Soil Dynamics and earthquake engineering Journal [34,64,69] Lecture notes on electrical engineering Conference [35] Advances in intelligent system and computing Journal [36] ISPR- International Journal of geo information Journal [37] Seismological Research Letter Conference [38,40] Geophysical Research Letter Conference [39,63] CEUR workshop procedings Conference [41] Geosciences Journal [42,59] Proceedings of SPIE-the international society of optical engineering Conference [43] Bulletin of seismological society of America Journal [44] Neural processing letter Conference [56] Proceedings-IEEE 4th International conference on big data, computing services and applications Conference [57] Lecture notes on computer science Conference [61] Geomagnatics, Natural hazards and risks Journal [62] International Journal of SWARM intelligence research Journal [66] Neural computing and application Journal [67] Proceedings- 14th international conference on frontiers of information technology Conference [68] Proceedings- 9th international conference on application of information and commucation technology Conference [70] Bollettino deGesfisica Teorica ed applicata Journal [71] Applied soft computing journal Journal [73] Journal of King Saud University Journal [74] Sustainability 2020, 12, 2420 14 of 32 Two dimensional studies (as described in Table 8) have been conducted to highlight seismically active zones of the world. In the first dimension, researchers have evaluated the data of specific regions, whereas in the second dimension, researchers have analyzed overall earthquake data of the world in their articles. Table 8 presented the zones (regional or global) with their geographical dimension and their location on the tectonic plates. Table 8 will facilitate the researchers in determining that which tectonic zone is being mostly explored and analyzed by the practitioners. Table 8. Target area representing the Geographic dimensions. Target Area Geographic dimension Ref. Worldwide Global [7,16,19–21,27,30,33,38–41,43,44, 56,57,60,63,70,74,77,79] Japan North American plate+Pacific plate+Pphilipine sea plate [61,85] California North American plate+Pacific plate [10,31,42,58,59,64,87] China Eurasian plate+Indian plate+Philipine sea plate [2,12,13,23,25,49,51,55,75,88] Taiwan Eurasian plate + Philipine sea plate [8] Pakistan Eurasian plate + Indian plate [3,51,53,54,68,89] Italy, Turkey Eurasian plate+African plate [15,32,70,76] Greece, Azarbaijan Eurasian plate+Arabian plate [86] [69] Morocco African plate [1] Nepal, Israel Indian plate+African plate [18,50] India, Indian plate [24,26,35,36,78] Iran Iranian plate [5,6,9,22,28,37,71,82,83] Saudi Arabia Arabian plate [52] Ethiopia Arabian plate+Somali plate+Nubian plate [86] Caraga Philipine sea plate [14] Vietnam, Malaysia Somali plate [80,84] Chile Nazca plate [10,17,58,71,72] Republic of Croatia Apulian Plate [65] Cyprus African plate+Eurasian Plate +Arabian plate [34] The researchers of the selected articles have evaluated their work using various tools and techniques including software, algorithms and other such as index normalization etc. These tools and techniques have been summarized in Table 9. Table 9. Tools and Techniques applied in the literature for ES development. Tools and Techniques % Reference MATrix Laboratory (MATLAB) 41 [1,3,6,7,9–14,16,19,22,26,30,32,33,76,79,81,84,87] Database Index normalization 4 [23,55] Generalized Langevin equation (GLE) 1.8 [51] Subsidence Coefficient calculator 1.8 [75] Predicate (PRED) in C++ 1.8 [88] Annealing, Sparsespike 1.8 [25] Classification and regression trees(CART) 1.8 [49] Fuzzy C-mean 4 [28,77] Upgraded IF THEN ELSE 4 [27,83] Normalized fuzzy peak ground acceleration (FPGA) 1.8 [8] Sustainability 2020, 12, 2420 15 of 32 Table 9. Cont. Tools and Techniques % Reference Predicate Logic 7 [17,24,50,86] Mean absolute error(MAE), Root mean square error(RMSE) 1.8 [54] Earth resources data analysis system (ERDAS) model maker 1.8 [51] 3Dimensional seismic tomography 1.8 [78] Mean square error(MSE) 4 [31,53] Rapid miner software, frequency, pattern growth algorithm 1.8 [20] Adobe 1.8 [89] Geological carbon storage (GCS) analyzer- Monecarle 1.8 [85] Fuzzy probablistic seismic hazard analyzer (FPSHA) 1.8 [2] FURIA 1.8 [80] AriGIS 1.8 [81] Saga 1.8 [83] Aeronautical reconnaissance coverage Geographic information system (ARC/INFO GIS) 1.8 [84] Geographic information system (GIS), Multi criteria decision analysis (MCDA) 4 [15,82] Multilayer Preceptron -Rule Based (MLP-RB) 1.8 [21] Nearest neighbor Invariant Riemannian metric (AIRM) 1.8 [52] WI (Weighted index) 1.8 [5] Knowledge extraction based on evolutionary learning (KEEL) 1.8 [10] Particle SWARM Optimization (PSO) 1.8 [56] Apache SPARK 1.8 [59] Kernal Fisher Discriminant Algoritthm (KFDA) 1.8 [60] Novel earthquake early warning system (NEEWS) 1.8 [64] Accuracy of results obtained through the proposed expert system for making earthquake predictions using a training set (TS) or independent test set (ITS) has been listed in Table 10. 4.2. Key Facts of Expert System Based Earthquake Prediction Publications Different publication channels, and frequency of publications per source have been presented in Table 7. Number of articles published per year from January 2010 to January 2020 along with the primary source where the article has been submitted is determined. Different publication channels have been identified, including conferences, journals, technical report and book chapters. Around 75 % of the selected papers have been published in peer reviewed journals, 24 % have presented at conferences, workshops, and symposia. 4% technical reports and book chapters have also included in this systematic mapping study. 4.3. ResearchType Facets Addressed by the Identified Publications For deep investigation, we have listed down the type of facets addressed by the selected publications given in Table 6. Type of research presented in the paper, its empirical type, the approach used and focused area has been inquired through RQ2 and listed in Table 6. As identified earlier, the research could be of evaluation type, solution type or experience type. In this mapping study, 23% of the research has presented a solution proposal to illustrate the novelty of the proposed solution for a problem or as an extension of an existing technique. 42% studies belonged to an evaluation type which represents the techniques has passed through the evaluation stages prior to its implementation or solutions have been evaluated after implementation. 22% of the mapping study presented experience Sustainability 2020, 12, 2420 16 of 32 papers which showed the personal experience of the author about earthquake prediction techniques to be used in practice. Table 10. Accuracy claimed in multiple articles. Reference Number of Records (EQ) Accuracy Magnitude Range Data Set [6] 60 78% 5.2–7.7 TS [9] 343 seismograms 99.71% ≥5.0 TS [10] 47 93.54 ≥5.5 TS [13] 12 indices 91% ≥4.5 TS [18] 9531 69.8% ≥2.0 ITS [20] 677245 87.85 3.6–9.1 TS [21] 12690 50.14% ≥3.0 ITS [22] 522 95.8% ≥4.0 TS [23] 1773 85.73% ≥3.5 TS [24] 337 63% ≥3.0 ITS [38] 1000 80.1% < 5.5 TS [43] 227 70% <5.0 TS [50] 77 80.11% ≥5.0 TS [55] 26481 78% 2.5–7.5 TS [63] 10567 40% 0.1–5.9 ITS [76] 100 99.99% 5.5–7.7 TS [86] 476 87.2% ≥5.0 TS [80] 248 84% ≥5.5 TS [83] 78 88% ≥5.0 TS [84] 1059846 86.28% ≥1.5 TS Empirically, the studies included in this systematic mapping study have been categorized into the survey, experiment and case study. 13% of the researchers have conducted surveys to study the already presented prediction models. 39% have conducted experiments on earthquake data retrieved from multiple seismic zones to make earthquake predictions and 32% have presented the case studies in which earthquake prediction task has been worked out using the data of specific areas. In this mapping study, the next type facet is the prediction approach which has been further categorized into model, method, guideline, framework and tool. 40% of the included researches have computational model for making earthquake predictions, 40%, researchers have proposed a method for earthquake prediction, 13% have given the guidelines for earthquake prediction, only 2% of the included studies have presented a framework for the earthquake prediction and analyzed earthquake prediction tools. Last type facet of this mapping study is the target area that has been classified as regional and global. 75% of the included researches have been focused to a certain specific regional area where as 24% of the researchers have used global data to forecast earthquake. Figure 3 shows the number of publications (n) according to the type facets given in RQ2. 4.4. ES type and System Specific Key Aspects of Proposed ES Three types of the proposed ES have been identified by RQ3 and presented in Figure 4. Fuzzy expert system (FES) has been proposed for making earthquake predictions by 29% of the studies. Neuro-Fuzzy expert system for making an earthquake prediction has been proposed by 11% of the studies. The rule based expert system has been proposed by 7% of the included studies. 36% of the articles have used Sustainability 2020, 12, 2420 17 of 32 machine learning for earthquake prediction and 12% of the studies have used other computational approaches like index development or classifier calculation for earthquake prediction. In this mapping study RQ3 also deals with exploring other key aspects of the proposed ESs like input domain, input attributes and their types, data type, prediction logic and the prediction technique covering the use of computational supporting software or algorithm for performing earthquake prediction task. Input domain can be the earthquake parameters like magnitude, b-value, etc. or precursors calculated from real time earthquake data. In this mapping study, 51% researches have directly consumed earthquake parameters (listed as quake variables in Table 6) whereas 38% have studied precursors that serve as indicators for an upcoming devastating earthquake event. Further, we have explored the type of above stated input attributes to analyze that whether predictions have been made by consuming single input attribute or combination of multiple attributes has been used for earthquake predictions. In this mapping study, 27%, researchers have made predictions on the basis of a single attribute where as 68% studies have used a combination of multiple input attributes for earthquake predictions. Through data type we have worked to find that input variables passed to the proposed ES are quantitative type (numeric form like magnitude, slope, etc.) or qualitative type (discrete form like water elevation, emission of radon gas etc.). It has been observed that 57% of researchers have used numeric inputs for earthquake forecast whereas 43% of the studies have performed the earthquake prediction task using a discrete type of input. This mapping study has also captured the data regarding prediction logic and prediction methodology. Through prediction logic, we have determined that the article have used the inductive or deductive logic for making earthquake predictions. Inductive logic has been used by 62% of the studies, whereas deductive logic has been presented in 32% of the included studies for giving absolute results. Next we have worked to determine the predictive methodology to describethe use of any software or an algorithm for evaluation of the proposed system. 42% of the studies have used multiple algorithms for evaluation of their predictions, 48% have used the MATLAB software for evaluation of the proposed ES and 10% have adopted other evaluation mechanisms. 4.5. Quality Assessment The quality assessment score obtained by each paper has been presented in Table 6. These articles have been collected in three distinct categories and presented in Table 11 according to the scores obtained by every article given Table 6. By observing the quality scores, 74% of the papers included in the mapping study had obtained above average score, 11% of the studies had obtained average score and 15% of the studies remained below average. This quality assessment may facilitate researchers in the selection of articles for their work. Table 11. Scores. Reference Score Average =1.5 Total % [1–14,17,19–22,25–27,31,32,34,35,37–39,42–44,49–52,55,56,58–72,74] Above average 55 79 [18,23,24,28,53,54,73] Average 7 10 [15,16,30,33,36,40,41,57] Below average 8 11 5. Discussion In this mapping study, main techniques used for earthquake prediction including rule-based, fuzzy, neuro-fuzzy and machine learning have been explored. Expert systems based approaches have been used for seismic risk assessment for landslide susceptibility through seismic hazard analysis [50–52] and soil classification [53]. ES have also been applied in earthquake engineering for seismic hazard analysis and assessment of bridges and buildings under multiple hazards [24,54,55]. The expert systems have been used to analyze multiple aspects of earthquake prediction, but due to the non-existence of the Sustainability 2020, 12, 2420 18 of 32 time-dependent global earthquake forecasting model, the regional earthquake likelihood models have been popular [87]. On the other hand, some initial research has been conducted to design earthquake early-warning systems that work globally [32,49]. A detailed discussion regarding the history and theory of accelerating seismic release and preparedness for an earthquake has been conducted in [90,91]. Extended earthquake sequences with stable features have been observed over long time periods and explained accelerated seismicity before the occurrence of devastating earthquakes with in time. The aftershock decay models proposed in the articles from 1894 till 2014 about accelerating seismology, has been analyzed in [92] and valuable information about multiple scientific process including earthquake cascaded events have been collected. Multiple anomalies have been investigated in [93] to formulate the criteria for identification of the genuine precursors from a preliminary list of identifying precursors, methods or case studies. A collection of one hundred articles related to accelerating seismology has been studied in [94] to classify the precursor into two types including critical processes such as cascading triggering of earthquake events or normal processes such as pressure on main fault. Many other studies investigated earthquake precursors like [95] examined multiple parameters collected from anomalies present in geophysical fields such as ionospheric disorder for short term earthquake prediction. Genetic algorithm (GA) has been used to optimize the hybrid artificial neural network model for the prediction of peak particle velocity in [96]. An artificial neural network has been applied to predict shear wave velocity in [97]. A seismic data-driven tool has been proposed for seismic fracture identification using large post-stack seismic dataset in [98]. The dynamic response of geogrid machine foundation bed has been studied in [99]. Rockfall hazard assessment using artificial neural network has been performed in [100]. The properties of the lower ionosphere have been examined in [101] using random matrix theory for the prediction of earthquakes. Both spatial and spatio temporal earthquake predictions have been made in [102] using machine learning methodologies. Reference [103] presented a natural time analysis of seismic-electric signal emerged before two earthquake events for the prediction of next expected earthquake events.The density of foreshocks and aftershocks has been analysed in [72] for prediction of future earthquakes that may occur in the selected seismically active regions. Reference [56] examined the impact of deep learning algorithms for classification of earthquake precursors for extraction of seismic patterns and unique features from big data. Reference [57] distinguished between seismic signals and non seismic signals using logistic regression method on the data collected from National Seismological Network of Colombia. Reference [58] applied Support Vector regression and Hybrid neural Network for earthquake prediction in Hindukush, Chile and Southern California regions with prediction accuracy rate of 82.7%, 84.9%, 90.6% respectively. Reference [59] analyzed earthquake magnitude prediction on the basis of regression algorithms and cloud based big data infrastructure. Reference [60] used grid-search method to construct support vector machine (SVM) based model for earthquake prediction. Reference [61] launched a web based platform for automatic calculation of seismic hazard fields to predict earthquakes. Reference [62] highlighted the vital role of temporal strong ground motion parameters in earthquake engineering and risk assessment using machine learning methods. Reference [63] used gradient boosted trees algorithm of machine learning to perform fivefold cross validation on training data set from earthquake catalogs to make earthquake predictions. References [64] developed an earthquake early warning system using image recognition techniques. Reference [65] studies multiple machine learning methods including random forest, artificial neural network, recurrent neural network, Naïve Bayesian and regression for earthquake prediction. Reference [66] examined the efficiency of bio-inspired algorithms for supervised classification of on real datasets to handle emergencies. Reference [67] discussed the emerging Regional Earthquake likelihood models in making earthquake predictions with improved accuracy. Reference [68] used tree-based ensemble methodologies for earthquake prediction within time period of 15 days and calculated seismic features of Hindukush region applying machine learning methods for macro earthquake prediction. Reference [69] proposed a multi-step prediction method for short-term prediction of strong Sustainability 2020, 12, 2420 19 of 32 earthquake with relatively high accuracy. Reference [70] proposed a monitoring system for preparing machine learning data-sets for earthquake prediction based on seismic-acoustic signals from stations in Azerbaijan. Reference [71] estimated the magnitudes of earthquake events recorded on daily bases using artificial neural network (ANN) to prove that training set of global data is more effective in earthquake prediction than making earthquake prediction using local data. References [72,73] studied the importance of machine learning methods in earthquake prediction and highlighted the impact of accurate prediction on country’s economy. Reference [74] presented a scheme for large earthquake prediction based on radial basis function (RBF) neural network (NN) models. This section summarizes and discusses the results of the systematic mapping study. Taxonomy of earthquake prediction is given in Figure 5. Sustainability 2020, 12, x FOR PEER REVIEW 8 of 33 prediction than making earthquake prediction using local data. [72,73] studied the importance of machine learning methods in earthquake prediction and highlighted the impact of accurate prediction on country’s economy. [74] presented a scheme for large earthquake prediction based on radial basis function (RBF) neural network (NN) models. This section summarizes and discusses the results of the systematic mapping study. Taxonomy of earthquake prediction is given in Figure 5. Figure 5. Taxonomy of earthquake prediction approaches. There are two distinct types of earthquake prediction approaches presented in Figure 5, i.e. deterministic forecasts and probabilistic forecasts. Deterministic forecasts are made on the base of earthquake characteristics like rupture length, modified Mercalli and return period where as probabilistic forecasts deals with precursors, implications of earthquake physics and elastic rebound. Table 12 clusters the articles according to the taxonomy given in Figure 5. Table 12. Approaches used to predict particular seismic phenomena. Deterministic Domain s Seismic Phenomena Approach Reference C ha ra ct er is ti c ea rt hq ua ke Rupture length Modified Mercalli Return Period Classification, Clustering, Machine Learning (ML), Neural network (NN) [2,6,8,10,16,17-– 20,28,33– 35,37,39,42,44,58– 62,65,69,70,72,74,80,86] Probabilistic P re cu rs or Animal behavior Predicate Logic, [27,66] Seismic velocity [9,32,57] Figure 5. Taxonomy of earthquake prediction approaches. There are two distinct types of earthquake prediction approaches presented in Figure 5, i.e., deterministic forecasts and probabilistic forecasts. Deterministic forecasts are made on the base of earthquake characteristics like rupture length, modified Mercalli and return period where as probabilistic forecasts deals with precursors, implications of earthquake physics and elastic rebound. Table 12 clusters the articles according to the taxonomy given in Figure 5. From Table 12 it is clear that multiple approaches have been used for eathquake prediction including Rule based, Fuzzy and Neuro-fuzzy and machine learning. For deterministic estimations, earthquakes have been grouped according to their magnitude range by applying classification, clustering and machine learning techiques. For probablistic forcasts, multiple artificial intelligence methods have been exercised for making earthquake predictions. Comparitive studies have been conducted [13], systematic methods of predicate logic have also been applied to analyze the precursor that may have vital importance for earthquake prediction [9,27,32,57,65]. To estimate the mutual relationship of precursors, regression line has been calculated [26]. Machine learnining approaches have shown greater improvement in prediction accuracy presented in Table 10. Multi criteria decision making (MCDM) approaches including Technique for order of preference by similarity of ideal solutions (TOPSIS) and Sustainability 2020, 12, 2420 20 of 32 Aggrigated indices randomization (AIRM) have also been applied for earthquake prediction. Multiple precursors may occur before catastrophic earthquakes which have been analyzed through MCDM methods by assigning weightage to every precurssor for the prediction of expected earthquake events. Pattern recognition techniques have also been exercised to analyse the seismic patterns generated by the energy released in previous earthquakes and radon emission recorded before an earthquake. Table 12. Approaches used to predict particular seismic phenomena. Deterministic Domains Seismic Phenomena Approach Reference Characteristic earthquake Rupture length Modified Mercalli Return Period Classification, Clustering, Machine Learning (ML), Neural network (NN) [2,6,8,10,16–20,28,33–35,37,39,42,44, 58–62,65,69,70,72,74,80,86] Probabilistic Precursor Animal behavior Predicate Logic, Aggregated Indices Randomization Method(AIRM), Regression, Comparison, Clustering, ML, NN [27,66] Seismic velocity [9,32,57] Seismic resistivity [84] Topography uplift [5,10,14,15,22,23,50,52,77] Radon emission [13,104] Seismic electric signal [21] Electromagnetic signals [30,63,64,71] Ground water elevation [105,106] Land sliding [41,82,83,86] Earthquake physics Earthquake light Ionosphere disorder ML, NN Technique for Order of Preference by Similarity to Ideal Solution [36,85] [26] Elsticrebound Seismic Gap Seismic Pattern Pattern recognition Clustering, ML [37,38,40,51] [49,88] 5.1. Comparitive Analysis of Methods Multiple techniques have been used for earthquake prediction in the literature. We have formulated the foundation of this mapping study on expert systems.However, there are many other approaches as well for the prediction of earthquake presented in the literature. We have compared expert system developed for earthquake prediction with other artificial intelligence techniques used for the same in Table 13. Table 13. Comparison of three Artificial intelligence techniques. Method Comparison Neural networks and Expert systems Expert system is about capturing and encoding (often manually) rules that experts use so as to develop a program that can mimic their behavior in a very specific domain. It often involved chaining these rules together. With ANN the rules are encoded automatically by presenting examples, good and bad, to the network. The network adjusts weightings over many iterative cycles, honing its output to the correct value. Feed Forward Neural Networks can predict long term and short term earthquakes but it cannot get feedback of output from multiple layers and Back Propagation Neural Network mostly trapped in different local conditions during the training phase of earthquake data sets. However, probability of getting desired output raises when it is tested with ideally designed inputs. Machine learning and Expert systems Machine learning (ML) focuses on modeling of data statistically and expert is involved at the time of decision. Supervised learning algorithms are used to copy the ending decisive behavior of the Expert systems are based upon set of rules prescribed by human expert and learn by directly injecting the domain level knowledge of human expert. The knowledge obtained from the expert is completely converted into membership functions and used in decision making. Explanation facility is also available as an expert describes all the steps till decision, the basis and exception handling procedures. A rigid system is developed that follows exact rules as described by the expert. Rigidness of the expert system makes it most suitable from all other techniques for predicting future earthquakes. Table 14 presented the comparision of multiple methods including fuzzy, neuro fuzzy, rule based and machine learning used for earthquake prediction in the literature.It can be observed that both deterministic and probablistic methods are being excercised for earthquake prediction. Many researchers have performed numerical experiments to achieve success in predicting earthquakes. Some have developed tools while others have explored multiple dimensions of application area. Table 14 clearly presents that most recent studies have worked on exploring machine learning based models for earthquake prediction. Sustainability 2020, 12, 2420 21 of 32 Table 14. Comparision of Methods. Method Ref. Prediction Approach Algorithm Defined Application Area Dataset Deterministic Probabilistic Analytical Work Global Approximation Numerical Experiment Exploration with actual forecasts Success Achieved Characteristic Earth quake Precursors Zone Studied Fuzzy Expert System [2] 4 4 4 4 China [6] 4 4 4 4 Iran [8] 4 4 Taiwan [9] 4 4 4 Iran [10] 4 4 4 4 4 California [13] 4 4 4 China [14] 4 4 4 Caraga [15] 4 4 4 Turkey [18] 4 4 Nepal [21] 4 4 4 4 [23] 4 4 4 4 China [24] 4 4 4 India [26] 4 4 4 India [50] 4 4 4 4 Nepal [51] 4 4 4 China [52] 4 4 Saudi Arabia [80] 4 4 4 4 4 Malaysia [82] 4 4 Iran [83] 4 4 4 4 Iran [84] 4 4 4 4 Malaysia [86] 4 4 Ethiopia Neuro Fuzzy Expert System (NFES) [5] 4 4 4 Iran [17] 4 4 Chile [19] 4 4 4 [20] 4 4 4 4 Sustainability 2020, 12, 2420 22 of 32 Table 14. Cont. Method Ref. Prediction Approach Algorithm Defined Application Area Dataset Deterministic Probabilistic Analytical Work Global Approximation Numerical Experiment Exploration with actual forecasts Success Achieved Characteristic Earth quake Precursors Zone Studied Neuro Fuzzy Expert System (NFES) [22] 4 4 4 Iran [27] 4 4 4 [28] 4 4 Iran [30] 4 4 4 4 [32] 4 4 Turkey [33] 4 4 4 4 [49] 4 4 4 4 China [77] 4 4 4 [81] 4 4 4 Greece Machine Learning (ML) [34] 4 4 4 Cyprus [35] 4 4 4 India [36] 4 4 India [37] 4 4 4 4 Iran [38] 4 4 4 4 [39] 4 4 4 4 4 [40] 4 4 4 [41] 4 4 4 California [42] 4 4 4 4 [44] 4 4 4 4 [56] 4 4 4 4 4 4 [57] 4 4 4 California [58] 4 4 4 4 California [59] 4 4 4 4 Sustainability 2020, 12, 2420 23 of 32 Table 14. Cont. Method Ref. Prediction Approach Algorithm Defined Application Area Dataset Deterministic Probabilistic Analytical Work Global Approximation Numerical Experiment Exploration with actual forecasts Success Achieved Characteristic Earth quake Precursors Zone Studied Machine Learning (ML) [60] 4 4 4 4 Japan [61] 4 4 4 4 [62] 4 4 4 [63] 4 4 4 4 4 [64] 4 4 4 California [65] 4 4 Croatia [66] 4 4 4 [68] 4 4 Pakistan [69] 4 4 Greece [70] 4 4 4 4 Turkey [71] 4 4 Iran [72] 4 4 Chile [74] 4 4 4 4 Sustainability 2020, 12, 2420 24 of 32 5.2. Principal Findings The goal of this systematic mapping study is to examine the current status of ES used to predict earthquakes by selecting the appropriate number of recently published papers. We have passed all the studies from our selection and quality criteria to classify the studies according to the scores obtained by every research work on the basis of rules listed in Table 6. The categories of the studies according to their quality scores has been listed in Table 6. In this mapping study, 74% articles have above average scores, 15% articles scored average quality results and 11% researches are below average.Table 11 will facilitate the researchers in selecting quality studies. In Figure 6, we have combined the answers from sub-parts of RQ2 for presenting an overview of earthquake prediction activity. This mapping has allowed us to obtain more information on how the results from each RQ relate to each others. Figure 6 presents the research type facet related to earthquake prediction, distributed over approaches and its empirical type. It allows us to conclude that only one model was proposed by the authors that mainly report their experience in the earthquake prediction process. However, guidelines have been provided by six authors. The majority of the solution proposals have developed models for earthquake prediction. However, just one researcher has developed a tool for earthquake prediction evaluation. Further information regarding the relationship between type facets is shown in Figure 6. Sustainability 2020, 12, x FOR PEER REVIEW 12 of 33 above average scores, 15% articles scored average quality results and 11% researches are below average.Table 11 will facilitate the researchers in selecting quality studies. In Figure 6, we have combined the answers from sub-parts of RQ2 for presenting an overview of earthquake prediction activity. This mapping has allowed us to obtain more information on how the results from each RQ relate to each others. Figure 6 presents the research type facet related to earthquake prediction, distributed over approaches and its empirical type. It allows us to conclude that only one model was proposed by the authors that mainly report their experience in the earthquake prediction process. However, guidelines have been provided by six authors. The majority of the solution proposals have developed models for earthquake prediction. However, just one researcher has developed a tool for earthquake prediction evaluation. Further information regarding the relationship between type facets is shown in Figure 6. Figure 6. Relationship between type facets given in Research Question 2 (RQ2). RQ3 deals with identification of the type of proposed ES. Figure 7 shows the relationship between the proposed ES with other system specific details. Machine / deep learning has emerged as most focused and recent trend in the earthquake prediction process. However, It can be clearly seen from Figure 7 that FES has been proposed by a majority of the researchers for a long time period. FES is representing a balance in the selection of all input parameters except the use of deductive logic in earthquake prediction in Figure 7. It might be due to the nature of FES basically reflecting uncertainty [15] that's why most of the researchers have used inductive logic to propose FES for the earthquake prediction. However, some of the researchers have also worked to proposed NFES and RBES but their frequency is very low. Figure 6. Relationship between type facets given in Research Question 2 (RQ2). RQ3 deals with identification of the type of proposed ES. Figure 7 shows the relationship between the proposed ES with other system specific details. Machine / deep learning has emerged as most focused and recent trend in the earthquake prediction process. However, It can be clearly seen from Figure 7 that FES has been proposed by a majority of the researchers for a long time period. FES is representing a balance in the selection of all input parameters except the use of deductive logic in earthquake prediction in Figure 7. It might be due to the nature of FES basically reflecting uncertainty [15] that’s why most of the researchers have used inductive logic to propose FES for the earthquake prediction. However, some of the researchers have also worked to proposed NFES and RBES but their frequency is very low. Sustainability 2020, 12, 2420 25 of 32 Sustainability 2020, 12, x FOR PEER REVIEW 13 of 33 Figure 7. Relationship between ES approaches and system specific parameters. The findings of our systematic mapping study havethe following implications for practitioners and researchers. This study will allow them to discover the existing use of expert system in the literature concerning earthquake prediction. In order to improve the reliability in earthquake prediction, researchers and practitioners may consider the following advices: 1. Globally, we need a program of identification and characterization of potentially hazardous faults in multiple seismic zones. From those studies, site-specific expected seismic shaking maps can be developed that would facilitate in developing expert system for earthquake prediction process. 2. By comparing different forecasts that are computed from common data, contrasts in performance can be tied to specific features of the computational prediction method. Enforcing the need to create a testable prediction, hypothesis that may reveal shortcomings or incomplete features of the prediction method is needed. 3. Activities focusing on comparative testing of computational prediction methods based on seismicity and fault information that provide probabilistic predictions of moderate magnitude earthquakes on a geographic grid are needed. This approach can be optimized to achieve useful statistics in a short time and can also advance the research field by providing insights into the computational predictability of earthquakes. However, visible hypotheses such as the M8/MSc predictions of global earthquakes, the “reverse detection of precursors” method, or the Retrograde Intravenous Pressure Infusion “RIPI” method, each of which analyze temporal and spatial variations in seismicity, or other methods based on observable quantities such as the electromagnetic field, ground temperature, gaseous emissions, geodetic deformation, or changes in seismic wave speed. Many of the most visible and influential earthquake predictions are posed as “alarms” or “times of increased probability” (TIPs) within some specified region rather than as probabilities on a grid of points. 4. Evaluation of emerging situations such as earthquake swarms, the likelihood of damaging aftershocks or triggered earthquakes following major quakes, or the likelihood of re-rupture of a fault following a major earthquake should be examined. Likewise, a broader suite of statistical tests, spanning the range from straightforward to sophisticated, would allow some prediction methods to be easily disproven in a way that’s clear to researchers, the media and the public, while providing the rigorous analysis required for comparative testing. These should include statistical tests applicable to alarm-based computational prediction methods. 5.3. Evolution of Tools and Techniques Figure 7. Relationship between ES approaches and system specific parameters. The findings of our systematic mapping study havethe following implications for practitioners and researchers. This study will allow them to discover the existing use of expert system in the literature concerning earthquake prediction. In order to improve the reliability in earthquake prediction, researchers and practitioners may consider the following advices: 1. Globally, we need a program of identification and characterization of potentially hazardous faults in multiple seismic zones. From those studies, site-specific expected seismic shaking maps can be developed that would facilitate in developing expert system for earthquake prediction process. 2. By comparing different forecasts that are computed from common data, contrasts in performance can be tied to specific features of the computational prediction method. Enforcing the need to create a testable prediction, hypothesis that may reveal shortcomings or incomplete features of the prediction method is needed. 3. Activities focusing on comparative testing of computational prediction methods based on seismicity and fault information that provide probabilistic predictions of moderate magnitude earthquakes on a geographic grid are needed. This approach can be optimized to achieve useful statistics in a short time and can also advance the research field by providing insights into the computational predictability of earthquakes. However, visible hypotheses such as the M8/MSc predictions of global earthquakes, the “reverse detection of precursors” method, or the Retrograde Intravenous Pressure Infusion “RIPI” method, each of which analyze temporal and spatial variations in seismicity, or other methods based on observable quantities such as the electromagnetic field, ground temperature, gaseous emissions, geodetic deformation, or changes in seismic wave speed. Many of the most visible and influential earthquake predictions are posed as “alarms” or “times of increased probability” (TIPs) within some specified region rather than as probabilities on a grid of points. 4. Evaluation of emerging situations such as earthquake swarms, the likelihood of damaging aftershocks or triggered earthquakes following major quakes, or the likelihood of re-rupture of a fault following a major earthquake should be examined. Likewise, a broader suite of statistical tests, spanning the range from straightforward to sophisticated, would allow some prediction methods to be easily disproven in a way that’s clear to researchers, the media and the public, while providing the rigorous analysis required for comparative testing. These should include statistical tests applicable to alarm-based computational prediction methods. Sustainability 2020, 12, 2420 26 of 32 5.3. Evolution of Tools and Techniques To reveal the development trend of researchers, various tools and techniques used to develop ES for earthquake prediction given in the literature have been listed in Table 10. It is clear from Table 10 that 41% of the researchers have used the MATLAB software for development of an ES. MATLAB is in the market since 1984. MATLAB is used for multi domain purposes like signal processing, image processing and automation, etc. It has Simulink, state flow, embedded coder and Simulink coder. Simulink facilitates the model based development where as code is automatically generated through embedded coder. Moreover, traceability of the code is much easier than legacy coding. Simulink helps in the development of a system in block diagrams by providing many elements like transfer function, summing junction, fuction generators and oscilloscopes which makes the work easier for the researchers. Through its debugging option, it has gained the trust of the researchers from last thirty five years. It is also clear from Table 6 that for earthquake prediction rule-based expert systems have been focused till 2013 then, till 2017 fuzzy and neur- fuzzy expert systems have been explored. But, now a prominent change in the research trend has been observed from 2018 onwards that earthquake predictions are being carried out using machine learning and deep learning approaches. Changing trend of earthquake prediction approaches has been shown in the timeline presented in the Figure 8. Sustainability 2020, 12, x FOR PEER REVIEW 14 of 33 To reveal the development trend of researchers, various tools and techniques used to develop ES for earthquake prediction given in the literature have been listed in Table 10. It is clear from Table 10 that 41% of the researchers have used the MATLAB software for development of an ES. MATLAB is in the market since 1984. MATLAB is used for multi domain purposes like signal processing, image processing and automation, etc. It has Simulink, state flow, embedded coder and Simulink coder. Simulink facilitates the model based development where as code is automatically generated through embedded coder. Moreover, traceability of the code is much easier than legacy coding. Simulink helps in the development of a system in block diagrams by providing many elements like transfer function, summing junction, fuction generators and oscilloscopes which makes the work easier for the researchers. Through its debugging option, it has gained the trust of the researchers from last thirty five years.It is also clear from Table 6 that for earthquake prediction rule-based expert systems have been focused till 2013 then, till 2017 fuzzy and neur- fuzzy expert systems have been explored. But, now a prominent change in the research trend has been observed from 2018 onwards that earthquake predictions are being carried out using machine learning and deep learning approaches. Changing trend of earthquake prediction approaches has been shown in the timeline presented in the Figure 8. 2013 2017 2020 Figure 8. Timeline of earthquake prediction approaches. 6. Conclusion and Future Directions This paper has presented a systematic mapping study to summarize the existing research on multiple approaches involving machine learning, neuro-fuzzy, fuzzy and rule based approaches that have been used for earthquake prediction. Out of 2137 studies, 70 articles published between January 2010 till January 2020 were carefully selected and classified on the basis of research type, empirical type, approach, target area, and other system specific parameters. Publication source and trend have also been identified. Most of the articles considered in this study have been selected from peer reviewed journals and (Computing Research and Education) CORE ranked conferences. Majority of the papers included in this mapping study involve empirical validation for the proposed solutions to predict earthquakes. The use of various types of ES presented in this study may help researchers to identify approaches that can be adopted in order to improve the quality of earthquake prediction in their work. For future research more attention should be given to theapplication of machine learning and deep learning methods for earthquake prediction. Due to ever increasing volume of data, there is a need to employ machine learning and deep learning for the prediction of earthquakes. Moreover , there is a need to conduct more of evaluation research for the validation of the already presented prediction models based on fuzzy logic. The analysis of the presented research works show that most of the approaches focused on analyzing the precursors generating direct warning of an earthquake. However, recent trends show that there is a need to extend this work by involving other factors which may include volcanic eruption, nuclear explosion, hurricanes, tsunamis etc. The results obtained showed that an increasing amount of attention has been paid to the use of an ES for earthquake prediction since 2011. It has been noticed that till 2013 the reserch has been more focused on rule based methods for earthquake prediction. Then the trend has changed towards using fuzzy and neuro fuzzy methods till 2018. Machine learning and deep learning has emerged as the most focused approach for earthquake prediction in the recent time. The classification developed in this mapping study has presented increased trend of applying FES and ML in making long term earthquake predictions. However, In the future, more advanced RBES FES / NFES Machine/ Deep Learning Figure 8. Timeline of earthquake prediction approaches. 6. Conclusion and Future Directions This paper has presented a systematic mapping study to summarize the existing research on multiple approaches involving machine learning, neuro-fuzzy, fuzzy and rule based approaches that have been used for earthquake prediction. Out of 2137 studies, 70 articles published between January 2010 till January 2020 were carefully selected and classified on the basis of research type, empirical type, approach, target area, and other system specific parameters. Publication source and trend have also been identified. Most of the articles considered in this study have been selected from peer reviewed journals and (Computing Research and Education) CORE ranked conferences. Majority of the papers included in this mapping study involve empirical validation for the proposed solutions to predict earthquakes. The use of various types of ES presented in this study may help researchers to identify approaches that can be adopted in order to improve the quality of earthquake prediction in their work. For future research more attention should be given to theapplication of machine learning and deep learning methods for earthquake prediction. Due to ever increasing volume of data, there is a need to employ machine learning and deep learning for the prediction of earthquakes. Moreover, there is a need to conduct more of evaluation research for the validation of the already presented prediction models based on fuzzy logic. The analysis of the presented research works show that most of the approaches focused on analyzing the precursors generating direct warning of an earthquake. However, recent trends show that there is a need to extend this work by involving other factors which may include volcanic eruption, nuclear explosion, hurricanes, tsunamis etc. The results obtained showed that an increasing amount of attention has been paid to the use of an ES for earthquake prediction since 2011. It has been noticed that till 2013 the reserch has been more focused on rule based methods for earthquake prediction. Then the trend has changed towards using fuzzy and neuro fuzzy methods till 2018. Machine learning and deep learning has emerged as the most focused approach for earthquake prediction in the recent time. Sustainability 2020, 12, 2420 27 of 32 The classification developed in this mapping study has presented increased trend of applying FES and ML in making long term earthquake predictions. However, In the future, more advanced deep learning based model should be designed to make pinpoint predictions. Moreover, there is a continuing need to develop a suite of basic tools and reference models to rapidly establish an unbiased framework to evaluate prediction methods, which enforces strict adherence to the scientific method, motivates investigators to accurately and unambiguously express prediction hypotheses, and provides guidance and tools for formal testing of those hypotheses. These features would lead to progress in evaluating seismicity-based models. Author Contributions: “conceptualization, R.T. and M.S.F.; methodology, R.T.; software, R.T.; validation, R.T., M.S.F and A.A.; formal analysis, R.T.; investigation, M.S.F; resources, A.A.; data curation, Rabia; writing—original draft preparation, R.T.; writing—review and editing, R.T., M.S.F. and A.A.; visualization, A.A.; supervision, M.S.F, A.A.; project administration, M.S.F.; funding acquisition, R.T.” All authors have read and agreed to the published version of the manuscript. Funding: This research received no external funding Acknowledgments: The authors appreciate the anonymous reviewers for their valuable feedback on the initial version of this paper. Conflicts of Interest: The authors declare no conflict of interest. References 1. Laasri, H.A.; Akhouayri, E.-S.; Agliz, D.; Zonta, D.; Atmani, A. A fuzzy expert system for automatic seismic signal classification. Expert Syst. Appl. 2015, 42, 1013–1027. [CrossRef] 2. Andrić, J.M.; Lu, D.-G. Fuzzy probabilistic seismic hazard analysis with applications to Kunming city, China. Nat. Hazards 2017, 89, 1031–1057. [CrossRef] 3. Asim, K.M.; Awais, M.; Martínez–Álvarez, F.; Iqbal, T. Seismic activity prediction using computational intelligence techniques in northern Pakistan. Acta Geophys. 2017, 65, 919–930. [CrossRef] 4. Asim, K.M.; Martínez-Álvarez, F.; Basit, A.; Iqbal, T. Earthquake magnitude prediction in Hindukush region using machine learning techniques. Nat. Hazards 2016, 85, 471–486. [CrossRef] 5. Aghdam, I.N.; Varzandeh, M.H.M.; Pradhan, B. Landslide susceptibility mapping using an ensemble statistical index (Wi) and adaptive neuro-fuzzy inference system (ANFIS) model at Alborz Mountains (Iran). Environ. Earth Sci. 2016, 75, 553. [CrossRef] 6. Ahmadi, M.; Nasrollahnejad, A.; Faraji, A. Prediction of Peak ground acceleration for earthquakes by using intelligent methods. In Proceedings of the 2017 5th Iranian Joint Congress on Fuzzy and Intelligent Systems (CFIS), Qazvin, Iran, 7–9 March 2017; pp. 7–12. 7. Bahrami, B.; Shafiee, M. Fuzzy Descriptor Models for Earthquake Time Prediction Using Seismic Time Series. Int. J. Uncertain. Fuzziness Knowl. -Based Syst. 2015, 23, 505–519. [CrossRef] 8. Ahumada, A.; Altunkaynak, A.; Ayoub, A. Fuzzy logic-based attenuation relationships of strong motion earthquake records. Expert Syst. Appl. 2015, 42, 1287–1297. [CrossRef] 9. Aboonasr, S.F.G.; Zamani, A.; Razavipour, F.; Boostani, R. Earthquake hazard assessment in the Zagros Orogenic Belt of Iran using a fuzzy rule-based model. Acta Geophys. 2017, 65, 589–605. [CrossRef] 10. Fernández-Gómez, M.J.; Asencio-Cortés, G.; Troncoso, A.; Martínez-Álvarez, F. Large Earthquake Magnitude Prediction in Chile with Imbalanced Classifiers and Ensemble Learning. Appl. Sci. 2017, 7, 625. [CrossRef] 11. Andrić, J.M.; Lu, D.G. Fuzzy–Based Method for the Prediction of Seismic Resilience of Bridges; Elsevier: Amsterdam, The Netherlands, 2016. 12. Azadeh, A.; Mohammadfam, I.; Khoshnoud, M.; Nikafrouz, M. Design and implementation of a fuzzy expert system for performance assessment of an integrated health, safety, environment (HSE) and ergonomics system: The case of a gas refinery. Inf. Sci. 2008, 178, 4280–4300. [CrossRef] 13. Cai, W.; Dou, L.; Zhang, M.; Cao, W.; Shi, J.-Q.; Feng, L. A fuzzy comprehensive evaluation methodology for rock burst forecasting using microseismic monitoring. Tunn. Undergr. Space Technol. 2018, 80, 232–245. [CrossRef] 14. Cerna, M.A.D.; Maravillas, E.A. Mamdani Fuzzy Decision Model for GIS-Based Landslide Hazard Mapping. In Transactions on Engineering Technologies; Springer: Singapore, 2017; pp. 59–73. http://dx.doi.org/10.1016/j.eswa.2014.08.023 http://dx.doi.org/10.1007/s11069-017-3007-z http://dx.doi.org/10.1007/s11600-017-0082-1 http://dx.doi.org/10.1007/s11069-016-2579-3 http://dx.doi.org/10.1007/s12665-015-5233-6 http://dx.doi.org/10.1142/S0218488515500221 http://dx.doi.org/10.1016/j.eswa.2014.09.035 http://dx.doi.org/10.1007/s11600-017-0055-4 http://dx.doi.org/10.3390/app7060625 http://dx.doi.org/10.1016/j.ins.2008.06.026 http://dx.doi.org/10.1016/j.tust.2018.06.029 Sustainability 2020, 12, 2420 28 of 32 15. D’Urso, M.G.; Masi, D.; Zuccaro, G.; Gregorio, D. Multicriteria Fuzzy Analysis for a GIS-Based Management of Earthquake Scenarios. Comput. Civ. Infrastruct. Eng. 2017, 33, 165–179. [CrossRef] 16. Dutta, P.K.; Mishra, O.P.; Naskar, M.K. A review of operational earthquake forecasting methodologies using linguistic fuzzy rule-based models from imprecise data with weighted regression approach. J. Sustain. Sci. Manag. 2013, 8, 220–235. 17. Martínez-Álvarez, F.; Reyes, J.; Morales-Esteban, A.; Rubio-Escudero, C. Determining the best set of seismicity indicators to predict earthquakes. Two case studies: Chile and the Iberian Peninsula. Knowl. -Based Syst. 2013, 50, 198–210. [CrossRef] 18. Last, M.; Rabinowitz, N.; Leonard, G. Predicting the Maximum Earthquake Magnitude from Seismic Data in Israel and Its Neighboring Countries. PLoS ONE 2016, 11, e0146101. [CrossRef] 19. Ikram, A.; Qamar, U. Developing an expert system based on association rules and predicate logic for earthquake prediction. Knowl. -Based Syst. 2015, 75, 87–103. [CrossRef] 20. Ikram, A.; Qamar, U. A rule-based expert system for earthquake prediction. J. Intell. Inf. Syst. 2014, 43, 205–230. [CrossRef] 21. Ghorbani, S.; Barari, M.; Hoseini, M. Presenting a new method to improve the detection of micro-seismic events. Environ. Monit. Assess. 2018, 190, 464. [CrossRef] 22. Ghorbanzadeh, O.; Rostamzadeh, H.; Blaschke, T.; Gholaminia, K.; Aryal, J. A new GIS-based data mining technique using an adaptive neuro-fuzzy inference system (ANFIS) and k-fold cross-validation approach for land subsidence susceptibility mapping. Nat. Hazards 2018, 94, 497–517. [CrossRef] 23. Meng, Q.; Miao, F.; Zhen, J.; Wang, X.; Wang, A.; Peng, Y.; Fan, Q. GIS-based landslide susceptibility mapping with logistic regression, analytical hierarchy process, and combined fuzzy and support vector machine methods: A case study from Wolong Giant Panda Natural Reserve, China. Bull. Int. Assoc. Eng. Geol. 2015, 75, 923–944. [CrossRef] 24. Tahernia, N. Fuzzy-Logic Tree Approach for Seismic Hazard Analysis. Int. J. Eng. Technol. 2014, 6, 182–185. [CrossRef] 25. Wang, S.; Liu, H.; Wang, S.; Tong, S.; Wang, R. Pseudo-acoustic inversion method and its application. In Proceedings of the 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery, Yantai, China, 10–12 August 2010; Volume 2, pp. 598–601. 26. Ratnam, D.V.; Vindhya, G.; Dabbakuti, J.R.K.K. Ionospheric forecasting model using fuzzy logic-based gradient descent method. Geodesy Geodyn. 2017, 8, 305–310. [CrossRef] 27. Hossain, M.S.; Al Hasan, A.; Guha, S.; Andersson, K. A Belief Rule Based Expert System to Predict Earthquake under Uncertainty. J. Wirel. Mob. Netw. Ubiquitous Comput. Dependable Appl. 2018, 9, 26–41. 28. Mirrashid, M. Earthquake magnitude prediction by adaptive neuro-fuzzy inference system (ANFIS) based on fuzzy C-means algorithm. Nat. Hazards 2014, 74, 1577–1593. [CrossRef] 29. Abayon, R.C.; Apilado, J.R.; Pacis, D.B.; Chua, M.G.; Aguilar, A.V.; Calim, J.; Padilla, S.M.A.; Puno, J.C.S.; Apsay, M.R.B.; Bustamante, R. A Weather Prediction and Earthquake Monitoring System. In Proceedings of the 2018 IEEE Conference on Systems, Process and Control (ICSPC), Malacca, Malaysia, 14–15 December 2018; pp. 203–208. 30. Shibli, M. A novel approach to predict earthquakes using adaptive neural fuzzy inference system and conservation of energy-angular momentum. Int. J. Comp. Inf. Syst. Ind. Manag. Appl. 2011, 2150, 371–390. 31. Torres, V.M.; Castillo, O. A Type-2 Fuzzy Neural Network Ensemble to Predict Chaotic Time Series. In Studies in Computational Intelligence; Springer: Berlin/Heidelberg, Germany, 2015; Volume 601, pp. 185–195. 32. Tosunoğlu, N.G.; Apaydin, A. A New Spatial Algorithm Based on Adaptive Fuzzy Neural Network for Prediction of Crustal Motion Velocities in Earthquake Research. Int. J. Fuzzy Syst. 2018, 20, 1656–1670. [CrossRef] 33. Kamath, R.S.; Kamat, R.K. Earthquake Magnitude Prediction for Andaman-Nicobar Islands: Adaptive Neuro Fuzzy Modeling with Fuzzy Subtractive Clustering Approach. J. Chem. Pharm. Sci. 2017, 10, 1228–1233. 34. Asim, K.M.; Moustafa, S.S.; Niaz, I.A.; Elawadi, E.A.; Iqbal, T.; Martínez-Álvarez, F. Seismicity analysis and machine learning models for short-term low magnitude seismic activity predictions in Cyprus. Soil Dyn. Earthq. Eng. 2020, 130, 105932. [CrossRef] 35. Vasti, M.; Dev, A. Classification and Analysis of Real-World Earthquake Data Using Various Machine Learning Algorithms. In Lecture Notes in Electrical Engineering; Springer: Singapore, 2019; pp. 1–14. http://dx.doi.org/10.1111/mice.12335 http://dx.doi.org/10.1016/j.knosys.2013.06.011 http://dx.doi.org/10.1371/journal.pone.0146101 http://dx.doi.org/10.1016/j.knosys.2014.11.024 http://dx.doi.org/10.1007/s10844-014-0316-5 http://dx.doi.org/10.1007/s10661-018-6837-6 http://dx.doi.org/10.1007/s11069-018-3449-y http://dx.doi.org/10.1007/s10064-015-0786-x http://dx.doi.org/10.7763/IJET.2014.V6.692 http://dx.doi.org/10.1016/j.geog.2017.05.003 http://dx.doi.org/10.1007/s11069-014-1264-7 http://dx.doi.org/10.1007/s40815-018-0483-6 http://dx.doi.org/10.1016/j.soildyn.2019.105932 Sustainability 2020, 12, 2420 29 of 32 36. Mukhopadhyay, U.K.; Sharma, R.N.K.; Anwar, S.; Dutta, A.D. Correlating Thermal Anomaly with Earthquake Occurrences Using Remote Sensing; Springer: Berlin/Heidelberg, Germany, 2019; pp. 863–875. 37. Karimzadeh, S.; Matsuoka, M.; Kuang, J.; Ge, L. Spatial Prediction of Aftershocks Triggered by a Major Earthquake: A Binary Machine Learning Perspective. ISPRS Int. J. Geo-Inf. 2019, 8, 462. [CrossRef] 38. Zhou, Z.; Lin, Y.; Zhang, Z.; Wu, Y.; Johnson, P. Earthquake Detection in 1D Time—Series Data with Feature Selection and Dictionary Learning. Seism. Res. Lett. 2019, 90, 563–572. [CrossRef] 39. Corbi, F.; Sandri, L.; Bedford, J.; Funiciello, F.; Brizzi, S.; Rosenau, M.; Lallemand, S. Machine Learning Can Predict the Timing and Size of Analog Earthquakes. Geophys. Res. Lett. 2019, 46, 1303–1311. [CrossRef] 40. Kong, Q.; Trugman, D.T.; Ross, Z.E.; Bianco, M.; Meade, B.J.; Gerstoft, P. Machine Learning in Seismology: Turning Data into Insights. Seism. Res. Lett. 2018, 90, 3–14. [CrossRef] 41. Galkina, A.; Grafeeva, N. Machine Learning Methods for Earthquake Prediction: A Survey. In Proceedings of the Fourth Conference on Software Engineering and Information Management (SEIM-2019), Saint Petersburg, Russia, 13 April 2019; full papers. p. 25. 42. Gitis, V.G.; Derendyaev, A. Machine Learning Methods for Seismic Hazards Forecast. Geosciences 2019, 9, 308. [CrossRef] 43. Al-Najjar, H.A.H.; Kalantar, B.; Pradhan, B.; Saeidi, V. Conditioning factor determination for mapping and prediction of landslide susceptibility using machine learning algorithms. In Earth Resources and Environmental Remote Sensing/GIS Applications X; International Society for Optics and Photonics: California, CA, USA, 2019; Volume 11156, p. 111560K. 44. Ganter, T.; Sundermier, A.; Ballard, S. Alternate Null Hypothesis Correlation: A New Approach to Automatic Seismic Event Detection. Bull. Seism. Soc. Am. 2018, 108, 3528–3547. [CrossRef] 45. Mosavi, A.; Salimi, M.; Ardabili, S.F.; Rabczuk, T.; Shamshirband, S.; Várkonyi-Kóczy, A.R. State of the Art of Machine Learning Models in Energy Systems, a Systematic Review. Energies 2019, 12, 1301. [CrossRef] 46. Mosavi, A.; Ozturk, P.; Chau, K.-W. Flood Prediction Using Machine Learning Models: Literature Review. Water 2018, 10, 1536. [CrossRef] 47. Dineva, A.; Mosavi, A.; Ardabili, S.F.; Vajda, I.; Shamshirband, S.; Rabczuk, T.; Chau, K.-W. Review of Soft Computing Models in Design and Control of Rotating Electrical Machines. Energies 2019, 12, 1049. [CrossRef] 48. Nosratabadi, S.; Mosavi, A.; Shamshirband, S.; Zavadskas, E.K.; Rakotonirainy, A.; Chau, K.-W. Sustainable Business Models: A Review. Sustainability 2019, 11, 1663. [CrossRef] 49. Zhang, L.; Si, L.; Yang, H.; Hu, Y.; Qiu, J. Precursory Pattern Based Feature Extraction Techniques for Earthquake Prediction. IEEE Access 2019, 7, 30991–31001. [CrossRef] 50. Kayastha, P.; Bijukchhen, S.; Dhital, M.R.; De Smedt, F. GIS based landslide susceptibility mapping using a fuzzy logic approach: A case study from Ghurmi-Dhad Khola area, Eastern Nepal. J. Geol. Soc. India 2013, 82, 249–261. [CrossRef] 51. Lu, J.; Hu, S.; Niu, Z.; You, R. The Application of Fuzzy Comprehensive Evaluation Model in Landslide Prediction. In Proceedings of the 2010 3rd International Conference on Information Management, Innovation Management and Industrial Engineering, Kunming, China, 26–28 November 2010; Volume 4, pp. 612–615. 52. Mallick, J.; Singh, R.K.; Alawadh, M.A.; Islam, S.; Khan, R.A.; Qureshi, M.N. GIS-based landslide susceptibility evaluation using fuzzy-AHP multi-criteria decision-making techniques in the Abha Watershed, Saudi Arabia. Environ. Earth Sci. 2018, 77, 276. [CrossRef] 53. Mohsin, S.; Azam, F. Computational seismic algorithmic comparison for earthquake prediction. Int. J. Geol. 2011, 5, 53–59. 54. Sengar, S.S.; Kumar, A.; Ghosh, S.K.; Wason, H.R.; Raju, P.L.N.; Murthy, Y.V.N.K. Earthquake-induced built-up damage identification using fuzzy approach. Geomat. Nat. Hazards Risk 2013, 4, 320–338. [CrossRef] 55. Sun, D.; Sun, B. Rapid prediction of earthquake damage to buildings based on fuzzy analysis. In Proceedings of the 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery, Yantai, China, 10–12 August 2010; Volume 3, pp. 1332–1335. 56. Huang, L.; Xiang, L.-Y. Method for Meteorological Early Warning of Precipitation-Induced Landslides Based on Deep Neural Network. Neural Process. Lett. 2018, 48, 1243–1260. [CrossRef] 57. Li, W.; Narvekar, N.; Nakshatra, N.; Raut, N.; Sirkeci, B.; Gao, J. Seismic Data Classification Using Machine Learning. In Proceedings of the 2018 IEEE Fourth International Conference on Big Data Computing Service and Applications (BigDataService), Bamberg, Germany, 26–29 March 2018; pp. 56–63. http://dx.doi.org/10.3390/ijgi8100462 http://dx.doi.org/10.1785/0220180315 http://dx.doi.org/10.1029/2018GL081251 http://dx.doi.org/10.1785/0220180259 http://dx.doi.org/10.3390/geosciences9070308 http://dx.doi.org/10.1785/0120180074 http://dx.doi.org/10.3390/en12071301 http://dx.doi.org/10.3390/w10111536 http://dx.doi.org/10.3390/en12061049 http://dx.doi.org/10.3390/su11061663 http://dx.doi.org/10.1109/ACCESS.2019.2902224 http://dx.doi.org/10.1007/s12594-013-0147-y http://dx.doi.org/10.1007/s12665-018-7451-1 http://dx.doi.org/10.1080/19475705.2012.746242 http://dx.doi.org/10.1007/s11063-017-9778-0 Sustainability 2020, 12, 2420 30 of 32 58. Asim, K.M.; Idris, A.; Iqbal, T.; Martínez-Álvarez, F. Earthquake prediction model using support vector regressor and hybrid neural networks. PLoS ONE 2018, 13, e0199004. [CrossRef] 59. Asencio–Cortés, G.; Morales-Esteban, A.; Shang, X.; Martínez–Álvarez, F. Earthquake prediction in California using regression algorithms and cloud-based big data infrastructure. Comput. Geosci. 2018, 115, 198–210. [CrossRef] 60. Hoang, N.-D.; Bui, D.T.T. Predicting earthquake-induced soil liquefaction based on a hybridization of kernel Fisher discriminant analysis and a least squares support vector machine: A multi-dataset study. Bull. Int. Assoc. Eng. Geol. 2016, 77, 191–204. [CrossRef] 61. Gitis, V.G.; Derendyaev, A. Web-Based GIS platform for automatic prediction of earthquakes. In Proceedings of the International Conference on Computational Science and Its Applications, Melbourne, Australia, 2–5 July 2018; Springer: Berlin/Heidelberg, Germany, 2018; pp. 268–283. 62. Thomas, S.; Pillai, G.; Pal, K. Prediction of peak ground acceleration using �-SVR, ν-SVR and Ls-SVR algorithm. Geomatics, Nat. Hazards Risk 2016, 8, 177–193. [CrossRef] 63. Rouet-Leduc, B.; Hulbert, C.; Lubbers, N.; Barros, K.; Humphreys, C.J.; Johnson, P.A. Machine Learning Predicts Laboratory Earthquakes. Geophys. Res. Lett. 2017, 44, 9276–9282. [CrossRef] 64. Rafiei, M.H.; Adeli, H. NEEWS: A novel earthquake early warning model using neural dynamic classification and neural dynamic optimization. Soil Dyn. Earthq. Eng. 2017, 100, 417–427. [CrossRef] 65. Asencio–Cortés, G.; Scitovski, S.; Scitovski, R.; Martínez–Álvarez, F. Temporal analysis of croatianseismogenic zones to improve earthquake magnitude prediction. Earth Sci. Inform. 2017, 10, 303–320. [CrossRef] 66. Rahmani, M.E.; Amine, A.; Hamou, R.M. A Novel Bio Inspired Algorithm Based on Echolocation Mechanism of Bats for Seismic States Prediction. Int. J. Swarm Intell. Res. 2017, 8, 1–18. [CrossRef] 67. Asencio-Cortés, G.; Martínez-Álvarez, F.; Troncoso, A.; Morales-Esteban, A. Medium–large earthquake magnitude prediction in Tokyo with artificial neural networks. Neural Comput. Appl. 2015, 28, 1043–1055. [CrossRef] 68. Asim, K.M.; Idris, A.; Martinez-Alvarez, F.; Iqbal, T. Short Term Earthquake Prediction in Hindukush Region Using Tree Based Ensemble Learning. In Proceedings of the 2016 International Conference on Frontiers of Information Technology (FIT), Islamabad, Pakistan, 19–21 December 2016; pp. 365–370. 69. Yang, D.; Yang, K. Multi-step prediction of strong earthquake ground motions and seismic responses of SDOF systems based on EMD-ELM method. Soil Dyn. Earthq. Eng. 2016, 85, 117–129. [CrossRef] 70. Vahaplar, A.; Tezel, B.T.; Nasiboglu, E.; Nasibov, E. A monitoring system to prepare machine learning data sets for earthquake prediction based on seismic-acoustic signals. In Proceedings of the 2015 9th International Conference on Application of Information and Communication Technologies (AICT), Rostov on Don, Russia, 14–16 October 2015; pp. 44–47. 71. Buscema, P.M.; Massini, G.; Maurelli, G. Artificial Adaptive Systems to predict the magnitude of earthquakes. Bollettino di GeofisicaTeorica ed Applicata 2015, 56, 227–256. 72. Kamogawa, M.; Nanjo, K.; Izutsu, J.; Orihara, Y.; Nagao, T.; Uyeda, S. Nucleation and Cascade Features of Earthquake Mainshock Statistically Explored from Foreshock Seismicity. Entropy 2019, 21, 421. [CrossRef] 73. Reyes, J.; Morales-Esteban, A.; Martínez-Álvarez, F. Neural networks to predict earthquakes in Chile. Appl. Soft Comput. 2013, 13, 1314–1328. [CrossRef] 74. Farrokhzad, F.; Choobbasti, A.; Barari, A. Liquefaction microzonation of Babol city using artificial neural network. J. King Saud Univ.-Sci. 2012, 24, 89–100. [CrossRef] 75. Gu, T.-F.; Wang, J.-D. Application of fuzzy neural networks for predicting seismic subsidence coefficient of loess subgrade. In Proceedings of the 2010 Sixth International Conference on Natural Computation, Yantai, China, 10–12 August 2010; Volume 3, pp. 1556–1559. 76. Korkmaz, K.A.; Demir, F. Ground Motion Data Profile of Western Turkey with Intelligent Hybrid Processing. Pure Appl. Geophys. 2016, 174, 293–303. [CrossRef] 77. Dutta, P.K.; Mishra, O.P.; Naskar, M.K. Decision analysis for earthquake prediction methodologies: Fuzzy inference algorithm for trust validation. Int. J. Comput. Appl. 2012, 45, 13–20. 78. Mishra, O.P. Seismological research in India. Proc. Indian Natl. Sci. Acad. 2012, 76, 361–375. 79. Pandit, A.R.; Biswal, K.C. Prediction of earthquake magnitude using adaptive neuro fuzzy inference system. Earth Sci. Inform. 2019, 12, 513–524. [CrossRef] 80. Pham, B.T.; Bui, D.T.T.; Pham, H.V.; Le, H.Q.; Prakash, I.; Dholakia, M.B. Landslide Hazard Assessment Using Random SubSpace Fuzzy Rules Based Classifier Ensemble and Probability Analysis of Rainfall Data: http://dx.doi.org/10.1371/journal.pone.0199004 http://dx.doi.org/10.1016/j.cageo.2017.10.011 http://dx.doi.org/10.1007/s10064-016-0924-0 http://dx.doi.org/10.1080/19475705.2016.1176604 http://dx.doi.org/10.1002/2017GL074677 http://dx.doi.org/10.1016/j.soildyn.2017.05.013 http://dx.doi.org/10.1007/s12145-017-0295-5 http://dx.doi.org/10.4018/IJSIR.2017070101 http://dx.doi.org/10.1007/s00521-015-2121-7 http://dx.doi.org/10.1016/j.soildyn.2016.03.015 http://dx.doi.org/10.3390/e21040421 http://dx.doi.org/10.1016/j.asoc.2012.10.014 http://dx.doi.org/10.1016/j.jksus.2010.09.003 http://dx.doi.org/10.1007/s00024-016-1379-8 http://dx.doi.org/10.1007/s12145-019-00397-w Sustainability 2020, 12, 2420 31 of 32 A Case Study at Mu Cang Chai District, Yen Bai Province (Viet Nam). J. Indian Soc. Remote. Sens. 2016, 45, 673–683. [CrossRef] 81. Polykretis, C.; Chalkias, C.; Ferentinou, M. Adaptive neuro-fuzzy inference system (ANFIS) modeling for landslide susceptibility assessment in a Mediterranean hilly area. Bull. Int. Assoc. Eng. Geol. 2017, 78, 1173–1187. [CrossRef] 82. Razifard, M.; Shoaei, G.; Zaré, M. Application of fuzzy logic in the preparation of hazard maps of landslides triggered by the twin Ahar-Varzeghan earthquakes (2012). Bull. Int. Assoc. Eng. Geol. 2018, 78, 223–245. [CrossRef] 83. Pourghasemi, H.R.; Pradhan, B.; Gokceoglu, C. Application of fuzzy logic and analytical hierarchy process (AHP) to landslide susceptibility mapping at Haraz watershed, Iran. Nat. Hazards 2012, 63, 965–996. [CrossRef] 84. Pradhan, B. Use of GIS-based fuzzy logic relations and its cross application to produce landslide susceptibility maps in three test areas in Malaysia. Environ. Earth Sci. 2010, 63, 329–349. [CrossRef] 85. Zhang, Y.; Oldenburg, C.M.; Finsterle, S. Percolation-theory and fuzzy rule-based probability estimation of fault leakage at geologic carbon sequestration sites. Environ. Earth Sci. 2009, 59, 1447–1459. [CrossRef] 86. Meten, M.; Bhandary, N.P.; Yatabe, R. Application of GIS-based fuzzy logic and rock engineering system (RES) approaches for landslide susceptibili mapping in Selelkula area of the Lower Jema River Gorge, Central Ethiopia. Environ, Earth Sci. 2015, 74, 3395–3416. [CrossRef] 87. Thomas, S.; Pillai, G.N.; Pal, K.; Jagtap, P. Prediction of ground motion parameters using randomized ANFIS (RANFIS). Appl. Soft Comput. 2016, 40, 624–634. [CrossRef] 88. Wu, A. Design and practice of a digital seismic waveform analyzing tool. In Proceedings of the 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, Chongqing, China, 29–31 May 2012; pp. 2299–2303. 89. Ismail, N.; Khattak, N. Reconnaissance Report on the Mw 7.5 Hindu Kush Earthquake of 26th October 2015 and the Subsequent Aftershocks; United Arab Emirates University: Al Ain, UAE, 2015. 90. Mignan, A. Retrospective on the Accelerating Seismic Release (ASR) hypothesis: Controversy and new horizons. Tectonophysics 2011, 505, 1–16. [CrossRef] 91. Keilis-Borok, V. Earthquake prediction: State-of-the-art and emerging possibilities. Annu. Rev. Earth Planet. Sci. 2002, 30, 1–33. [CrossRef] 92. Mignan, A. The debate on the prognostic value of earthquake foreshocks: A meta-analysis. Sci. Rep. 2014, 4, 4099. [CrossRef] [PubMed] 93. Wyss, M. Evaluation of proposed earthquake precursors. EOS Trans. Am. Geophys. Union 1991, 72, 411. [CrossRef] 94. Mignan, A. A preliminary text classification of the precursory accelerating seismicity corpus: Inference on some theoretical trends in earthquake predictability research from 1988 to 2018. J. Seism. 2019, 23, 771–785. [CrossRef] 95. Cicerone, R.D.; Ebel, J.E.; Britton, J. A systematic compilation of earthquake precursors. Tectonophysics 2009, 476, 371–396. [CrossRef] 96. Azimi, Y.; Khoshrou, S.H.; Osanloo, M. Prediction of blast induced ground vibration (BIGV) of quarry mining using hybrid genetic algorithm optimized artificial neural network. Measurement 2019, 147, 106874. [CrossRef] 97. Mehrgini, B.; Izadi, H.; Memarian, H. Shear wave velocity prediction using Elman artificial neural network. Carbonates Evaporites 2017, 34, 1281–1291. [CrossRef] 98. Udegbe, E.; Morgan, E.; Srinivasan, S. Big data analytics for seismic fracture identification using amplitude-based statistics. Comput. Geosci. 2019, 23, 1277–1291. [CrossRef] 99. Sharma, S.; Venkateswarlu, H.; Hegde, A. Application of Machine Learning Techniques for Predicting the Dynamic Response of Geogrid Reinforced Foundation Beds. Geotech. Geol. Eng. 2019, 37, 4845–4864. [CrossRef] 100. Fanos, A.M.; Pradhan, B. A Novel Hybrid Machine Learning-Based Model for Rockfall Source Identification in Presence of Other Landslide Types Using LiDAR and GIS. Earth Syst. Environ. 2019, 3, 491–506. [CrossRef] 101. Břizová, L.; Kříž, J.; Studnička, F.; Slegr, J. Methods for the Evaluation of the Stochastic Properties of the Ionosphere for Earthquake Prediction—Random Matrix Theory. Atmosphere 2019, 10, 413. [CrossRef] http://dx.doi.org/10.1007/s12524-016-0620-3 http://dx.doi.org/10.1007/s10064-017-1125-1 http://dx.doi.org/10.1007/s10064-018-1235-4 http://dx.doi.org/10.1007/s11069-012-0217-2 http://dx.doi.org/10.1007/s12665-010-0705-1 http://dx.doi.org/10.1007/s12665-009-0131-4 http://dx.doi.org/10.1007/s12665-015-4377-8 http://dx.doi.org/10.1016/j.asoc.2015.12.013 http://dx.doi.org/10.1016/j.tecto.2011.03.010 http://dx.doi.org/10.1146/annurev.earth.30.100301.083856 http://dx.doi.org/10.1038/srep04099 http://www.ncbi.nlm.nih.gov/pubmed/24526224 http://dx.doi.org/10.1029/90EO10300 http://dx.doi.org/10.1007/s10950-019-09833-2 http://dx.doi.org/10.1016/j.tecto.2009.06.008 http://dx.doi.org/10.1016/j.measurement.2019.106874 http://dx.doi.org/10.1007/s13146-017-0406-x http://dx.doi.org/10.1007/s10596-019-09890-z http://dx.doi.org/10.1007/s10706-019-00945-7 http://dx.doi.org/10.1007/s41748-019-00114-z http://dx.doi.org/10.3390/atmos10070413 Sustainability 2020, 12, 2420 32 of 32 102. Andrén, M.; Stockmann, G.; SkeltoniD, A.; Sturkell, E.; Mörth, C.; Guðrúnardóttir, H.R.; Keller, N.S.; Odling, N.; Dahrén, B.; Broman, C.; et al. Coupling between mineral reactions, chemical changes in groundwater, and earthquakes in Iceland. J. Geophys. Res. Solid Earth 2016, 121, 2315–2337. [CrossRef] 103. Sarlis, N.; Skordas, E.S.; Varotsos, P. Natural Time Analysis: Results Related to Two Earthquakes in Greece during 2019. Proceedings 2019, 24, 20. 104. Tareen, A.D.K.; Asim, K.M.; Kearfott, K.; Rafique, M.; Nadeem, M.S.A.; Iqbal, T.; Rahman, S.U. Automated anomalous behaviour detection in soil radon gas prior to earthquakes using computational intelligence techniques. J. Environ. Radioact. 2019, 203, 48–54. [CrossRef] [PubMed] 105. Orihara, Y.; Kamogawa, M.; Nagao, T. Preseismic Changes of the Level and Temperature of Confined Groundwater related to the 2011 Tohoku Earthquake. Sci. Rep. 2014, 4, 6907. [CrossRef] 106. SkeltoniD, A.; Andrén, M.; Kristmannsdóttir, H.; Stockmann, G.; Mörth, C.-M.; Sveinbjörnsdóttir, A.; Jónsson, S.; Sturkell, E.; Guðrúnardóttir, H.R.; Hjartarson, H.; et al. Changes in groundwater chemistry before two consecutive earthquakes in Iceland. Nat. Geosci. 2014, 7, 752–756. [CrossRef] © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). http://dx.doi.org/10.1002/2015JB012614 http://dx.doi.org/10.1016/j.jenvrad.2019.03.003 http://www.ncbi.nlm.nih.gov/pubmed/30861489 http://dx.doi.org/10.1038/srep06907 http://dx.doi.org/10.1038/ngeo2250 http://creativecommons.org/ http://creativecommons.org/licenses/by/4.0/. Introduction Background Fuzzy Expert System (FES) Rule Based Expert System (RBES) Neuro Fuzzy Expert System (NFES) Machine Learning (Ml) Research Methodology Defining Research Questions Search and Selection Strategy Identification of Search String Screening and Selection Criteria Data Extraction and Synthesis Analysis Basic Analysis Key Facts of Expert System Based Earthquake Prediction Publications ResearchType Facets Addressed by the Identified Publications ES type and System Specific Key Aspects of Proposed ES Quality Assessment Discussion Comparitive Analysis of Methods Principal Findings Evolution of Tools and Techniques Conclusion and Future Directions References