A novel Artificial Immune System for fault behavior detection Expert Systems with Applications 38 (2011) 6957–6966 Contents lists available at ScienceDirect Expert Systems with Applications j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / e s w a A novel Artificial Immune System for fault behavior detection C.A. Laurentys a, R.M. Palhares b,⇑, W.M. Caminhas b a Department of Electrical Engineering, Federal University of Minas Gerais, Av. Antônio Carlos 6627, 31270-010 Belo Horizonte, MG, Brazil b Department of Electronics Engineering, Federal University of Minas Gerais, Av. Antônio Carlos 6627, 31270-010 Belo Horizonte, MG, Brazil a r t i c l e i n f o a b s t r a c t Keywords: Artificial intelligence Artificial Immune System Fault detection Immune system Model-based fault diagnosis 0957-4174 � 2010 Elsevier Ltd. doi:10.1016/j.eswa.2010.12.019 ⇑ Corresponding author. Tel.: +55 (31) 3409 3457; E-mail address: palhares@cpdee.ufmg.br (R.M. Pal Open access under the El This paper presents an error detection methodology to enable fault detection inspired on recent immune theory. The fault detection problem is a challenging problem due to processes increasing complexity and agility necessary to avoid malfunction or accidents. The key challenge is determining the difference between normal and potential harmful activities. A promising solution is emerging in the form of Artifi- cial Immune System (AIS). In this article, Natural Killer (NK) immune cells mechanisms inspired an AIS. The AIS proposed uses recent biological mechanism such as: NK activation and education machinery. DAMADICS benchmark was applied to compare the proposed AIS performance to others fault detection algorithms. The results show that the novel approach developed provides better detection rate and false alarms tradeoff when compared to other methods in literature. � 2010 Elsevier Ltd. Open access under the Elsevier OA license. 1. Introduction Brazil has been facing an increasing rate of incidence related to manufacturing facilities, reaching more than 234 thousands in 2007. According to MPS studies between 1999 and 2007, the an- nual accidents related to manufacturing lays around 35% of total accidents (MPS, 2009). In 2007, for instance, manufacturing facilities accidents were the major cause of accidents as depicted by Fig. 1. Industrial statistics shows that 70% of the accidents are caused by human mistakes and operators need decision making tools to create a more trustable industrial environment (Venkatasubrama- nian et al., 2003). In this context, fault detection is one of the vital components for the Abnormal Event Management (AEM) Luo, Zhang, Zhao, & Zhang, 2004; Venkatasubramanian et al., 2003. AEM becomes challenging due to the size and complexity of proce- dures and the broad activities scope, encompassing a variety of factors such as: degradation of the process, inadequate, incomplete and not reliable measurements. The AEM has been turning into a challenging problem due to the size and complexity of procedures and the broad activities scope. Today’s challenge is the AEM fault detection automation, and the usage of computer systems to improve it, applying artificial intelligence methods such as fuzzy (Mendonca, Sousa, & Sa da Costa, 2009; Saravanan, Cholairajan, & Ramachandran, 2009) and neural networks (Tang & Yao, 2008; Wu & Liu, 2009). Natural Computing (De Castro, 2005; Tang & Yao, 2008), more specifically, Artificial Immune System (AIS) Timmis et al., 2008, has been sup- porting this context (Gan, Zhao, & Chow, 2009; Ji & Dasgupta, fax: +55 (31) 3409 4850. hares). sevier OA license. 2009). The AIS have emerged from attempts to model and apply immunological principles or inspirations into the development of new computational tools. Artificial Immune Systems (AIS) are de- fined as a new computational paradigm based on metaphors of the biological immune systems. Immune-based techniques have been applied in a wide area of applications such as intrusion detection (Powers & He, 2008), opti- mization algorithms (Paquete & Stutzle, 2009), robust control (Gui- maraes, Palhares, Campelo, & Igarashi, 2007) and others (DeCastro and Von Zuben, 2009). The powerful information processing capabil- ity, pattern recognition, learning, memory and immune distributive nature provide rich metaphors for its artificial (computational) counterpart. The novel technique is based on a synergy of Immunological inspiration, its formalization using Natural Computing , and the Fault detection context as shown by Fig. 2. In this article the Fault detection block shown by Fig. 2 defined the performance indicators to allow the comparison with literature algorithms; immunology provided the inspiration to develop the AIS fault detection scheme named NKC Algorithm and Natural Computing act a driver to translate the immune inspirations into algorithms. Firstly, it is important to stress that the key article contribution is an AIS inspired on recent literature of Natural Killer (NK) cell mechanisms (Lanier, 2008; Larrea, Alvarez, Soto, Vazquez, & Gonz- alez, 2008; Luci & Tomasello, 2008) to enable AEM fault detection. Secondly, the novel AIS is a population-based algorithm. The immune system has a set of NK cells that dynamically interacts to reach an overall outcome and it is not based on a single NK cell as proposed by other literature models (Elmeziane, Berrada, & Kas- sou, 2008). http://dx.doi.org/10.1016/j.eswa.2010.12.019 mailto:palhares@cpdee.ufmg.br http://dx.doi.org/10.1016/j.eswa.2010.12.019 http://www.sciencedirect.com/science/journal/09574174 http://www.elsevier.com/locate/eswa http://www.elsevier.com/open-access/userlicense/1.0/ http://www.elsevier.com/open-access/userlicense/1.0/ 0 100 200 300 400 500 600 700 19 99 20 00 20 01 20 02 20 03 20 04 20 05 20 06 20 07 # Total #Manufacturing 35.91% 15.29% 7.07% 6.62% 5.93% 5.31% 12.90% 2.72% 8.25% 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 35.00% 40.00% Ma nu fac tur ing Se rvi ce s Co mm erc e Bu ild ing He alt hy Tr an sp or tat ion Ign or ed Pr im ary Se cto r Ot he rs (a) (b) T ho us an ds Fig. 1. (a) Accidents chart showing the number of total accidents and only the ones related to the manufacturing environment during 1999 and 2007 (last year published by MPS). (b) MPS accidents reasons pareto chart regarding 2007: Manufacturing leads the key root cause of Brazilian accidents. Fig. 2. Immune inspired fault detection: A dynamic feedback between fault detection, immunology and natural computing are the major components adopted in this article. 6958 C.A. Laurentys et al. / Expert Systems with Applications 38 (2011) 6957–6966 Finally, this novel proposition is applied to a DAMADICS bench- mark (Barty, 2006) resulting with a better detection rate and false alarms tradeoff when compared to other methods in literature. The article is structured in the following sections: � Natural Killer Immune Cell: describes Natural Killer Cells features that inspired the AIS, � Materials and Methods: presents analogies between the pro- posed AIS and NK cells, � Calculation: describes the DAMADICS benchmark used for AIS tests, � Results and Discussions: discusses algorithm results and compare its performance, � Conclusions: points out the major benefits and AIS improvements. 2. Natural killer immune cell This section describes Natural Killer cells recent immune litera- ture and details key features that inspired the proposed AIS. 2.1. NK cell immune overview NK cells are an essential component of immunity against tu- mour and viral infected cells (Raulet, Vance, & McMahon, 2001). NK cells are involved in early defenses (Luci & Tomasello, 2008). In the context of engineering, i.e., in error detection, this specific characteristic is useful once it is desired to detect a fault as soon as it began, especially in the case of incipient faults. The NK cells triggering is regulated by activating and inhibitory surface cell receptors (Lanier, 2008; Vivier, Tomasello, Baratin, Walzer, & Ugolini, 2008). It means that NK cell machinery is com- posed by activating and inhibitory molecules that are able to con- nect them into target cells surface molecules. NK cells detect changes in the target cell surface – malignant transformation or infection – resulting in loss or gain of molecules that are detected by the NK cell surface receptors. Based on the features described, the NK cell immune model that defines the NK lifecycle is described in the following steps: � Initiation of receptors expression: is described by generation of NK cells inhibitory and activation receptors through a stochastic process to distribute the NK Cell receptors – this procedure will generate immature NK cells, � Education process: aims to maximize the discriminatory proper- ties of the immature NK Cells receptors maximizing its discrim- inatory properties – this procedure will generate mature NK cells, � Recognition phase: mature NK cells interacts with others cells applying the balance between negative and positive signals; they may be triggered (generating activated NK cells) or not (generating inactivated NK cells). If it is triggered, it releases molecular substances called cytokine to signal it to other cells. These key steps are further detailed in the next sections. 2.2. NK Immune cell receptors initiation The NK Cell receptors initiation is a process that will produce the NK Cells receptors. According to literature the receptors are generated by a stochastic mechanism. A model found in the litera- ture (Lanier, 2008) describes it as an adjustment mechanism based on ‘‘At least one theory’’ and ‘‘Zero sum balance’’. The ‘‘At least one’’ hypothesis is imposed to describe NK cells developmental process. According to this theory, to achieve self- tolerance (Raulet & Vance, 2006) (does not activate improperly) NK cells must have at least one inhibitory receptor. A process must equip the NK cell with at least one inhibitory receptor; otherwise it will be activated improperly. C.A. Laurentys et al. / Expert Systems with Applications 38 (2011) 6957–6966 6959 The ‘‘Zero sum balance’’ fundamentals are illustrated by Fig. 3. The shaded section indicates a stimulatory signal. According to (Luci & Tomasello, 2008), the NK cells development states are: � Mature – has already achieved ‘‘Zero sum balance’’ (e.g. see Fig. 2 cell ‘‘A’’) through inhibitory and activating receptors balance, � Immature or hyporesponsive (e.g. see Fig. 2 cell ‘‘B’’) – NK cell will not activate due to absence of inhibitory or activating receptors, � Possible mature (e.g. see Fig. 2 cell ‘‘C’’) – might achieve the ‘‘Zero sum balance’’ through a further education process. 2.3. NK immune cell education process Although a stochastic mechanism appears to underlie the initial expression of the receptors, the final functional NK cell repertoire is shaped by an education process (Raulet & Vance, 2006). The goal of the education process is to maximize the discrimina- tory properties of the NK cells. Their ultimate effects are to ensure a repertoire of NK cell is both self-tolerant and useful. The self-toler- ant property avoids NK cell inappropriate activation, specifically to activate in a normal situation, destroying cells of the own organism. There are evidences that the education mechanism maximizes the NK cells discrimination properties but this process and its machinery is still poorly understood (Raulet et al., 2001). 2.4. NK immune cell recognition phase NK cells functions are regulated by a dynamic balance between negative and positive signals initiated after engagement of cell sur- face inhibitory and activating receptors (Raulet et al., 2001). A dy- namic balance is illustrated by Fig. 4. The states of a mature NK cell are: � Triggered by Absence of Inhibitory Signals: NK Cell receptors interacted with target cells receptors and it was triggered by absence of target cell inhibitory receptors, � Triggered by Activating Signals: NK Cell receptors interacted with target cell stimulatory receptors and the NK cell was triggered by the presence of stimulatory receptors, � Not Triggered Region: NK cell receptors interacted with target cell inhibitory receptors and the NK cell will not be triggered. Fig. 3. Model for stimulatory and inhibitory initiation. Cell ‘‘A’’ has achieved balance in the amplitudes of positive and negative signals (influenced by the number of receptors, ligand affinities) and is therefore maximally sensitive to changes in host cell. Cell ‘‘B’’ lacks inhibitory receptors; these cells are rendered hyporesponsive. Cell ‘‘C’’ has both stimulatory and inhibitory receptors but has not reach Zero sum balance. It is important to stress the role of innate cytokines – a molec- ular communication substance released by innate cells – is a mech- anism that might alter a predefined dynamic balance. In general, NK cells are able to release different types of cytokine depending on the molecular interaction with other cells. The dynamic balance of a particular NK cell might be altered by innate cytokines re- leased by other cells as illustrated by Fig. 4 (d) facilitating the trig- ger or turning it more difficult. 3. Materials and methods The Section 2 described NK cells immune mechanisms and its machinery that inspired an Artificial Immune System (AIS) for fault detection. The AIS proposed in this article was inspired on the NK cells described on item II. The sources of inspiration were: (a) Initiation of the NK Cells receptors: It is generated stochastically using the ‘‘At least one’’ hypothesis and the ‘‘Zero sum balance’’, (b) Education process: NK immune response is guided by a dynamic balance of two signal receptors – activating and inhibitory. NK receptors are adjusted by an education mech- anism to maximize its performance, (c) Recognition phase: The generated NK cells will monitor the other cells in other to verify if they are activated based on their inhibitory and activating receptors. The NK cells immune context is interpreted as an analogy to propose a fault detection AIS. In this scenario, the normal and anomalous available patterns will behave as inhibitory and activat- ing signal of the fault detection methodology. Therefore, the nor- mal patterns are understood as inhibitory receptors and the anomalous ones as activating receptors. A stochastic mechanism will initiate NK cells population recep- tors based on the available patterns (normal and abnormal). Fur- ther, an education process will be necessary to optimize the NK cell population. Fig. 5 summarizes key analogies of the NK cells im- mune system mechanisms inspirations. The Methodology section will detail how each of these steps were artificially developed generating the AIS proposed. 3.1. Initiation of the NK cells receptor This phase applies a metaphor for how stimulatory and inhibi- tory interactions of the NK cell receptors are integrated during the receptor acquisition phase of NK cell development generating immature NK cells. Fig. 6 describes the key steps of the Artificial NK Receptors Ini- tiation algorithm which is immune inspired in the initiation of the NK Cells receptors. It is important to mention the biological analogy with the algo- rithm using the ‘‘At least one’’ hypothesis and the ‘‘Zero sum mod- el’’ for initiating the NK Cells receptors expression (Raulet et al., 2001). The artificial At least one algorithm represented by block (1) at Fig. 6 forces a NK Cell receptor repertoire with at least one inhibitory receptor in order to create responsive NK cells (Matured or possible Matured). The artificial Zero sum algorithm represented by block (2) at Fig. 6 defines the NK cell activation region. It is firstly based on the number of receptors as a first indicative of Zero sum balance. In this case the type of the receptor alternate between {F} and {N} using a random choice, producing a balanced set of inhibitory and activating receptors. Secondly, it considers the affinity be- tween the receptors of inhibitory and activating to define NK cell response. The Zero sum model is represented by a line equation that fits the average Euclidian distance points between a specific inhibitory receptor and all activating receptors. Fig. 4. NK cell recognition phase. Left panels show the NK cell balances. Activating target cells often down regulate inhibitory – polygons (b) and up regulate activating receptors – circles (c). They also induce innate cytokines that bias NK-cell recognition and signaling in favor of triggering (d). Innate cytokines (stars and sliding weight) alter the equilibrium signaling machinery. Right panels shows interaction between the two cells. 6960 C.A. Laurentys et al. / Expert Systems with Applications 38 (2011) 6957–6966 These cells might need a further educational process to balance the strengths of each inhibitory and activate signaling and there- fore are considered immature NK cells population. In order to illustrate the stochastic mechanism of NK cell forma- tion a two class problem, a two dimensional space, is depicted at Fig. 7. It should be stressed that the number of NK cells activating and inhibitory receptors will be equal and the NK cells generated will have at least one inhibitory receptor and lines to define the Zero sum balance of it. The algorithm takes as input: � Fault Set ‘‘f’’ – [x1 . . . xn]: These are the n process variables val- ues affected when the fault occurs. These components will gen- erate the NK cells activating receptors. � Normal Set ‘‘n’’ - [x1 . . . xn]: These are the same n process vari- ables values in normal conditions. These components will gen- erate the NK cells inhibitory receptors. The key algorithm parameters are: � Cells_Pop –‘‘m’’: It is a constant real number defined by the user. It defines the number of Immature Artificial NK cells that the population will have. � ReceptorsbyCell – ‘‘r’’: It is a constant real number defined by the user. It represents the number of receptors (inhibitory and acti- vating) that the NK cells will posse. � Size_Receptors – ‘‘s’’: It is a constant real number defined by the user. It represents the length of the receptor generated. Fig. 5. Comparison of the NK cell immune model and the AIS key steps. C.A. Laurentys et al. / Expert Systems with Applications 38 (2011) 6957–6966 6961 3.2. Education mechanism The education mechanism goal is to maximize the NK popula- tion discriminatory properties such as NK cell self-tolerance and used by stochastic mechanisms. The education mechanism gener- ates mature NK Cells through the immature population. The Artificial Education Mechanism algorithm proposed consid- ers the false positive rate as a self-tolerance measurement and individual/population performance as usefulness measurement. The algorithm is presented at Fig. 8 along its key parameters. The key of the Artificial Education Mechanism algorithm is repre- sented by block (1) in Fig. 8, where the discriminatory properties (self-tolerance, performance and maturation) are assessed. Fig. 6. Artificial NK Receptors Initiation algorithm – Implements the stochastic initiation of the NK cells receptors. Firstly, a threshold mature is checked to verify if the minimum performance is reached otherwise the cell will be discarded. Secondly, the Artificial Education Mechanism algorithm considers the NK cell self-tolerance, meaning in this context that cells will not be triggered improperly, generating false alarms. Therefore, a mechanism of avoiding a NK cell detecting own inhibitory recep- tors as activating receptors is implemented. In this case, the algo- rithm will erase the artificial Zero sum model (line equations) that triggers in inappropriate situation. If all zero sum models are not self tolerant, the cell is deleted. Finally, the performance is assessed as being the NK cell useful- ness. NK cell correct triggers for normal and fault situation, consid- ering own cell receptors and a disjoint validation set of known receptors, are applied. Fig. 7. Example of receptors generated through Artificial NK Receptors Initiation algorithm. The activating receptors marked with ‘‘+’’ refer to anomalous conditions and the inhibitory receptors (‘‘�’’) to normal conditions. The circles around the receptors indicate the stochastic receptors selected by Artificial NK Receptors Initiation algorithm (activating and inhibitory receptors). The lines marked with ‘‘�’’ indicate the zero sum lines generated. Fig. 8. Artificial Education Mechanism algorithm – Implement the educational process to maximize performance of the NK cell population. Fig. 9. (a) Illustrative example of a mature NK cell generated. The activating receptors ‘‘+’’ refers to possible anomalous situation and the inhibitory receptors marked with ‘‘�’’. The circles in the lines (zero sum models) indicated the Zero sum self-tolerant marked with circles. (b) A NK cell is only added to the mature population if it increases overall population performance. In this example, the population has four mature NK cells. 6962 C.A. Laurentys et al. / Expert Systems with Applications 38 (2011) 6957–6966 The NK cell will be added to the mature population if it im- proves the overall population performance, turning into a mature NK cell (see Fig. 9 for details). This constraint aims to create a pop- ulation that does not have redundant knowledge, and always increasing its capacity to improve performance with a restricted number of NK cells (parameter Max_cell). The key algorithm parameters are: � Max_Cells – ‘‘max’’: its constant real number defined by the user. It defines the maximum number of Immature Artificial NK cells that the mature population will have. � Threshold_mature – ‘‘tr’’: its constant real number defined by the user. It represents minimum NK cell performance to join the mature NK cell population. � Attempts_to_join – ‘‘aj’’: its constant real number defined by the user. It represents the number of attempts to maturate NK cells. It is a stop criterion of the education mechanism. 3.3. Recognition phase The mature NK cells population will interact with unknown cell receptors that it desires to classify in safe or dangerous situa- tion using the interaction machinery described by Fig. 3. The cells will interact with the stimulatory and inhibitory signal from the target cell generating: � Innate cytokine – based on the Euclidian distance of the target cell receptors and its Zero sum balances: positive (above Zero sum balance) and negative (below it) signals. � Status (triggered or not triggered) – triggered when positive innate cytokines are greater than negative innate cytokines, otherwise set as not triggered. The fault detection AIS considers fault situation using the majority vote of the mature NK cells status indicates triggered, otherwise the fault detection AIS indicates normal condition as shown by Fig. 10. 4. Calculation In this paper the DAMADICS benchmark is used to validate the proposed methodology and to compare its performance to other fault detection methods. The benchmark consists of an actuator controlling the water in- put flow in a boiler. The boiler presented is part of a process of evaporation comprised of five stages of a sugar factory in Poland. The benchmark was developed by a research group in Europe called DAMADICS: Development and Application Methods for Diagnosis of Actuator in Industrial Control Systems. The process actuator is composed of three elements: a control valve V, the pneumatic servo motor S and the positioner P. The valve V controls the flow of water that passes through a pipe feed- ing the boiler. The pneumatic servo motor S set the place of valve V plug’s in order to act on the rate of flow of water. The positioner is the device used to correct a mispositioning of the shaft of the en- gine, caused by internal or external sources, such as: friction, vari- ations in pressure supplied to the servo motor and so on. Fig. 11 illustrates the schematic diagram of the actuator. Fig. 10. (a) The proposed AIS is a population-based algorithm that allows interaction between the NK cells output. Fig. 11. DAMADICS industrial process schematic diagram. Table 1 Input variables applied to the model. Input Range Unit Description GV [0 1] % Output signal from controller PI – Pa The value of the input pressure P2 – Pa The value of the output pressure Tl – C Fluid temperature Table 2 Output variables applied to the model. Output Range Unit Description F – t/h Medium flow X [0 100] % Disturbed value of rod displacement C.A. Laurentys et al. / Expert Systems with Applications 38 (2011) 6957–6966 6963 4.1. Normal operation process modeling The process described was modeled using input and output variables defined by Tables 1 and 2. In order to provide a dynamic system model to simulate normal conditions, a neural network was applied. The neural network Fig. 12. Feed-forward multilayer perceptron ap structure was generated by a threshold validation error and it is depicted by Fig. 12. For training phase 5,000 points in normal operation were used: 50% for training and last 50% for test. The sampling time applied was 1 s. The sample time was picked in order to avoid aliasing. The stop training criterion was the medium square error below 0.1%. The neural network parameters generated by training are de- tailed in Appendix A. plied to model the normal plant dynamics. Table 3 Artificial fault available at the DAMADICS. Fault Description Control valve faults fl Valve clogging f2 Valve plug or valve seat sedimentation f3 Valve plug or valve seat erosion f4 Increased cf valve or hush ng friction f5 External leakage (leaky bushing, covers, terminals) f6 Internal leakage (valve tightness) f7 Medium evaporation or critical flow Pneumatic servo-motor faults f8 Twisted servo-motor’s piston rod f9 Servo-motor’s housing or terminals tightness f10 Servo-motor’s diaphragm perforation f11 Servo-motor’s spring fault Positioner faults f12 Electro-pneumatic transducer fault fl3 Rod displacement sensor fault f14 Pressure sensor fault f15 Positioner feedback fault General faults/external faults f16 Positioner-supply pressure drop f17 Unexpected pressure change across the valve f18 Fully or partly opened bypass valves f19 Flow rate sensor fault Fig. 13. k-fold cross-validation using k = 10. Table 4 List of parameters and corresponding levels considered for the experiments. Id Algorithm and parameter name Levels � + A Artificial NK Receptors Initiation: Size_Receptors 10 30 B Artificial NK Receptors Initiation: ReceptorsbyCell 3 5 C Artificial Education Mechanism: Threshold_mature 40 60 D Artificial Education Mechanism: Attempts_to_join 3 5 E Artificial Education Mechanism: Max_Cells 3 5 6964 C.A. Laurentys et al. / Expert Systems with Applications 38 (2011) 6957–6966 It is important to stress that the generated model was achieved by neural network training using normal data. It means that the model is able to describe the plant behavior without faults. Therefore, the expected behavior of the residue (difference be- tween process outputs and model outputs) is to be near to zero in absence of faults. Also, none filtering approach was applied to the residue. Fig. 14. Main effect plot for the exp 4.2. DAMADICS artificial faults The DAMADICS simulation tools allowed several artificial faults simulation, to validate the AIS proposed. Table 3 presents the faults available to be applied in this context. In order to compare the proposed AIS performance to others found in the literature, the following faults were considered: f1, f7, f15. The restricted faults set are used once other authors only provided their result for this set. For each fault a 5000 points database is available to train and assess the AIS overall performance. The output residues (the differ- ence between the neural network model and the real measured outputs) from X and F in normal and abnormal situations were ap- plied to generate the inhibitory and activating receptors. The resi- due in normal and abnormal situation was picked to represent NK cell receptors once faults manifest as errors and will enable further fault detection. Therefore, the selected model has a key role on the proposed algorithm. 5. Results and discussions In order to compare the proposed AIS performance to other methods in the Literature applied to the DAMADICS, a cross-valida- tion method was applied. The following DAMADICS faults were as- sessed: f1, f7, f15, f17. A train and test disjoint set was generated for each fault. The database used a stratified k-fold cross-validation where k-1 folds were training folds and one for test. A stratified 10-fold cross-val- idation (Demsar, 2006) was applied to estimate the average error as depicted by Fig. 13. eriments defined in this article. Fig. 15. Interaction plot for the experiments defined in this article. Table 5 Performance indicators for DAMADICS fault detection. Id AIS proposed Paim Previdia dr (%) fa (%) dr (%) fa (%) dr (%) fa (%) f1 98.3 ± 1.1 0 ± 0 98.22 0 83.0 0.02 f7 98.5 ± 1.2 0 ± 0 98.89 0 83.0 0.02 f15 98.7 ± 2.8 0 ± 0 98.89 0 92.0 0.02 f17 95.9 ± 5.6 0 ± 0 98.78 0 99.0 0 C.A. Laurentys et al. / Expert Systems with Applications 38 (2011) 6957–6966 6965 The normal and fault data were uniformly distributed in order to generate balanced folds. Each of the 10-folds has a total of 1000 points (500 belonging to normal dataset and 500 to fault dataset). Since only one 10-fold cross-validation run may not be enough to obtain a reliable error estimate, the overall performance of the AIS was computed as the average validation error for 100 runs of the stratified 10-fold cross-validation. An important decision that needs to be discussed is about the algorithm parameters definition. Design of Experiments (DOE) is needed for experiment with real-life systems, and with either deterministic or random simulation models (Kleijnen, 2008). DOE was applied to verify the proposed AIS parameter sensitiv- ity analysis and define what set of parameters produce best performance. The experimental design considered all AIS algorithm parame- ters that were described in the Methodology section. Table 4 lists the algorithm parameters and their associated values (or levels) applied on the experiments stressing the best ones for the DAMA- DICS benchmark. The simulations average results for each set of parameters are shown in Appendix B. In order to understand the effect of each parameter in the detection rate the main effect plot was generated for the simulations and is shown by Fig. 14. Regarding the algorithm sensibility analysis, the parameter ‘‘Size_Receptors’’ has shown to be the most sensitive one (differ- ence of 5.2%) for the levels experimented followed by the Recep- torsbyCell reaching 0.7% difference as shown by Fig. 14. Also to verify how each parameter affects the others the inter- action plot was generated and it is shown by Fig. 15 The AIS proposed performance using best parameters levels (indicated in bold at Table 4) are shown by Table 5 along with its standard deviations. Table 5 also presents the results found in lit- erature using the following performance indicators: � dr – Average detection rate – as the % of correct classifications, � fa – Average false alarm – as the % of false positive classifications. Analyzing faults ‘‘f1’’, ‘‘f7’’ and ‘‘f15’’ the proposed algorithm achieved higher detection rate than the Previdia (Previdia & Parisin, 2006) algorithm. In general aspects, the proposed AIS detection rate is very high when compared to Previdia algo- rithm with statistically the same false alarms rate, except the fault ‘‘f17’’. Compared the assessed result for fault ‘‘f17’’, it was verified that the algorithms have statistically the same detection rate (confi- dence level of 95%) with the proposed AIS presenting significantly more false alarm rate (almost reaching 1% of false alarm) than Pre- vidia algorithm. This difference is due to the residue analysis does not clearly provide separation between normal and abnormal situation, gener- ating for the AIS proposed an increased false alarm rate. This fact reinforces the coupling of the modeling procedures to the AIS proposed algorithm performance. 6. Conclusions This paper has proposed and validated a novel Artificial Im- mune System (AIS) based approach for fault detection inspired on the recent NK cell immune theory. The approach validation used a stratified 10-fold cross-validation and DAMADICS fault detection database. The DOE was applied to check sensibility analysis and define the best parameter performance. Comparing the proposed AIS perfor- mance to the Previdia algorithm as shown in Table 4, it is clear that the proposed AIS is able to provide a better detection rate with the same or in some cases lower false alarm rate for several DAMADICS database faults. The results show that the proposed AIS automates the AEM fault detection step providing a better tradeoff among detection rate and false alarm rate through the new approach proposed. Acknowledgment This work has been supported by the Brazilian agencies CNPq and FAPEMIG. 6966 C.A. Laurentys et al. / Expert Systems with Applications 38 (2011) 6957–6966 Appendix A. Neural network parameters IWð1; 1Þ¼ �44:7848 15:2394 1:3094 �4:4656 8:8382 �619705 216:2751 65:2473 �40:1468 32:6456 24:0152 42:3434 32:4858 �30:7896 41:79 62:4357 2 6664 3 7775 LWð2; 1Þ¼ �0:65707 0:0073388 �0:074369 0:088101 0:085571 �0:02992 0:2211 �0:079982 � � bð1Þ¼ 16:695 �107:0308 �31:901 �264309 2 6664 3 7775 bð2Þ¼ 0:52836 0:59037 � � Appendix B. Individual experiments results Id A B C D E dr (%) fa (%) 1 + + + + � 98.24 ± 2.38 0 ± 0 2 + � + + + 93.73 ± 4.23 0.01 ± 0.01 3 � � � + � 91.92 ± 5.79 0.24 ± 0.48 4 � � + � + 93 ± 4.65 0 ± 0 5 + + + � + 98.8 ± 1.39 0 ± 0 6 + � + � � 92.14 ± 5.29 0.09 ± 0.18 7 + + � � 98.1 ± 1.32 0 ± 0 8 � � � + + 93.41 ± 4.64 0 ± 0 9 � + + + + 98.07 ± 2.26 0 ± 0 10 � � + + � 94.09 ± 3.95 0 ± 0 11 + + � + � 98.72 ± 1.09 0 ± 0 12 � � + + + 91.77 ± 5.57 0.6 ± 1.21 13 � + � � � 96.88 ± 2.37 0 ± 0 14 � + � + + 97.09 ± 4.04 0 ± 0 15 + + � � + 98.29 ± 1.32 0 ± 0 16 � + � � 96.58 ± 3.51 0 ± 0 17 + � + � + 93.35 ± 4.58 0 ± 0 18 + + + + + 99.33 ± 1.32 0 ± 0 19 + + � + + 98.3 ± 1.44 0 ± 0 20 + � � + � 93.63 ± 4.54 0.03 ± 0.06 21 � + + � + 97.96 ± 1.41 0 ± 0 22 � + � � + 98.02 ± 1.38 0 ± 0 23 + � � � � 92.51 ± 5.07 0.09 ± 0.18 24 + � � � + 91.96 ± 5.49 0.09 ± 0.15 25 + � � � 98.15 ± 1.27 0 ± 0 26 � + + � � 96.95 ± 2.82 0 ± 0 27 � � � � + 92.74 ± 4.89 0 ± 0 28 + � + � 92.08 ± 5.26 0.09 ± 0.18 29 � + + + � 97.83 ± 1.49 0 ± 0 30 � � + � � 92.81 ± 4.94 0.03 ± 0.05 31 + � � + + 92.98 ± 4.73 0 ± 0 32 � � � � � 92.54 ± 5.18 0.09 ± 0.18 References Barty, M. Z. (2006). In Specification of actuators intended to use for benchmark definition. Warszawa, Poland: Warsaw University of Technology, Institute of Automatic Control and Robotics. De Castro, L. N. (2005). Fundamentals of natural computing: Basic concepts, algorithms, and applications. Chapman and Hall/CRC Computer and Information Sciences, Taylor & Francis, Inc. ISBN: 1584886439. DeCastro, P. A. D., & Von Zuben, F. J. (2009). BAIS: A Bayesian Artificial Immune System for the effective handling of building blocks. Information Sciences, 179, 1426–1440. Demsar, J. (2006). Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research, 7, 1–30. Elmeziane, R., Berrada, I., Kassou, I. (2008). A new artificial immune system for the detection of abnormal behaviour, studies in computational intelligence. Springer Berlin/Heidelberg, 149, 113–122. doi:10.1007/978-3-540-70560-410. Gan, Z., Zhao, M., & Chow, T. (2009). Induction machine fault detection using clone selection programming. Expert Systems with Applications, 36(4), 8000–8012. ISSN 0957-4174, doi:10.1016/j.eswa.2008.10.058. Guimaraes, F. G. R., Palhares, M., Campelo, F., & Igarashi, H. (2007). Design of mixed control systems using algorithms inspired by the immune system. Information Sciences, 177, 4368–4386. Ji, Z., & Dasgupta, D. (2009). V-detector: An efficient negative selection algorithm with ‘probably adequate’ detector coverage. Information Sciences, 179, 1390–1406. Kleijnen, J. P. C. (2008). Design of experiments: Overview. In S. J. Mason, R. R. Hill, L. Mönch, J. O. Rose, & J. W. Fowler (Eds.), Proceedings of the 2008 Winter Simulation Conference (pp. 479–488). The Netherlands: Tilburg. Lanier, L. L. (2008). Up on the tightrope: natural killer cell activation and inhibition. Nature Immunology, 5, 495–506. Larrea, C. L., Alvarez, B. S., Soto, A. L., Vazquez, A. L., & Gonzalez, S. (2008). The NKG2D receptor: sensing stressed cells. Cell Press - Elsevier Ltd. Luci, C., & Tomasello, E. (2008). Natural killer cells: Detectors of stress. The International Journal of Biochemistry & Cell Biology, 40, 2335–2340. Luo, M., Zhang, D. H., Zhao, Y. Z., & Zhang, J. B. (2004). A structured methodology for the development of anticipative event management systems with self-recovery capability, IECON 2004. 30th Annual Conference of IEEE (vol. 3(2–6), pp. 2614– 2619). Mendonca, L. F., Sousa, J.M.C., & Sa da Costa, J.M.G. (2009). An architecture for fault detection and isolation based on fuzzy methods. Expert systems with applications (vol. 36(2), part 1, pp. 1092–1104). ISSN 0957-4174, doi: 10.1016/ j.eswa.2007.11.009. MPS, Brazilian Official Work on Statistics (2009). . Paquete, L. T., & Stutzle, L. (2009). Design and analysis of stochastic local search for the multiobjective traveling salesman problem. Computers & Operations Research, 36, 2619–2631. Powers, S. T., & He, J. (2008). A hybrid artificial immune system and Self Organising Map for network intrusion detection. Information Sciences, 178, 3024–3042. Previdia, F., & Parisin, T. (2006). Model-free actuator fault detection using a spectral estimation approach: the case of the DAMADICS benchmark problem. Control Engineering Practice, 14, 635–644. Raulet, D. H., & Vance, R. E. (2006). Self-tolerance of natural killer cells. Nature Reviews, 6, 520–531. Raulet, D. H., Vance, R. E., & McMahon, C. W. (2001). Regulation of the Natural Killer Cell Receptor Repertoire. Annual Review of Immunology, 19, 291–330. Saravanan, N., Cholairajan, S., & Ramachandran, K.I. (2009). Vibration-based fault diagnosis of spur bevel gear box using fuzzy technique. Expert systems with applications (vol. 36(2), part 2, pp. 3119–3135). ISSN 0957-4174, doi:10.1016/ j.eswa.2008.01.010. Tang, K., & Yao, X. (2008). Special issue on ‘Nature inspired problem-solving’. Information Sciences, 178, 2983–2984. Timmis, J., Hart, E., Hone, A., Neal, M., Robins, A., Stepney, S., et al. (2008). Immuno- engineering. In 2nd IFIP International Conference on Biologically Inspired Collaborative Computing (pp. 3–17). Milan, Italy: 20th IFIP World Computer Congress. Venkatasubramanian, V. et al. (2003). A review of process fault detection and Diagnosis. Computers and Chemical Engineering, 27, 293–311. Vivier, E., Tomasello, E., Baratin, M., Walzer, T., & Ugolini, S. (2008). Functions of natural killer cells. Nature Immunology, 9, 503–510. Wu, J., & Liu, C. (2009). An expert system for fault diagnosis in internal combustion engines using wavelet packet transform and neural network. Expert systems with applications (vol. 36(3), part 1, pp. 4278–4286). ISSN 0957-4174, doi:10.1016/j.eswa.2008.03.008. http://www.creme.dataprev.gov.br/AEAT/GLiq/Liq01/Liq01.htm http://www.creme.dataprev.gov.br/AEAT/GLiq/Liq01/Liq01.htm A novel Artificial Immune System for fault behavior detection Introduction Natural killer immune cell NK cell immune overview NK Immune cell receptors initiation NK immune cell education process NK immune cell recognition phase Materials and methods Initiation of the NK cells receptor Education mechanism Recognition phase Calculation Normal operation process modeling DAMADICS artificial faults Results and discussions Conclusions Acknowledgment Neural network parameters Individual experiments results References