This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues. Other uses, including reproduction and distribution, or selling or licensing copies, or posting to personal, institutional or third party websites are prohibited. In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier’s archiving and manuscript policies are encouraged to visit: http://www.elsevier.com/copyright An optimum feature extraction method for texture classification Engin Avci *, Abdulkadir Sengur, Davut Hanbay Firat University, Department of Electronic and Computer Science, 23119, Elazig, Turkey a r t i c l e i n f o Keywords: Pattern recognition Texture classification Optimum feature extraction Discrete wavelet transform Entropy Energy Genetic algorithm Neural networks Intelligent systems a b s t r a c t Texture can be defined as a local statistical pattern of texture primitives in observer’s domain of interest. Texture classification aims to assign texture labels to unknown textures, according to training samples and classification rules. In this paper a novel method, which is an intelligent system for texture classifi- cation is introduced. It used a combination of genetic algorithm, discrete wavelet transform and neural network for optimum feature extraction from texture images. An algorithm called the intelligent system, which processes the pattern recognition approximation, is developed. We tested the proposed method with several texture images. The overall success rate is about 95%. � 2008 Published by Elsevier Ltd. 1. Introduction Texture classification aims to assign texture labels to unknown textures, according to training samples and classification rules. Two major issues are critical for texture classification: the texture classification algorithms and texture feature extraction. The main aim of texture classification is to find a best matched category for a given texture among existing textures. During the past decades wavelet analysis has become a power- ful tool for multi-resolution analysis. Important applications can be found in various fields, ranging from remote sensing to biomedical imaging. Intuitively, multi-scale wavelet analysis is an ideal ap- proach to analyze texture because it is well recognized that scale is one of the most important aspects of texture information. Many previous works are proposed to solve texture classification prob- lem (Chellappa, Chatterjee, & Bagdazian, 1985; Chen & Chen, 1999; Jain & Farrokhnia, 1991). A comparison of textural features from Fourier power spectrum, second-order gray level statistics and first-order statistics of gray level differences were shown in Weszka, Dyer, and Rosenfeld (1976). Other textural features including co-occurrence features, Gabor features, Markov random fields (MRF) based features and fractal features were compared in another work (Chen & Chen, 1999). In Muneeswaran, Ganesan, Arumugam, and Ruba Soundar (2005), a new rotational and scale invariant feature set for textural image classification was pre- sented and a combined invariant feature (CIF) set was proposed. It is an integration of the crude wavelets like Gaussian, Mexican Hat and orthogonal wavelets like Daubechies to achieve a high quality rotational and scale invariant feature set. Also it is added with features obtained using the newly proposed weighted smoothening Gaussian filter masks to improve the classification results. Li et al. proposed a dyadic wavelet, wavelet frame, Gabor wavelet, and steerable pyramid for texture classification problem with individual and combined multi-resolution features (Shutao & Shawe-Taylor, 2005). They used support vector machines (SVM) as classifier. An SVM for texture classification, using transla- tion-invariant features generated from the discrete wavelet frame transform is also proposed in Shutao, James, Kwok Hailong, and Yaonan (2003). In Arivazhagan and Ganesan (2003), Arivazhagan and Ganesan used wavelet statistical features, wavelet co-occur- rence features and combination of wavelet statistical features and co-occurrence features of one-level wavelet transformed images with different feature databases. A distance classifier is used for measurement of similarity and consequent labeling. Moj- silovic et al. investigated in their paper whether the properties of decomposition filters play an important role in texture description, and which feature was dominant in the selection of an optimal fil- ter bank (Mojsilovic, Rackov, & Popovic, 2000). They performed classification experiments with 23 Brodatz textures. The experi- mental results indicated that the selection of the decomposition filters has a significant influence on the result of texture character- ization. Finally, the paper ranked 19 orthogonal and biorthogonal filters, and establishes the most relevant criteria for choice of decomposition filters in wavelet-based texture characterization algorithms. Chen and Chen gave a comparative study for filtering methods for texture classification according to their discrimination ability (Chen & Chen, 1999). Their main aim was to evaluate the performance of four filtering methods including Fourier transform, spatial filter, Gabor filter and wavelet transform for classifica- tion. The superiority of the wavelet transform is shown by their 0957-4174/$ - see front matter � 2008 Published by Elsevier Ltd. doi:10.1016/j.eswa.2008.06.076 * Corresponding author. Tel.: +90 4242370000x4257; fax: +90 4242367064. E-mail address: enginavci23@hotmail.com (E. Avci). Expert Systems with Applications 36 (2009) 6036–6043 Contents lists available at ScienceDirect Expert Systems with Applications j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / e s w a experimental study. Wang et al; proposed a multi-resolution MRF (MRMRF) modeling to describe textures (Lei & Jun, 1999). MRMRF modeling is a method trying to fuse filtering theory and MRF models. In this model, both high pass and low pass components are considered. Huang and Aviyente analyzed the dependence of energy values from different subbands, which may be from the same wavelet basis or different wavelet bases (Huang & Aviyente, 2006). They proposed and information-theoretic measure, mutual information, to select subbands for sparse representation and texture classification. Sengur et al.; used wavelet packet neural networks (WPNN) for texture classification (Sengur, Turkoglu, & Ince, in press).The proposed schema composed of a wavelet packet feature extractor and a multilayer perceptron classifier. Entropy and energy features are integrated wavelet feature extractor. The performed experimental studies show the effectiveness of the WPNN structure. In this paper, a novel method which is an expert system for tex- ture classification is introduced. It used a combination of genetic algorithm, discrete wavelet transform and neural network for opti- mum feature extraction from texture images. An algorithm called the intelligent system, which processes the pattern recognition approximation, is developed. In texture image classification area, the novelties presented of this paper can be summarized as follows: 1. The presented first novelty in this study is, the using an effec- tively optimum feature extraction method that increases per- centage of the correct texture image classification. 2. The presented second novelty in this study is the using of genetic-discrete wavelet-neural network model for selecting of the feature extraction method and finding the optimum wavelet filter and wavelet entropy parameter values for opti- mum feature extraction from texture images. This wavelet filter type and entropy parameters are used for obtaining the wavelet entropy values from wavelet layer as a newness and efficiently method in texture image classification area. The paper is organized as follows. In Section 2, discrete wavelet transform (DWT), artificial neural networks (ANN) are mentioned. Methodology and applications using genetic-discrete wavelet-neu- ral network (GDWNN) is described in Section 3. This GDWNN method enables a large reduction of the texture image data while retaining problem specific information, which facilitates an effi- cient texture classification process. The effectiveness of the pro- posed method for classification of texture images is demonstrated in Section 4. Finally, Section 5 presents discussion and conclusion. 2. Theory 2.1. Discrete wavelet transform Discrete wavelet transform (DWT) is finding inverse use in fields as diverse as telecommunications and biology. Because of their suitability for analyzing non-stationary signals, they have be- come a powerful alternative to Fourier methods in many medical applications, where such signals abound (Daubechies, 1988). The main advantages of wavelets is that they have a varying window size, being wide for slow frequencies and narrow for the fast ones, thus leading to an optimal time-frequency resolution in all the fre- quency ranges. Furthermore, owing to the fact that windows are adapted to the transients of each scale, wavelets lack the require- ment of stationary. The (continuous) wavelet transform of a 1-D signal f(x) is de- fined as ðW wfÞða; bÞ¼ hf ; wða; bÞi¼ Z þ1 �1 fðxÞw�ða;bÞðxÞdx; wða;bÞ ¼ a �1=2w x � b a � � ; ð1Þ where a is the scaling factor, b is the translation parameter related to the location of the window, and w*(x) is the transforming func- tion. The latter is also called the mother wavelet, which is the pro- totype for generating the other window functions. The extension to 2-D is usually performed by using a product of 1-D filters. The transform is computed by applying a filter bank as shown in Fig. 1. L and H to denote the 1-D low pass and high pass filter, respectively. The rows and columns of image are processed separately and down sampled by a factor of 2 in each direction, resulting in one low pass image LL and three detail images HL, LH, and HH. (Fig. 2)a shows the one-level decomposition of Fig. 1 in the spatial domain. The LH channel contains image information of low horizontal frequency and high vertical frequency, the HL channel contains high horizontal frequency and low vertical fre- quency, and the HH channel contains high horizontal and high ver- tical frequencies. Three-level frequency decomposition is shown in Fig. 2b. Note that in multi-scale wavelet decomposition only the LL subband is successively decomposed. 2.2. Genetic algorithms The genetic algorithms use an evolutionary process for solving a problem. The genetic algorithm begins with a set of solutions which are represented by individuals. These sets of solution are called as population. It is taken the solutions from one population and use to form a new population. This iterative process is main- tained during the new population will be better than the old one. Solutions that are then selected to form new solutions are selected according to their fitness values. If fitness value of an individual is better than other’s, this individual is more lucky than other for be reproduced at next population. This iterative process is repeated until some conditions (for example, number of populations or improvement of the best solution, etc.) are satisfied (http:// cs.felk.cvut.cz/~xobitko/ga/ (accessed: 20.8.04)). Outline of the ba- sic genetic algorithm can be given as below: � It is generated random population of n individuals which pro- vide suitable solutions for the problem. � It is evaluated the fitness f(x) of each individual x in the popula- tion (http://cs.felk.cvut.cz/~xobitko/ga/ (accessed: 20.8.04)). � It is created a new population by repeating following steps until the new population is complete. Fig. 1. A one-level wavelet analysis filter bank. E. Avci et al. / Expert Systems with Applications 36 (2009) 6036–6043 6037 � It is selected two parent individuals from a population according to their fitness. The better fitness is interpreted as the bigger chance to be selected for next population. � It is crossed over the parents with a crossover probability to form new individuals. If crossover wasn’t performed, individual would be the exact copy of parents. � It is mutated new individual with a mutation probability at each locus which is position in individual. � It is placed new individuals in the new population. � It is used new generated population for a further run of the algorithm. � It is stop the genetic algorithm, if the end condition is satisfied, and return the best solution in current population. � It is gone to step second. 2.3. The artificial neural networks An artificial neural network (ANN) is a mathematical model consisting of a number of highly interconnected processing ele- ments organized into layers, the geometry and functionality of which have been likened to that of the human brain. The ANN may be regarded as processing learning capabilities. It has natural propensity for storing experimental knowledge. By virtue of its parallel distribution, an ANN is generally robust for tolerant of faults and noise, able to generalize well and capable of solving non-linear problems (Haykin, 1994). 3. Methodology and applications Feature extraction is the key for pattern recognition so that it is the most important component of designing the intelligent system based on pattern recognition since the best classifier will perform poorly if the features are not chosen well (Avci, Turkoglu, & Poyraz, 2005). A feature extractor should reduce the pattern vector (i.e., the original waveform) to a lower dimension, which contains most of the useful information from the original vector. 3.1. Entropy and energy values as features Entropy is a quantity that is widely used in information theory and is based on probability theory (Tzanakou, 2000). Entropy is a common concept in many fields, mainly in mechanic, image pro- cessing and signal processing. The general form of the entropy function is shown as follows: HðXÞ¼� Xn i¼1 pilog2pi ð2Þ where X is a random variable which can be one of the values x1, x2, . . . , xn with probability p1, p2, . . . , pn. Note that if pi = 0, then 0log2 0 is defined to be 0. Thus, H(X) can be interpreted as represent- ing the amount of uncertainty that exists in the value of X. In infor- mation theory, entropy value is considered to be an average amount of information received when the value of X is observed. In this pa- per, we used norm entropy and Sure entropy, respectively. The Norm entropy HN(s) is defined as follows: HNðxÞ¼ Xn i¼0 jxij p for ð1 6 p < 2Þ ð3Þ where x is signal and xi is ith component of the given x variable (Coifman & Wickerhauser, 1992; MATLAB, 2003). The Sure entropy HS(x) is defined as below: jxij6 e ) HSðxÞ¼ Xn i¼0 minððx2i ; e 2Þ ð4Þ There, e is a positive threshold value (Coifman & Wickerhauser, 1992; MATLAB, 2003). Where x is variable and xi is ith component of the given variable. Energy is one of the most commonly used features for texture analysis. In this study we used the averaged l2-norm which is de- fined as follows: El2 ¼ 1 N Xn i¼1 ðxiÞ 2 ð5Þ where N is the number of the components in the x variable. 3.2. Optimum feature extraction and classification by using GDWNN In this study, the genetic algorithm is used for obtaining the optimum wavelet filter and values of the entropy parameter. For this reason, the feature extraction methods are generated using db2, db3, db5, db10, sym2, sym3, sym5, sym7, sym8, bior1.3, bior2.2, bior2.8, bior6.8, coif1, coif2, coif3, coif5 wavelet filters sep- arately. These feature extraction methods can be named as the following: (1) The feature extraction method by using db2 wavelet filter (FEM- 1): For wavelet decomposition of the various texture images, the decomposition and reconstruction structures at level 1 Fig. 2. Wavelet frequency decomposition. 6038 E. Avci et al. / Expert Systems with Applications 36 (2009) 6036–6043 are shown in Fig. 2. DWT is applied to the texture images by using the db2 wavelet decomposition filters. Thus, three detail images and one approximate image are obtained: one-approximation coefficients LL and three detail coeffi- cients LH, HL, HH. A representative example of the wavelet decomposition of the texture image is shown in Fig. 2. One-level DWT decomposition of the Brickwall texture using db2 is shown in Fig. 3. (2) The feature extraction method by using db3 wavelet filter (FEM- 2): In FEM-2, the previous method’s (FEM-1) descriptions are valid. In this method, only db3 wavelet filter is used differ- ently from the FEM-1 for DWT decomposition stage of GDWNN. (3) The feature extraction method by using db5 wavelet filter (FEM- 3): In FEM-3, the descriptions of FEM-1 are valid. In this method, only db5 wavelet filter is used differently from the FEM-1 for DWT decomposition stage of GDWNN. (4) The feature extraction method by using db10 wavelet filter (FEM-4): In FEM-4, the descriptions of FEM-1 are valid. In this method, only db10 wavelet filter is used differently from the FEM-1 for DWT decomposition stage of GDWNN. (5) The feature extraction method by using sym2 wavelet filter (FEM-5): In FEM-5, the previous method’s (FEM-1) descrip- tions are valid. In this method, only sym2 wavelet filter is used differently from the FEM-1 for DWT decomposition stage of GDWNN. (6) The feature extraction method by using sym3 wavelet filter (FEM-6): In FEM-6, the previous method’s (FEM-1) descrip- tions are valid. In this method, only sym3 wavelet filter is used differently from the FEM-1 for DWT decomposition stage of GDWNN. (7) The feature extraction method by using sym5 wavelet filter (FEM-7): In FEM-7, the previous method’s (FEM-1) descrip- tions are valid. In this method, only sym5 wavelet filter is used differently from the FEM-1 for DWT decomposition stage of GDWNN. (8) The feature extraction method by using sym8 wavelet filter (FEM-8): In FEM-8, the previous method’s (FEM-1) descrip- tions are valid. In this method, only sym8 wavelet filter is used differently from the FEM-1 for DWT decomposition stage of GDWNN. (9) The feature extraction method by using bior1.3 wavelet filter (FEM-9): In FEM-9, the previous method’s (FEM-1) descrip- tions are valid. In this method, only bior1.3 wavelet filter is used differently from the FEM-1 for DWT decomposition stage of GDWNN. (10) The feature extraction method by using bior2.2 wavelet filter (FEM-10): In FEM-10, the previous method’s (FEM-1) descriptions are valid. In this method, only bior2.2 wavelet filter is used differently from the FEM-1 for DWT decompo- sition stage of GDWNN. (11) The feature extraction method by using bior2.8 wavelet filter (FEM-11): In FEM-11, the previous method’s (FEM-1) descriptions are valid. In this method, only bior2.8 wavelet filter is used differently from the FEM-1 for DWT decompo- sition stage of GDWNN. (12) The feature extraction method by using bior6.8 wavelet filter (FEM-12): In FEM-12, the previous method’s (FEM-1) descriptions are valid. In this method, only bior6.8 wavelet filter is used differently from the FEM-1 for DWT decompo- sition stage of GDWNN . (13) The feature extraction method by using coif1 wavelet filter (FEM-13): In FEM-13, the previous method’s (FEM-1) descriptions are valid. In this method, only coif1 wavelet fil- ter is used differently from the FEM-1 for DWT decomposi- tion stage of GDWNN. (14) The feature extraction method by using coif2 wavelet filter (FEM-14): In FEM-14, the previous method’s (FEM-1) descriptions are valid. In this method, only coif2 wavelet fil- ter is used differently from the FEM-1 for DWT decomposi- tion stage of GDWNN. (15) The feature extraction method by using coif3 wavelet filter (FEM-15): In FEM-15, the previous method’s (FEM-1) descriptions are valid. In this method, only coif3 wavelet fil- ter is used differently from the FEM-1 for DWT decomposi- tion stage of GDWNN. (16) The feature extraction method by using coif5 wavelet filter (FEM-16): In FEM-16, the previous method’s (FEM-1) descriptions are valid. In this method, only coif5 wavelet fil- ter is used differently from the FEM-1 for DWT decomposi- tion stage of GDWNN. The feature extraction methods and binary equivalent of the feature extraction methods are shown in Table 1. Structural and operational process of the GDWNN: The proposed GDWNN algorithm is developed for determining the most efficient method of 16 texture feature extraction methods for texture dis- crimination which was explained in Section 3.2. Besides choosing the most appropriate wavelet decomposition filter, the proposed algorithm has the ability of selecting the optimum P-parameter va- lue of the norm entropy and optimum e-parameter value of the sure entropy for efficient features which characterize the texture Fig. 3. One-level DWT decomposition of the Brickwall image using db2. Table 1 The feature extraction methods and binary equivalent of the feature extraction methods Feature extraction method Binary equivalent of the feature extraction method Feature extraction method Binary equivalent of the feature extraction method FEM-1 0 0 0 0 FEM-9 1 0 0 0 FEM-2 0 0 0 1 FEM-10 1 0 0 1 FEM-3 0 0 1 0 FEM-11 1 0 1 0 FEM-4 0 0 1 1 FEM-12 1 0 1 1 FEM-5 0 1 0 0 FEM-13 1 1 0 0 FEM-6 0 1 0 1 FEM-14 1 1 0 1 FEM-7 0 1 1 0 FEM-15 1 1 1 0 FEM-8 0 1 1 1 FEM-16 1 1 1 1 E. Avci et al. / Expert Systems with Applications 36 (2009) 6036–6043 6039 images. The operational processes for employing the genetic algo- rithm are listed below: 3.2.1. Step 1. Composing of the initial population The initial population of the genetic algorithm is consisted of 20 individuals. Each individual is a word of 10 bits. These words are consisted of three segments. The first segment of an individual rep- resents the wavelet filter type and four bits are enough for repre- senting this segment because we have 16 wavelet decomposition filters. Fifth, sixth, and seventh bits of the each individual represent the P-parameter value of the norm entropy which is mentioned un- der Section 3.2.1. Moreover, eighth, ninth, and tenth bits of each individual represent e-parameter value of the sure entropy. In norm entropy, P is the power and should be chosen between [1, 2). In sure entropy, e is the threshold and it also should be se- lected between [1, 8). Sensitivity for P-parameter is 1/7 since P- parameter is represented by using three bits for each of individual of the population. Thus, the P-parameter gets one of the values from the set Xp. Xp is consist of the following components: 1, 1.142, 1.285, 1.426, 1.568, 1.71, 1.852 and 1.994. e-Parameter is represented by three bits for each individual of the population. The e-parameter gets one of the following values 1–8. According to this, we can convert these values to binary form in Table 2. For example, two individuals of population may be shown as below: 0 0 0 0 FEM-1 0 0 1 value of the P parameter = 1.142 0 1 1 value of the parameter = 4 a random individual = 0 0 0 0 0 0 1 0 1 1 FEM-2 0 1 0 value of the P parameter = 1.285 0 0 1 value of the parameter = 2 a random individual = 0 0 0 1 0 1 0 0 0 1 0 0 0 1 3.2.2. Step 2. Feature extraction mechanism 1. Each individual of the population represent one of the feature extraction methods which are given in Section 3.2. P-parameter value is used for norm entropy and e-parameter value is used for sure entropy. When a random individual is selected, the appropriate wavelet filter type and the pre-defined parameter values are assigned to the related parameters (P and e). This individual is then sent to the feature extraction mechanism. 2. The feature extraction mechanism tries to obtain the most appropriate feature vector which characterizes the input tex- ture images by adjusting the P and e-parameter values of the related entropy functions. The proposed feature extraction mechanism is tested with several Brodatz texture images, which are shown in Fig. 4. A number of randomly selected tex- ture images of size 96 � 96 are formed. Thus, totally 1000 tex- ture images are processed in the feature extraction mechanism. For each of this 1000 texture image, the norm entropy, the sure entropy and the energy values of the each wavelet decomposition coefficients were obtained by using the feature extraction method. We calculated the norm entropy values of the texture images at the terminal node obtained from wavelet decomposition as defined in Eq. (1). Where, the wavelet entropy H is a real number, x is the termi- nal node signal and (xi) i is the waveform of terminal node. In norm entropy, P is the power and should be such that 1 6 P < 2. We also calculated the sure entropy and energy val- ues as defined in Eqs. (3), (4) and (5). In sure entropy, e is the threshold and must be such that 1 6 e 6 8. All obtained entropy values are normalized by dividing to N=1000. Thus, total normalized 6 entropy values and 3 energy values are found for each of these 10 texture images. 3.2.3. Step 3. Classification and calculation of the fitness function for an individual mechanism This mechanism is realized the intelligent classification by using features obtained from feature extraction mechanism. The training parameters and the structure of the MLP are shown in Ta- ble 3. These parameters are selected for ANN structure after several different experiments. In these experiments, the ANN is employed with different parameters such as the number of hidden layers, the size of the hidden layers, value of the moment constant and learn- ing rate, and type of the activation functions. The operations in this stage are ordered as below: 1. 200 � 9 feature vector which is obtained in feature extraction mechanism is given to input of the artificial neural network (ANN) classification. The decision space at the output of ANN classifier is formed from 10 texture images. 2. The mean square error (MSE) of the ANN is obtained at the final of training of the ANN classifier. 3. We call a constant desired error rate (DER) for the ANN classi- fier. The individuals are selected as winner if MSE is equal to DER or less than DER. If this status is occurred, this means that individual has high fitness value. Therefore, the comparison between MSE and DER is used for our genetic algorithm as fit- ness function of an individual of the population. In this applica- tion, fitness values of the all individuals of population are calculated in the same way. Later, obtained fitness values of all the individuals at the population are ordered from 1 to 20. If fitness value of an individual is less than 11, fitness value of this individual is low. The fitness values, which are higher than 11, are evaluated as high. The individuals which have high fit- ness values at the current population are saved for composing of the next generation. The individuals which have low fitness values at the current population are eliminated. If there are individuals which have same fitness value, one of these individ- ual is selected as random for population in next generation. As a result, optimum 10 individuals of current population are saved for next generation. 3.2.4. Step 4. Crossover operation The 30 % portion of the optimum 10 individuals obtained in Stage-4 are randomly selected and subjected to crossover operator. Namely, 3 individuals are subjected to crossover operator. 2 bits of the each of the random 2 individuals are randomly selected and Table 2 P-parameter value of the norm entropy and e-parameter value of the sure entropy P-parameter value of the norm entropy Binary equivalent of the P-parameter value of the norm entropy e-parameter value of the sure entropy Binary equivalent of the e-parameter value of the sure entropy 1 0 0 0 1 0 0 0 1.142 0 0 1 2 0 0 1 1.285 0 1 0 3 0 1 0 1.426 0 1 1 4 0 1 1 1.568 1 0 0 5 1 0 0 1.71 1 0 1 6 1 0 1 1.852 1 1 0 7 1 1 0 1.994 1 1 1 8 1 1 1 6040 E. Avci et al. / Expert Systems with Applications 36 (2009) 6036–6043 replaced each other for crossover operations. At the final of the crossover operations, 6 new individuals are obtained. Structures of feature extraction and classification mechanism of the GDWNN are shown in Fig. 5. 3.2.5. Step 5. Mutation operation In there, the bit inversion method is used for mutation operator (http://cs.felk.cvut.cz/~xobitko/ga/ (accessed: 20.8.04)). The muta- tion operation is realized by using the 0.3% portion of the total bits numbers of other seven individuals. If a bit is equal to 1, it is chan- ged to 0. If it is equal to 0, it is changed to 1. At the final of the mutation operation, seven new individuals are obtained. There are total 20 individuals in population of next generation at the final of Step 4 and Step 5. The processes in Steps 1–5 are realized respectively for each of the 20 individuals in population for a generation. Aim of using GDWNN in this study is the selecting the most appropriate feature extraction method from among four different feature extraction methods, optimum value of P-norm entropy parameter, and optimum value of e sure entropy parameter. The combination of selected these values are given to feature extrac- tion mechanism. 4. Experimental results Experiments are conducted with 10 Brodatz texture images, each of size 512 � 512 obtained from http://sipi.usc.edu/data- base/database.cgi?volume=textures&image=1#top. All texture images which are used in this study are shown in Fig. 4. We make 20 sub images of size 96 � 96 by randomly chosen from the each original input texture images. Thus, we obtain totally 200 images before feature extraction and training of the GDWNN. 100 texture images are used for test. Each of texture image pass through the process which is explained in Section 3.2. Each texture image is decomposed to 4 subbands according to the 1 level wavelet decomposition using various filters. The experimental results for proposed system are given in Table 4. Here, only the best 10 meth- od results, which the proposed algorithm was found, are given. We also compared the proposed system with several randomly chosen wavelet filters and entropy parameter values. These randomly cho- sen wavelet types, the entropy parameter values and the classifica- tion performance of the texture images were indicated in Table 5. Fig. 4. Texture images. Table 3 MLP architecture and training parameters The number of layers 3 The number of neuron on the layers Input: 9 Hidden: 15 Output: 1 The initial weights and biases Random Activation functions Tangent-sigmoid Training parameters Learning rule Levenberg-Marquart Back-propagation Sum-squared error 0.0000001 E. Avci et al. / Expert Systems with Applications 36 (2009) 6036–6043 6041 As it can be seen from Table 4, the overall classification rates for all texture types are over 90% except one method because the GDWNN could select the most appropriate parameters and the wavelet type. Population An individual in the population is randomly selected The Feature Extraction Method Represented by This Individual Value of the P Norm Entropy Parameter Represented by This Individual Value of the Sure Entropy Parameter Represented by This Individual ε Calculating of The Wavelet Entropies and Wavelet Energies 200 x 9 Feature Vectors Multi Layer Neural Network Classifier No If MSE <= DER Yes The individual is saved for population of the next generation Evaluating of Fitness Function of The GDWNN Fig. 5. Structure of feature extraction and classification mechanism of the GWNN. Table 4 The obtained optimum values by using GDWNN algorithm and classification performance of the GDWNN. Obtained optimum feature extraction method and the related parameters The correct classification rates for texture images (%) Method Wavelet Filter P e Brick wall Grass Gravel Bark Herringbone weave Plastic bubbles Rough wall Straw Woolen cloth Wood grain Total correct classification rates (%) FEM-14 coif2 1.142 8 78 72 99 100 100 100 88 95 100 100 93.2 FEM-3 db5 1.571 1 80 100 100 97 100 100 82 97 100 100 95.6 FEM-15 coif3 1.427 8 75 71 100 85 91 98 78 100 98 100 89.6 FEM-1 db2 1.142 3 78 96 100 100 98 100 71 96 98 100 93.7 FEM-13 coif1 1.713 5 71 100 100 98 82 100 86 86 91 100 91.4 FEM-10 bior2.2 1 5 82 88 100 92 82 100 100 98 100 100 94.2 FEM-15 coif3 1.285 6 78 100 100 82 100 98 96 98 100 100 95.2 FEM-2 db3 1 3 75 87 100 90 100 100 98 93 100 100 94.3 FEM-6 sym3 1.142 8 78 88 92 96 96 96 100 75 100 100 92.1 FEM-7 sym5 1.285 2 75 92 100 97 98 100 86 84 100 100 93.2 Table 5 The randomly chosen wavelet filters and entropy parameter values and their classification performance Randomly selected wavelet filter and parameter values The correct classification rates for texture images (%) Wavelet filter P e Brick wall Grass Gravel Bark Herringbone weave Plastic bubbles Rough wall Straw Woolen cloth Wood grain Total correct classification rates (%) bior2.2 1.1 8 74 80 96 100 93 98 86 90 100 100 82.7 db4 1.5 2 83 93 100 68 93 100 64 88 100 100 88.9 6042 E. Avci et al. / Expert Systems with Applications 36 (2009) 6036–6043 5. Discussions and conclusion This work indicates the use of GDWNN for texture image classi- fication. The feature choice was motivated by a realization that wavelet transform essentially is a representation of textures at a variety of resolutions. In brief, the wavelet decomposition has been demonstrated to be an effective tool for extracting information from the texture images. In this paper, the application of the wavelet entropy and energy in the wavelet layer of GDWNN to the feature extraction from tex- ture images was shown. Wavelet energy proved to be very useful features for characterizing the texture images; furthermore the information obtained from the wavelet entropy is related to the energy. The most important aspect of the intelligent system is the ability of self-organization of the GDWNN without require- ments of programming and the immediate response of a trained net during real-time applications. GDWNN finds optimum wavelet filter type and the optimum parameter values that are used for norm and sure entropy. The disadvantage of the GDWNN for tex- ture classification is that it needs more computation time than a standard MLP structure. Based on theoretical analysis and experimental results, the GDWNN features and classification is shown to be capable of cap- turing image structures which are crucial for texture characteriza- tion. It is shown that GDWNN can classify the given texture images with an overall 93.25% success rate. References Arivazhagan, S., & Ganesan, L. (2003). Texture classification using wavelet transform. Pattern Recognition Letters, 24, 1513–1521. Avci, E., Turkoglu, I., & Poyraz, M. (2005). Intelligent target recognition based on wavelet packet neural network. Experts Systems with Applications, 29(1). Chellappa, R., Chatterjee, S., & Bagdazian, R. (1985). Classification of textures using Gaussian Markov random fields. IEEE Transactions on Acoustics, Speech and Signal Processing, 30, 959–963. Chen, C. C., & Chen, C. C. (1999). Filtering methods for texture discrimination. Pattern Recognition Letters, 20, 783–790. Coifman, R. R., & Wickerhauser, M. V. (1992). Entropy-based algorithms for best basis selection. IEEE Transactions on Information Theory, 38(2), 713–718. Daubechies, I. (1988). Orthogonal bases of compactly supported wavelets. Communications on Pure and Applied Mathematics, 41, 909–996. Haykin, S. (1994). Neural networks. A comprehensive foundation. New York: Macmillan College Publishing Company Inc.. Huang, K., & Aviyente, S. (2006). Information-theoretic wavelet packet subband selection for texture classification. Signal Processing, 86(7), 1410–1420. Jain, A. K., & Farrokhnia, F. (1991). Unsupervised texture segmentation using Gabor filters. Pattern Recognition, 24, 1167–1186. Lei, Wang, & Jun, Liu. (1999). Texture classification using multiresolution Markov random field models. Pattern Recognition Letters, 20(2), 171–182. MATLAB (2003). 5.3 version Wavelet Toolbox, MathWorks Company. Mojsilovic, A., Rackov, D., & Popovic, M. (2000). On the selection of an optimal wavelet basis for texture characterization. IEEE Transactions on Image Processing, 9(12). Muneeswaran, K., Ganesan, L., Arumugam, S., & Ruba Soundar, K. (2005). Texture classification with combined rotation and scale invariant wavelet features. Pattern Recognition, 38(10), 1495–1506. Sengur, A., Turkoglu, I., Ince, M.C., in press. Wavelet packet neural networks for texture classification. Expert Systems with Applications, Uncorrected proof. Available online 13 January , 2006. Shutao, Li., James, T., Kwok Hailong, Zhu, & Yaonan, Wang (2003). Texture classification using the support vector machines. Pattern Recognition, 36(12), 2883–2893. Shutao, Li., & Shawe-Taylor, John (2005). Comparison and fusion of multiresolution features for texture classification. Pattern Recognition Letters, 26(5), 633–638. Tzanakou, E. M. (2000). Supervised and unsupervised pattern recognition feature extraction and computation. CRC Press LLC. Weszka, J. S., Dyer, C. R., & Rosenfeld, A. (1976). A comparative study of texture measures for terrain classification. IEEE Transactions on Systems, Man and Cybernetics, 6, 269–285. E. Avci et al. / Expert Systems with Applications 36 (2009) 6036–6043 6043 EXPERT SYSTEMS WITH APPLICATIONS An International Journal EDITOR-IN-CHIEF Jay Liebowitz, D.Sc. 966 Farm Haven Drive, Rockville, Maryland 20852, USA EDITORIAL ADVISORY BOARD Edward Feigenbaum Stanford University, USA Lance B. Eliot University of Southern California, USA Dianne C. Berry University of Reading, UK Walter Reitman Rensselaer Polytechnic Institute, USA Shri K. Goyal GTE Laboratories, USA Daniel E. O’Leary University of Southern California, USA Richard G. Vedder University of North Texas, USA James L. Rash NASA Goddard Space Flight Center, USA Randall Shumaker Naval Research Laboratory, USA David Bendel Hertz University of Miami, USA A. Desai Narasimhalu Singapore Management University Patricia Lightfoot NASA Goddard Space Flight Center, USA Mark Fox Toronto, Ontario, Canada Jae Kyu Lee Korea Advanced Institute of Science and Technology I. Burhan Turksen University of Toronto, Canada Ching Suen Concordia University, Canada Ernest H. Forman George Washington University, USA Roar Fjellheim Computas A.S., Norway Jan Vanthienen Catholic University, Leuven, Belgium Jon R. Wright AT&T Bell Laboratories, USA Roy Rada University of Maryland, USA John P. Coyne George Washington University, USA John H. Carson George Washington University, USA Daniel Schutzer Citicorp Investment Bank, USA Paul Harmon Harmon Associates, USA Francisco J. Cantu Instituto Tecnologico y de Estudios Superiores de Monterrey, Mexico Brian Gaines University of Calgary, Canada Michel Clerget Cognitech, France Kenneth J. Fordyce IBM, USA James R. Slagle University of Minnesota, USA Eliot Weinman Software Productivity, USA Randall Davis Massachusetts Institute of Technology, USA E. Balagurusamy Mahareer Academy of Technology and Sciences, India Dieter Specht Technische Universitat Cottbus, Germany Vladimir Milacic University of Belgrade, Yugoslavia Hans Bergkvist ATTEXOR S. A., Switzerland Laura C. Davis U.S. Navy Center for Applied Research in Artificial Intelligence, USA Jerald Feinstein MITRE, USA Efraim Turban University of Hawaii, 3435 Kehala Dr., Kiehi HI 96753, USA Bernt Bremdal GeoKnowledge, Norway Milton White Datanamics Inc., USA Editorial Office: Jay Liebowitz, 966 Farm Haven Drive, Rockville, Maryland 20852, USA. Tel./Fax: (+1) 301-770-2978; e-mail: jliebow1@jhu.edu Author enquiries: For enquiries relating to the submission of articles (including electronic submission where available) please visit this journal’s homepage at http://www.elsevier.com/ locate/eswa. You can track accepted articles at http://www.elsevier.com/trackarticle and set up e-mail alerts to inform you of when an article’s status has changed. Also accessible from here is information on copyright, frequently asked questions and more. Contact details for questions arising after acceptance of an article, especially those relating to proofs, are provided after registration of an article for publication. Publication information: Expert Systems with Applications (ISSN 0957-4174). For 2009, volumes 36, 37 are scheduled for publication. Subscription prices are available upon request from the Publisher or from the Regional Sales Office nearest you or from this journal’s website (http://www.elsevier.com/locate/eswa). Further information is available on this journal and other Elsevier products through Elsevier’s website: (http://www.elsevier.com). Subscriptions are accepted on a prepaid basis only and are entered on a calendar year basis. Issues are sent by standard mail (surface within Europe, air delivery outside Europe). Priority rates are available upon request. Claims for missing issues should be made within six months of the date of dispatch. Advertising information: Advertising orders and enquiries can be sent to: James Kenney, Elsevier Ltd., 32 Jamestown Road, London NW1 7BY, UK; phone: (+44) 207 424 4216; fax: (+44) 1865 853 136; e-mail: j.kenney@elsevier.com. Customers in the US and Canada can also contact: Mr Tino DeCarlo, Advertising Department, Elsevier Inc., 360 Park Avenue South, New York, NY 10010-1710, USA; phone: (+1) (212) 633 3815; fax: (+1) (212) 633 3820; e-mail: t.decarlo@elsevier.com. Orders, claims, and journal enquiries: please contact the Regional Sales Office nearest you: St. Louis: Elsevier, Customer Service Department, 11830 Westline Industrial Drive, St. Louis, MO 63146, USA; phone: (877) 8397126 [toll free within the USA]; (+1) (314) 4537076 [outside the USA]; fax: (+1) (314) 5235153; e-mail: JournalCustomerService-usa@elsevier.com Amsterdam: Elsevier, Customer Service Department, PO Box 211, 1000 AE Amsterdam, The Netherlands; phone: (+31) (20) 4853757; fax: (+31) (20) 4853432; e-mail: JournalsCustomer- ServiceEMEA@elsevier.com Tokyo: Elsevier, Customer Service Department, 4F Higashi-Azabu, 1-Chome Bldg, 1-9-15 Higashi-Azabu, Minato-ku, Tokyo 106-0044, Japan; phone: (+81) (3) 5561 5037; fax: (+81) (3) 5561 5047; e-mail: JournalsCustomerServiceJapan@elsevier.com Singapore: Elsevier, Customer Service Department, 3 Killiney Road, #08-01 Winsland House I, Singapore 239519; phone: (+65) 63490222; fax: (+65) 67331510; e-mail: JournalsCustomer- ServiceAPAC@elsevier.com USA mailing notice: Expert Systems with Applications (ISSN 0957-4174) is published 10 issues per annum in January, March, April, May, July, August, September, October, November, December by Elsevier Ltd. (The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, UK). Periodical postage paid at Rahway NJ and additional mailing offices. USA POSTMASTER: Send change of address: Expert Systems with Applications, Elsevier, Customer Service Department, 11830 Westline Industrial Drive, St. Louis, MO 63146, USA. Periodical postage paid at Rahway NJ and additional mailing offices. AIRFREIGHT AND MAILING in USA by Mercury International Limited, 365, Blair Road, Avenel, NJ 07001.