哈尔滨工程大学学报模板 2018 International Conference on Sensor Network and Computer Engineering (ICSNCE 2018) 62 Multiple Vehicle License Plate Location in Complex Background Yaxin Zhao School of Computer Science and Engineering Technological University Xi’an, China e-mail: 953383250@qq.com Li Zhao School of Computer Science and Engineering, Xi’an Technological University, Xi’an, China e-mail: 332099732@qq.com Ya Li ChongQing Water Resources And Electric Engineering College Chong Qing, China e-mail: 498108151@qq.com Abstract—In order to expand the application range of the intelligent traffic management system, and to solve the problem that the license plate positioning accuracy is low in the changing of the scene. On the basis of the analysis of previous methods advantages and disadvantages, applying deep learning model orientation method is proposed. The image expressed as graph of graph theory .Based on the principle of minimum spanning tree preliminary separate target objects in image of vehicle. Combined with the color, dimension, texture and match the similarity to choose, search and merger area in the image, suspicious area of the license plate is obtained. Using visual word package to express rectangular profile after coarse positioning. Using support vector machine (SVM) to classify and identify rectangular area of license plate. Accurate positioning license plate location is positioned accurately. The method of accuracy is 96.4% for 135 pieces of test sample positioning, strong anti-jamming. Keywords-License Plate Position; Complex Background; Bag-of-view; Support Vector Machine; License Plate Vertical Texture At present , There is the main algorithm is based on the morphological features for the license plate location problem . The algorithm is mainly using the texture characteristics of car license plate . After processing on the morphology , It takes exclusion of a part of the interference of the noise in the background of license plate images , and then based on the geometric features of license plate, such as aspect ratio condition to locate the license plate location . Due to the use of the algorithm is only a single plate texture feature, ignoring the license plate of the other characteristics of the information . So when the image background and license plate area is in the same objects with the texture feature , just rely on geometric features such as aspect ratio decision condition is difficult to determine whether to license plate area. Eventually led to the wrong location; The positioning algorithm based on color image [5].This algorithm makes full use of the license plate in the image color information, through the characteristics of the different color space to locate the license plate. But the color model is operate in the image on the multi-channel models . It has large amount of calculation and poor real-time performance. When the license plate region color is very similar to surroundings, the algorithm positioning error rate increase. And the color of the license plate image information is susceptible to interference illumination change, which made an impact on the extraction of license plate character. Eventually lead to the wrong location or position; The algorithm based on BP neural network. The algorithm is divided the image grid precessed into many blocks ,using neural network for each block region extracted feature descriptor to classify and locate the license plate. But the extraction of license plate image characteristics of image block area size has nothing to do with the actual license plate image size. The global characteristics can not truly reflect the global features of the license plate. Convergence time is long, and BP algorithm easy to fall into local optimum. I. VISUAL WORD PACKAGE MODEL OF LICENSE PLATE LOCALIZATION ALGORITHM Analysis of previous license plate localization algorithm based on texture feature. After the license plate image by the morphological processing can effectively make the license plate area rectangle connected area. But it is difficult to further improve the accuracy of this method. If the interference area of the image and texture features of license plate is similar, then interference region to form a rectangular connected domain also. So just rely on license plate aspect ratio to determine conditions to distinguish the license plate region and the license plate region is difficult. This makes the poor anti-jamming, misjudgment rate is high. Aimed at the problem in rectangular area filter link using the visual word packet and support vector machine (SVM) to improve filtering accuracy. Basic principle of visual word package features is to use characteristic descriptors to express images, and 2018 International Conference on Sensor Network and Computer Engineering (ICSNCE 2018) 63 put the picture as a different set of feature points. Through the statistics of each feature point frequency in the single photo to vectors to a photograph. Namely in the form of a histogram to represent the photograph [7].As shown in figure 1 package basic process for visual words. Due to the different types of pictures’ vector representation from it’s visual word package are different, so it can choose the appropriate classifier using sample set for training. Then use the trained classifier to classify test images. Figure 1. Flow chart of BOW model Considering the license plate images will get affected by Angle, light and other factors. In order to reduce such factors on the characteristics of the descriptor to describe the influence of the license plate area, and can achieve rapid positioning of the target. This paper adopted the surf feature descriptor. The descriptor is characterized by has fast calculation and partial invariance. That is to say, it has the scale of the robustness of a certain range scaling, image rotation, the change of perspective, illumination changes and image blur, can effectively eliminate from light, Angle and other factors. Extraction algorithm of surf feature descriptor is Hessian matrix is used to determine the candidate points. Not greatly restrain and realize the feature point detection. In the image pixel Hessian matrix are defined as follows:             222 222 ,H yfyxf yxfxf yxf (1) Hessian matrix discriminant such as type (2).The value of the discriminant is the eigenvalues of the Hessian matrix. It classifies all points using symbols of determining structure and according to the discriminant value plus or minus identifying whether this point is the value of the pole.   2 2 2 2 2 2 Hdet                 yx f y f x f (2) This algorithm use image pixel L (x, y) to replace function f (x, y) and use type (1) through specific nuclear convolution between to computation the second order partial derivative, it is concluded that the Hessian matrix of the matrix elements is xx L 、 xy L 、 yy L .According to the type (2) the discriminant, a approximate Hessian determinant map can be obtained. Compared with Hessian matrix of each pixel point and the pyramid image the size of the 26 points in the field of 3 d. The maximum as the characteristics of the initial point. As shown in figure 2. Local eigenvalues of x is greater than the surrounding pixels. It is the characteristic points of the region. Figure 2. Feature points In order to guarantee the rotation invariance, we need to calculate Centered on feature points a-nd the sum of the corresponding Harr wavelet for 60 degrees within the sector is all points in the direction of the x and y. Close to feature points of the weight is big , away from is small. To get the main direction of each feature point. This process is shown in figure 3. 2018 International Conference on Sensor Network and Computer Engineering (ICSNCE 2018) 64 Figure 3. Determine the main direction Although can Surf characteristics describe a image, an image contains a large number of Surf feature points.If the training directly used in classification, calculation will be very big. We need through the clustering algorithm to cluster the vector data.Using a cluster of clusters represent visual words in a visual word, and then map the surf feature points to the visual word package generated code.In this paper, using K means clustering algorithm in constructing the visual word package .The principle of algorithm is simple and easy to implement and there is a good clustering effect.Such as type (3) the calculation formula.E is sum of square error for all clustering objects.P is the clustering objects. i C is the number of i C class of clustering objects .     i Cp C p pE i (3) The problem of License plate location only need to classify an outline of the license plate and the plate profile picture. We chose SVM ,which is the classifier of dichotomy.SVM is a kind of machine learning methods based on the theory of VC dimension and the theory of structural risk minimization . It is outstanding in solving nonlinear problems and it was originally designed for binary classification problems [8].Principle of SVM is put data map to high-dimensional space, then finding largest classification interval hyperplane in high dimensional space, and using the hyperplane to classify. Such as type (4) calculation formula.   ii yx , is the training sample. l for the sample. n for the dimensions of the input.C is punish parameters.  as though laser coefficient.   ji xxk , for the selection of kernel function.      n ji jijiii n i i xxkyy 1,1 , 2 1 max s.t. C i 0 , i =1,...,n (4) 0 1   n i ii y II. THE REALIZATION OF THE ALGORITHM A. The suspicious area extraction of license plate Image contains abundant information.The object shape, size, color, texture, and other characteristics.The image is defined as the graph of graph theory, are defined as follows: EvveVvv EVG jiiji   ),(,, ),( (5) Among them, i v is a picture point in the image and a vertex of graph in the graph theory. i e is the weight of an edge of graph in the graph theory, is i v and j v between the picture points of the gray level difference or the distance between the.Using Graph Cuts algorithm [] to divided graph G into a series of disjoint independent subgraphs ' G .The algorithm is mainly using the minimum Spanning Tree (MST, Minimun Spanning Tree), realizing regional segmentation.For segmentation region contains the largest edge weights are defined as follows: )(max)( ),( eCInt ECMSTe    (6) Among them, C is a segmented regions. ()w is weight. The minimum edge weights of vertices are connected between the two regional segmentation are defined as follows: )),((min),( ),(,, 21 21 ji EvvCvCv vvwCCDif jiji   (7) For no edge connection between segmentation part, is S.For the segmentation boundary judgment are defined as follows:      otherwisefalse CCMIntCCDififtrue CCD ),(),(, ),( 2121 (8) The smallest division of internal difference are defined as follows: ))()(),()(min(),( 221121 CCIntCCIntCCMInt   (9) Among them, () as the threshold function control segmentation area of consolidation degree, defined as   CKC / , C is segmentation part of the number of vertices, k is coarsness of parameter control image segmentation area .Using Graph Cuts algorithm to 2018 International Conference on Sensor Network and Computer Engineering (ICSNCE 2018) 65 Conduct a preliminary regional segmentation for image, the segmentation result is shown in figure 4. (a) (b) Figure 4. Image segmentation In the image object is hierarchical. After initial segmented regions, using similarity to calculate diversification and region merging. For area merger adopt four similarity in the image as follows: 1) Color similarity. The normalized image obtains 25 bins in the histogram for each color channel. Each area can be expressed into a 75 - dimensional vector .Regional color of the computation formula is as follows: )()( )()( ),min(),( 1 ji jjii t n k k j k ijicolour rsizersize CrsizeCrsize C ccrrS       (10) 2) Texture similarity. To calculate gaussian differential for each color channel of eight different direction. The normalized image acquires 10 bins in the histogram for each channel and for each color. Making areas represent a 240 - dimensional vector . computation formula is as follows:    n k k j k ijitexture ttrrS 1 ),min(),( (11) )( )()( 1),( imsize rsizersize rrS ji jisize   (12) 3) Size similarity. Refers to the number of pixels of similarity in the region. Small area for priority merger. Computation formula is as follows: 4) Consistent with similarity. Refers to the marked rectangle the smaller alignment is higher for the combined area. The calculation is as follows: )( )()()( 1),( imsize rsizersizeBBsize rrfill iiij ji   (13) To combine the calculated four kinds of similarity. Similar set is S .Formula is as follows: ),(),( ),(),(),( 43 21 jifilljisize jitexturejicolourji rrsarrsa rrsarrsarrs   (14) The  1,0 i a .Finding the similarity of the two biggest area is i r and j r from S that is similarity of collection. merging it into a region t r ,as well as dividing calculation of similarity that is in i r and j r between neighboring areas in S that is a collection. Calculate t r generated area and its adjacent area of similarity, and added to the similarity of set S . t r is added to the collection area R and use rectangular box to mark area. The suspicious area is extracted by using the algorithm for vehicle image as shown in figure 5. 2018 International Conference on Sensor Network and Computer Engineering (ICSNCE 2018) 66 Figure 5. The suspicious area extraction B. The location of license plate 1) Putting the pixels in the image into type (1).The Hessian matrix on dimension can be got such as type (15). Among is convolution results that is gaussian filtering second derivative. and above meaning. Gaussian filter function such as type (6);            ),(),( ),(),(   xLxL xLxL H yyxy xyxx (15) 2) On the original image, by expanding the size of the box to form the different scales of image pyramid. For example 9 * 9 box value of the filter template as shown in figure 6.The grey part template value is 0.Corresponding to the second order gauss filter is=1.2.is the scale of the corresponding values. After the convolution of the value of , ,.Get F of Hessian matrix expression is as follows: 2 )9.0( xyxyxx DDDH  (16) (a)x direction (b)y direction (c)xy direction Figure 6. 9*9 filtering template 3) Building scale image pyramid. In every order, The four layers of the scale of the image is chosen. The four order of building of parameters as shown in figure 7.The grey number is the size of the filter template the box. If the image size is greater than the template size, we need to continue to increase the order. Such as the size of the filter template for N * N. The scale of the corresponding is N/s = 1.2 * 9;Using Hessian matrix to calculate the extremum, and then in the 3 * 3 * 3 within the territory of three-dimensional take non-maximum suppression. When has compared to last scale, the next scale and the scale of around 26 field values are big or hours, there are candidate features. In the scale space and image space take a interpolation arithmetic. Get the stable feature points 2018 International Conference on Sensor Network and Computer Engineering (ICSNCE 2018) 67 location and the dimension values. Figure 7. The filter size of Octave 4) With feature points as the center, s is the scale value of the feature points. For calculating radius of 6 s points in the field in the x and y direction Harr wavelet response. And giving the response value assigned a gaussian weight coefficient, this allows the response contribution near the feature point is big , away from the feature point is small. Adding the 60 degrees within the scope of the response to form a new vector. Traverse the entire circle area, selecting the longest vector for the feature points in the direction of the principal direction. As shown in figure 3. 5) Rotating coordinate axis to the main direction. According to the main direction to choose side length of 20 s square area. The window area was divided into 4 * 4 sub area. Computing wavelet response in each area 5 s * 5 s within the scope. Forand. As shown in figure 8.Each subregion of response and the absolute value of response together to form  xd ,  yd ,  xd ,  yd .In each subdomain form 4 d weight o f vector =（ xd 、 yd 、  xd 、 yd ）.So each feature point is 4 * 4 * 4 = 64 dimension of description of the vector. And then to vector normalization. The extraction results as shown in figure 9. Figure 8. Generate surf feature points Figure 9. Surf feature points Surf feature points are extracted. Using the k - means clustering algorithm to generate visual word package. [10] All surf feature points' samples extracted in the training library are     mxx ,...,1 , each   64  i x . the generation process is as follows: a) Then select 1000 cluster centroid point are ,,...,, 64 21  k  ; b) To calculate each sample i should belong to the class that is   2 minarg: j i j i xC  , recalculating each type j  of the centroid        m i i m i ii j jc xjc 1 )( 1 )()( }{ }{ : is until convergence. The training of the classifier. Using visual word package to express the picture of the training set. That is to say, extracting surf feature points of images and mapping it to the corresponding word package. To generate the code of the picture. Inputting into SVM to take training, the process is as follows: a) Gaving each sample   1000  i x mark i y . License plate sample tagged value is 1.The license tag of value is -1; b) To select the gaussian kernel function as a conversion function that cast onto the n dimensional vector. Selecting its  value control the dimension of the projection. Selection of penalty factor C is to optimize decision surface; c) All the sample are substituted into type (4)and to calculation, get the classification of the decision function in the form below: } 2 1 {min 1 2 ,    n i i C   , iii bxy   1 , 0i (17) In the recognition phase, through visual word package said the image into vector form. Using the trained SVM classifier to classify its license plate images .Locating the license plate location. III. CONCLUSION Aimed at the phenomenon of laminated object in the image, using based on graph search algorithm to obtain the suspicious area of vehicle license plate in the image. 2018 International Conference on Sensor Network and Computer Engineering (ICSNCE 2018) 68 Extracting its feature points of surf for coarse positioning of the rectangular profile area. According to generated bag of Visterms represented the candidate images as codebook . Using decisions classification function are obtained by training SVM to classify rectangular area, locating the license plate. This method have higher recognition rate and anti-jamming is strong under complex background. The collection of 140 photos of the result: its accuracy is 135 pieces, accuracy is 96.4%, has the strong robustness. REFERENCES [1] ZhiQiang Li, Yongbin Li. The development and research status of license plate recognition technology. Science&Technology Information, 2012,5:110-125. [2] Geng Qing-tian, Zhao Hong-wei. License plate recognition based on fractal and hidden Markov feature. Optics and Precision Engineering, 2013,21(12):3198-3203 [3] Qin Zhong, Shi Sheng li, Xu Jianmin et al. Method of license plate location on comer feature[C]//Proceeding of the 6th World Congress on Intelligent Control and Automation Piscataway, NJ, USA: IEEE, Press, 2006: 8645-8649 [4] Guo Z Q, Wang Y J, Dong Z, et al. License plate detection method base on paired morphological operator[C]//Proceeding of the 16th National Conference on Image and Graphics. Beijing: Tsinghua Vniversity Press, 2011,292-295. [5] Zhang Yin, Pan Yunhe. A new approach for vehicle license plate locating from color image[J]. Journal of Image and Graphics. 2001,6(4):374-377. [6] Zheng L H, He X J, Samali B, et al. An algorithm for accuracy enhancement of license plate recognition[J]. Journal of Computer and System Sciences, 2013,79(2):245-255. [7] Qiwei Wang, Shouhong Wan, Lihua Yue, Che Wang. Visual Attention based Bag-of-Words Model for Image Classification. Sixth International Conference on Digital Image Processing. 2014 [8] Corinna Cortes, Vladimir Vapnik. Support-Vector Networks. Machine Learning, 1995,20(3):273-297 [9] HE Chunhua , ZHANG Xuefei , HU Yingchun. A study on the improved algorithm for Sobel on image edge detection. OPTICALTECHNIQUE,2013,38(3):324 [10] A.Bolovinou, I.Pratikakis, S.Perantonis. Bag of spatio-visual words for context inference in scene classification[J]. Pattern Recognition . 2013,43(3):1039-1053 http://www.cnki.net/kcms/detail/detail.aspx?filename=SJES14010600107401&dbcode=SSJD&v= http://www.cnki.net/kcms/detail/detail.aspx?filename=SJES14010600107401&dbcode=SSJD&v=