key: cord-0784451-9ijrbt5b authors: Rizvi, Syed Khurram Jah; Azad, Muhammad Ajmal; Fraz, Muhammad Moazam title: Spectrum of Advancements and Developments in Multidisciplinary Domains for Generative Adversarial Networks (GANs) date: 2021-04-02 journal: Arch Comput Methods Eng DOI: 10.1007/s11831-021-09543-4 sha: fbd287bf4db445162d702b92956deb19dd43a50f doc_id: 784451 cord_uid: 9ijrbt5b The survey paper summarizes the recent applications and developments in the domain of Generative Adversarial Networks (GANs) i.e. a back propagation based neural network architecture for generative modeling. GANs is one of the most highlighted research avenue due to its synthetic data generation capabilities and benefits of representations comprehended irrespective of the application. While several reviews for GANs in the arena of image processing have been conducted by present but none have given attention on the review of GANs over multi-disciplinary domains. Therefore, in this survey, use of GAN in multidisciplinary applications areas and its implementation challenges have been done by conducting a rigorous search for journal/research article related to GAN and in this regard five renowned journal databases i.e. “ACM Digital Library”,” Elsevier”, “IEEE Explore”, “Science Direct”, “Springer” and proceedings of best domain specific conference are considered. By employing hybrid research methodology and article inclusion and exclusion criteria, 100 research articles are considered encompassing 23 application domains for the survey. In this paper applications of GAN in various practical domain and their implementation challenges its associated advantages and disadvantages have been discussed. For the first time a survey of this type have been done where GAN with wide range of application and its associated advantages and disadvantages issue have been reviewed. Finally, this article presents several diversified prominent developing trends in the respective research domain which will provide a visionary perspective regarding ongoing GANs related research and eventually help to develop an intuition for problem solving using GANs. The inculcation of Machine learning in almost every domain of life being the most popular and effective research domain is a universal reality. It is becoming a dominating and driving domain with novel research avenues with undoubtedly significant numerous applications e.g., computer vision, autonomous driving, speech recognition, image identification, drug discovery, cyber security, anomaly detection etc. The representation of data in machine learning approaches requires appropriate feature extraction and therefore researchers proposed a representation learning [1] approach for feature mining and its onward application for classification. Deep learning [2] is an autonomous feature learning approach by extracting high level abstract features and framing it into very intuitive and meaningful representations. In deep learning [3] , numerous tactics have been presented to overcome the challenges and limitations related to hidden layer by employing cascaded hidden layers. There are many well-known deep learning approaches [4] such as Deep Belief Network (DBN) and Boltzmann Machine (DBM) [5] , Deep Neural Network (DNN) [6] , Auto Encoder with their variants and Convolutional Neural Network (CNN) [5] . Generative adversarial Networks(GANs) have recently accumulated substantial importance in the arena of Generative Models by assuring the ability to produce novel content such that one generative model can generate only data while others can provide an estimate of density function. However, there are some generative models which can generate both data and density function. Generative models unlike discriminative models, successfully aim to understand the underpinning data distribution by learning the fundamental parameters which enables model data analysis, extraction of novel intuitions and synthetic data generation such that conversion of a low dimension input to a high dimension output. The recent emergence of GANs as most dominating category of generative models is based on the aggregate or collective optimization of two neural networks with mutually exclusive objectives. The first network i.e. generator strives to generate synthetic but perceptually convincing data points from some random uniform distribution akin to input representation from a low dimension source space to a high dimension target domain. The second network i.e. discriminator strives to distinguish between the original and fake sample. The behavioral optimization of both the generator and discriminator is based on the discriminators' output such that weight adjustment for the generator will be required if the discriminator can easily differentiate the fake and real samples. The training process for generator and discriminator is conducted separately such that initially real samples are submitted to discriminator with correct labels, updating the discriminator by adjusting its stochastic gradient while keeping the generator fixed. Subsequently, the generator is updated by adjusting its stochastic gradient while keeping discriminator fixed and the process is repeated until Nash equilibrium is achieved. The model optimization is performed wr.t. a combined loss function for generator as well as discriminator which is given in Eq. 1. Increasing use of GANs in image processing and image creation [7] can be employed in fashion industry to create imaginary fashion models which obviates the need to hire real fashion model, photographer and makeup artists resulting in significant economy of cost. GANS have also been used in simulating images [8] of astronomical bodies and in 2018 they were used for successful modeling of dark matter distribution in space. They have also been used to visualize effects of climate change [9] , face recognition over the years, cartography as well as in fields as diverse as Image Processing to Machine learning and natural language processing (Table 1) . While going through the extensive literature review regarding the GANs it was revealed that several reviews for GANs in the area of image vision and processing have been conducted by present but none have given attention on the review of GANs over multi-disciplinary domains Therefore, in this survey, use of GAN in multidisciplinary applications areas and its implementation challenges have been done by conducting a rigorous search for journal/research article related to GAN and in this regard five renowned journal databases i.e. "ACM Digital Library"," Elsevier", "IEEE Explore", "Science Direct", "Springer" and proceedings of best domain specific conference are considered. By employing hybrid research methodology and article inclusion and exclusion criteria, 62 research articles are considered encompassing 23 application domains for the survey. The major benefactions are: a. This article presents several diversified prominent developing trends in the respective research domain which will provide a visionary perspective regarding ongoing GANs related research and eventually help to develop an intuition for problem solving using GANs. b. This paper provides a unique view of GANs in impressive cross and intersection domains. This research work is also an attempt to provide a unique survey of its kind which has never been studied before in the best of our knowledge and it is not only restricted to domain of natural language and image processing. The paper organization is as follows: Sect. 2 narrates methodology including article search strategy and its selection criteria. Section 3 narrates the application of GAN in diverse portfolio of practical domains. Section 4 describe the review analysis along with the problems of GAN. Section 5 narrates the concluding remarks followed by the future direction. There are three fundamental approaches to conduct research i.e. qualitative, quantitative and mixed keeping in view problem statement, domain expertise of the researcher, nature of data being analyzed and reporting audience. Qualitative research covers descriptive and brief approach to yield significance and interpretation of the experimental phenomena and have further classification including narrative, ethnography, case study and grounded theory. However, quantitative research employs statistics with further classification including descriptive, cross-sectional, correlational and experimental approaches. Moreover, mixed methodology combines quantitative and qualitative approaches to attain in-depth understanding of the phenomena being probed with further classification including concurrent triangulation, sequential exploratory design, and concurrent nested. The most frequently used research approach is mixed for generative modeling and its application for better output and evaluation. GAN has emerged to be a revolutionary approach for generative modeling in recent years which is evident by the significant number publications covering scope of its advancements in domain of image processing and computer vision. Despite the existing studies, limited research is done regarding its application in other domains and therefore following questions are addressed in this research work: • Which domains can be the candidate for the application of GAN? • What approaches and tools are being used in the development of GAN based learning? • What are the advantages and limitations of discussed novel GAN based approaches? In this research work, the following strategy is used for article searching and selection (Fig. 1) . The critical step is the manuscript selection for appropriate review and therefore to keep the review process methodologically strong and uniform, a criterion is defined for article selection. Different journals relevant to large number of domains are selected from specific libraries including five renowned journal databases i.e. "ACM Digital Library"," Elsevier", "IEEE Explore", "Science Direct", "Springer" and proceedings of subject specific top conferences. The collection of aforementioned sources is performed on respective impact factor value and relative ranking. Moreover, keywords used for paper searching comprised of "Applications of GANs", "Review of GANs applications", "Benefits of Generative Modeling", "Advancements of GANs in combination to domains keywords and other tags. The criteria for inclusion or excluding the article is summarized in the Table 2 . In this research, 23 practical applications domains were selected and based on inclusion and exclusion criteria, over 100 studies from selected journals were included in this analysis. The applications domains selected for the application of GAN's are summarized in the Table 3 . The process of data extraction, analysis and synthesis was done with due diligence and in depth study of all the selected articles with appropriate supervisory review controls. The findings of the review are presented in the following section in the context of established questions keeping in view that given papers are from different domain without any standard theme (Fig. 2 ). The following section explaining the applications of GAN into computer vision, healthcare, IoT, agriculture, Intelligent transportation, software development, natural language processing, entertainment and cyber security have been summarized (Table 4 ). In [18] , authors proposed a model which uses simulated and unsupervised learning instead of using random vectors and synthetic images as inputs. The major challenge faced in image processing is the use of synthetic images in the learning or training phase as it doesn't produce reliable results when applied over real images. This deficiency is the result of characteristics peculiar to synthetic data as the network learns details that are present only in synthetic images and there is substantial difference between generalizations of actual and synthetic images and therefore results in production of images which are highly realistic. These images are later on evaluated using both qualitative measures as well as through user study. The result of both of these evaluations show a significant improvement over synthetic images without the use of any labeled real data. In [19] the authors presented a strategy for reconstruction of compressed sensing magnetic resonance imaging (CS-MRI). Compared to MRI, CS-MRI uses much less raw data to reconstruct MRI images. CS-MRI has an edge as it can rebuild images from randomly collected fewer samples without suffering consequent quality degradation in images due to nonlinear optimization. The major obstacle for CS-MRI is finding an appropriate reconstruction procedure of a noise free image from random under sampled In [20] the authors use an approach different from normalization methods but close to style transfer approach for image transformation in GANs which resulted in better efficiency and image quality compared to images produced through normalization techniques. One of the major problems in image processing is the image translation process which requires use of labeled data which is time consuming process. If the image translation can be performed on unlabeled data, it can result in significant cost of economy. In order to address this problem in [21] a two-step method is used for image translation. This method has the advantage that it can perform image translation between two different domains without the use of labeled data which results in significant cost of economy. Zhang et al. [22] a new GAN model StackGAN is proposed for text to image generation which is comprised of dual stages whereas in former stage the model creates an image of low resolution from the textual description while later phase performs the refinement of the image followed by removal of various defects and this results in a high resolution image which is also more realistic. StackGAN uses novel conditioning augmentation technique to achieve smoothness and better results. Reed et al. [23] a new GAN model that is capable of producing images from text descriptions is proposed. The experiments conducted by authors produce result in which images of animal and plants are produced from in depth text deails. The capability of the model to create image with multiple objects and variable backgrounds was also demonstrated. In [24] the authors introduce a new speech recognition model which relies on a data driven approach instead of relying on domain expertise or signal processing assumptions. The framework is scalable and increases robustness as it uses invariance at the encoder embedding level. Experiments show that the new approach has demonstrated good speech recognition capabilities of plain sequential models without requiring particular preprocessing. In [25] the authors increase the effectiveness of GANs to improve speech containing noise of additive kind as well as reverberant noise. Previously one of the main problems faced in the field of speech recognition was that application of GAN only improved the output of a noise free trained automatic speech recognition (ASR) system however failed to increase performance compared to those achieved by conventional multistyle training (MTR). In order to overcome this deficiency, the authors demonstrate that applying GANs on Log-Mel filterbank spectra results in better performance, less computation requirements, increased robustness and achieved a 7% WER improvement relative to the MTR system. The increased robustness is achieved by increasing resistance to reverberant noise whereas improved WER is achieved by appending the GAN-enhanced features to the noisy inputs and retraining. Pascual et al. [26] a new GAN for speech enhancement is proposed which operates at waveform level and the model has the ability to include 28 speakers which was tested against diverse noise conditions. The results demonstrated that the new model was capable of handling the dataset in an efficient manner. UAVs or drones are used for a variety of purposes which include military, agriculture, recreation, science, ArƟcle Screening Image processing Shrivastava et al. [18] 2017 Simulated and unsupervised learning for synthetic image generation Use image as input instead of random vector Yang et al. [19] 2017 Conditional GAN for reconstruction of compressed or corrupted sensing magnetic resonance imaging(CS-MRI) from random under sampled data Generate image with better textures and edges Cho et al. [20] 2019 Style transfer approach for image transformation Achieved efficiency and image quality Dong et al. [21] 2017 Image translation as well as intra-domain image translation Zhang et al. [22] 2016 Conditional augmentation technique for generation of low resolution image from text followed by noise removal Achieved image smoothness and improved quality Reed et al. [23] 2016 Synthetic image generation from text description Advantage: image generation with high quality and with multiple objects and variable background Speech recognition Sriram et al. [24] 2017 Data driven approach that introduced invariance in encoder embedding with no specialized pre-processing Achieved scalability and robustness Donahue et al. [25] 2017 Enhancement of contaminated speed by additive and reverberant noise by employing GAN with Log-mel filter banks instead of wavelet Attained performance and robustness Pascual et al. [26] 2017 Application of GAN at waveform level Unmanned aerial vehicles (UAV's) Wang et al. [27] 2018 Noise filtering to avoid information loss during remote sending Benefit: image de-noising Qiuhong et al. [28] 2019 High volume data redundancy by applying compression NLP-natural language processing (NLP) Li et al. [29] 2018 Text regression model for association of text data and social outcome Advantage: data analysis with limited labelling Lin et al. [30] 2017 Rank-Gan for data analysis and quality assessment using rank metric Qian et al. [31] 2018 Event factuality identification using Ac-GAN by learning syntactic inform and address imbalance among factuality values Advantage: reduced reliance over annotated text Health Che et al. [32] 2017 Hergan for synthetic health data generation with limited electronic health record (HER) Hwang et al. [33] 2017 Disease prediction using AC-GAN and stacked auto-encoder Rezaei et al. [34] 2018 Semantic segmentation and disease classification by selective weighted loss Advantage: address Data imbalance Fake audio, video and image generation Choi et al. [35] 2018 Stargan for fake image generation by using deep CNN Achieved high classification accuracy Nataraj et al. [36] 2019 Detection of fake images using co-occurrence matrices along with deep learning Achieved good generalization and very high classification accuracy Agriculture Suarez et al. [37] 2017 Strength assessment of vegetation against normalized difference vegetation index (NDI) by applying Conditional GAN Barth et al. [38] 2017 Cyclegan for gap reduction between synthetic and empirical image data set Advantage: ease of translation of color and textures Music Yang et al. [39] 2017 Midinet-generation of musical notes by using CNN GAN Comparison of midinet was also made with Google's melodyrnn from scratch Advantage: combine existing melodies as well as generate melodies from multiple channels Dong et al. [40] 2018 Misegan-generates symbolic music i.e. piano-rolls of five tracks and four bars i.e. Bass, drums, guitar, piano and strings Proposed three models known as call jamming model, composer model and hybrid model for music generation Yu et al. [41] 2019 Simultaneous generation of lyrics-conditioned melody and association alignment between syllables of given lyrics by using conditional deep Lstm generator and discriminator Deep generative model for generation of melody and notes of predicted melody Weather forecasting Chen et al. [42] 2018 Scenario generation used for weather forecasting, however errors become more pronounced when the typhoons move into deep sea Advantage: generates wind patterns and weather forecasts based on historic data Ruttgers et al. [43] 2018 Predict track of typhoons by using satellite image. If information about surface pressure, velocity and sea surface temperature are added the results can become more accurate Advantage: predict the typhoon center as well as the movement of clouds with certain margins for error Sports Jiao et al. [44] 2018 Distinguishes correct performed golf swings Achieved accuracy and precision both in identification as well as classification of golf swings Deverall et al. [45] 2017 Conditional GAN for designing athletic shoes based on google gnet Achieved shoes categorization according to their physical attributes as well as functional type Internet of things (IoT) Wang et al. [46] 2018 Use of Bayesian methods for Radio Frequency (RF) sensing for IoT Advantage: overcome limitation of limited data availability by introducing an offline stage Zhao et al. [47] 2018 Individual identity authentication by applying open-categorical classification model based on gan (occ-gan) Advantage: better results are achieved than other methods like one-class support vector machine (oc-svm) and one-versus-rest support vector machine (ovr-svm) Genetic engineering Dizaji et al. [48] 2018 Gene expression profiling by using semi-supervised GAN for expression inference Use landmark genes instead of whole gene expressions Simulation and modeling Hassouni et al. [49] 2018 Generating realistic simulation environments that simulates daily activities of users Advantage: generate realistic sensory data that related to daily activities of users Pöpperl et al. [50] 2019 Synthetic ultrasonic signal simulation using conditional gans (cgans) Advantage: real like data augmentation for automotive ultrasonic and also adaptive to external influences Market prediction and forecasting Tian et al. [51] 2019 A technique for predicting the consumption of energy Advantage: outperforms the standard approaches i.e. information diffusion technology (idt), the heuristic mega-trend-diffusion (hmtd) technology and the bootstrap technique Advantage: scalable to perform forecast for demand of electricity and the traffic supply Luo et al. [52] 2018 A technique for predicting the prices of the crude oil using adaptive scales continuous wavelet transform (as-cwt) Advantage: more accurate forecasts as compare to naive forecast (nf) model and other nonlinear models i.e. deep belief networks (dbns) Drug discovery Zhavoronkov et al. [53] 2019 Drug discovery using generative modelling. i.e. generative tensorial reinforcement learning Useful for the discovery of new micro molecule kinase inhibitors and DNA damage response (DDR1) inhibitors [54] 2018 Floor plan image identification and creation Floor design images get translated into programmatic patches of colors Wang et al. [55] 2019 Double P-buried layers MISFET (DP-MISFET) is proposed Simulated and characteristics are analysed by the Sentaurus TCAD tool Road network generation and path planning Albert et al. [56] 2018 Novel technique to simulate real like urban designs fine-tuned with urban land-use inventory Advantage: synthetic urban pattern is formulated to qualitatively regenerate the spatial structures perceived in urban designs Mohammadi et al. [57] 2018 Precise and reliable paths for navigation software including wayfinding for disabled people, route identification for evacuations, robotic navigations for autonomous vehicles Advantage: high accuracy of the classification task with high quality of the generated paths is achieved Testing and validation Zhang et al. [58] 2018 Unsupervised model for automatic verification and validation of the consistent behavior of autonomous vehicle driving systems Real time validation is also achieved Segura et al. [59] 2016 Metamorphic verification and validation approach for identifying unusual behaviors of autonomous vehicle systems along with input validation Zhihui Li et al. [60] 2019 Create fuzzing data using Wasserstein GANs (wgans) Advantage: does not require specification of input data Significant for testing of industrial control systems (icss) Software designing and development Li et al. [61] 2019 Layoutgan-Wireframe designing i.e. layouts generation of relational graphic elements to wireframe images by modelling geometric relations of different types of two dimensional elements Advantage: introduction of wireframe rendering layer which produce a set of relational graphic controls Liu et al. [62] 2018 Treegan for source code generation Advantage: syntax-aware sequence generation Fault prediction Gao et al. [63] 2019 ASM1D-GAN a model to identify the faults by extracting features related to faults from real fault samples and create the similar one Advantage: integration of data creation and fault determination Zhou et al. [64] 2019 Synthesize vibrational fault samples using a technique of global optimization Advantage: feature extraction of feature using limited number of samples and its effective representation using auto-encoder Filter the non-compliant synthetic samples which are not useful for reliable fault diagnosis Zheng et al. [65] 2019 Gan-fp utilizes multiple GANs to create training samples and an inference network in parallel to predict failures for newly crafted samples Improved performance as well as significant socio-economic impact Text generation Subramanian et al. [66] 2018 Ability to create sentence outlines using an adversarial model which learns the distribution of sentences in a hidden space persuaded by sentence encoder Advantage: produce real like samples with multinomial sampling Liang et al. [67] 2017 Create useful distractors Advantage: achieves comparable performance to a frequently used word2vec-based method for the Wiki dataset Malware detection Dahl et al. [68] 2013 Employ random projections to decrease the dimension of the original latent space Achieved improved classification results commerce and in capturing data for remote sensing images. Wang et al. [27] the authors try to solve the problem of noise in UAVs. Noise causes loss of content during remote sensing of images by UAVs. The authors discovered the association between noisy and benign images and establish a theoretical model that helps in getting better images by removing noise. The resultant images have clearer and well defined texture and edges. The deniosed images were compared with other images and the results clearly indicated that this method has achieved better results. Another problem faced in image processing by UAVs is the presence of large amount of unnecessary data. The occurrence of hefty volume of data results in problems during post processing phase. In [28] the authors try to overcome this issue through the use of compression techniques to compress the image taken by UAV. This compression is done by an encoder which is placed in front of the generator whereas to improve the quality of compressed image, discriminator is used. This is achieved by training the compressed image along with the real image. The experiments conducted in the paper show in comparison with traditional compression methods this method has achieved better compression results and is also able to achieve hierarchical synthesis of non-essential parts of the image (trees and rivers) with essential components like houses and roads. [70] 2017 IDSGAN used to generate malware attacks which can bypass the different intrusion detection systems (IDS) Achieved high degree of evasion against IDS Heusel et al. [71] 2017 Framework to target portable executable (PE) anti malware systems in an offensive way Advantage: proved to be an effective model to identify the vulnerabilities of the anti-malware systems Arjovsky et al. [72] 2017 Model to generate malware instances for Black-Box Attacks Based on GAN Gulrajani et al. [73] 2019 Adversarial sample generation to launch attack against malware classifiers Singh et al. [74] 2019 Generative model for malware images that could be used to boost classifier's performance by performing data augmentation Advantage: leveraged to generate malware images which would alleviate the problem of public sharing of the dataset Odena et al. [75] 2016 Class-conditional image synthesis model to segregate generated samples to their respective malware category without any manual intervention Anderson et al. [76] 2016 Model to bypass a detector of web domain generation algorithm Rigaki et al. [77] 2018 To adapt malware communication to force misclassification of new generation Intrusion Prevention Systems (IPS) Advantage: effective at modifying malware traffic in order to remain undetectable Labaca et al. [78] 2019 GAN to inject automatic byte-level perturbations into PE files to fool the classifier Kawai et al. [79] 2020 Bypass malware defenders by adding benign to the original malicious code Advantage: resolve the problem of creating an huge collection of APIs to bypass the detectors BlockChain Zheng [80] 2020 GANs based technology for exchange of secret key which also overcome the block chain problems of security, recovery of lost key and communication inefficiency Advantage: a new avenue is opened the exchange of secret key which us reliable and adaptive as well as efficient In [29] the authors propose a text regression model which associates text data and social outcomes. The advantage of this model is its ability to work with datasets which have limited labeling. In [30] a new GAN RankGAN is proposed which uses policy gradient technique. RankGAN has the ability to analyze and rank human and machine written sentences. RankGAN is thus able to analyze a set of data and indicate their quality through a ranking system. The semantic task of identifying event factuality is of paramount significance in NLP and traditional research deeply depends on annotated texts. The research work [31] proposes a model for mining indispensable features associated with event factuality from raw texts, used those features as input, and subsequently recognizing event factuality by employing GANS with Auxiliary Classification (AC-GAN) which helps to acquire supplementary syntactic content and overcome the disparity among factuality figures. Recently GAN generated molecules were used to attack protein targets in critical circumstances like cancer, fibrosis and inflammation. GAN generated molecules have also been introduced into mice recently to help in health related experiments. Most of the GAN models used in health sector rely on huge volume of labeled data. In order to overcome this reliance on labeled data in [32] a new model ehrGAN is proposed which can work with limited electronic health records (HER). EhrGAN mimicks real patient records and is able to data and provide plausible labeled data sets. Hwang et al. [33] GANs are used for disease prediction. A combination of auxiliary classifier (AC-GAN) and stacked autoencoder is used for disease prediction with 98.5% accuracy results. Finally, Rezaei et al. [34] the problem of data imbalance is tackled with the help of a new model. Most of the models have a tendency to be biased towards healthy data which needs to be avoided in clinical applications. In the new model this data imbalance is tackled through selective weighted loss and the use of semantic segmentation and disease classification. Choi et al. [35] a new scalable approach StarGAN for image translation is proposed and the core benefit of StarGAN as compared to other models is that can be used for multiple domains whereas previously a new model had to be built for every image pair belonging to different domains. Experiments conducted show the flexibility as well as better quality of images produced by StarGAN. A major problem faced in Image generation by GANS is the creation of fake images. Nataraj et al. [36] a new method has been devised for detecting fake images by devising a deep CNN based model. Experiments conducted on a dataset of more than 56,000 images show that this model was able to achieve 99% accuracy in classification of images. GAN's have also been applied in agriculture. In [37] in acquiring a precise estimate of NDVI however, one drawback was that a comparison with results obtained from previously used techniques couldn't be made due to the novelty of this approach. In the field of agriculture robotics there is a requirement for large synthetic datasets. Manual modeling is used to optimize such large synthetic datasets by improving the color, texture and geometry of these 3D modeled plants with a residual gap between synthetic and empirical content. In [38] the authors have used a cycle GAN to bridge the gap between synthetic and empirical image dataset of agricultural crops. This is done by increasing the realism of synthetic images. The authors used a dataset of 10,500 synthetic images, 225 unlabeled empirical images and 50 empirical annotated images. Experiments conducted by authors showed that there was improvement in the optimization of synthetic data in both qualitative and quantitative terms. It was also shown that certain factors like color and texture of synthetic images can be more easily translated as compared to certain other features like plant morphology which can't be translated. GANs are also increasingly applied in the field of music. More specifically the have been applied to generate musical notes as Yang et al. [39] where authors suggest a CNN based GAN i.e. Midinet for the generation of musical notes. MIDINET has also the capability of learning (bothe generator and distributor) the distribution of musical notes as an iterative process. The new music generation model can generate melodies from scratch, combine existing melodies and is also capable of generating melodies from multiple channels. Depending upon the input and specifications this model is capable of generating different types of music. A comparison of MIDINET was also made with Google's MelodyRNN model and it was found that MIDINET gives comparable performance. Dong et al. [40] another musical model (MiseGAN) that generates symbolic music is proposed. The authors propose three models which they call Jamming model, composer model and hybrid model. The dataset used for the training of these models consist of one hundred thousand bars of rock music. The model is capable of generating piano-rolls of five tracks and four bars i.e. bass, drums, guitar, piano and strings. However as admitted by authors themselves the proposed model needs a lot of improvement as musically and aesthetically it is still far behind music generate by humans. Yu and Canales [41] a novel model for melody generation from lyrics is proposed, comprising of a conditional deep LSTM generator and discriminator subject to lyrics. Furthermore, simultaneous generation of lyrics-conditioned melody and association alignment between syllables of given lyrics and notes of predicted melody is also achieved. The proposed lyrics-tomelody generative model made it possible to infer plausible and tuneful sequences from lyrics. GAN's are now also increasingly used in the field of weather forecasting. A recent example of this is [42] in which a new data driven (consisting of wind and solar time data sets) method is proposed for scenario generation used for weather forecasting. This method generates wind patterns and weather forecasts based on different sets of input consisting of labeled data. Both event based (windy or rainy day) and time based (solar generation on a particular day of the year) event sets can be used as inputs. GAN was able to generate scenarios based on historic data sets of weather collected from multiple sites. The results of the paper clearly demonstrate that GANS are very suitable for scenario generation of wind and solar. A further use of GAN was made Rüttgers et al. [43] where authors propose a model which can predict track of typhoons by using satellite images. These satellite images taken at 6 h interval and submitted as inputs for GAN. The model was able to predict the typhoon center as well as the movement of clouds with certain margins for error. In case of typhoon center prediction, the errors observed were less than 80 km in 42:4% cases, within a range of 80-120 km for 32:1% cases and above 120 km for 25.5% cases. These errors become more pronounced when the typhoons move into deep sea, are about to hit mainland or sudden change of their course. If information about surface pressure, velocity and sea surface temperature are added the results can become more accurate. Jiao et al. [44] a new GAN model is proposed which takes as data input swings performed by different golf players and distinguishes correct performed golf swings from incorrectly performed golf swings. The input data consists of 13 signal and each signal in turn comprises of 1500 data samples. The model tells in the output the player to which the golf swing belongs and whether the swing belongs to correct or incorrect category. The evaluation results prove that the proposed approach has achieved accuracy and precision in identification of golf swings as well as in its classification of them in the category of correct or incorrect swings. Deverall [45] a conditional GAN is used to design athletic shoes. A dataset consisting of 50,025 images of shoes is used. In the dataset shoes are further categorized according to their physical attributes as well as functional type. In the first step a GAN is designed and then trained on several athletic shoes. In the second step, a classifier is built based on GoogLegNet. This classifier helps to classify the shoes according to their functional types. In the third step a binary classifier for each value of dataset was created. The biggest hurdle faced in this entire process was the vague and subjective nature of attribute comparison dataset. The authors emphasize the need for an effective attribute classifier as future work which can help to enhance the functionality of this model. Wang et al. [46] a GAN is proposed for RF sensing for IoT and proposed model was evaluated for fingerprinting, event recognition and sign monitoring. The framework is a two phased model where the first is offline stage in which training data is used to train GAN. When the training has been completed then in the second stage this trained data is fed as input to GAN which uses this data for prediction by using different methods such as Bayesian methods. In [47] authors construct an open-categorical classification model based on GAN (OCC-GAN) to solve the problem faced by traditional classifiers in individual identity authentication. This model achieves better recognition results due to auxiliary training and also increases the efficiency. Through experiments it is proven that OCC-GAN achieves 90 percent recognition results with a recall rate of 97 percent. These results are better than other methods like one-class support vector machine (OC-SVM) and one-versus-rest support vector machine (OvR-SVM) but major limitation is small volume of training data. Moreover, to improve the training there is a need to consider other methods like Auxiliary Classifier GAN (ACGAN) for training purposes. GAN has also been used for gene expression profiling and the biggest challenge faced in the field gene profiling is cost associated with genome profiling of very large libraries. A remedy for aforementioned problem is to use landmark genes instead of complete gene expressions due to its complexity and the cost of process. Dizaji et al. [48] the authors propose a new semi-supervised GAN for expression inference model and therefore instead of taking a completely supervised approach which is dependent upon a huge volume labeled dataset consisting of paired landmark and target gene expressions, inexpensive unlabeled content i.e. landmark genes are used. The GAN synthetic data is used as additional training pairs to bring enhancement in the training performance and the use of semi supervised learning also results in more reliable and better training of inference data. GAN has also been used for simulation and modeling and it is known fact for reinforcement learning (RL) algorithms, simulated environments are very worthwhile testbeds but they essentially require to have an appropriate level of realism. The simulator shall have access to real content to generate real while appropriately addressing the privacy issues. In [49] the authors propose to use GANs for generating realistic simulation environments and applied a prevailing simulator to simulate routine tasks of users and to create realistic sensory logs that supplements such routine tasks. The simulator can easily be trained and tested by the GAN based approach and assessment demonstrated same level of performance on synthetic data as delivered real dataset. In [50] the authors presented a novel using conditional GANs (cGANs) method for synthetic ultrasonic signal simulation and as per the authors prerogative, it is the foremost data augmentation technique for automotive ultrasonics which is also adaptive to external influences. The performance of cGANs will surely uplift realistic environment simulation to a new horizon. GAN has also been used for prediction of consumption and market trends including price. Prediction of energy consumption is very vital for good energy management, power distribution planning and optimized resource utilization. In [51] , the authors proposed a parallel systematic prediction scheme for the calibrating consumption of energy which makes use of a limited number of the original content to generate the synthetic data. Moreover, it constitutes a hybrid dataset comprising the original and synthetic data to train the model to be used for prediction. The results established that the proposed technique outperforms the prevailing baseline schemes i.e. information diffusion technology (IDT), the heuristic Mega-trend-diffusion (HMTD) as well as bootstrap method. The proposed organization can also be enhanced to predict the load of electricity load as well as its flow. The paper [52] proposes a novel approach based on a supervised GANS to predict the price of crude oil using Adaptive Scales Continuous Wavelet Transform (AS-CWT). The supervised GAN based model predicted relatively precise and accurate forecasts in comparison to naive forecast (NF) and other nonlinear model i.e. Deep Belief Networks (DBNs) particularly while interacting with low volume of data. In this approach Continuous Wavelet Transform (CWT) is applied to decompose price of oil commodity into various time oriented components, such as the months, years weeks and sequence of days so that the resultant values of time series can be utilized as inputs for prediction model. The forecasting performance is improved by the application of adaptive scales in the CWT method. In [81] authors presented a GAN based chest x-ray detection based model for detection of novel coronavirus COVID-19 associated pneumonia for a small dataset. To validate the usefulness of the model, authors use limited volume of the original dataset and utilized 90% of synthetic images for model training. Drug discovery is an exhaustive and time taking process. It starts with testing thousands of molecules to just explore a few lead-like compounds which are likely to be useful. But, among the explored ones very small percentage of those qualified clinical trials. The advanced utilization of GANs in the domain of drug discovery is an inspirational illustration of cutting edge application of AI in all facets of life specially the pharmaceutical industry. The article [53] presents a substantial advancement drug discovery using Generative modeling. The authors devised a model i.e. generative tensorial reinforcement learning (GENTRL), has enabled the discovery of new minor molecule i.e. kinase inhibitors, DNA damage response (DDR1) inhibitors. The advancements in artificial intelligence has also introduced into the design field, resulting in the improvement of computer-aided design. The authors in [54] proposes generation and recognition of floor plan using GAN such that images of the floor plan processed by GAN based model can be translated into programmatic patches of colors. Wang et al. [55] , a performance-improved AlGaN-/ GaN-Based metal-insulator-semiconductor field effect transistor (MISFET) with double P-buried layers MISFET (DP-MISFET) is proposed, simulated and attribute analysis is conducted by the Sentaurus TCAD tool. In [56] the author proposes a novel approach to simulate hyperrealistic urban designs using GANS which is trained with a dataset of inventory used by urban land globally. A synthetic urban pattern is formulated that qualitatively duplicates the spatial organization logged in urban designs. A collection of 30, 0000 training samples of cities was used and results demonstrated that a basic, unconditional GAN based model can create real like urban designs that capture the impressive diversity of urban forms. In [57] , the author proposed the use of GANs to acclaim correct and reliable tracks for navigation applications including wayfinding for disabled people, path planning for evacuations, robotic navigations as well as navigation for autonomous vehicles. The model equipped users with synthetic paths that assisted navigation from local environment to reach a desired destination and notably impressive classification accuracy is achieved with reliable synthetic paths. In [58] authors proposed a Deep neural network model that works in unsupervised manner for online verification and validation of self-governing vehicle driving system. The model generates various driving scenes along with various weather conditions next to reality using GAN. In [59] , the authors applied metamorphic verification and validation approaches for the identification of unusual patterns and behaviors of self-directed vehicle driving systems as well as validation of input images. Fuzzy testing is another important approach for the identification of software vulnerabilities and errors and in such kind of testing a huge volume of input data is essentially required by the target program and therefore, a critical requirement of fuzzing evaluation is the generation of fuzzing samples. Several customary approaches to create fuzzing data samples have devised including model and random fuzzing data generation. These practices essentially demand the detailed description of input data and its corresponding representation or perform reverse engineering to conduct the said task which is also a challenge. In [60] , the authors proposed a novel scheme for generation of fuzzing data using Wasserstein GANS which does not require specification of input data and corresponding representation. Thus, the proposed method is significant for industrial control systems (ICSs) testing. The software graphical design is imperative for graphical interface and is n significant interactive tool for communication between development team and software user analogous to magazine layouts to web design. A wireframe is a 2D visual and graphical illustration regarding the layout of web page with special attention page space utilization addressing the content allocation and prioritization, availability and placement of features as well as overall behaviors. In [61] , the authors proposed GANS for generating graphical design layouts of associated interface controls by learning representation of geometric relations of various kind of graphical controls or elements. The model can generate images in pixel-level i.e. can render a set of relational graphic controls unlike customary GANs. Moreover, a rendering layer was devised to rasterize the synthetic visual interface controls to wireframe images, leaving it appropriate to influence CNNs as discriminator for optimized layout. Beside wireframe layout structuring, GAN can impressively be utilized for generation of source code generation to assist the job of software programming and development. In numerous real-life applications, there is a requirement to create sequences in a formal computer language with the limitation of language grammar. For instance, to validate the operational efficiency of a database, a collection of SQL queries may be generated similar real queries but also comply the language syntax of the target database. Creating such sequences is a tedious job and Liu et al. [62] , the authors devised a GAN based syntaxaware i.e. TreeGAN for generation of sequences. The experimental results showed that the proposed technique outperforms the state of the art adversarial learning approaches for generation of syntax-compliant sequences. The of the proposed approach is well demonstrated. In [65] the author proposed a GAN based method for prediction of failure and the GAN-FP model employs the concept of multiple GAN networks for simultaneous generation of input samples and construction of an inference network for prediction of failures for synthetic samples. The tuning of inference network is performed by optimizing an objective function of weighted loss objective considering only original failure and non-failure samples followed by the tuning of model using a next GAN to ensure the uniformity between real and synthetic samples. This novel approach will not only improve performance of modeling process but also have substantial socio-economic impact. Latest developments in generative models is determined by two standards i.e. autoregressive and adversarial models. In [66] the authors devised a hybrid approach with the intent of learning generative models of textual processing. The model creates an abstract outline for sentence and afterwards create words in a sequentially manner while depending on outline as well as the previous outputs. The sentence outlines are created using an adversarial model which learns the distribution of sentences in a hidden space persuaded by sentence encoder which ensures robust and informative conditioning for the autoregressive phase. The quantitative assessments showed that attributes extracted from synthetic outlines can drive the autoregressive model to generate real like samples, comparable to stat of the are existing models even with multinomial sampling for high temperatures. Distractor creation is a critical activity for generation of questions like fill-in-the-blank. In [67] , the authors propose a GAN based generative model to generate intelligent distractors by utilizing context information and not relying on the correct answer and therefore is entirely different from existing ontology or similarity based method. The training is done over on the corpus of Wikipedia, predict Wiki entities as distractors and is validated on two datasets comprising of biology questions acquired from Wikipedia as well real educational institutional exams. Experimental results showed that context-based approach attained impressive performance in comparison to a commonly used word2vec-based approach for the Wiki dataset. GAN's are also increasingly used in the field of malware detection i.e. to attack protected systems and highlight the various weaknesses and blind spots of the anti-malware defense systems. An adversary can bypass the malware classifier or the defense system by making slight perturbation to malware file that retains its malicious nature but made to be classified as benign [68, 69] [75] proposed mechanism of class-conditional image synthesis model to segregate generated samples to their respective malware category without any manual intervention, where the generator is conditioned with class label and the discriminator is tasked to predict the class label. In DeepDGA [76] the authors attempt to bypass a detector of web domain generation algorithm that identifies humangenerated domains from its automatically generated counterparts. Rigaki et al. [77] proposed to adapt malware communication to force misclassification of advanced systems of Intrusion Prevention. The malicious code was customized to duplicate the network traffic of the chat application and proposed work suggested that GANs can be successful at modifying malware traffic in order to remain undetectable. The authors implemented a GAN to inject automatic bytelevel perturbations into PE files [78] . The authors proposed an approach to bypass detectors by inculcating benign features to the malicious code [79] . The approach introduced a loss calculation Layer to Generator to resolve the issue of generating massive number of APIs for detector avoidance. The authors [80] presented a GANs based key secret-sharing technology to overcome blockchain problems of security, lost key recovery and communication efficiency. The proposed approach deals the secret-sharing process as a classification subject and main theme is to consider visualization of secret as an image for the exchange of secret key. The proposed scheme provides a novel avenue for the sharing of secret key scheme and also demonstrated results show that the apporach is reliable, adaptive and efficient in communication. In this research paper, contrary to other research surveys which are restricted to domains of computer vision, image processing and natural language processing, holistic view of GAN applications have been presented to date. A gap was realized that several surveys have been done regarding GANs but none have focused on the review of GANs over multi-disciplinary domains. Therefore, an extensive survey is conducted in this research work and utility of GANs in diversified domains such as healthcare, IoT, agriculture, Intelligent transportation, drug discovery and pandemic control like covid-19 software development, natural language processing, entertainment and cyber security have been summarized. In this section, we highlight the problems of the generative models faced by the researchers. a. Training of GANs is a challenge due to establishment of the equilibrium [82] , however, methods have been proposed for making it possible but it is still an active research area [83] and a problem b. Lack of benchmarking evaluation criteria for assessment of the stability of the GAN [84] and similarly the establishment of equilibrium progress is underway in this regard but it is still a menace [85] . c. Convergence is difficult to achieve as both discriminator and the generator models updates their cost function independently. Significant research has focused on identifying improved training algorithms [86] for GANs as well as achieving improved conceptual comprehension of training dynamics [87] . d. The generator flops due to production of limited varieties of samples. It is called mode collapse [88] . A partial is common but a complete collapse is rare. Reference [89] [90] [91] integrate samples created by various models to overcome the limitation. e. The problem of Diminished gradients [82] exists. It is possible that the discriminator performs in an outstanding manner that generator's gradient diminished and ends up with no learning or learning is very slow. On the other side, if discriminator does not perform well then the generator does not have precise and reliable input and the loss function cannot present the true picture. The paper reference [92] proposed a model for the problem of diminishing gradient. f. Highly sensitive to the hyper parameter selection [93] . Reference [94] addressed the issue. g. Overfitting due to in equilibrium [95, 96] unbalance between both discriminator and the generator models. Therefore, detecting [97, 98] and overcoming overfitting is a good problem area. h. Lack of a good objective function is another problem of GAN [99] . The lack of god objective functions this makes it very challenging to perform comprehensive evaluation of models. GAN has tremendous potential in generative modeling and its application in numerous domains and therefore this review focuses on the identification of those applications. Although a lot has been done in this domain, there is still a desperate need for research in the GAN based applications (Fig. 3 ). Keeping in view literature studied, it is found that generative models has the following challenges: a. The research [100] in adversarial machine learning, its goal and capability was proposed in [101] . The authors [102] crafted an inversion attack based on the linear regression over a bespoke but vulnerable drug information system. Current research has shown that the coun- terintuitive attributes of DNNs affects their security and regardless of considering adversarial samples in training models to improve the robustness of model [103] , solutions are still vulnerable to overcome the aforementioned risk. Therefore, secure generative model is a dire need of time and research on such avenue will be very promising e.g., Bayes deep networks with prior information [104] . Overdue security introduces overheads and also affect generalization performance of model that subsequently challenges their application. Hence, moderate balance is recommended to facilitate the practical usage. b. Currently, formally standardizing [105] assessment techniques for generative models is still at the initial stage. There are no standard metric available as well. Therefore, it is a challenge for establishing a generally accepted and well-articulated standard defining the evaluation procedure of generative models as well evaluation criteria. c. GANs are very famous for synthetic data generation, however, models designed for synthetic data generation have notable limitations. The synthesized data can represent multiple attributes of real data, but does not exact clone of the real data. Moreover, any processing over synthetic data requires verification [106] against the counterpart real dataset. Furthermore, there is requirement of a standard verification procedure to conduct test and perform the quantitative comparative analysis of real and synthetic outputs. The proper training of model must be ensured to guarantee that it is not creating the expected outputs because of assumptions made for synthetic data. Due to the large variety, volume and velocity of data, it is not possibility to generate universal synthetic data generation tool however generation of bespoke synthetic datasets can be the achievable target. d. Generative models learn patterns in the real data during the data generation but presence of any noise may be invisible to them that can be a critical issue [106] . e. Robustness of generative model is also a big challenge. The quality of synthesized data heavily relies on the superiority of the model being used for the process as the generative models can be vulnerable to statistical noise i.e. modifications that can fool the model to misclassify the data [107] . Therefore, such models cannot be utilized in production environment if they are not immune to adversarial attacks. f. Optimization of generative models [108] , generalization performance and overhead is required to design optimized learning algorithms. g. Maturity of generative adversarial learning [109, 110] in the context of quantum computing has been theoretically presented and demonstrated to possess the potential of revealing an unsual benefit over its conventional counterpart can be a future challenge. Representation learning: a review and new perspectives Deep learning A survey of data mining and machine learning methods for cyber security intrusion detection Theano: new features and speed improvements Backpropagation applied to handwritten zip code recognition Neocognitron: a self-organizing neural network model for a mechanism of visual pattern recognition. In: Competition and cooperation in neural nets Threat of adversarial attacks on deep learning in computer vision: a survey Non-uniformity correction for photoncounting detectors using double GANs Artificial intelligence based cloud distributor (AI-CD): probing low cloud distribution with a conditional generative adversarial network A survey of image synthesis and editing with generative adversarial networks Generative adversarial networks: introduction and outlook Stable and improved generative adversarial nets (GANS): a constructive survey Comparative study on generative adversarial networks An empirical study on evaluation metrics of generative adversarial networks Recent progress on generative adversarial networks (GANs): a survey Generative adversarial networks in computer vision: a survey and taxonomy A review on generative adversarial networks: algorithms, theory, and applications Learning from simulated and unsupervised images through adversarial training Dagan: deep dealiasing generative adversarial networks for fast compressed sensing mri reconstruction Image-toimage translation via group-wise deep whitening and coloring transformation Unsupervised image-to-image translation with generative adversarial networks Stackgan: text to photo-realistic image synthesis with stacked generative adversarial networks Generative adversarial text to image synthesis Robust speech recognition using generative adversarial networks Exploring speech enhancement with generative adversarial networks for robust speech recognition SEGAN: speech enhancement generative adversarial network An effective image denoising method for UAV images via improved generative adversarial networks UAV image high fidelity compression algorithm based on generative adversarial networks under complex disaster conditions Semi-supervised text regression with conditional generative adversarial networks Adversarial ranking for language generation Zhu Q Event factuality identification via generative adversarial networks with auxiliary classification Boosting deep learning risk prediction with generative adversarial networks for electronic health records Disease prediction from electronic health records using generative adversarial networks Multi-task generative adversarial network for handling imbalanced clinical data. CoRR Stargan: unified generative adversarial networks for multi-domain imageto-image translation Detecting GAN generated fake images using co-occurrence matrices Learning image vegetation index through a conditional generative adversarial network Optimising realism of synthetic agricultural images using cycle generative adversarial networks MidiNet: a convolutional generative adversarial network for symbolic-domain music generation Musegan: multi-track sequential generative adversarial networks for symbolic music generation and accompaniment Conditional LSTM-GAN for melody generation from lyrics Model-free renewable scenario generation using generative adversarial networks Typhoon track prediction using satellite images in a generative adversarial network Golf Swing data classification with deep convolutional neural network Using generative adversarial networks to design shoes: the preliminary steps RF sensing for internet of things: a general deep learning framework Research on the opencategorical classification of the internet-of-things based on generative adversarial networks Semi-supervised generative adversarial network for gene expression inference Using generative adversarial networks to develop a realistic human behavior simulator Realistic ultrasonic environment simulation using conditional generative adversarial networks Data driven parallel prediction of building energy consumption using generative adversarial nets Oil price forecasting using supervised GANs with continuous wavelet transform features Deep learning enables rapid identification of potent DDR1 kinase inhibitors Architectural drawings recognition and generation through machine learning Technology computer aided design study of GaN MISFET with double p-buried layers Modeling urbanization patterns with generative adversarial networks Path planning in support of smart mobility applications using generative adversarial networks Dee-pRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems A survey on metamorphic testing An intelligent fuzzing data generation method based on deep adversarial learning Layoutgan: generating graphic layouts with wireframe discriminators TreeGAN: syntax-aware sequence generation with generative adversarial networks ASM1D-GAN: an intelligent fault diagnosis method based on assembled 1d convolutional neural network and generative adversarial networks Deep learning fault diagnosis method based on global optimization GAN for unbalanced data Generative adversarial networks for failure prediction Towards text generation with adversarially learned neural outlines Distractor generation with generative dversarial nets for automatically creating fill-in-the-blank questions Large-scale malware classification using random projections and neural networks Adversarial perturbations against deep neural networks for malware classification Wasserstein generative adversarial networks GANs trained by a two time-scale update rule converge to a local Nash equilibrium Towards principled methods for training generative adversarial networks 73 The thirty-third AAAI conference on artificial intelligence Conditional image synthesis with auxiliary classifier GANs Deepdga: adversarially tuned domain generation and detection Bringing a gan to a knife-fight: adapting malware communication to avoid detection ARMED: how automatic malware modifications can evade static detection? Improved MalGAN: avoiding malware detector by leaning cleanware features GAN-based key secret-sharing scheme in Blockchain Elghamrawy S Detection of coronavirus (COVID-19) associated pneumonia based on generative adversarial networks and a fine-tuned deep transfer learning model using chest X-ray dataset Idsgan: generative adversarial networks for attack generation against intrusion detection Evading machine learning malware detection Intriguing properties of neural networks Generating adversarial malware examples for black-box attacks based on GAN The numerics of gans Deep neural networks are easily fooled: high confidence predictions for unrecognizable images Multi-agent diverse generative adversarial networks Boundaryseeking generative adversarial networks Unrolled generative adversarial networks Loss-sensitive generative adversarial networks on lipschitz densities Twostage transfer surrogate model for automatic hyperparameter optimization A large-scale study on regularization and normalization in GANs Optimizing the latent space of generative networks Improved techniques for training gans Stargan: unified generative adversarial networks for multidomain imageto-image translation Quantitatively evaluating GANs with divergences proposed for training Generalization and equilibrium in generative adversarial nets (GANs) Adversarial learning The security of machine learning Privacy in pharmacogenetics: an end-to-end case study of personalized warfarin dosing Towards Bayesian deep learning: a framework and some existing methods Learning to repair software vulnerabilities with generative adversarial networks Analysis of classifiers robustness to adversarial perturbations Anomaly generation using generative adversarial networks in host-based intrusion detection Distillation as a defense to adversarial perturbations against deep neural networks Deep generative design: integration of topology optimization and generative models Quantum generative adversarial learning Woerner S Quantum generative adversarial networks for learning and loading random distributions Proceedings of the international conference on neural information processing systems (NIPS 2014) Multiresolution grayscale and rotation invariant texture classification with local binary patterns Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations Acknowledgements The authors would like to express their gratitude for the institutional support and research grant of Higher Education Commission (HEC) of Pakistan under International Research Support Initiative Program (IRSIP). Conflict of interest The authors declared that they have no conflict of interest.