key: cord-0054834-rgukgs6u authors: Raychaudhuri, Soumya Jyoti; Manjunath, Soumya; Srinivasan, Chithra Priya; Swathi, N.; Sushma, S.; Nitin Bhushan, K. N.; Narendra Babu, C. title: Prescriptive analytics for impulsive behaviour prevention using real-time biometrics date: 2021-01-06 journal: Prog Artif Intell DOI: 10.1007/s13748-020-00229-9 sha: ed729818ba5a9a426c16993d72874a1afa48a841 doc_id: 54834 cord_uid: rgukgs6u The present biometric market segment has been captured by compact, lightweight sensors which are capable of reading the biometric fluctuations of a user in real-time. This biometric market segment has further facilitated rise of a new ecosystem of wearable devices helpful in tracking the real-time physiological data for Healthcare-related analysis. However, the devices in the smart-wearable ecosystem are limited to capturing and displaying the biometrics without any prescriptive analytics. This paper addresses this gap to analyse the human emotion space based on an individual’s state of mind over the past 60 min and employs Deep Learning and Bayesian prediction techniques to predict the possibility of an impulsive outburst within upcoming few minutes. A lightweight smart processing device mounted with sensors captures the biometrics of the user and calibrate the same to the mental state of the user on a scale of zero to hundred. The results reveal that the deep learning algorithm along with the Bayesian probability module can predict the future mood fluctuations of the user with lower error than the other contemporary models. The predicted mood fluctuations has matched with the actual mood changes of the experimental subject within [Formula: see text] min of the predicted time index in 93% of the cases and within [Formula: see text] min in 82% of the cases. Fitness and getting towards healthy lifestyle is the key resolution of today's millennial generation. The youth have started paying attention to the diet being consumed, with nutrition being centre of focus supplemented by the healthy workout regimen followed by them religiously. The rise in the number medical assisting devices have helped them to keep track of the calorie intake, distance walked, calorie burnt during the workout, heartbeat, temperature and other biometric data and helped them to keep a check on their physical fitness. While these measures have helped the people to maintain their physical fitness, the hectic lifestyle and untimely work schedules have taken a toll on the mental fitness. There is a rise in cases of anxiety disorders, depression, stress-induced psychosomatic disorders like hypertension, variations in blood sugar levels, migraines, peptic ulcers, heart diseases [38] and stroke. When a person is exposed to stressful environment over a long duration, it leads to a breakdown point and the person starts taking Impulsive decisions [6] . Impulsiveness also occurs when people are ambushed with sudden change in circumstances. Impulsivity leads a person to perform actions incautiously. Impulsiveness reduces rational thinking even in pragmatic individuals. These actions may have lasting impact which may be irrevocable. This may not only cause irreparable damage to on-self but also people around and sometime to the entire society in question. Impulsive rulings and amendments signed by leaders of states, the actions taken by an armed personnel, ruining collaboration between countries, friends, irrational step taken when a bad result or news is heard and even punishments given by impulsive parents are some of the instances that illustrate the severity in which things can take turn due to impulsiveness. These unpropitious situations could be prevented if there was a method to control the impulsivity and redirect the nutrition, calorie intake and also track the number steps walked. But there is lack of devices focusing on the mental health and well being along with prescriptive analytics. There is a dearth for devices that not only monitor the biometrics but also provide an analysis on the temperament of the individual based on variations in the physiology. Focusing on the diagnostic and prescriptive analytics for mental health tracking, this research attempts to build a wearable device by utilizing the variations in the biological parameters to track the rise in the impulsivity and alert the individual about the possibility of reaching a point of agitation. This can be very important step which can help people in calming down. On long-term usage, this would help the individuals in resolving psychical issues like anger management, reducing anxiety and agitation and take decisions rationally. This paper attempts to implement a prescriptive analyticsbased system that can predict the possibility of an impulsive behaviour by closely monitoring few physiological markers. In this context, the contributions of this paper are as follows. (i) Implementation of a biometric data sampling system which performs a synchronous sampling of heart beat, skin temperature and EEG signals along with the climatic parameters and user displacement data into a raspberry pi processor empowered data acquisition system (DAS) for deep learning-based analysis. (ii) The proposed system is capable of analysing the user emotional states over consecutive time frames to predict the possibility of an impulsive behaviour within a following time window and prescribes the user to act carefully. Long-term usage of such devises may even change the actual behavioural patterns of a user and help him deal with anxiety disorders and anger management problems more effectively. The rest of the paper is organized as follows: A detailed survey on the machine learning algorithms used to analyse the biometrics, contemporary work on healthcare analytics and wearable devices catering to healthcare is presented in Sect. 2. This is followed by the detailed discussion on the algorithms for processing biometric data acquired from sensors in Sect. 3. Section 4 elaborates on the proposed approach. Results and discussion on the analysis of the results is elaborated in Sect. 5. Section 6 concludes the paper. This section details on the contemporary work done on wearable devices, the sensors used in data acquisition of biometrics and the machine learning algorithms used in analysing the human biometric data. The primary objective of this literature survey is to find out the most significant biometric parameters that vary with rise and fall of agitation in human beings. Irritation, frustration, agitation, anger and getting startled are some of the different states of mind that may lead to impulsive behaviour causing increase in blood pressure [34] and higher pulse count per minute. Sudden rage and frustration may also get manifested as an emotional fever and a sudden rise in temperature has been observed [7] . Hyperhidrosis is one of the important indicators of anxiety, rage and impulsiveness getting built up in an individual [3] . Shortness of breath, intermittent sighing and wheezing is also a highly dependable indicator of emotional disturbance [32] . Electromyogram (EMG) which measures the electric signals flowing in muscles and nerve cells has also been proved to be one of the important indicators of emotional changes [22] . A disturbed mind filled with frustration causes increase in testosterone levels and reduces cortisol levels [26] . Human endocrine system has a defence mechanism for stressful situations and it triggers production of cortisol hormone to calm the person down. This volume of cortisol production acts as a very good index of impulsiveness. Brainwaves are one of the important biomarkers which show direct variations with the changes in emotions [35] . EEG which gives an index of neuronal activity, very efficiently indicates the mood swings of a person. [33] There are many other biological parameters that show variations with impulsive behaviour. Out of these available biological parameters, few parameters having reasonable correlation with increase in agitation and impulsive behaviour, resorts to non-invasive techniques and are easily available have been chosen for the current study. Temperature, EEG, heart rate have been chosen for this research as biomarkers for impulsiveness. Second objective of this literature survey is to identify the sensors that have the highest efficiency in measuring the biometric data accurately. The EEG signals from the human brain can be measured using dry electrode brainwave recording system [10] . Neurosky is a leading global manufacturer of such noninvasive EEG measurement devices which captures the variations in alpha, beta, gamma and delta waves accurately and maps them to meditation and attention signals [24, 25] and provides a very simple interface for further processing. There is a wide variety of temperature sensors available in the market with different methods of sensing temperature namely thermocouples, resistance temperature detectors, thermistors, etc. DS18B20 is a one-wire programmable resolution digital thermometer which generates voltage changes corresponding to temperature changes and gives the digital output which can be an ideal choice to interface with the digital DAS [20] . Heart beat rates can be measured using various techniques like Photoplethysmography (PPG), Phonocardiographic and Electrocardiography (ECG). Among these, PPG method using SEN-11574 [18] has the best accuracy in measuring the human heart rate while being least expensive and is easy to use. However, sensors based on Cortisol, Skin Conductance Response (SCR) and Galvanic Skin Response have been avoided to make the proposed invention non-invasive and reasonably priced. Various algorithms can be used to process the biometric data acquired using the biometric sensors to identify the patterns that may lead towards impulsiveness. K-Nearest neighbours (KNN), Support Vector Machine (SVM), Artificial Neural Network are some of the popular algorithms used to classify biometric data collected from multiple sources to identify diseases, detect tumours, etc. Deep Neural Networks are known for their high accuracy in retrieving hidden patterns in data and trends over time series data and thus have proven to have nearly 80% efficiency in prediction and classification depending on the type of biometrics and the class of related disease [15] . Studies have been attempted to achieve prediction of pathology by processing medical data as fuzzy time series data [2] Contemporary work in detecting mood changes, development of wearable devices and their efficiency in the classification of human tendency towards impulsive behaviour has been discussed in the following paragraph. Wickramasuriya et al. used Hilbert-Huang transform on offline EMG data to segregate negative states of mind and positive states of mind [39] . SVM-based emotion detection device is another contemporary device which utilized skin conductance response coupled with heart rate variation [19] . Researchers have tried to build a wearable emotion detecting device for the aid of the elderly [30] . Rafealle Grain et al. attempted to build a model that predicts emotion to identify the mood changes arising due to sedentary lifestyle [27] . Most of these devices were built based on skin conductance, cortisol collection, blood pressure variations, etc. and many of these devices employed invasive techniques. The methodology proposed in this paper focuses on building a wearable device that collects the biometric data using noninvasive approach and predicts the probability of impulsiveness using a deep neural network approach. This detailed literature survey gives clarity on the biometrics to be used to capture the changes in the emotional levels of a person. Clarity on the sensors needed to measure these biometric data has been achieved by this study and most relevant sensors can be selected to design the data acquisition system to interface with a processor. The processor will be designed with the most relevant Deep Neural network approach deduced from the deep study on different machine learning approaches used in the classification of biometric data for accurate prediction of Impulsiveness factor The following section elaborates on the Algorithms implemented during this research Machine learning algorithms can be trained using available data, and these algorithms are capable of analysing available scenarios on its own to improve the model performance with experience without any further human involvement. This section explores few deep learning algorithms used in this research to estimate a probable emotional landscape of an experimental subject immediately following a specific time instance by analysing the state of mind of the same subject for an hour preceding the specific timestamp. CNN algorithms have been highly acclaimed for their extraordinary capabilities in analysing images. A deep CNN architecture consists of several layers of neurons which accepts an image as a multi-dimensional matrix and tries to identify precise features in the image by focusing on smaller sections of the image at a time. A typical CNN structure as shown in Fig. 1 consists of an input layer accepting an image and passes the same through a convolution layer which applies several kernels to extract prominent features from different subsections of the image. The convolutional layer comprising multiple kernels is generally followed by a pooling layer aggregating the features extracted by the kernels which further relays the mined information to the output through fully connected dense layers nonlinearized with suitable activation functions. Convolutional neural nets have the ability to extract the Spatial and Temporal features in the input data by taking smaller patches or samples of the data and thus achieves reduction in dimensions leading to reduction in training parameters and allowing reusability of weights. The kernel function strides through the data input and breaking the data down to extract the high level features [16] . With the addition of every conv layer, the resolution of feature extraction increases and the CNN would be able to identify hidden patterns in the input data set. This is followed by the Pooling Layer which reduces the Spatial size of the extracted feature which is an important step in processing as it helps to reduce the computational complexity and time. This layer also helps to extract the dominant features in the input. Different types of pooling are employed to achieve different types of results-Maxpooling, Average Pooling, etc. Max-pooling helps in noise reduction as well. Fully connected layer follows the Pooling layer which basically consists of several layers of neurons connected to each other which undergo training and present the results to the final pooling layer which is usually a Softmax layer or Sigmoid Activation layer to obtain the final predictions. The advantages of using a Convolutional Neural Network are that they require minimal pre-processing in comparison with other classification models. Various trained models of CNN (ResNet, AlexNet, VGG, LeNet, GoogleNet, etc.) are available in the open source community which can be used to perform transfer learning and obtain better results. The current research work focuses on analysing the realtime biometrics of an experimental subject calibrated on a [0-100] scale where zero refers to a calmest temperament whereas hundred refers to a quite stressed condition and high chance of an impulsive behaviour. The emotion topology of the subject over the last 1 h has been portrayed as an image Fig. 9 . The emotion topology image has been analysed using a CNN network such that the trained CNN would be able to analyse the minutest topological features in the last 1 h data of a time instance and try to estimate the possible emotion score 20 min after the given timestamp. Bayesian inferencing refers to a statistical technique useful in analysing a sequence of events using Bayes' theorem [37] . Bayesian Inference is the process of determining the Posterior Probability of occurrence of an event given the Prior Probability of a hypothesis and a Likelihood of a function. Setting up parameters and models are fundamental principles of Bayesian inference. Parameters are the features in the model influencing the observed input and models are the numeric expression of observed input. Two mathematical models to represent prior beliefs and likelihood function are used to define the model as shown in Fig. 2 . Posterior belief distribution is the product of prior and likelihood. The following mathematical formulation facilitates the derivation of a posterior probability by analysing a pair of antecedents (prior and likelihood) along with an available evidence. Figure 3 illustrates the Bayesian Inference Posterior probability distribution of an event, given the Prior distribution and likelihood of an event [13] . The difference between the peaks of Prior and Likelihood distributions is the Prediction error. Variance in the likelihood function introduces the noise component while Uncertainty is directly proportional to the variance in the Prior. The mindset of a person is highly influenced by external stimuli and keeps changing over time depending on the stimulus. The emotional topology of a person changes over small intervals of time giving rise to unique distribution of emotional topologies over a span of time i.e. over the period of time, the proposed research deals with many consecutive frames of emotional surface similar to the one presented in Fig. 9 representing the terrain within a single time frame. Consider a specific time index t . In the current scenario, consider: (i) Event a represents the occurrence where the calibrated emotion has exceeded a specific threshold Th over a span of 60 min. (ii) Let, P(a) represent the distribution of occurrences of Event a over the 60-min interval. (iii) Event b represents the occurrence where the calibrated emotion has exceeded a specific threshold Th over a span of 20 min immediately succeeding Event a. (iv) Let, P(b) represent the distribution of occurrences of Event b over the following 20-minute interval. In the current context, if an algorithm could be suitably trained on the distributions P(a) and P(b) to learn their interdependence i.e. the Likelihood, then the algorithm should be able to estimate the probability of an occurrence of the Event b given that the evidence i.e. an Event a has already occurred. An AE refers to an unsupervised deep learning algorithm capable of self-learning from a set of given data. More precisely, an AE can act like a black-box system wherein, [13] through the adjustment of internal weights the trained AE imitates the target system and reacts in the same manner given the same stimulus. The aim of an Autoencoder is to achieve the dimensional reduction by learning the hidden patterns in the data and ignoring the noise embedded in the data. Figure 4 represents the architecture of a typical AE where the deep neural net accepts an input through a large number of neurons and transmits them through a much compact bottleneck (smaller neural layer) in order to capture a suitable interpretation of the input data to train its internal weights accordingly so as to be able to reconstruct the same input from the internal interpretation alone. This ensures the imitating capacity of the AE as given a scenario, the AE can react in the same way as the original system. In the current scenario, the emotional conscience of a person remains the same, but, the same person reacts differently to different external stimulus based on their own reasoning capabilities. In this context, an AE has been trained with the distributions P(a) and P(b) to capture the Likelihood i.e. P(a∕b) and train the deep net to imitate the conscience of the concerned subject so that given an evidence i.e. an Event[a], the AE can estimate the Posterior i.e. P(b∕a) for the concerned subject. It has been experimentally observed that an agitated state of human mind is accompanied by several biometric changes within an individual. These observations form the ground truth of this research where the variations of the biometrics of an experimental subject has been captured using wearable sensors in real-time to estimate the possibility of an emotional outburst of the individual within a following time interval. The overall experimental setup comprised of an array of sensors to capture the different biometrics as in Fig. 5 . Table 1 presents the biometric sensor stack [1, 4, 5, 18, 20, 24, 25, 28] along with their operating characteristics. Furthermore, it has also been observed that the same biometrics of an individual fluctuates depending on external conditions like climatic factors and physical activities. Therefore, external sensors have been introduced to include a negation effect of climatic changes and physical exertion induced fluctuations. A Raspberry Pi processor has been chosen to collect all the biometric data from the different sensors and process them in situ to be able to provide prescriptive analytics to the user to caution him of possibilities of an impulsive behaviour in near future. The following sections highlight the approaches undertaken for data acquisition, pre-processing, emotion calibration and prescriptive analytics. The scope of the current research considers two aspects of the human emotions i.e. relaxed and agitated. Depending on the changes in the biometrics, the user emotions have been calibrated from a calm and composed mindset to an agitated state where the user might be prone to making loose statements or taking impulsive decisions that may lead to adverse impacts. Ethics statement The volunteers/subjects were chosen and written consent was obtained to record their biometric data without any conflict. The agitated and relaxed states of the experimental subject were simulated by conducting experiments in different scenarios like watching funny cartoons, romantic content, horror and disturbing videos. Other simulated environments included answering different thought-provoking questions, facing arithmetic puzzles, rapid fire games and so on. The data acquisition process also involved subjects like teachers conducting classes, people performing yoga, undergoing physical exertion, meditation, listening to music, relaxing and students writing exams. The biometrics of the volunteered subjects were recorded along with their honest feedback on their emotional states. The data collection was conducted under indoor and outdoor conditions through different climatic effects where the same subjects were physically exhausted, or they had been exposed to extreme heat in the mid-day leading to rise in heart rates and body temperatures. Over multiple rounds of experiments and based on the observed phenomenon, approximate penalty factors were levied on the biometric data such as the body temperature and heart rate to nullify the effect of physical stress within predefined scope when the experimental subjects had undergone some sort of physical hardship. The raw data collected from the sensors were in encrypted format and could not be used directly for further processing. The raw data were converted into machine readable format. Appropriate data wrangling techniques have been applied on the raw output from the sensors to obtain meaningful information [8] . Raspberry Pi requires input in digital format to perform any further operations on the data. Few of the sensors gave out analog output like in case of pulse sensor. An analog to digital converter [ADC] was introduced to facilitate this conversion. Equation (3) represents the mathematical formulation of the conversion strategy for the ADC. To perform the further analysis on the raw data from the biometric sensors, the data were subjected to pre-processing. Due to the malfunctioning of sensors, lack of connectivity in the circuitry and loss of contact with the skin caused discontinuities in the biometric raw data. Since, the actual biometrics would not alter within a short span of time in any individual, in cases of loss of skin contact, the last captured biometric was used to cover up the inconsistencies for the loss of connectivity of the sensor to the processor. The data obtained after wrangling were not uniform and had variations in their range. This could impact the performance of the proposed machine learning algorithm by inappropriately adjusting the internal weights. This has been taken care by identifying the sensor with lowest sampling rate and the data from other sensors being sampled at this rate, thus ensuring heterogeneous (Table 1 ) data synchronization. Further, the values were max-min normalized (4) to scale all features on a uniform range of (0-100): The data from different sensors were observed to be differently oriented when the subjects were stimulated to agitation conditions. With elevation in agitation levels of a subject, few sensor values increased from lower to higher range while few other sensors behaved conversely. Therefore, scale inversion (5) was applied to bring uniformity in the data: (3) Voltage Analog = ADC reading * SystemVoltage Resolution ADC (4) data normalized = 100 * data − data min data max − data min (5) data normalized = 100 * 1 − data − data min data max − data min Eight different features have been collected by integrating biometric sensors with the Raspberry Pi. The actual ranges of all these features are shown in Table 2 . The eight-feature model is converted into a four-feature model in which DS18B20-Body temp, Pulse, Meditation and Attention has a direct impact on the estimation accuracy of the system, and the other four features are used to reduce the impact as follows: 1. DHT22-Humidity and temperature-these two features are used to negate the fluctuation due to environmental changes. Negation happens only if humidity is ≥ 75 and ≤ 100 and temperature is ≥ 35 and ≤ 45 °C. 2. ADXL345-Euclidian-using the x, y and z-axis displacement collected from ADXL345 we calculate the Euclidean distance, which when exceeds a particular threshold say 52, we nullify its impact by reducing the values of Body temperature and pulse by 20%. 3. Quality describes the signal strength of the mindwave over a scale of 0-200 where 0 and 200 represents very poor and excellent signal strength, respectively. We nullify its impact on Attention and Meditation by 25% only when qualities range is ≥ 0 and ≤ 70 [5, 25] . Depending on multiple experimental outcomes, the preprocessed data has been calibrated into the emotion score. This score occurs with an interval of every one second and is averaged over a minute. Figure 6 presents the overall architecture of the proposed system. Figure 7 represents a flow diagram highlighting the interactions between the individual entities within the developed ecosystem. reflecting the convenience of the user by choosing between either pattern analytics or probabilistic estimation of future emotional topology. Arrow 5. Depending on user convenience, the devised system either employs a CNN-based approach for emotion pattern analysis or an AE-based approach for probabilistic inference. Arrow 6. The system sends an intimation to the user by alarming him of the possibility of an impulsive outburst and prescribes him to act carefully to evade extreme situations. In either case, there can be two scenarios that the user may face: (i) Case 1 CNN/AE predicts an impulse The trained CNN can send a prescriptive intimation to the user or the AE can predict a more than 60% (say) chance of an impulsive outburst based on the trends observed in the user's temperament dataset. Precisely, this prediction would indicate that the user is likely to get agitated given his current state of mind, but the same agitation can be averted by taking precautionary measures. (ii) Case 2 CNN/AE predicts NO impulse The trained CNN may not predict the possibility of an impulsive outburst, and the AE may predict a less than 30% (say) chance of an impulsive outburst based on the user's historical trends. Precisely, this prediction indicates that the user is less likely to achieve an agitated state within the following 20 min, but there can always be some sudden external situation like a sudden news of a stock market crash which could momentarily change his mental state. Lastly, as the prescriptive analytics forecasts a probability of occurrence of an event, in extreme cases the values of probability could be 1 or zero indicating that the event must happen or there is no chance of an event to take place. Considering the classical rule of probability, a mathematical definition can be formulated as below (6); Now, consider a situation where an event occurs 5 times out of 20, i.e. But in an extreme case, if the system foresees zero occurrences of an event, the proposed system would project a zero per cent chance which would be a misleading information, since in reality all events are fully random in nature. Therefore, the mathematical formulation of probability can be slightly modified to rectify the extreme cases i.e. instead of predicting a zero per cent chance, the system may predict as follows (6) by slightly modifying the formula; Again, in another extreme case, if a system foresees a continuous threshold breach over the entire time window, the proposed system would project 100 per cent chance of (6) Probability = times occured total cases probability of occurrence 5 20 . 0 + 0.2 20 = 0.01 ≅ 1% chance of occurrence. Fig. 7 Flow of events occurrence which is unlikely to happen. Therefore, modifying the same formula, we can evade the upper bound errors as follows; By combining the results obtained above, the mathematical formulation of probability can be updated as Eq. (7); This section discusses on the challenges encountered with suitable mitigation plans in the light of the results obtained from analysing the emotions of the user using different deep neural algorithms. The proposed CNN model comprised of four [9-36-36-81] convolutional layers followed by a fully connected [360] feedforward layer at 10% dropout. The proposed model was able to analyse the emotion topology and estimate future landscapes with reasonably low error. Figure 8 shows a comparison between the actual and the predicted curves with red dotted circles rounding the matched impulsive peaks. The predicted emotional impulses matched with the actual impulses in 93% of the cases within ±10 min of the predicted time index and in 82% of the cases within ±5 min of the predicted timestamp. Further, the proposed neural net was able to predict emotional impulses 5 min in advance which matched with actual impulsive trends of the user in 90% of the cases. Since the biometric data are collected from multiple sensors, the range and distribution of the data are different for different sensors and therefore have been scaled fit into a common range of [0 − 100] as mentioned in feature scaling above. Further, the scaled biometric data has been segmented into equal-sized buckets as shown in Table 3 . 20 + 0.2 20 + 0.4 = 0.99 ≅ 99% chance of occurrence. (7) Pr updated = times occurred + 0.2 total cases + 0.4 The data from each of these biometric sensors have been modelled to calibrate the collective impact of these parametric changes on the emotional state of the experimental subject on a scale of zero to hundred. Now, based on the discretization as per Table 3 , all biometric parameters would vary within categories [A-E], collectively giving rise to calibrated emotional states on a similar scale of [0-100]. Extensive experimentation was conducted to observe the changes in the biometric readings of an experimental subject under the influence of different situations such as when the person is relaxing with music, reading funny stories, writing exams, etc. Similar experiments were conducted in laboratory environments or after the subject has been exposed to physical stress to observe the natural biometric fluctuations due to physical activity. It was observed that depending on high movement or working in higher temperature and humidity conditions caused a natural inflation in the heart rate and slight increase in the temperature. Therefore, the sensor readings for heartbeat and skin temperature were nullified by 2-5% of the reading depending on the higher categories of climatic parameters and physical displacement. When all the biometrics correspond to higher buckets with lower levels of climatic influence and physical displacement, the consecutive emotional state is marked on the higher category whereas with all biometrics at lower buckets even with some influence of the external stimuli, the resulting emotional state is labelled on the lower categories based on experimentation and self-judgement of the subject. The calibration factors such as the emotional state labels and the nullifying percentages of the respective subjects were uniformly modified over multiple iterative experiments to finalize the best fit. The biometric sensor data have been experimentally calibrated to emotional states, and the observed emotional responses have been plotted on a surface as shown in Fig. 9 . The second image in Fig. 9 illustrates how a threshold plane can be introduced to filter out only the peaks which are greater than the threshold. As per the visual representation of the emotion surface plotted in Fig. 9 , the portions where the emotion levels were calmest have been highlighted in deep blue colour in contrast to the impulsive peaks represented by red colour. The emotional levels in between are represented by lighter shades of blue, green and yellow. Needless to mention that the human emotions vary as a temporal function depending on relevant external stimuli. Since a human emotional state can be represented in terms of a visual pattern as presented in Fig. 9 , the emotional fluctuations over time can be considered by an equivalent array of visual frames. A CNN-based deep neural network has been employed to study the temporal patterns of these visual frames and map the states to the changes in visual patterns up to 20 min in advance to predict the possible emotional state of an individual. The predicted outcomes of the CNN analysis have been projected by a blue line in Fig. 8 tracing the possible fluctuations of human emotions 20 min in advance. The orange line in Fig. 8 represents the actual emotional changes in the human subject which has been found to closely follow the predicted path 20 min after the trend was predicted. The predicted emotional surface has been plotted in Fig. 10 for two different time spans. The predicted emotional surface varies from deep blue for the calmest emotions to red for the potentially impulsive landscapes. In general for time sequenced data, variants of deep neural architectures are capable of studying the data trends and predict the future patterns. But, human emotions being a very complex entity influenced by a variety of external stimuli, the emotional trends vary almost within interval of seconds. As a result, the emotional plots are quite zigzag in nature making it difficult to predict the future trends accurately. On the contrary, since a forecast is a matter of probability, it can be more convenient to predict the possibility of an emotional outburst following a specific emotional snapshot at a particular time step. An autoencoder has been employed for this purpose to act as a black-box system imitating the emotional state of an individual. The variational autoencoder has been trained samples of emotional surfaces of an experimental subject such that, by taking the emotional snapshot of the individual at an instance of time, the system is able to determine the probability with which an emotional outburst may follow. Table 4 presents the error metrics of the developed system. With respect to the emotion dataset, the error calculation module computes the number of times the emotion topology encountered peaks which are greater than a given threshold and compares the same with the predicted peak count. Consider, A i represents the actual number of peaks in a given time interval and P i represents the predicted peak count in the same interval. The prediction error in peak count has been computed using the MAPE error metric. The proposed CNN-based model was able to predict the impulsive peaks with less than 5% error while predicting 5 min in advance and with less than 15% error while predicting 20 min in advance. Figure 11 shows the ROC curves obtained by comparing the probability of emotional outbursts of an individual against the actual impulsive peaks obtained from experimental data. When predicting an outburst 20 min prior to the actual occurrence, the AUC was found to be 0.77 (77% overall cases matched) and 0.93 (93% overall cases matched) and for predicting an outburst 5 min in advance, the AUC was found to be 0.90 (90% overall cases matched). Based on the type of algorithm employed, a user notification can be sent for the user to be more cautious or try to be more careful before taking any high-impact decision. If a CNN predicts a peak greater than a specific threshold, a user notification alert may be sent in the form of a vibration of the wearable or a led blink. Alternatively, the system may also use a Bayesian implementation using autoencoder to show a probability of impulsive burst on the screen. The choice of the algorithm may also be left for user selection. More advanced variants of CNN such as the ResNet, VGGNet and better alternatives to the variational AE can be used for further reducing the prediction error of the system and this forms the future scope of this work. Innovations in healthcare industry have played a major role in improving the lifestyle of people worldwide by providing gadgets to monitor biometrics. Nevertheless, they fail to address the root cause of these ailments, which is mental fitness. Stress and anxiety due to high-impact decision-making roles have led to various types of disorders affecting major organs which in some cases may even become incurable and fatal. The proposed smart-wearable analyses the emotional state of an individual based on biometrics to predict the possibility of a following impulsive outburst. The findings of the current research work highlight the correlation between the changes in human emotions and the variations in their body biometrics. This work aims at infusion of artificial intelligence with smart-wearable gadgets that help the subject to handle real-world problems with calm and focused mindset. This invention challenges the current diagnostic healthcare system and proposes a step forward towards a new era of prognostic healthcare. The analysis of the results obtained from this research have confirmed that this device can predict the impulsiveness with 95% accuracy and 20 min in advance which is sufficient for a person to regain composure and take action. This invention not only provides prescriptive alerts but also helps in anger management by bringing about positive behavioural changes over prolonged usage. Further, as this device can be customized to store the daily biometrics at real-time, the data obtained could prove to be very informative for historical analysis of various ongoing and future ailments. Furthermore, in the current Covid-19 perspective, such smart wearables could also be a useful tool to create a time lapse digital footprint of the users which can be extremely useful in contact tracing and isolation in future pandemic conditions. This research work suggests how AI empowered smart textiles can serve as a mobile medicare facilitator and can keep a regular check on an individual's biometrics to understand the current medical and psychological conditions. DHT22. Retrieved A refined method of forecasting based on highorder intuitionistic fuzzy time series data iAware: a real-time emotional biofeedback system based on physiological signals Sensitivity of EEG upper alpha activity to cognitive and affective creativity interventions Impact of stress on different components of impulsivity in borderline personality disorder Assessment of emotions: anxiety, anger, depression, and curiosity Data wrangling: making data useful again Toward Autonomous, Adaptive, and Context-Aware Multimodal Interfaces. Theoretical and Practical Issues A mobile EEG system with dry electrodes Emotions and hydrochloric acid secretion during psychoanalytic hours Review on psychological stress detection using biosignals Modeling emotions associated with novelty at variable uncertainty levels: a Bayesian approach The psychological significance of changes in skin temperature Deep learning analysis of mobile physiological, environmental and location sensor data for emotion detection An Introduction to Convolutional Neural Networks Blood, sweat, and fears, the autonomic architecture of emotion www.pulse senso r.com. Retrieved 26 Emotion Detection in Aging Adults Through Continuous Monitoring of Electro-Dermal Activity and Heart-Rate Variability MaximIntegrated: DS18B20-Programmable Resolution 1-Wire Digital Thermometer The effects of measuring emotion: physiological reactions to emotional situations depend on whether someone is asking Stress emotion recognition based on RSP and EMG signals Partitioning and interpolation based hybrid ARIMA-ANN model for time series forecasting Neurosky: MindWave Mobile: User Guide. Retrieved NeuroSky's eSense™ Meters and Detection of Mental State What happens when we get angry? Hormonal, cardiovascular and asymmetrical brain responses Posture and Gesture Analysis Supporting Emotional Activity Recognition A compact embedding for facial expression similarity Speech signal analysis for the estimation of heart rates under different emotional states Emotional stress as a trigger of myasthenic crisis and concomitant takotsubo cardiomyopathy: a case report Human emotion detection via brain waves study by using electroencephalogram (EEG) Physiological, psychological, and behavioral factors and white coat hypertension Development system for emotion detection Emotion classification based on brain wave: a survey Bayesian inference: an introduction to principles and practice in machine learning Sex differences in mental stress-induced myocardial ischemia in young survivors of an acute myocardial infarction Online and offline anger detection via electromyography analysis Recognizing human facial expressions from long image sequences using optical flow Acknowledgements We express our sincere gratitude and thankfulness to Dr. Pallaviram Sure, Associate Professor, Dept. of ECE, MSRUAS for her immense help and support in setting up the hardware.