key: cord-0435483-iath2232
authors: Sharma, Renu; Ross, Arun
title: Periocular Biometrics and its Relevance to Partially Masked Faces: A Survey
date: 2022-03-29
journal: nan
DOI: nan
sha: ad7be453bee95c5db1ab4a069ee6ab22a71b5eaf
doc_id: 435483
cord_uid: iath2232

The performance of face recognition systems can be negatively impacted in the presence of masks and other types of facial coverings that have become prevalent due to the COVID-19 pandemic. In such cases, the periocular region of the human face becomes an important biometric cue. In this article, we present a detailed review of periocular biometrics. We first examine the various face and periocular techniques specially designed to recognize humans wearing a face mask. Then, we review different aspects of periocular biometrics: (a) the anatomical cues present in the periocular region useful for recognition, (b) the various feature extraction and matching techniques developed, (c) recognition across different spectra, (d) fusion with other biometric modalities (face or iris), (e) recognition on mobile devices, (f) its usefulness in other applications, (g) periocular datasets, and (h) competitions organized for evaluating the efficacy of this biometric modality. Finally, we discuss various challenges and future directions in the field of periocular biometrics.

Biometrics is the automated or semi-automated recognition of individuals based on their physical (face, iris), behavioral (signature, gait), or psychophysiological (ECG, EEG) traits Ross et al., 2019) . The COVID-19 pandemic has ushered in a number of considerations for biometric systems. For example, in the context of fingerprint recognition, researchers are now investing more effort in designing contactless fingerprint systems (Yin et al., 2020; Lin and Kumar, 2019) . Similarly, the prevalent use of face masks and social distancing protocols has refocused attention on occluded face recognition and, inevitably, ocular biometrics. The ocular region refers to the anatomical structures related to the eyes, and biometric cues in this region include pupil, iris, sclera, conjunctival vasculature, periocular region, retina, and oculomotor plant (comprising eye globe, muscles, and the neural control signals).

The term "periocular" has been used to refer to the region surrounding an eye consisting of eyelids, eyelashes, eye-folds, eyebrows, tear duct, inner and outer corner of an eye, eye shape, and skin texture (Figure 1 ). While many articles in the biometric literature include the sclera, iris, and pupil in the context of * * Corresponding author: e-mail: sharma90@cse.msu.edu (Renu Sharma) periocular recognition (Park et al., 2009; De Marsico et al., 2017; Smereka and Kumar, 2017; Luz et al., 2018) , others have excluded these regions (Woodard et al., 2010a,b; Park et al., 2011; Proena and Neves, 2018) . The periocular region may be biocular (the periocular regions of both eyes are considered to be a single unit) Juefei-Xu and Savvides, 2012; , monocular (either left or right periocular) (Park et al., 2009 (Park et al., , 2011 , or a fusion of the two monocular regions (combination of left and right periocular regions) (Bharadwaj et al., 2010; Woodard et al., 2010a; Boddeti et al., 2011) . Earlier work (Park et al., 2009 (Park et al., , 2011 on periocular biometrics studied its feasibility as a standalone biometric trait. Other researchers (Woodard et al., 2010b; Park et al., 2011; Juefei-Xu and Savvides, 2012) established its relevance by comparing it with the face and iris modalities. In some non-ideal conditions, the periocular region even shows higher performance than face Park et al., 2011; Juefei-Xu and Savvides, 2012) and iris (Boddeti et al., 2011) modalities. Hollingsworth et al. ascertained its usefulness as a biometric trait by conducting human analysis on near-infrared (Hollingsworth et al., 2010; Hollingsworth et al., 2011; Hollingsworth et al., 2012) and visible (Hollingsworth et al., 2012) spectrum images. ing theaters where physicians wear surgical masks; (b) occupations where people wear helmets that occlude faces (e.g., military, astronauts, firefighters, bomb diffusion squads); (c) sporting events requiring a helmet (cricket, football, car racing); (d) use of veils to cover one's face due to cultural or religious purposes ; and (e) robbers masking their faces to avoid being recognized. Periocular biometrics has various advantages, which further motivate its usage:

1. Periocular modality can be acquired using the sensors that capture face and iris modalities. So, there is typically no additional imaging requirement. 2. Compared to iris or other ocular traits (e.g., retina or conjunctival vasculature), periocular images can be captured in a relatively non-invasive, less constrained, and noncooperative environment. They are also less prone to occlusions due to eyelids, eyeglasses, or deviated gaze. In contrast to the face modality, the periocular region (which is, of course, a part of the face) is relatively more stable as it is less affected by variations in pose, aging, expression, plastic surgery, and gender transformation. It is also seldom occluded when face images are captured in close quarters (e.g., selfies) or in the presence of scarves, masks, or helmets. 3. The periocular modality can complement the information provided by the iris and face modalities. So, it can be combined with the iris (Santos and Hoyle, 2012; Tan and Kumar, 2012; Raghavendra et al., 2013; and face Mahalingam et al., 2014) modalities to increase the performance of the biometric system without any modification to the acquisition setup. 4. It can also be used for other tasks such as presentation attack detection (Hoffman et al., 2019) , and soft-biometrics extraction (Merkow et al., 2010; Lyle et al., 2012; Alonso-Fernandez et al., 2018) . 5. It can help with cross-spectral iris recognition as iris images captured in different spectra depict different features, while periocular features (shape of the eye, eyelashes, eyebrows) are relatively stable. It also facilitates cross-modal (face-iris) matching (Jillela and Ross, 2014) as it is the common region present in both the modalities.

In the literature, there are previous surveys that focused on periocular biometrics (Santos and Proena, 2013; Nigam et al., 2015; Alonso-Fernandez and Bigun, 2016; Badejo et al., 2019; Behera et al., 2019; Kumari and Seeja, 2019) . Santos and Proena (2013) summarized the significant papers on periocular recognition before 2013. The authors in (Nigam et al., 2015; discussed the research advances of various ocular biometric traits such as iris, periocular, retina, conjunctival vasculature and eye movement, and their fusion with other modalities. Alonso-Fernandez and Bigun (2016) described periocular recognition methodologies in terms of pre-processing, feature extraction, fusion, soft-biometric extraction, and other applications. Behera et al. (2019) focused on cross-spectral periocular recognition. Zanlorensi et al. (2021) provided detailed information on periocular and iris datasets, and competitions based on some of these datasets. A recent survey on periocular biometrics is in (Kumari and Seeja, 2019) . Figure 3 shows a visualization of various research work on periocular biometrics. The main contributions of this paper lie in the detailed categorization of periocular techniques. We also describe periocular techniques specifically useful for human recognition in the COVID-19 pandemic.

In this paper, we present a comprehensive review of periocular biometrics. We discuss different categorizations based on (a) different anatomical cues utilized for recognition, (b) feature extraction or matching methodologies, (c) different spectral input images, and (d) fusion with different modalities. We then discuss periocular recognition techniques for mobile devices, in other applications (e.g., soft biometrics, iris presentation detection) and for recognition in special circumstances (cross-modal, gender transformation, long-distance). We also describe various periocular datasets and competitions held. Considering the COVID-19 pandemic situation, we also provide a brief review of recent face and periocular techniques specifically designed to recognize humans wearing a face mask.

The rest of the paper is organized as follows: Section 2 describes various face and periocular techniques specially applied on the masked faces for human identification, Section 3 categorizes periocular techniques based on anatomical cues utilize for recognition, Section 4 describes various periocular features extraction and matching techniques, Section 5 categorizes techniques based on input images of different spectra, Section 6 discusses fusion techniques with other biometric modalities, Section 7 provides details of periocular authentication on mobile devices, Section 8 describes periocular recognition in specific scenarios and other applications, Section 9 details periocular datasets and competitions, Section 10 focuses on various challenges and future directions, and Section 11 concludes the paper.

Periocular recognition has gained relevance during the COVID-19 pandemic as some reports have documented a drop in performance of existing face recognition methods in the presence of facial masks Ngan et al., 2020a,b) . Fig. 2 . Various scenarios where the periocular region has increased significance: (a) girl with a mask during the covid pandemic, (b) doctors and nurses in the surgical room, (c) women in niqab, (d) partial faces in the crowd, (e) occluded face while drinking, (f) robber with face covering, (e) military personnel with face paint, (f) football player wearing a helmet, (g) cricket player wearing a helmet, (h) F1 race player in a helmet, (i) face-covering in cold weather, (j) astronauts in suit, (k) dancer with a face veil, (l) firefighter in uniform, and (m) people in motorbike helmets.

The tests conducted by NIST applied digitally tailored masks to face images for evaluation (6.2 million images from 1 million people). The first report (Ngan et al., 2020a) presented the performance of 89 algorithms that were submitted to NIST before the COVID-19 pandemic on the masked images. All 89 face recognition algorithms showed an increase in False Non-Match Rate (FNMR) by about 5% -50% at a 0.001% False Match rate (FMR) -higher than NISTs prior study on unmasked images. The second report (Ngan et al., 2020b) published the performance of 65 new algorithms submitted to NIST after mid-March 2020 along with their previous submissions (cumulative results for 152 algorithms). The new algorithms include masked images during the enrollment stage. However, the report showed increased FNMR (5% -40%) for all the newly submitted algorithms, though the new algorithms showed improved accuracy compared to the pre-pandemic algorithms. An earlier work by Park et al. (2011) also showed a drop in rankone accuracy of a commercial face recognition software from 99.77% (full-face images) to 39.55% when the lower region was occluded.

In an era of masked faces necessitated by the pandemic, periocular information can be helpful for human recognition in two ways, either by generating a full face from the periocular region or by matching using only the periocular region. Juefei- Xu et al. (Juefei-Xu et al., 2014; Juefei-Xu and Savvides, 2016) hallucinated the entire face from the periocular region using dictionary learning algorithms. Ud Din et al. (2020) detected the masked region from a face image and then performed image completion on the masked region. They used a GAN-based network for image completion, which consists of two discriminators; one learns the global structure of the face and the other focuses on learning the missing region. Li et al. (2020) also performed face completion to recover the content under the mask through the de-occlusion distillation framework. Hoang et al. (2020) Anwar and Raychowdhury (2020) presented an open-source tool, MaskTheFace, to create masked faces images. Moreover, research work on face detection in the pres-ence of masks (Opitz et al., 2016; Ge et al., 2017) , face mask detection (Chowdary et al., 2020; Qin and Li, 2020; Loey et al., 2021) and face recognition under occlusion (Song et al., 2019; Ding et al., 2020; Geng et al., 2020; Boutros et al., 2021a; Damer et al., 2021; Hariri, 2021; Montero et al., 2021) would be helpful in human identification on masked face images. Various competitions are also conducted to benchmark face recognition techniques on masked faces Boutros et al., 2021b; Zhu et al., 2021) .

Woodard et al. (2010b) classified periocular anatomical cues into two levels: level-one cues comprise eyelids, eye folds, eyelashes, eyebrows, and eye corners, while level-two includes skin texture, fine wrinkles, color, and skin pores. Specifically, level-one cues represent geometric nature of the periocular region, while level-two embodies textural and color attributes. The authors in (Hollingsworth et al., 2010; Hollingsworth et al., 2011; Park et al., 2011; Beom-Seok Oh et al., 2012) studied the significance of various periocular components in recognizing individuals. Earlier work on face recognition (Sadr et al., 2003) suggested the eyebrows to be the most salient and stable feature of the face. Hollingsworth et al. (Hollingsworth et al., 2010; Hollingsworth et al., 2011) conducted a human analysis to identify the discriminative cues on near-infrared (NIR) images and found that eyelashes, tear ducts, shape of the eye, and eyelids are the most frequently used cues in verifying the two images of a person. The studies in (Park et al., 2011; Beom-Seok Oh et al., 2012) utilized automatic feature descriptors to determine important regions on visible (VIS) spectrum images and concluded that eyebrows, iris, and sclera are the most significant cues for periocular performance. In a subsequent work (Hollingsworth et al., 2012) , the authors applied both human and machine approaches to identify discriminative regions on both NIR as well as VIS periocular images. They observed that humans and computers both focus on the same periocular cues for identification: in VIS images, blood vessels, skin region, and eye shape are more salient, whereas in NIR images, eyelashes, tear ducts, and eye shape are more promising. Other authors (Smereka and Kumar, 2013; Smereka and Kumar, 2017) also drew similar con- clusions on the relevance of periocular cues in VIS and NIR images. In summary, level-one cues are more useful for NIR images, whereas level-two cues aid in VIS images.

Researchers have also analyzed the utility of periocular cues as a standalone biometrics, for instance, using only the eyebrows (Dong and Woodard, 2011; Le et al., 2014; Hoang et al., 2020) , or periocular skin , or eyelids (Proena, 2014) . Details of the other ocular biometrics traits closely related to periocular can be found in the following studies: iris (Bowyer et al., 2008 (Bowyer et al., , 2013 , sclera (Zhou et al., 2012; Das et al., 2013) , conjunctiva vasculature (Derakhshani and Ross, 2007; Crihalmeanu and Ross, 2012) , eye movements (Rigas et al., 2012; Holland and Komogortsev, 2013; Sun et al., 2014) , occulomotor plant characteristics (Komogortsev et al., 2010) , and gaze analysis (Cantoni et al., 2015) . The description of these ocular traits is out of the scope of this paper.

A typical periocular recognition system consists of the following steps: acquisition, pre-processing of the acquired image, localization of region-of-interest (ROI), feature extraction, post-processing of extracted features, and matching of two feature sets. In the acquisition step, the periocular image is captured using a sensor or camera. We provide details of vari-ous sensors used to capture periocular images along with their datasets in Section 9. The pre-processing step aims to enhance the visual quality of an image. Commonly, pre-processing techniques are applied to normalize illumination variations, such as anisotropic diffusion (Juefei- Xu and Savvides, 2012) and Multiscale Retinex (MSR) (Juefei- Xu et al., 2014; Nie et al., 2014) . Karahan et al. (2014) applied histogram equalization for contrast enhancement. Juefei- Xu et al. (2011) performed pre-processing schemes for pose correction, illumination, and periocular region normalization. Proena and Briceo (2014) investigated an elastic graph matching (EGM) algorithm to handle nonlinear distortions in the periocular region due to facial expressions.

The localization step extracts the periocular region from the acquired or pre-processed image. As the definition of the periocular region has not yet been standardized, the ROI used for periocular recognition varies across the literature. The authors in (Tan and Kumar, 2012; Park et al., 2009; Mahalingam et al., 2014) considered the iris center as a reference point to determine the periocular rectangular region. The authors in (Padole and Proena, 2012; Nie et al., 2014) used the geometric mean of eye corners to localize the ROI since the iris center is affected by gaze, pose, and occlusion. Bakshi et al. (2013) localized the periocular region based on the anthropometry of the human face. Park et al. (2011) studied the effect of including eyebrows in ROI on the recognition performance by performing both manual localization (based on the centers of the eyes) and automatic localization (based on the anthropometry of the human face). Algashaam et al. (2017b) analyzed the influence of varying periocular window sizes on periocular recognition performance. Kumari and Seeja (2021b) proposed an approach to extract optimum size periocular ROIs of two different shapes (polygon and rectangular) by using five reference points (inner and outer canthus points, two end points and the midpoint of eyebrow). described an integrated algorithm for labeling seven components of the periocular region in a single-shot: iris, sclera, eyelashes, eyebrows, hair, skin, and glasses. Deep learning techniques have also been used to detect the periocular region, such as ROI-based object detectors (Reddy et al., 2018b) and supervised semantic mask generators (Zhao and Kumar, 2018) . Reddy et al. (2020) proposed spatial transformer network (STN), which is trained in conjunction with the feature extraction model to detect the ROI.

The feature extraction step involves the extraction of discriminative and robust features from the localized periocular region. Alonso-Fernandez and Bigun (2016) categorized feature extraction techniques into global and local approaches. We group deep learning-based approaches separately. Table  1 lists the various feature extraction techniques corresponding to these categories, along with research papers utilizing these techniques. The description of all three approaches are provided below.

1. Global Feature Approaches: The global feature extraction approaches consider the entire periocular ROI as a single unit and extract features based on texture, color, or shape. Texture in a digital image refers to the repeated spatial arrangement of the image pixels. Commonly used techniques to capture the textural features from the periocular region are Local Binary Patterns (LBP) and its variants (Park et al., 2009; Adams et al., 2010; Bharadwaj et al., 2010; Juefei-Xu et al., 2010; Xu et al., 2010; Juefei-Xu and Savvides, 2012; Beom-Seok Oh et al., 2012; Padole and Proena, 2012; Santos and Hoyle, 2012; Uzair et al., 2013; Cao and Schmid, 2014; Mahalingam et al., 2014; Nie et al., 2014; Sharma et al., 2014; Santos et al., 2015) , Histogram of Oriented Gradients (HOG) (Park et al., 2009 (Park et al., , 2011 , Gabor filters (Juefei- Xu et al., 2010; Alonso-Fernandez and Bigun, 2012; Joshi et al., 2014; Cao and Schmid, 2014; , and Binarized Statistical Image Features (BSIF) (Raghavendra et al., 2013; Raja et al., 2014a) . The LBP descriptor computes the binary patterns around each pixel by comparing the pixel value with its neighborhood. The binary patterns are then quantized into histograms, which on concatenation form a feature vector. In the HOG descriptor, gradient orientation and magnitude around each pixel are binned into histograms and histograms are then concatenated to form a feature vector. The Gabor filters extract features by applying textural filters of different frequencies and orientations on an image. The BSIF descriptor convolves the image with a set of filters learned from natural images, and then the responses are binarized. Other texture-based features include Bayesian Graphical Models (BGM) (Boddeti et al., 2011) , Probabilistic Deformation Models (PDM) Smereka and Kumar, 2013) , Discrete Cosine Transform (DCT) (Juefei- Xu et al., 2010) , Discrete Wavelet Transform (DWT) (Juefei- Xu et al., 2010; Joshi et al., 2014) , Force Field Transform (FFT) (Juefei- Xu et al., 2010) , GIST perceptual descriptors (Bharadwaj et al., 2010) , Joint Dictionarybased Sparse Representation (JDSR) (Raghavendra et al., 2013; Jillela and Ross, 2014; Moreno et al., 2016) , Laws masks (Juefei- Xu et al., 2010) , Leung-Mallik filters (LMF) (Tan and Kumar, 2012) , Laplacian of Gaussian (LoG) (Juefei- Xu et al., 2010) , Correlation-based methods (Boddeti et al., 2011; Juefei-Xu and Savvides, 2012; Ross et al., 2012; Jillela and Ross, 2014) , Phase Intensive Global Pattern (PIGP) (Smereka and Kumar, 2013; Bakshi et al., 2014) , Structured Random Projections (SRP) (Oh et al., 2014) , Walsh masks (Juefei- Xu et al., 2010) , Higher Order Spectral (HOS) (Algashaam et al., 2017b) , Gaussian Markov random field (Smereka et al., 2015) , and Maximum Response (MR) .

The color features of the periocular region correspond to the wavelengths of light reflected from its constituent parts. Woodard et al. (2010b) utilized the color features by applying histogram equalization on the luminance channel and then calculating the color histogram on the spatially salient patches of the image. Lyle et al. (2012) also extracted color features using local color histograms. Moreno et al. (2016) defined color components using linear and nonlinear color spaces such as red-green-blue (RGB), chromaticity-brightness (CB), and huesaturation-value (HSV) and then applied a re-weighted elastic net (REN) model. The authors in (Woodard et al., 2010b; Moreno et al., 2016) utilized both textural and color features from the periocular recognition. Regarding shape features, the work in (Dong and Woodard, 2011; Le et al., 2014) utilized eyebrow shape-based features, while Proena (2014) extracted eyelid shape features. Ambika et al. (2016) employed LaplaceBeltrami operator to extract periocular shape characteristics. All aforementioned techniques use 2D image data of the periocular region. Chen and Ferryman (2015) combined 3D shape features extracted using the iterative closest point (ICP) method and fused them with 2D LBP textural features at the score-level. One of the major advantages of using global feature approaches is that they generate feature vectors of fixed-length, and matching of fixed-length vectors is computationally effective. However, global feature approaches are more susceptible to image variations, such as occlusions or geometric transformations.

2. Local Feature Approaches: The local feature extraction approaches first detect salient or key points from the ROI and then extract features from their local neighborhood to create a feature descriptor. Commonly used local feature approaches are Scale Invariant Feature Transformation (SIFT) Park et al., 2011; Padole and Proena, 2012; Ross et al., 2012; Santos and Hoyle, 2012; Smereka and Kumar, 2013; and Speeded-up Robust Features (SURF) (Juefei- Xu et al., 2010; Xu et al., 2010; Raja et al., 2015b) . The SIFT feature extractor defines key locations as extrema points on the difference of Gaussians (DoG) images obtained from a series of smoothed and rescaled images. Feature descriptor is then formed by concatenating orientation histograms defined around each key point. On the other hand, SURF detects key points by utilizing the Hessian blob detector, and the key points are then described using Haar wavelet features. SURF utilizes integral images to speed up the computation. Other local feature descriptors are Binary Robust Invariant Scalable Keypoints (BRISK) (Mikaelyan et al., 2014) , Oriented FAST and Rotated BRIEF (ORB) (Mikaelyan et al., 2014) , Phase Intensive Local Pattern (PILP) (Bakshi et al., 2015) , Symmetry Assessment by Feature Expansion (SAFE) (Mikaelyan et al., 2014; , and Dense SIFT (Ahuja et al., 2016a) . Since the number of detected key points varies among images, the resulting feature vectors also vary in length, making the process computationally expensive in some cases. However, local feature approaches are more robust to occlusions, illumination variations, and geometric transformations compared to global feature approaches.

3. Deep Learning Approaches: With the success of deep learning in computer vision and biometrics, this approach has also been applied to periocular recognition. Earlier work (Nie et al., 2014 ) based on learning approaches introduced an unsupervised convolutional version of Restricted Boltzman Machines (CRBM) for periocular recognition. Raja et al. (Raja et al., 2016b extracted features from Deep Sparse Filters using transfer learning methodology and input them into a dictionary-based approach for classification. On the other hand, Raghavendra and Busch (2016) extracted texture features using Maximum Response (MR) filters and input them into deep coupled autoencoders for classification. Other studies that utilized transfer learning methodologies can be found in Silva et al., 2018; Kumari and Seeja, 2020) . Proena and Neves (2018) utilized deep CNN to emphasize the importance of the periocular region for recognition by training the network with augmented periocular images having inconsistent iris and sclera regions. The training procedure causes the network to implicitly disregard the iris and sclera region. The authors in (Zhao and Kumar, 2018; Wang and Kumar, 2021) integrated attention model to the deep architecture in order to highlight the significant regions (eyebrow and eye) of the periocular image. Some researchers utilized existing off-the-shelf CNN models to extract deep features at various convolutional levels (Hernandez-Diaz et al., 2018; Kim et al., 2018; Hwang and Lee, 2020; Seeja, 2020, 2021a) . The authors in (Zhang et al., 2018; Reddy et al., 2018a) proposed compact and custom deep learning models for use in mobile devices. Other deep learning-based methods include PatchCNN (Reddy et al., 2018b) , In-Set CNN Iterative Analysis (Proena and Neves, 2019) , unsupervised convolutional autoencoders (Reddy et al., 2019) , compact Convolutional Neural Network (CNN) (Reddy et al., 2020) , VisobNet (Ahuja et al., 2017) , semantics assisted CNN (Zhao and Kumar, 2017) , heterogeneity aware deep embedding (Garg et al., 2018) , and Generalized Label Smoothing Regularization (GLSR)-trained networks (Jung et al., 2020) . Deep learning approaches provide state-of-theart recognition performance, but their performance are heavily data-driven.

After the feature extraction step, some researchers further processed the feature vector, which generally includes the application of feature selection, subspace projection, or dimen-sional reduction (Beom-Seok Oh et al., 2012; Joshi et al., 2014) techniques. These techniques aim to transform the feature set into a condensed representative feature set such that it improves the accuracy and reduces the computational complexity. Various post-processing techniques used in periocular recognition are Principal Component Analysis (PCA) (Beom-Seok Oh et al., 2012) , Direct Linear Discriminant Analysis (DLDA) (Joshi et al., 2014) , and Particle Swarm Optimization (Silva et al., 2018) . Finally, the processed features are compared using similarity or dissimilarity metrics such as Bhattacharya distance (Woodard et al., 2010a) , Hamming distance (Oh et al., 2014) , I-Divergence metric (Cao and Schmid, 2014) , Euclidean distance (Ambika et al., 2016) , or Mahalanobis distance (Nie et al., 2014) .

Different imaging spectra have been described in the literature for capturing the periocular region, including Near-Infrared (NIR), Visible (VIS), Short Wave Infrared (SWIR), Middle Wave Infrared (MWIR), and Long Wave Infrared (LWIR). The most commonly used imaging spectra are NIR and VIS. This is because most research in periocular biometrics is based on face images (VIS) or iris images (NIR). Further, even as a standalone biometric, periocular images are captured using existing face or iris sensors. The NIR spectrum, which operates in the 700-900nm range, predominantly captures the iris pattern, eye shape, outer and inner corner of the eye, eyelashes, eyebrows, and eyelids. Often there is saturation in the area around the eye, skin, and cheek regions. On the other hand, the VIS spectrum (400-700nm) captures textural details of the periocular skin region, conjunctiva vasculature, eye shape, eyelashes, eyebrows, and eyelids. The VIS imaging fails to capture the textural nuances of the iris pattern, especially for dark-colored irides. Examples of periocular recognition techniques in the NIR spectrum are (Monwar et al., 2013; Uzair et al., 2013; Hwang and Lee, 2020; Mikaelyan et al., 2014) , and in the VIS spectrum are (Adams et al., 2010; Bharadwaj et al., 2010; Park et al., 2009; Juefei-Xu et al., 2010; Woodard et al., 2010b; Xu et al., 2010; Park et al., 2011; Beom-Seok Oh et al., 2012; Padole and Proena, 2012; Santos and Hoyle, 2012; Joshi et al., 2014; Nie et al., 2014; Proena and Briceo, 2014; Bakshi et al., 2015; Santos et al., 2015; Hernandez-Diaz et al., 2018; Luz et al., 2018; Reddy et al., 2019) . provided a detailed survey of ocular techniques in the VIS spectrum. The researchers in (Hollingsworth et al., 2010; Smereka and Kumar, 2017) suggested that VIS images provide more discriminative information for periocular recognition compared to NIR images. Hollingsworth et al. (2012) made the same conclusion using human volunteers. The authors in (Alonso-Fernandez and Bigun, 2012; Ross et al., 2012; Smereka et al., 2015; Ambika et al., 2016; Zhao and Kumar, 2017) proposed periocular recognition techniques that can be applied to both NIR and VIS images. Other researchers Vetrekar et al., 2018; Ipe and Thomas, 2020) fused information obtained from both NIR and VIS images. Table 2 provides a summary (features extraction, datasets, (Park et al., 2009; Bharadwaj et al., 2010; Xu et al., 2010; Beom-Seok Oh et al., 2012) (Padole and Proena, 2012; Santos and Hoyle, 2012; Uzair et al., 2013) (Mahalingam et al., 2014; Nie et al., 2014; Sharma et al., 2014; Bakshi et al., 2015) (Juefei- Xu et al., 2010; Santos et al., 2015) (Juefei- Xu and Savvides, 2012; Cao and Schmid, 2014) Bayesian Graphical Models (BGM) (Boddeti et al., 2011) Gabor filters (Jung et al., 2020) and performance) of various techniques applied on NIR, VIS, both spectrum, and multi-spectral (fusion of NIR and VIS) images.

A vast amount of research has also focused on crossspectrum matching, where enrolled images are in one spectrum, while probe images are in another spectrum. The crossspectrum evaluation scenario implicitly encapsulates the crosssensor scenario (enrolled and probes images are from different sensors) as well. Examples of papers discussing the crossspectrum scenario are (Cao and Schmid, 2014; Sharma et al., 2014; Ramaiah and Kumar, 2016; Behera et al., 2017; Raja et al., 2017; Hernandez-Diaz et al., 2019; Alonso-Fernandez et al., 2020; Behera et al., 2020; Hernandez-Diaz et al., 2020; Vyas, 2022) . Behera et al. (2019) provided a detailed survey on cross-spectrum periocular recognition. A more difficult evaluation scenario is when testing is performed on different datasets (cross-dataset) as it has to account for the variations due to different sensors, data acquisition environments, and subject population. Examples of cross-dataset evaluation can be found in (Reddy et al., 2019 (Reddy et al., , 2020 . Table  3 summarizes various cross-spectrum and cross-dataset techniques. The cross-sensor techniques are mainly evaluated on different mobile devices, so we provide these details in Section 7 (Periocular Recognition on Mobile Devices).

Simultaneous acquisition of periocular with the iris modality, and its complementary nature with respect to iris, has motivated researchers to fuse periocular with iris to improve the overall recognition performance. The authors in (Woodard et al., 2010a; Ross et al., 2012) proposed the fusion of periocular with iris to improve the performance when acquired iris images are of low quality due to partial occlusions, specular reflections, off-axis gaze, motion and spatial blur, non-linear deformations, contrast variations, and illumination artifacts. The fusion is also helpful when iris images are captured from a distance as the periocular region is relatively stable even at a distance (Tan and Kumar, 2012) . It is also advantageous when iris images are acquired in the visible spectrum (Santos and Hoyle, 2012; Tan and Kumar, 2013; Proena, 2014; Jain et al., 2015; Silva et al., 2018) , or using mobile devices Ahuja et al., 2016b) . The iris texture is better discernible in NIR illumination, whereas periocular features become more perceptible in VIS illumination (Alonso-Fernandez and Bigun, 2015; . The overall performance obtained on the fusion of iris and periocular traits is generally better than using the iris only as shown in (Komogortsev et al., 2012; Raghavendra et al., 2013; Raja et al., 2014a; Ahmed et al., 2017; Verma et al., 2016) . The fusion of iris and periocular is mainly performed at the score-level (Woodard et al., 2010a; Tan and Kumar, 2012; Raghavendra et al., 2013; Tan and Kumar, 2013; Proena, 2014; Jain et al., 2015; Santos et al., 2015; Verma et al., 2016; Ahuja et al., 2016b; Algashaam et al., 2021) , though there is some work on feature-level (Jain et al., 2015; Stokkenes et al., 2017; Silva et al., 2018) and decision-level (Santos and Hoyle, 2012) fusion also. Ogawa and Kameyama (2021) proposed Multi Modal Selector that adaptively selects a iris and periocu- (Park et al., 2011) Fusion of HOF, LBP and SIFT FRGC: RR is 87.32% (Padole and Proena, 2012) HOG, LBP, SIFT UBIPr: EER is ∼20%(HOG + LBP + SIFT) (Santos and Hoyle, 2012) LBP, SIFT UBIRIS v2: EER is 31.87% and RR is 56.4% (Joshi et al., 2014) Gabor-PPNN, DWT, LBP, HOG MBGC: EER is 6.4%, GTDB: EER is 5.9%, IITK: EER is 15.5%, PUT: EER is 4.8% (Nie et al., 2014) PCA to: CRBM, SIFT, LBP, HOG UBIPr: EER is 6.4% and RR is 50.1% (Proena and Briceo, 2014) GC-EGM to: LBP + HOG + SIFT FaceExpressUBI: EER is 16% (Hernandez-Diaz et al., 2018) Fusion of off-the-shelf CNN (AlexNet, GoogLeNet, ResNet, and VGG) features and traditional features UBIPr: EER of 5.1% and FRR is 11.3% at 1% FAR (Jung et al., 2020) Generalized label smoothing regularization-trained networks ETHNIC, PUBFIG, FACESCRUB, AND IMDB WIKI: avg.RR is 88.7% and EER of 10.4% NIR and VIS Spectrum (Woodard et al., 2010b) Tessellated image + Histograms of texture and color FRGC (VIS): RR is 91%, MBGC (NIR): RR is 87% Fusion of GOH, PDM, SIFT features at the score-level FOCS (NIR): EER is 18.8%, FRGC (VIS): EER is 1.59%

(Alonso-Fernandez and Bigun, 2015) Gabor features 4 NIR datasets: Accuracy is 97% 2 VIS datasets: Accuracy is 27% (Bakshi et al., 2015) Raw pixels, LBP, PCA, LBP + PCA MGBC: NIR-RR is 99.8%, VIS-RR is 98.5% CMU Hyperspectral: RR is 97.2%, UBIPr: RR is 99.5% (Ambika et al., 2016) LaplaceBeltrami based shape features CASIA FV1: accuracy is 95%, Basel 3D: Accuracy is 97% 3D periocular: Accuracy is 97.5% (Smereka et al., 2015) Periocular probabilistic deformation models 2 NIR and 3 VIS images datasets (Zhao and Kumar, 2017) Semantics-assisted convolutional neural networks UBIRIS.V2: RR is 82.43%, FRGC: RR is 91.13%, FOCS: RR is 96.93%, CASIA.v4-distance: RR is 98.90% Multi-spectrum Multimodal compact multi-linear pooling feature fusion IMP: Accuracy is 91.8% (Vetrekar et al., 2018) HOG, GIST, Log-Gabor transform and BSIF + CRC Proprietary: RR is 96.92% (Ipe and Thomas, 2020) Fusion of the off-the-shelf CNN feature IMP: Accuracy is 97.14% Table 3 . A chronological overview (description, datasets, and performance) of periocular techniques working under cross-spectrum and cross-dataset scenarios. Here, GAR is Genuine Acceptance Rate, GMR is Genuine Match Rate, FMR is False Match Rate, and d is the separation between the mean of genuine and impostor distributions. The acronyms used in the 'Description' column are defined in the text or in the referenced papers.

Datasets and Performance Cross-spectrum (Cao and Schmid, 2014) Gabor LBP, Generalized LBP, Gabor Weber descriptors Pre-Tinders,TINDERS, PCSO (GAR at 0.1 FAR): SWIR-VIS: 0.75, NIR-VIS: 0.35, MWIR-VIS: 0.35 Combined neural network architecture IMP (GAR @ 1% FAR): VIS-NV: 71.93%, VIS-NIR:47.08%, NV-NIR:48.21% (Ramaiah and Kumar, 2016) Three patch LBP + MRF (GAR @ 0.1% FAR) IMP (NIR-VIS): 18.35%, PolyU (NIR-VIS): 73.2% Steerable pyramids + SVM + Fusion of different scales CROSS-EYED2016 (NIR-VIS): GMR is 100% at 0.01% FMR (Behera et al., The fusion of periocular with the face modality is also a viable option as periocular is a part of the face, and no additional acquisition is required. Though the periocular region is already accounted in face recognition as a part of the face, isolating the periocular and performing region-specic feature extraction provides an overall improvement in recognition performance. The fusion of face with periocular is also beneficial when face images are occluded, having large pose variations, or captured at a very close distance (e.g., a selfie). The work of periocular fusion with face in the context of plastic surgery , gender transformation (Mahalingam et al., 2014) and mobile devices (Raja et al., 2015a; Pereira and Marcel, 2015) shows improved recognition accuracy. Table 4 summarizes various techniques that fuse periocular with iris and face modalities. In another research work, Oh et al. (2014) fused periocular features (structured random projections) with binary sclera features at the score-level for identity verification. Nigam et al. (2015) provided a detailed survey on the fusion of various ocular biometrics.

The extensive usage of mobile devices motivates the need for human authentication on mobile devices for various purposes, such as access control, digital payments, or mobile banking. Several mobile devices are now emerging with integrated biometric sensors -iPhone 12 has a Touch ID fingerprint sensor and Face ID cameras, and the Samsung Galaxy S20 series has an in-display fingerprint sensor and an iris scanner. Periocular images are generally acquired using the front or rear camera of mobile devices in the visible spectrum. The challenges in mobile biometrics are low-quality input images and relatively limited computational power. The low-quality images are due to hardware limitations and less constrained capturing environments. Raja et al. (2014b) explored periocular recognition on smart devices using well known feature extraction techniques (SIFT, SURF, and BSIF) and achieved a Genuine Match Rate (GMR) of 89.38% at 0.01% False Match Rate (FMR). There is some work on NIR images captured from mobile devices (Bakshi et al., 2018; Zhang et al., 2018) . Bakshi et al. (2018) utilized a reduced version of Phase Intensive Local Pattern (PILP) features, whereas (Zhang et al., 2018) fused compact CNN features of iris and periocular through a weighted concatenation. Majority of the periocular-based mobile biometrics are performed on VIS images (Pereira and Marcel, 2015; Raja et al., 2015b; Ahuja et al., 2016a; Keshari et al., 2016; Raja et al., 2016b; Raghavendra and Busch, 2016; Ahmed et al., 2017; Ahuja et al., 2017; Stokkenes et al., 2017; Boutros et al., 2020; Krishnan et al., 2020; Raja et al., 2020) . Keshari et al. (2016) investigated periocular recognition on pre-and post-cataract surgery mobile images. Krishnan et al. (2020) investigated the fairness of mobile ocular biometrics methods across gender. The work in (Pereira and Marcel, 2015; Raja et al., 2015b; Ahmed et al., 2017; Zhang et al., 2018) used fusion of different modalities for mobile biometrics - Raja et al. (2015b) fused iris, face and periocular modalities, (Pereira and Marcel, 2015) combined face and periocular, whereas Ahmed et al., 2017; Zhang et al., 2018) combined iris and periocular. Recent work on mobile biometrics used deep learning features (Raja et al., 2016b; Raghavendra and Busch, 2016; Ahuja et al., 2017; Raja et al., 2020) . Boutros et al. (2020) verified an individual wearing Head Mounted Display (HMD) using four handcrafted feature extraction methods and two deeplearning strategies. Generalizability across different mobile sensors (cross-sensor) are also evaluated in Raja et al., 2016a,c; Garg et al., 2018; Reddy et al., 2018b;  Alonso . Table 5 provides a brief description of various mobile-based periocular recognition techniques.

Softbiometrics refer to attributes used to classify individuals in broad categories such as gender, ethnicity, race, age, height, weight, or hair color. The periocular region has also been used for automatically estimating age, gender, ethnicity, and facial expression information. An exploration of gender information contained in the periocular region is performed in (Merkow et al., 2010; Lyle et al., 2012; Bobeldyk and Ross, 2016; Castrilln-Santana et al., 2016; Tapia and Arellano, 2019) . Tapia and Arellano (2019) synthesized NIR periocular images using a conditional GAN based on gender information, and then identify gender using the synthesized periocular images. The work in (Lyle et al., 2012; Woodard et al., 2017) extracted race information from the periocular region, while the work in determined an individual's age from the periocular region. Alonso-Fernandez et al. (2018) investigated the feasibility of using the periocular region for facial expression recognition. 2. Long Distance Recognition: Bharadwaj et al. (2010) showed the degradation of iris recognition performance with an increase in standoff distance and suggested the use of the periocular region on long-distance images. The authors in (Tan and Kumar, 2012; Verma et al., 2016) proposed fusion approaches (iris and periocular) for human recognition at a distance (NIR images). Kim et al. (2018) presented CNN-based periocular recognition in a surveillance environment. Ross (2014) presented the challenging problem of matching face in VIS spectrum against iris images in NIR spectrum (cross-modal) using periocular information. They utilized LBP, Normalized Gradient Correlation (NGC), and Joint Dictionary-based Sparse Representation (JDSR) methods to accomplish cross-modality matching. 5. Periocular Forensics: The authors in (Marra et al., 2018; Banerjee and Ross, 2018) deduced sensor information from the periocular images. In another work, Banerjee and Ross (2019) suppressed the sensor-specific information (sensor anonymization) and also incorporated the sensor pattern of a different device (sensor spoofing) in periocular images. 6. Other Applications: Du et al. (2016) utilized the periocular region to correct mislabeled left and right iris images in a diverse set of iris datasets. The work in Hoffman et al., 2019) Raja et al., 2016a) utilized the periocular region to identify individuals after they undergo facial plastic surgery. Mahalingam et al. (2014) introduced a medically altered gender transformation face dataset and proposed the fusion of periocular (patched-based LBP) with face, which outperformed standalone commercial-off-theshelf face matchers. Keshari et al. (2016) investigated periocular recognition on pre-and post-cataract surgery images.

In early literature, periocular recognition was performed using face and iris datasets as there were limited datasets available that contained the periocular region only. Commonly used face datasets to perform periocular recognition research on VIS images are FRGC, FERET, FG-NET, MobBIO, and on NIR images are IIT Delhi v1.0, CASIA Interval, BioSec. The iris datasets used for periocular recognition research are UBIRIS v2 (VIS), MBGC (NIR), and PolyU cross-spectral datasets. Table 6 describes the datasets specifically collected for periocular recognition. Figures 4, 5 , and 6 show a few images from these periocular datasets. The datasets used to perform periocular recognition research under variable stand-off distance are FRGC, UBIRIS v2, and UBIPr. Examples of datasets providing video data of subjects for periocular biometrics research are MBGC, FOCS, and VSSIRIS. Other datasets provide special evaluation scenarios such as aging (MORPH, FG-NET), plastic surgery (Raja et al., 2016a) , gender transformation (Mahalingam et al., 2014) , expression changes (FaceExpressUBI), face occlusion (AR, Compass), cross-spectral matching (CMU-H, IMP, CROSS-EYED 2016, CROSS-EYED 2017), or mobile authentication (CASIA-Iris-Mobile-V1.0, CSIP, MICHE I and II, VSSIRIS, VISOB 1.0 and 2.0, CMPD). Various competitions focusing on periocular recognition can be found in Sequeira et al., 2016; De Marsico et al., 2017; Sequeira et al., 2017) . The competitions in De Marsico et al., 2017) are on mobile periocular images, while the competitions in Sequeira et al., 2017) evaluated the cross-spectrum (matching of VIS and NIR images) scenario. Table 7 summarizes details about these competitions. Zanlorensi et al. (2021) surveyed various ocular datasets and discussed popular ocular recognition competitions. The authors described 36 iris, 4 iris/periocular, 4 periocular, and 10 multimodal datasets.

The definition of the periocular region is not standardized. What is the actual boundary around the eye? Should we consider a single eye or both eyes to be in the periocular region? These questions about the scope of the periocular region is yet to be answered. Apart from these definitional concerns, issues around standardization has to be resolved for groundtruth segmentation, and the minimum resolution needed for recognition. 2. Generalizability: Periocular biometric solutions should be generalizable, which refers to the matching of periocular images under cross-sensor (images from different sensors), cross-spectrum (images from different spectra), cross-dataset (images from different datasets), crossresolution (images at multiple distances), and cross-modal (images from different modalities) scenarios. 3. Non-ideal Conditions: Researchers need to focus on periocular matching under non-ideal conditions, i.e., pose variations (Park et al., 2011; Karakaya, 2021) , expression, non-uniform illumination, low-resolution, occlusions (eyeglasses, eye-blinking, different types of masks, scarfs or helmets or eye makeup), or large stand-off distance. 4. Effects of Aging: With age, wrinkles and folds around the eye could change the overall appearance of the periocular region. The effects of aging on periocular recognition are yet to be comprehensively studied (Ma et al., 2019) .

Anti-spoofing Measures: While periocular region has been utilized to detect iris spoof attacks (Alonso-Fernandez and Bigun, 2014; Hoffman et al., 2019), we should also be vigilant about spoof attacks directed at the periocular region.

6. Explainability and Interpretability: Increasing use of deep learning-based techniques in periocular biometrics opens another direction which involves explainability of these deep learning models (Brito and Proena, 2021) .

This article provided a survey on periocular biometrics in the wake of its importance due to the increased use of face masks. Firstly, we reported recent face and periocular recognition techniques specifically designed to recognize humans wearing a face mask. Subsequently, we provided details on various aspects of periocular biometrics, viz., anatomical cues in the periocular region used for recognition, various feature extraction and matching techniques, cross-spectral recognition, its fusion with other biometrics modalities (face or iris), authentication in mobile devices, usefulness of this biometric in other applications, periocular datasets, and competitions. Finally, we discussed the various challenges and future directions to work on. The applicability of the periocular biometrics is likely to extend to other scenarios where only the ocular region of the face may be visible. This could be due to cultural etiquette (e.g., women covering their face) or safety precautions (e.g., surgeons or construction workers covering their nose and mouth). Table 7 . A summary (datasets and performance achieved) of various competitions on periocular recognition. Here, GFRR is Generalized False Rejection Rate, and GFAR is Generalized False Acceptance Rate.

Dataset Performance MICHE-II (De Marsico et al., 2017) MICHE-I and MICHE-II EER is 2.74% and FRR is 9.13% @ 0.1% FAR Ahmed et al., 2017 ) ICIP VISOB EER is 0.06% -0.20% and GMR is 92% @ 0.1% FMR ( (Zhang et al., 2018) .

Genetic-based type II feature extraction for periocular biometric recognition: Less is more

Using fusion of iris code and periocular biometric for matching visible spectrum iris images captured by smart phone cameras

Combining iris and periocular biometric for matching visible spectrum eye images

Isure: User authentication in mobile devices using ocular biometrics in visible spectrum

A preliminary study of CNNs for iris and periocular verification in the visible spectrum

Convolutional neural networks for ocular smartphone-based biometrics

Hierarchical fusion network for periocular and iris by neural network approximation and sparse autoencoder

Multispectral periocular classification with multimodal compact multi-linear pooling

Elliptical higher-order-spectra periocular code

Periocular recognition using retinotopic sampling and Gabor decomposition

Best regions for periocular recognition with NIR and visible images

Exploting periocular and RGB information in fake iris detection. International Convention on Information and Communication Technology

Near-infrared and visible-light periocular recognition with Gabor features using frequency-adaptive automatic eye detection

A survey on periocular biometrics research

Expression recognition using the periocular region: A feasibility study

Comparison and fusion of multiple iris and periocular matchers using near-infrared and visible images

Cross-sensor periocular biometrics for partial face recognition in a global pandemic

Periocular authentication based on FEM using LaplaceBeltrami eigenvalues

Masked face recognition for secure authentication

Survey of periocular recognition techniques

Optimized periocular template selection for human recognition

Fast periocular authentication in handheld devices with reduced phase intensive local pattern

Phase intensive global pattern for periocular recognition

A novel phase-intensive local pattern for periocular recognition under visible spectrum

Impact of photometric transformations on PRNU estimation schemes: A case study using near infrared ocular images

Smartphone camera de-identification while preserving biometric utility

Periocular recognition in cross-spectral scenario

Cross-spectral periocular recognition: A survey

Variance-guided attention-based twin deep network for cross-spectral periocular recognition. Image and Vision Computing (IVC) 104, 104016

Periocular biometrics: When iris recognition fails. International Conference on Biometrics: Theory, Applications and Systems (BTAS)

Iris or periocular? exploring sex prediction from near infrared ocular images

A comparative evaluation of iris and ocular recognition methods on challenging ocular images

Self-restrained

(g) UFPR-Periocular (Zanlorensi et al., 2020). triplet loss for accurate masked face recognition

MFR 2021: masked face recognition competition

Iris and periocular biometrics for head mounted displays: Segmentation, recognition, and synthetic data generation

A survey of iris biometrics research

Image understanding for iris biometrics: A survey

A short survey on machine learning explainability: An application to periocular recognition

GANT: Gaze analysis technique for human identification

Matching heterogeneous periocular regions: Short and long standoff distances

On using periocular biometric for gender classification in the wild

Combining 3D and 2D for less constrained periocular recognition

Multispectral scleral patterns for ocular biometric recognition

Masked face recognition: Human vs. machine

The effect of wearing a mask on face recognition performance: an exploratory study

Sclera recognition -a survey

Results from MICHE II Mobile Iris CHallenge Evaluation II

Mobile Iris Challenge Evaluation (MICHE)-I: biometric iris dataset and protocols

Masked face recognition challenge: The InsightFace track report

A texture-based neural network classifier for biometric identification using ocular surface vasculature

Masked face recognition with latent part detection

Eyebrow shape-based features for biometric recognition and gender classification: A feasibility study

Automated classification of mislabeled near-infrared left and right iris images using convolutional neural networks

Examples of periocular images from Multi-spectral datasets: (a) IIITD Multispectral Periocular (IMP

Heterogeneity aware deep embedding for mobile periocular recognition

Detecting masked faces in the wild with LLE-CNNs

Masked face recognition with generative data augmentation and domain constrained ranking

Efficient masked face recognition method during the COVID-19 pandemic

Periocular recognition using CNN features off-the-shelf

Cross spectral periocular matching using ResNet features

Cross-spectral periocular recognition with conditional adversarial networks

Eyebrow deserves attention: Upper periocular biometrics

Iris + ocular: Generalized iris presentation attack detection using multiple convolutional neural networks

Complex eye movement pattern biometrics: The effects of environment and stimulus

Identifying useful features for recognition in near-infrared periocular images

Useful features for human verification in near-infrared periocular images. Image and Vision Computing (IVC)

Human and machine performance on periocular biometrics under near-infrared light and visible light

Near-infrared image-based periocular biometric method using convolutional neural network

CNN based periocular recognition using multispectral images

Person identification at a distance via ocular biometrics

Introduction to Biometrics

Mitigating effects of plastic surgery: Fusing face and ocular biometrics

Matching face against iris images using periocular information

Periocular recognition based on Gabor and Parzen PNN. International Conference on Image Processing

Robust local binary pattern feature sets for periocular biometric identification

vestigating age invariant face recognition based on periocular biometrics. International Joint Conference on Biometrics (IJCB)

Hallucinating the full face from the periocular region via dimensionally weighted K-SVD

Unconstrained periocular biometric acquisition and recognition using COTS PTZ camera for uncooperative and noncooperative subjects

Fastfood dictionary learning for periocularbased full face hallucination

Periocular recognition in the wild with generalized label smoothing regularization

On identification from periocular region utilizing SIFT and SURF

Iris-ocular-periocular: toward more accurate biometrics for off-angle images

Mobile periocular matching with pre-post cataract surgery

Convolutional neural network-based periocular recognition in surveillance environments

Biometric identification via an oculomotor plant mathematical model

Multimodal ocular biometrics approach: A feasibility study. International Conference on Biometrics: Theory, Applications and Systems (BTAS)

Probing fairness of mobile ocular biometrics methods across gender on visob 2

Periocular biometrics: A survey

Periocular biometrics for non-ideal images: with off-the-shelf deep CNN & transfer learning approach

A novel periocular biometrics solution for authentication during Covid-19 pandemic situation

An optimal feature enriched region of interest (ROI) extraction for periocular biometric system

A novel eyebrow segmentation and eyebrow shape-based identification

Look through masks: Towards masked face recognition with de-occlusion distillation

A CNN-based framework for comparison of contactless to contact-based fingerprints

A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic

Deep periocular representation aiming video surveillance

Soft biometric classification using local appearance periocular region features

Effect of aging in periocular appearances by comparison of anthropometry between early and middle adulthoods in chinese han population

Investigating the periocularbased face recognition across gender transformation

A deep learning approach for iris sensor model identification

An exploration of gender identification using only the periocular region

Periocular recognition by detection of local symmetry patterns

Performance evaluation of local appearance based periocular recognition

Personal identification using periocular skin texture

Boosting masked face recognition with multi-task arcface

Rank information fusion for challenging ocular image recognition

Robust periocular recognition by fusing sparse representations of color and geometry information

NISTIR 8311 -Ongoing FRVT Part 6A: Face recognition accuracy with face masks using pre-COVID-19 algorithms

NISTIR 8331 -Ongoing FRVT Part 6B: Face recognition accuracy with face masks using post-COVID-19 algorithms

Periocular recognition using unsupervised convolutional rbm feature learning

Ocular biometrics: A survey of modalities and fusion approaches

Adaptive selection of classifiers for person recognition by iris pattern and periocular image

Combining sclera and periocular features for multi-modal identity verification

Grid loss: Detecting occluded faces. European Conference on Computer Vision (ECCV)

Periocular recognition: Analysis of performance degradation factors

Periocular biometrics in the visible spectrum

Periocular biometrics in the visible spectrum: A feasibility study. International Conference on Biometrics: Theory, Applications, and Systems (BTAS)

Evaluation of periocular features for kinship verification in the wild

Periocular biometrics in mobile environment. International Conference on Biometrics Theory, Applications and Systems (BTAS)

Ocular biometrics by score-level fusion of disparate experts

Periocular biometrics: constraining the elastic graph matching algorithm to biologically plausible distortions

Deep-PRWIS: Periocular recognition without the iris and sclera using deep learning frameworks

A reminiscence of mastermind: Iris/periocular biometrics by in-set CNN iterative analysis

Segmenting the periocular region using a hierarchical graphical model fed by texture / shape information and geometrical constraints

Identifying facemask-wearing condition using image super-resolution with classification network to prevent COVID-19s

Learning deeply coupled autoencoders for smartphone based robust periocular verification

Combining iris and periocular recognition using light field camera

Binarized statistical features for improved iris and periocular recognition in visible spectrum

Biometric recognition of surgically altered periocular region: A comprehensive study. International Conference on

Collaborative representation of deep sparse filtered features for robust verification of smartphone periocular images

Dynamic scale selected laplacian decomposed frequency response for cross-smartphone periocular verification in visible spectrum

Scale-level score fusion of steered pyramid features for cross-spectral periocular verification

Collaborative representation of blur invariant deep sparse features for periocular recognition from smartphones

Smartphone authentication system using periocular biometrics. International Conference of the Biometrics Special Interest Group (BIOSIG

Fusion of face and periocular information for improved authentication on smartphones

Multi-modal authentication system for smartphones using face, iris and periocular

Smartphone based visible iris recognition using deep sparse filtering

On matching cross-spectral periocular images for accurate biometrics identification

Ocular biometrics in the visible spectrum: A survey

On fine-tuning convolutional neural networks for smartphone based ocular recognition

ICIP 2016 competition on mobile ocular biometric recognition. International Conference on Image Processing

Convolutional neural network for age classification from smart-phone based ocular images

Comparison of deep learning models for biometric-based mobile user authentication

OcularNet: Deep patch-based ocular biometric recognition

Robust subject-invariant feature learning for ocular biometrics in visible spectrum

Generalizable deep features for ocular biometrics. Image and Vision Computing (IVC) 103

Human eye movements as a trait for biometrical identification

Some research problems in biometrics: The future beckons. International Conference on Biometrics (ICB)

Matching highly non-ideal ocular images: An information fusion approach

The role of eyebrows in face recognition

Fusing iris and periocular information for cross-sensor recognition

A fusion approach to unconstrained iris recognition

Periocular biometrics: An emerging technology for unconstrained scenarios

Cross-eyed 2017: Crossspectral iris/periocular recognition competition

Cross-eyed -cross-spectral iris/periocular recognition database and competition. International Conference of the

On cross spectral periocular recognition

Multimodal feature level fusion based on particle swarm optimization with deep transfer learning

Probabilistic deformation models for challenging periocular image verification

What is a "good" periocular region for recognition?

Occlusion robust face recognition based on mask learning with pairwise differential siamese network

Feature level fused templates for multi-biometric system on smartphones

Toward statistical modeling of saccadic eye-movement and visual saliency

Human identification from at-a-distance images by simultaneously exploiting iris and periocular features

Towards online iris and periocular recognition under relaxed imaging constraints

Noisy iris image matching by using multiple cues

Soft-biometrics encoding conditional GAN for synthesis of NIR periocular images

A novel GAN-based network for unmasking of masked face

Periocular biometric recognition using image sets

At-a-distance person recognition via combining ocular features

Multispectral imaging for robust ocular biometrics

Enhanced near-infrared periocular recognition through collaborative rendering of hand crafted and deep features

Periocular-assisted multi-feature collaboration for dynamic iris recognition

Masked face recognition dataset and application

On the fusion of periocular and iris biometrics in non-ideal imagery

Periocular region appearance cues for biometric identification

Periocular-based soft biometric classification, in: Iris and Periocular Biometric Recognition, IET Digital Library

Robust local binary pattern feature sets for periocular biometric identification

Contactless fingerprint recognition based on global minutia topology and loose genetic algorithm

Ufpr-periocular: A periocular dataset collected by mobile devices in unconstrained scenarios

Ocular recognition databases and competitions: A survey

Deep representations for cross-spectral ocular biometrics

The BTAS competition on mobile iris recognition

Deep feature fusion for iris and periocular biometrics on mobile devices

Accurate periocular recognition under less constrained environment using semantics-assisted convolutional neural network

Improving periocular recognition by explicit attention to critical regions in deep neural network

A new human identification method: Sclera recognition

Masked face recognition challenge: The WebFace260M track report

 Table 6 . Description of periocular datasets (NIR, VIS, and multi-spectrum), along with representative research papers utilizing these datasets. Datasets Description Papers NIR Spectrum NIST-Face and ocular challenge series (FOCS) 9,588 total images, 136 subjects, 750x600 resolution, monocular images IOM sensor (Boddeti et al., 2011; Ross et al., 2012; Monwar et al., 2013 ) (Smereka and Kumar, 2013; Smereka et al., 2015 ) (Verma et al., 2016; Zhao and Kumar, 2017) MIR 2016 (Zhang et al., 2016) 16,500 total images, 550 subjects, 1968x2014 resolution, biocular images IrisKing mobile sensor (Zhang et al., 2016) CASIA-Iris-Mobile-V1.0 (Zhang et al., 2018) 11,000 total images, 630 subjects, biocular images, CASIA NIR mobile camera (Zhang et al., 2018) CASIA-IrisV4-Distance 2,567 total images, 142 subjects, 2352x1728 resolution, biocular images, CASIA long-range iris camera Kumar, 2012, 2013; Verma et al., 2016) Visible SpectrumUBIPr ( (Sequeira et al., 2017) 5,600 total images, 175 subjects, 900800 resolution, monocular images (Sequeira et al., 2017 ) QUT Multispectral