key: cord-0320446-m0dirkcs
authors: Gupta, Roopam K.; Hempler, Nils; Malcolm, Graeme P. A.; Dholakia, Kishan; Powis, Simon J.
title: High throughput hemogram of T cells using digital holographic microscopy and deep learning
date: 2021-12-23
journal: bioRxiv
DOI: 10.1101/2021.12.23.473983
sha: 6fe5fb58fcb9741def20050d1c24ffc0a5eb4fcf
doc_id: 320446
cord_uid: m0dirkcs

T cells of the adaptive immune system provide effective protection to the human body against numerous pathogenic challenges. Current labelling methods of detecting these cells, such as flow cytometry or magnetic bead labelling, are time consuming and expensive. To overcome these limitations, the label-free method of digital holographic microscopy (DHM) combined with deep learning has recently been introduced which is both time and cost effective. In this study, we demonstrate the application of digital holographic microscopy with deep learning to classify the key CD4+ and CD8+ T cell subsets. We show that combining DHM of varying fields of view, with deep learning, can potentially achieve a classification throughput rate of 78,000 cells per second with an accuracy of 76.2% for these morphologically similar cells. This throughput rate is 100 times faster than the previous studies and proves to be an effective replacement for labelling methods.

The adaptive immune response comprises white blood cells including T and B cells 2 that can recognise and respond in an antigen-specific manner to a vast array of potential 3 human pathogens. Of great significance, residing within this same subset of cells is the 4 ability to generate memory cells, which can produce faster and stronger secondary 5 responses. Vaccination/immunisation relies almost exclusively on the generation of 6 such memory T and B cells to protect both individuals and populations [1] .

Methods such as support vector machines (SVMs), random forests (RFs) or artificial 23 neural networks (ANNs) have been popularly employed for these purposes. However, 24 due to the inherent linearity, the methods of SVM and RF have proven to be less ef-25 ficient for classification than deep learning based ANNs [12] . Hence deep learning is 26 being ever more widely applied to solve the classification problem in biophotonics [13] . 27 Another aspect of the mentioned systems is their throughput rate. While Raman 28 spectroscopy provides high molecular specificity, it is slow and lacks the aspect of 29 throughput [14] . DHM when combined with convolutional neural networks, on the 30 other hand, provides the capability to differentiate morphologically similar cells with 31 a recent demonstration of throughput rate of more than 100 cells/s [15] . This through-32 put rate is still too low: the gold standard flow cytomtery may allow a throughput rate 33 of 70,000 cells/s [16] .The throughput rate of the DHM system can be enhanced by 34 reducing the magnification and numerical aperture (NA) of the microscopic objective 35 which may in-turn result in a lower resolution of images. These lower resolution im- 36 ages system can be transformed into ones of higher resolution using the single image 37 super resolution (SISR) method of deep learning [17] . Recently, deep learning has 38 been widely applied more broadly in photonics to improve the resolution of bright field 39 optical microscope [18], to enable cross-modality super resolution in fluorescence mi-40 croscopy [19] , to facilitate pixel super-resolution in coherent imaging systems Liu et al. 41 [20], and to enhance the resolution of scanning electron microscopy [21] . high throughput of 78,000 cells/s using a combination of DHM with CNNs, which is nearly two orders of magnitude in excess of previous reports [ We acquired the data for the two cell types separately for each optical configuration. The phase images calculated using the three optical configurations demonstrate dif-100 ferent degrees of resolutions. Hence to identify and understand the degree of granular-101 ity discovered using each configuration, we implemented a method of k-means cluster-102 ing based image segmentation [23] . We considered the phase images corresponding to best position attained by any member of their topological neighborhood. This approach is used to minimize the error output of an objective function. In our case, we consider the objective function as the classification sensitivity and specificity achieved using a 121 given network geometry (particle).

To identify the best CNN geometries, we divided the complete dataset for each op- 144

Here, in Eq. 1a and 1b, TP is true positive, TN is true negative, FP is false positive and 145 FN is false negative. 146 We considered a total of 40 particles and a single swarm (optimized from 2 to 60 147 in the steps of one unit to avoid divergence) to find isolate an optimal architecture 148 of CNN for each image size. Each particle's position and velocity were initialized randomly. After the calculation of cost for all the particles, the particle with least cost 150 was considered as the reference such that the position and velocity of all the other 151 particles were updated relatively to the reference. For each training instance, we calculated a generative adversarial loss for both the 175 generative networks. We also calculated a cycle consistency loss using the combination 176 of two networks. The generative adversarial loss was evaluated as:

Here, i ∈ {a, b}. The cycle consistency loss L cyc (G a , G b ) is computed, to satisfy the

Here, the variables x and y represent the input and output images for the given network 180 configuration. The combined loss was calculated as:

Here, λ is a hyperparameter which we chose as 10 for this application. During the 182 training, the objective is to minimize the combined loss for the generator networks 183 while maximizing the loss for the discriminator networks:

To achieve minimum training loss and avoid divergence during training, we consid- 

We captured bright field and fringe images using all the three configurations de- The images accumulated using the three optical configuration exhibit varying FOVs 207 and resolutions. Fig. 4 demonstrate the automatic cellular detection for various FOVs.

As summarised in Table 2 , the FOV achieved by using the 20X objective was greatest at using the test data were 82.5 % ± 3.96 % and 73.18 % ± 7.55 % respectively.

The interesting aspect of this comparison is the trend of increasing classification 262 accuracy (Fig. 6) The single image super resolution transformation was implemented on the phase 280 images acquired using 20X configuration to convert them into the phase images ac-281 quired using 100X configuration. To achieve this the DL models were trained using 282 the cycle GAN training method as explained before.

The trained generative models resulted in astounding transformations of the phase 

Immunological mechanisms of vaccination

The regulation of cd4 and cd8 coreceptor 330 gene expression during t cell development

In vitro differentia-333 tion of effector cd4+ t helper cell subsets

Cd8+ t cell states in 336 human cancer: insights from single-cell analysis

Dissecting how cd4 t cells are lost during hiv infection

Adoptive immunotherapy for 341 cancer: harnessing the t cell response

The blockade of immune checkpoints in cancer immunotherapy

Reduction and functional exhaustion of t cells in patients with 347 coronavirus disease 2019 (covid-19)

Classification of t-cell activation 354 via autofluorescence lifetime imaging

Multimodal 357 discrimination of immune cells using a combination of raman spectroscopy and 358 digital holographic microscopy

Comparison of support vector machine, random forest 360 and neural network classifiers for tree species classification on airborne hyper-361 spectral apex images

Deep learning a 363 boon for biophotonics?

Towards automated cancer 365 screening: label-free classification of fixed cell samples using wavelength modu-366 lated raman spectroscopy

Label-free optical hemogram of granulocytes enhanced by artificial neural net-369 works

Flow cytometry: ret-371 rospective, fundamentals and recent instrumentation

Deep learning for 374 single image super-resolution: A brief review

Deep 377 learning microscopy

Deep learning enables cross-modality super-resolution in 380 fluorescence microscopy

Deep 382 learning-based super-resolution in coherent imaging systems

Resolution enhance-385 ment in scanning electron microscopy using deep learning

Particle swarm optimization

ICNN'95-International Conference on Neural Networks

The advantages of careful seeding

SODA '07

Adam: A method for stochastic optimization

Unpaired image-to-image translation 397 using cycle-consistent adversarial networks

Cycle-consistent 400 deep learning approach to coherent noise reduction in optical diffraction tomog-401 raphy

The use of wavelength modulated raman spectroscopy in label-404 free identification of t lymphocyte subsets, natural killer cells and dendritic cells

Multimodal 407 discrimination of immune cells using a combination of raman spectroscopy and 408 digital holographic microscopy

Detect circles with various radii in grayscale image via hough transform