key: cord-0020004-nof634tc
authors: Kittichai, Veerayuth; Kaewthamasorn, Morakot; Thanee, Suchansa; Jomtarak, Rangsan; Klanboot, Kamonpob; Naing, Kaung Myat; Tongloy, Teerawat; Chuwongin, Santhad; Boonsang, Siridech
title: Classification for avian malaria parasite Plasmodium gallinaceum blood stages by using deep convolutional neural networks
date: 2021-08-19
journal: Sci Rep
DOI: 10.1038/s41598-021-96475-5
sha: 153b123ab7d61a2accb72d0dbbc56f0876549eab
doc_id: 20004
cord_uid: nof634tc

The infection of an avian malaria parasite (Plasmodium gallinaceum) in domestic chickens presents a major threat to the poultry industry because it causes economic loss in both the quality and quantity of meat and egg production. Computer-aided diagnosis has been developed to automatically identify avian malaria infections and classify the blood infection stage development. In this study, four types of deep convolutional neural networks, namely Darknet, Darknet19, Darknet19-448 and Densenet201 are used to classify P. gallinaceum blood stages. We randomly collected a dataset of 12,761 single-cell images consisting of three parasite stages from ten-infected blood films stained by Giemsa. All images were confirmed by three well-trained examiners. The study mainly compared several image classification models and used both qualitative and quantitative data for the evaluation of the proposed models. In the model-wise comparison, the four neural network models gave us high values with a mean average accuracy of at least 97%. The Darknet can reproduce a superior performance in the classification of the P. gallinaceum development stages across any other model architectures. Furthermore, the Darknet has the best performance in multiple class-wise classification, with average values of greater than 99% in accuracy, specificity, and sensitivity. It also has a low misclassification rate (< 1%) than the other three models. Therefore, the model is more suitable in the classification of P. gallinaceum blood stages. The findings could help us create a fast-screening method to help non-experts in field studies where there is a lack of specialized instruments for avian malaria diagnostics.

www.nature.com/scientificreports/ biology approach could not always be affordable in low-income and resource-limited nations. As a result, an automated identification tool is desired. Artificial intelligence (AI) technology is currently a positive integration success in a diverse areas of interest, including agriculture, medicine and veterinary medicine [8] [9] [10] [11] [12] [13] [14] [15] . Computer-aided diagnosis, an AI subdivision, has been developed to identify human malaria infections and to classify their blood stage of the parasite growth. These can be used to assist clinical decision-making. Machine learning applications have also been studied in the documented veterinary field 16 . Previous research has suggested a diagnostic tool in veterinary medicine focused on image analysis using machine learning techniques 17, 18 , such as microscopic examination used to help diagnose disorder and disease in fish farm 19 . The analysis referred to above is intended to enhance the inspection of the pathogen region in the image by means of object detection, which includes various image processing techniques, including noise reduction, edge detection, morphological operations and context extraction.

Deep learning is a revolutionary and groundbreaking approach that has been incorporated into microscopic analysis for the veterinary medicine field 20, 21 . The methods are combined and tailored for individual datasets that differ in the size of the region of interest. In specific, deep learning technology is applied to end-to-end methods of extraction of features and self-discovery. The deep learning algorithm is very popular and useful with the emergence of a high-power computing machine to be used to study the classification of images and the recognition of clinical issues. Several neural network models have been used to contend with the animal sector, the Single-Shot MultiBox Detector (SSD) model used to evaluate the percentage of reticulocytes in cat's samples 22 , Alexnet for classification of fish disease such as Epizootic ulcerative syndrome (EUS), Ichthyophthirius (Ich) and Columnaris 19 . Deep learning is a revolutionary and groundbreaking approach that has been incorporated recently into microscopic analysis for the veterinary medicine field. The work described above shows that deep learning algorithms can be applied successfully in the field of veterinary medicine.

Previously, several techniques for image-based identification and classification of malaria parasite infections have been discussed, such as dark stretching technique 23 , modified fuzzy divergence technique, segmentation techniques 24 , adaptive color segmentation and tree-based decision classification 25 , segmentation, feature extraction, and SVM classifier 26 , convolutional neural classification 27, 28 . Moreover, deep CNN research has been conducted under the channel color space segmentation method 29 , deep belief network technique 30 , the Faster Region-based Convolutional Neural Network (Faster R-CNN) 31 , and the MATLAB-based Zach Threshold process for segmentation technique 32 . Successfully, several studies have reported the use of deep learning models to classify malaria parasites as automatic, rapid, and accurate approaches [33] [34] [35] . Interestingly, more than 95% of the accuracy is recorded in the detection of malaria-infected cells using three well-known CNNs, including LeNet, AlexNet, and GoogLeNet 36 . The previous work demonstrated micro-platforms to study and identify the infected avian red blood cells by using morphological modifications on the RBC surface to reveal the phases of P. gallinaceum. Since malaria has been described as a disease of blood and blood-forming tissues, the blood film sample has been diagnosed to better understand different degrees of disease 37 . Early rapid screening of parasite infection with reliable and also low-cost development is required, which could help us deter the spread of the disease. Therefore, timely identification of the malaria parasite in a blood smear test is crucial because it needs reliable and early diagnosis for successful containment.

A hybrid platform (VGG19 and SVM) recently demonstrated high performance in detecting infected and non-infected malaria parasite images, as observed follows: 93.44 per cent sensitivity; 92.92 per cent specificity; 89.95 per cent precision, 91.66 per cent F-score and 93.13 per cent accuracy 38 . The outstanding performance of the hybrid algorithms mentioned previously motivates us to develop a hybrid object detection and classification method for further classifying avian malaria in Thailand.

In this work, we employ two-state learning techniques, which combine an object detection model based on YOLOv3 with one of four classification models, namely Darknet, Darknet19, Darknet19-448, and Densenet201, to characterize the avian malaria blood stages of P. gallinaceum. The primary contribution is a comparison of several image classification methods utilizing blood protozoa images as qualitative and quantitative data for evaluating the proposed models. We compared the effectiveness of the proposed model in predicting parasitized and healthy chicken RBCs in thin blood film images. Furthermore, the suggested model can predict the stage of malaria development, which impacts both the severity of illness and the likelihood of malaria transmission. Because of the medicinal relevance of human malaria parasites, this type of technique has been used more extensively. To the best of our knowledge, this is the first study to use CNNs deep learning model in the categorization of clinical datasets related to clinical issues in P. gallinaceum-infected chickens. This work contributes significantly to the field since conventional identification relies heavily on microscopist experts. Such experts require years of training and practice. Therefore, it would be very helpful to have CNNs deep learning aid and could be done by technicians without intensive training.

Ethics statement. Archived Giemsa-stained thin-blood films have been collected from previous studies 1, 39 .

This study was reviewed and approved by the Chulalongkorn University Faculty of Veterinary Science Biosafety Committee (Approval No. 1931011) and was approved by the Institutional Animal Care and Use Committee in accordance with university regulations and policies governing the care and use of laboratory animals (Approval No. 1931091) . In this study, we strictly adhered to the university's relevant guidelines and regulations. All procedures were carried in accordance with the ARRIVE guidelines.

In the present study, blood films were prepared immediately after withdrawal of blood samples from the infected chickens. This results in the proper shape of an erythrocyte's nuclei. The parasite infection was confirmed by three well-trained investigators using microscopic diagnosis. In addition, all P. gallina- www.nature.com/scientificreports/ ceum positive slides used in this study were also confirmed by PCR and sequencing as described in Xuan et al. 40 . Ten P. gallinaceum-infected chicken blood films were randomly chosen. A total of 432 images of chicken blood cells at various stages of malarial growth were taken at 1000 × using an oil immersion magnification mounted to a light microscope (Olympus CX31, Tokyo, Japan).The digitized chicken blood cells were captured using an Olympus DP21-SAL digital camera (Tokyo, Japan). An individual RBC image from each blood film was selected to be used in an image with a region of interest to the so-called monolayer area.

A hybrid two-stage model: RBC detection and classification models. The proposed methodology for classifying the blood stages of avian malaria, P. gallinaceum, uses a hybrid two-stage model (object identification YOLOv3 and Darknet/Densenet201 classification algorithms). Previously, the combination of two CNN models was reported to have increased prediction accuracy 41, 42 . It was more beneficial when a combination of two deep learning models such as using two-state learning strategies of concatenated YOLO models for identifying genus, species and gender of the mosquito vector 42 . In this work, the first stage of the proposed model is the YOLOv3-based object detection, which aims to distinguish a single RBC image from those inside a microscopic image. Among other object detection models, the YOLOv3 model outperformed the others, in terms of localization and classification accuracy 14 . Model inference, in particular, can be accomplished in real-time by processing 45-155 frames per second. It can also recognize up to 9000 object classes. After encoding the circumstantial information of the relative classes, the model can detect the entire area of an input image set with only a few false positives 43 and shows a balance of accuracy and speed in practical applications that time is not restrictive 44 .

Cropped images (single RBC) from its first stage model inference were used as inputs for the second stage model. The classification model was used in the second stage to categorize the single RBC detected. CNN model candidates were widely used for studying image classification, from a top-5 pre-trained model which was measured as single-crop validation accuracy at 94.7% 45 . The models employed in this study are the top ranking of the ILSVRC2016 ImageNet competition to identify pathological tasks. These versions are in the following sizes; Densenet201, Darknet, Darknet19, and Darknet19-448 45 . The model prediction was automatically masked and colored with a JET color map generated by the Class Activation Map (CAM) algorithm 46 . Dataset preparation. Two datasets were developed by a team of experts who labeled all 432 microscopic examination images for RBCs 1,39 . The first dataset is obtained from microscopic images of chicken RBCs scattered around the whole image. A rectangular bounding box closely fitting to the RBC image region was manually designated for each individual RBC. The ground truth data file was saved as a collection of RBCs with designated bounding boxes. The first dataset was then set as the ground truth file for the object detection model (YOLOv3). The ground truth was deposited as shown in this link as followed; https:// git. cira-lab. com/ cira-medic al/ class ifica tion-of-avian-malar ia-paras ite. Using the first dataset, the YOLOv3 model was subsequently employed to learn and detect individual RBC. By feeding all microscopic images to the trained YOLOv3, the model output reported all the image regions that contained any chicken's RBC. Where an RBC was detected, the image region of a single RBC (both normal and parasitized-RBCs) was cropped using the standard image crop technique. Each single RBC image was then used for the preparation of the second dataset (Fig. 1) .

The deep learning classification model was used in the second stage to identify a specific relative class within the cropped image extracted from the captured image containing a single chicken's RBC. The second dataset consists of 12,761 cropped images containing a significant proportion of regions of interest (ROI). Pooled single cropped-cell images were grouped and assigned labels according to four classes based on their physiological and morphological stages, including: (i) normal RBC for 6724, (ii) trophozoite for 5343, (iii) schizont for 657, and (iv) gametocyte for 37, respectively. Each class above was randomly divided into training (90 per cent) and testing (10 per cent) sets, minimizing potential similarities between these two sets. This protocol can be trustable for preventing sample selection bias from the same patient's slides. While disproportionate sample size between classes has been identified and can trigger biased-prognosis against a class with a large number of image sets, a deep learning approach with multi-layered layers and data annotation can be explored prior to model training. To speed up model convergence, rescale the image from raw pixels to 448 × 448 pixels before training with a chosen neural network model. Data augmentation and model training. Data augmentation techniques were introduced to the dataset prior to training to avoid over-fitting of the training image collection, and it was also used in the case of an unbalanced class. The technique involves rotations, brightness/contrast, blurriness and Gaussian noise under the following conditions: www.nature.com/scientificreports/ graphic processor unit. Furthermore, the qualified models were trained for at least 100,000 epochs in order to record the learned parameters. The likelihood of a threshold greater than and equal to 50% is considered to be a true positive value, which incurs no cost 42, 44 . Otherwise, the result of image classification would produce false positive values that are unexpected in medical diagnosis.

The performance quality of the model-and class-wise prediction was evaluated in terms of the following statistical metric parameters: accuracy, sensitivity, specificity, precision and misclassification rate 11, 42, 47 .

(1) 

Accompanying those protocols in the method part, we combined YOLOv3 and image classification (Darknet/ Densenet201) models in order to localize, identify and classify a single RBC cell from any microscopic examination image with multiple-RBCs. Their relative RBC's classes were then classified as both normal and that infection varied based on pathological features (Supplementary Fig. S1 ). Besides, the hybrid platform of YOLOv2 with ResNet-50 detector helps improve the average precision of the proposed detector up to 81% compared to a previous single model 41 .

Performance comparison of classification models. In this analysis, the model-wise performance was assessed as to whether the classification model was the best-selected model based on an attention map and used to estimate P. gallinaceum-infected blood phases (Supplementary Fig. S1 ). These models included Darknet, Darknet19, Darknet19-448 and Densenet201 as described above. For classification models for avian malaria parasite phases, performance metrics such as average precision, uncertainty confusion matrix table, and ROC curve were estimated and compared. In addition, we randomly split the single-cell images for training and compared the image sets to determine if the different models generated the same classifier performance. All welltrained models provided us with high average accuracy values of more than 97%, which increased to 99.2% for the Darknet algorithm (Table 1 ). In addition, all other two statistics also support the Darknet network as a superior performance at 99.2 per cent for both sensitivity and specificity, respectively (Tables 2, 3) . Interestingly, www.nature.com/scientificreports/ all models used showed less than 2% error rate, which was particularly impressive considering that the Darknet network gave less than 1% error rate (Table 4 ). On average, the precision of the Darknet also showed outstanding performance at 99.0% more than others (Table 5) . General accuracy, obtained from the confusion matrix table, showed outstanding values for all four models (Table 6 ). In addition, the Darknet model outperformed all other models by more than 97% (Table 6 (1)-(4)). Overall, the general accuracy from the confusion matrix gave greater than 95 per cent, except for the Dark-net19-448 for classifying the normal RBC class and followed by the DenseNet201 for classifying the trophozoite class gave us at 91.21 and 94.01 per cent, respectively (Table 6 (3), (4)). Specifically, this manifested in an overall AUC ranking of 0.986-1.000 (Fig. 2) . Hence, the Darknet models can reproduce superior performance in the classification of the development phases of P. gallinaceum in every other model architecture. Table 6 . Comparison of the studied model performance using the confusion matrix table. (Tables 1, 2, 3) . This is because our best-chosen model can distinguish malaria phases with high precision but a low rate of misclassification of less than 1% found (Tables 4, 5 ). This may be one of the benefits of applying the class-balancing data augmentation protocol to the prepared dataset. This study's findings indicate that the model could be validated using multiple blood smears derived from realworld contexts.

In this study, the robustness of deep neural network models led to the discovery of a new approach for more rapid screening of avian malaria under a microscope. Asymptomatic diseases, in particular, can lead to disease transmission and even death if not adequately prevented 48, 49 . This study may contribute to the main comparison of several image classification models based on images of avian malaria, P. gallinaceum. Also, both qualitative and quantitative data were used to evaluate the performance of the proposed models. Several CNNs have also been developed to provide a number of pre-trained ImageNet classification models, including AlexNet, Darknet, VGG-16, Resnet 50 , ResNext, and Densenet201 51 , for use as effective in image classification applications. According to an evaluation of the performance of these pre-trained models, the Darknet model has a higher accuracy than the Densenet201 model and also provides the fastest processing time in both CPU and GPU 45, 52 . Even though we trained network models with actual patient-level yet unbalanced class sizes, www.nature.com/scientificreports/ the performance of well-trained proposed models has shown an outstanding outcome based on several statistical parameters, even in a multi-class comparison. Since the project was a significant success, this would help to advance the development of innovative technologies in order to implement and validate them in a real-world environment.

We would like to illustrate our approach to developing deep learning models from concatenated neural network architectures to make clinical applications by viewing the video as follows: https:// youtu. be/--pzvwS rUdc. Nonetheless, before the technology is implemented, the inter-and intra-variability examinations should be performed in order to validate the degree of consensus between the predictive model's functional performance metrics and the human expert 53 .

The limitation of the study is deeply based on the determination of three key points for the preparation of the dataset 54 , including; (i) differences in the parasite-blood stages 24 , (ii) infection status that can induce either single or multiple infections, and (iii) co-infections in any single blood cell. It is worth noting that in this study, only fully grown gametocyte were counted as gametocytes because young gametocytes are transformed from late trophozoites. The imbalance data is another consequence of dataset preparation, though it is less severe than the others listed above. In the case of differences in the blood stage of each parasite, an increasing number of samples is a potential solution. The analysis has influenced model efficiency under the research design. This may be because the image set is a special attribute between classes that helps to increase class differentiation for the well-trained models. We have different sizes of test set including 610, 484, 60 and 5 images for normal RBC-, trophozoite-, schizont-, and gametocyte classes, respectively. Although our datasets used are imbalanced, in the study, the overall accuracy shows more than 98-100% in classifying between the independently different classes. Since our models were trained utilizing the well-planned dataset preparation described in the method section, the role of data augmentation and data balancing for environmental simulation improved the classification efficiency 47 . Additionally, five-fold cross validation was also studied to help confirm whether our selected model gave consistently general accuracy and no predictive bias 54, 55 . According to our results, cross-validation solves the problem of overfitting. This is because the cross-validation applied can help minimize the cross-validated error to build the optimal model and result in indifferent statistical parameters used between any experiment (Supplementary Tables S1, S2). Furthermore, we appropriately chose an image of a single species of parasite infection to be examined. Since the staining of artifacts/impurities of uninfected blood cells can interfere with the prediction, the expert's simple staining of color under the Giemsa staining process also provides us with the ideal dataset 56 . It is important to note that all images to be included in this application should be prepared properly, otherwise the abnormal host cells such as segmented erythrocyte nuclei or other debris might lead to false identification. Confirmation by a technician is still necessary. Nevertheless, the model must be validated in a specific environment of multiple pathogens and co-infections, which could already exist in field samples.

In this study, the application of a deep neural learning network to identify and verify avian malaria and to assess it by using truly invisible images that have never been published. Based on our results, we demonstrated the superior performance of a novel hybrid two-stage model (object detection YOLOv3 and Darknet/Densenet201 classification algorithms). Although we trained network models with real patient-level yet unbalanced class sizes, the performance of well-trained proposed models was exceptional based on many statistical parameters, including in a multi-class comparison. Furthermore, we can deduce that only a model learning methodology and data preparation should be used in the research design. Fortunately, the study showed a high value of the various statistical measurements listed above, which could be useful for cytological technicians in making decisions. It would be useful to diagnose malaria infections wherever there is due to a shortage of molecular biology and veterinary knowledge.

The data that support the findings of this study are available from the corresponding author's GitHub repository: URL: https:// git. cira-lab. com/ cira-medic al/ class ifica tion-of-avian-malar ia-paras ite. www.nature.com/scientificreports/

Molecular detection of the avian malaria parasite Plasmodium gallinaceum in Thailand

The development of Plasmodium gallinaceum infections in chickens following single infections with three different dose levels

A review of malaria diagnostic tools: Microscopy and rapid diagnostic test (RDT)

Avian malaria: Clinical and chemical pathology of Plasmodium gallinaceum in the domesticated fowl Gallus gallus

Evaluation of three different DNA extraction methods from blood samples collected in dried filter paper in Plasmodium subpatent infections from the Amazon region in Brazil

Orthologous gene sequences of merozoite surface protein 1 (MSP1) from Plasmodium reichenowi and P. gallinaceum confirm an ancient divergence of P. falciparum alleles

Mitochondrial DNA targets increase sensitivity of malaria detection using loop-mediated isothermal amplification

On deep learning for medical image analysis

A deep learning-based automatic mosquito sensing and control system for urban mosquito habitats. Sensors (Basel)

A robust algorithm for tomato detection based on YOLOv3. Sensors (Basel)

Pre-trained convolutional neural networks as feature extractors toward improved malaria parasite detection in thin blood smear images

Deep learning in medical image analysis

Skin lesion segmentation in dermoscopic images with combination of YOLO and GrabCut algorithm. Diagnostics (Basel)

Deep learning approach to peripheral leukocyte recognition

Cardiac VFM visualization and analysis based on YOLO deep learning model and modified 2D continuity equation

A review of machine learning applications in veterinary field

Validation of digital microscopy compared with light microscopy for the diagnosis of canine cutaneous tumors

Review of medical decision support and machine-learning methods

Automatic recognition of fish diseases in fish farms

Deep learning-based quantification of pulmonary hemosiderophages in cytology slides

Deep learning algorithms out-perform veterinary pathologists in detecting the mitotically most active tumor region

Using Convolutional Neural Networks for Determining Reticulocyte Percentage in Cats

An automatic device for detection and classification of malaria parasite species in thick blood film

Detection of plasmodium falciparum in peripheral blood smear images

Identification of plasmodium falciparum development phase in malaria infected red blood cells using adaptive color segmentation and decision tree based classification

Automated detection of plasmodium falciparum from Giemsa-stained thin blood films

Automatic detection of Malaria infected RBCs from a focus stack of bright field microscope slide images

CNN-based image analysis for malaria diagnosis

A fast and accurate detection of Schizont plasmodium falciparum using channel color space segmentation method

Malaria parasite detection from peripheral blood smear images using deep belief networks

Applying faster R-CNN for object detection on malaria images

Computer based diagnosis of malaria in thin blood smears using thresholding based approach

Deep learning based classification of malaria from slide images

Malaria parasite detection using deep learning: (Beneficial to humankind)

Deep learning based automatic malaria parasite detection from blood smear and its smartphone based application. Diagnostics (Basel) 10, 329

Evaluations of deep convolutional neural networks for automatic identification of malaria infected cells

Keys to the avian malaria parasites

Deep learning approach to detect malaria from microscopic images

Vector competence of Aedes albopictus (Skuse) and Aedes aegypti (Linnaeus) for Plasmodium gallinaceum infection and transmission

Development and application of a novel multiplex PCR assay for the differentiation of four haemosporidian parasites in the chicken Gallus gallus domesticus

Fighting against COVID-19: A novel deep learning model based on YOLO-v2 with ResNet-50 for medical face mask detection

Deep learning approaches for challenging species and gender identification of mosquito vectors

Deep learning for generic object detection: A survey

An evaluation of deep learning methods for small object detection

Open Source Neural Networks in C

Learning deep features for discriminative localization

Human-level recognition of blast cells in acute myeloid leukaemia with convolutional neural networks

Evidence of asymptomatic submicroscopic malaria in low transmission areas in Belaga district

High levels of imported asymptomatic malaria but limited local transmission in KwaZulu-Natal, a South African malaria-endemic province nearing malaria elimination

Deep residual learning for image recognition

Densely connected convolutional networks

An incremental improvement

Automated microscopy for routine malaria diagnosis: A field comparison on Giemsa-stained blood films in Peru

Performance evaluation of deep neural ensembles toward malaria parasite detection in thin-blood smear images

Cross-validation

Deep learning based automatic malaria parasite detection from blood smear and its smartphone based application. Diagnostics (Basel)

We are grateful to Thailand Science and Research Innovation Fund and King Mongkut's Institute of Technology Ladkrabang Fund [RE-KRIS-016-64] for providing the financial support for this research project. We also thank the College of Advanced Manufacturing Innovation, King Mongkut's Institute of Technology Ladkrabang which provided the deep learning platform and software to support the research project.

S.B., V.K. and S.C. conceived and designed the research study; S.B., V.K., M.K., K.M.N. and S.C. wrote the manuscript; V.K., K.M.N., R.J., K.K. and S.C. performed the computational experiment; M.K. and S.T. performed animal inoculation and the parasite investigation; T.T., S.C. and S.B. conducted the computer's platform. S.B. read and approved the final manuscript.

The authors declare no competing interests.

The online version contains supplementary material available at https:// doi. org/ 10. 1038/ s41598-021-96475-5.Correspondence and requests for materials should be addressed to S.B.Reprints and permissions information is available at www.nature.com/reprints.Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.