key: cord-0290893-cuv50ulp
authors: Li, Xiang; Siahpour, Shahin; Lee, Jay; Wang, Yachao; Shi, Jing
title: Deep Learning-Based Intelligent Process Monitoring of Directed Energy Deposition in Additive Manufacturing with Thermal Images
date: 2020-12-31
journal: Procedia Manufacturing
DOI: 10.1016/j.promfg.2020.05.093
sha: 3fde26fd88149394bfa77e29171eecb0754cf5f5
doc_id: 290893
cord_uid: cuv50ulp

Abstract Additive manufacturing (AM) techniques have been successfully developed in the past years with the great potential of overcoming the existing obstacles in traditional manufacturing. In order to improve the quality of the manufactured parts and reduce costs, it is important to timely and accurately monitor the AM process during manufacturing. However, it remains a challenging task due to the high complexity of the AM process and the difficulty in processing the condition monitoring data. This paper proposes a deep learning-based process monitoring method for directed energy deposition in AM. The thermal images collected during manufacturing are used to identify the process condition, and a deep convolutional neural network model is proposed to build an end-to-end condition monitoring framework. Experiments on a real directed energy deposition dataset in AM are carried out for validation. The results suggest the proposed method offers a promising approach in process monitoring based on the industrial images. Furthermore, little prior knowledge on signal processing and AM is required, that largely facilitates the potential applications in the real industrial scenarios.

In the recent years, additive manufacturing technologies have been successfully developed and gaining increasing attention from both academia and industries. A large number of application scenarios can be significantly benefited from the techniques [19, 17, 5] , such as robotics, electronics, automotive, aerospace etc. Additive manufacturing is also known as 3D printing or rapid prototyping, which is a process of manufacturing parts layer-by-layer based on the CAD models. Generally, the objects with complicated structures can be manufactured, which can not be accomplished using the traditional approaches [3] . The current restrictions on the geometrical design can be largely reduced and the cost can be also minimized. Therefore, the AM has great potential to overcome the existing obstacles in traditional manufacturing field.

At present, five categories are generally included in the AM methods, i.e. directed energy deposition (DED), powder bed fusion (PBF), sheet lamination process, material jetting and

In the recent years, additive manufacturing technologies have been successfully developed and gaining increasing attention from both academia and industries. A large number of application scenarios can be significantly benefited from the techniques [19, 17, 5] , such as robotics, electronics, automotive, aerospace etc. Additive manufacturing is also known as 3D printing or rapid prototyping, which is a process of manufacturing parts layer-by-layer based on the CAD models. Generally, the objects with complicated structures can be manufactured, which can not be accomplished using the traditional approaches [3] . The current restrictions on the geometrical design can be largely reduced and the cost can be also minimized. Therefore, the AM has great potential to overcome the existing obstacles in traditional manufacturing field.

At present, five categories are generally included in the AM methods, i.e. directed energy deposition (DED), powder bed fusion (PBF), sheet lamination process, material jetting and * Corresponding author. Tel.: +1-513-394-9787.

E-mail address: li5xi@ucmail.uc.edu (Xiang Li). binder jetting. Among the approaches, the powder bed fusion has been popularly developed since the products can be manufactured with high density and resolution [15, 1, 23, 9] . In the PBF process, the laser beam or electron beam is used as the energy source to melt the concerned area of the powder bed. In this way, the metallic bonding between different layers can be achieved. The directed energy deposition (DED) has been popularly adopted in AM, which generally achieves high accuracy in manufacturing and good quality in both mechanical and geometrical properties. In this paper, the DED method in AM is investigated.

While the AM technologies have been widely developed and promising results have been obtained, the mechanical quality of the manufactured parts generally can not be well guaranteed, and it is of great importance to monitor the AM process and inspect the part quality during manufacturing [17] . Basically, the part quality can be largely affected by the material microstructure, and the possible defects such as cracks, porosity, residual stress etc. generally result from some manufacturing factors, including laser scanning speed, hatch distance, laser power etc. Therefore, accurate monitoring of the manufacturing process is important in AM field in order to improve the part quality and reduce cost [9, 8] . However, this task is challenging in the real implementations due to the high complexity of the process and the difficulty in analysing the noisy condition monitoring data.

In the recent years, additive manufacturing technologies have been successfully developed and gaining increasing attention from both academia and industries. A large number of application scenarios can be significantly benefited from the techniques [19, 17, 5] , such as robotics, electronics, automotive, aerospace etc. Additive manufacturing is also known as 3D printing or rapid prototyping, which is a process of manufacturing parts layer-by-layer based on the CAD models. Generally, the objects with complicated structures can be manufactured, which can not be accomplished using the traditional approaches [3] . The current restrictions on the geometrical design can be largely reduced and the cost can be also minimized. Therefore, the AM has great potential to overcome the existing obstacles in traditional manufacturing field.

At present, five categories are generally included in the AM methods, i.e. directed energy deposition (DED), powder bed fusion (PBF), sheet lamination process, material jetting and * Corresponding author. Tel.: +1-513-394-9787.

E-mail address: li5xi@ucmail.uc.edu (Xiang Li). binder jetting. Among the approaches, the powder bed fusion has been popularly developed since the products can be manufactured with high density and resolution [15, 1, 23, 9] . In the PBF process, the laser beam or electron beam is used as the energy source to melt the concerned area of the powder bed. In this way, the metallic bonding between different layers can be achieved. The directed energy deposition (DED) has been popularly adopted in AM, which generally achieves high accuracy in manufacturing and good quality in both mechanical and geometrical properties. In this paper, the DED method in AM is investigated.

While the AM technologies have been widely developed and promising results have been obtained, the mechanical quality of the manufactured parts generally can not be well guaranteed, and it is of great importance to monitor the AM process and inspect the part quality during manufacturing [17] . Basically, the part quality can be largely affected by the material microstructure, and the possible defects such as cracks, porosity, residual stress etc. generally result from some manufacturing factors, including laser scanning speed, hatch distance, laser power etc. Therefore, accurate monitoring of the manufacturing process is important in AM field in order to improve the part quality and reduce cost [9, 8] . However, this task is challenging in the real implementations due to the high complexity of the process and the difficulty in analysing the noisy condition monitoring data.

In the recent years, additive manufacturing technologies have been successfully developed and gaining increasing attention from both academia and industries. A large number of application scenarios can be significantly benefited from the techniques [19, 17, 5] , such as robotics, electronics, automotive, aerospace etc. Additive manufacturing is also known as 3D printing or rapid prototyping, which is a process of manufacturing parts layer-by-layer based on the CAD models. Generally, the objects with complicated structures can be manufactured, which can not be accomplished using the traditional approaches [3] . The current restrictions on the geometrical design can be largely reduced and the cost can be also minimized. Therefore, the AM has great potential to overcome the existing obstacles in traditional manufacturing field.

At present, five categories are generally included in the AM methods, i.e. directed energy deposition (DED), powder bed fusion (PBF), sheet lamination process, material jetting and * Corresponding author. Tel.: +1-513-394-9787.

E-mail address: li5xi@ucmail.uc.edu (Xiang Li). binder jetting. Among the approaches, the powder bed fusion has been popularly developed since the products can be manufactured with high density and resolution [15, 1, 23, 9] . In the PBF process, the laser beam or electron beam is used as the energy source to melt the concerned area of the powder bed. In this way, the metallic bonding between different layers can be achieved. The directed energy deposition (DED) has been popularly adopted in AM, which generally achieves high accuracy in manufacturing and good quality in both mechanical and geometrical properties. In this paper, the DED method in AM is investigated.

While the AM technologies have been widely developed and promising results have been obtained, the mechanical quality of the manufactured parts generally can not be well guaranteed, and it is of great importance to monitor the AM process and inspect the part quality during manufacturing [17] . Basically, the part quality can be largely affected by the material microstructure, and the possible defects such as cracks, porosity, residual stress etc. generally result from some manufacturing factors, including laser scanning speed, hatch distance, laser power etc. Therefore, accurate monitoring of the manufacturing process is important in AM field in order to improve the part quality and reduce cost [9, 8] . However, this task is challenging in the real implementations due to the high complexity of the process and the difficulty in analysing the noisy condition monitoring data.

In the recent years, additive manufacturing technologies have been successfully developed and gaining increasing attention from both academia and industries. A large number of application scenarios can be significantly benefited from the techniques [19, 17, 5] , such as robotics, electronics, automotive, aerospace etc. Additive manufacturing is also known as 3D printing or rapid prototyping, which is a process of manufacturing parts layer-by-layer based on the CAD models. Generally, the objects with complicated structures can be manufactured, which can not be accomplished using the traditional approaches [3] . The current restrictions on the geometrical design can be largely reduced and the cost can be also minimized. Therefore, the AM has great potential to overcome the existing obstacles in traditional manufacturing field.

At present, five categories are generally included in the AM methods, i.e. directed energy deposition (DED), powder bed fusion (PBF), sheet lamination process, material jetting and * Corresponding author. Tel.: +1-513-394-9787.

E-mail address: li5xi@ucmail.uc.edu (Xiang Li). binder jetting. Among the approaches, the powder bed fusion has been popularly developed since the products can be manufactured with high density and resolution [15, 1, 23, 9] . In the PBF process, the laser beam or electron beam is used as the energy source to melt the concerned area of the powder bed. In this way, the metallic bonding between different layers can be achieved. The directed energy deposition (DED) has been popularly adopted in AM, which generally achieves high accuracy in manufacturing and good quality in both mechanical and geometrical properties. In this paper, the DED method in AM is investigated.

While the AM technologies have been widely developed and promising results have been obtained, the mechanical quality of the manufactured parts generally can not be well guaranteed, and it is of great importance to monitor the AM process and inspect the part quality during manufacturing [17] . Basically, the part quality can be largely affected by the material microstructure, and the possible defects such as cracks, porosity, residual stress etc. generally result from some manufacturing factors, including laser scanning speed, hatch distance, laser power etc. Therefore, accurate monitoring of the manufacturing process is important in AM field in order to improve the part quality and reduce cost [9, 8] . However, this task is challenging in the real implementations due to the high complexity of the process and the difficulty in analysing the noisy condition monitoring data.

In the recent years, additive manufacturing technologies have been successfully developed and gaining increasing attention from both academia and industries. A large number of application scenarios can be significantly benefited from the techniques [19, 17, 5] , such as robotics, electronics, automotive, aerospace etc. Additive manufacturing is also known as 3D printing or rapid prototyping, which is a process of manufacturing parts layer-by-layer based on the CAD models. Generally, the objects with complicated structures can be manufactured, which can not be accomplished using the traditional approaches [3] . The current restrictions on the geometrical design can be largely reduced and the cost can be also minimized. Therefore, the AM has great potential to overcome the existing obstacles in traditional manufacturing field.

At present, five categories are generally included in the AM methods, i.e. directed energy deposition (DED), powder bed fusion (PBF), sheet lamination process, material jetting and * Corresponding author. Tel.: +1-513-394-9787.

E-mail address: li5xi@ucmail.uc.edu (Xiang Li). binder jetting. Among the approaches, the powder bed fusion has been popularly developed since the products can be manufactured with high density and resolution [15, 1, 23, 9] . In the PBF process, the laser beam or electron beam is used as the energy source to melt the concerned area of the powder bed. In this way, the metallic bonding between different layers can be achieved. The directed energy deposition (DED) has been popularly adopted in AM, which generally achieves high accuracy in manufacturing and good quality in both mechanical and geometrical properties. In this paper, the DED method in AM is investigated.

While the AM technologies have been widely developed and promising results have been obtained, the mechanical quality of the manufactured parts generally can not be well guaranteed, and it is of great importance to monitor the AM process and inspect the part quality during manufacturing [17] . Basically, the part quality can be largely affected by the material microstructure, and the possible defects such as cracks, porosity, residual stress etc. generally result from some manufacturing factors, including laser scanning speed, hatch distance, laser power etc. Therefore, accurate monitoring of the manufacturing process is important in AM field in order to improve the part quality and reduce cost [9, 8] . However, this task is challenging in the real implementations due to the high complexity of the process and the difficulty in analysing the noisy condition monitoring data.

In the recent years, additive manufacturing technologies have been successfully developed and gaining increasing attention from both academia and industries. A large number of application scenarios can be significantly benefited from the techniques [19, 17, 5] , such as robotics, electronics, automotive, aerospace etc. Additive manufacturing is also known as 3D printing or rapid prototyping, which is a process of manufacturing parts layer-by-layer based on the CAD models. Generally, the objects with complicated structures can be manufactured, which can not be accomplished using the traditional approaches [3] . The current restrictions on the geometrical design can be largely reduced and the cost can be also minimized. Therefore, the AM has great potential to overcome the existing obstacles in traditional manufacturing field.

At present, five categories are generally included in the AM methods, i.e. directed energy deposition (DED), powder bed fusion (PBF), sheet lamination process, material jetting and * Corresponding author. Tel.: +1-513-394-9787.

E-mail address: li5xi@ucmail.uc.edu (Xiang Li). binder jetting. Among the approaches, the powder bed fusion has been popularly developed since the products can be manufactured with high density and resolution [15, 1, 23, 9] . In the PBF process, the laser beam or electron beam is used as the energy source to melt the concerned area of the powder bed. In this way, the metallic bonding between different layers can be achieved. The directed energy deposition (DED) has been popularly adopted in AM, which generally achieves high accuracy in manufacturing and good quality in both mechanical and geometrical properties. In this paper, the DED method in AM is investigated.

While the AM technologies have been widely developed and promising results have been obtained, the mechanical quality of the manufactured parts generally can not be well guaranteed, and it is of great importance to monitor the AM process and inspect the part quality during manufacturing [17] . Basically, the part quality can be largely affected by the material microstructure, and the possible defects such as cracks, porosity, residual stress etc. generally result from some manufacturing factors, including laser scanning speed, hatch distance, laser power etc. Therefore, accurate monitoring of the manufacturing process is important in AM field in order to improve the part quality and reduce cost [9, 8] . However, this task is challenging in the real implementations due to the high complexity of the process and the difficulty in analysing the noisy condition monitoring data.

In the recent years, additive manufacturing technologies have been successfully developed and gaining increasing attention from both academia and industries. A large number of application scenarios can be significantly benefited from the techniques [19, 17, 5] , such as robotics, electronics, automotive, aerospace etc. Additive manufacturing is also known as 3D printing or rapid prototyping, which is a process of manufacturing parts layer-by-layer based on the CAD models. Generally, the objects with complicated structures can be manufactured, which can not be accomplished using the traditional approaches [3] . The current restrictions on the geometrical design can be largely reduced and the cost can be also minimized. Therefore, the AM has great potential to overcome the existing obstacles in traditional manufacturing field.

At present, five categories are generally included in the AM methods, i.e. directed energy deposition (DED), powder bed fusion (PBF), sheet lamination process, material jetting and * Corresponding author. Tel.: +1-513-394-9787.

E-mail address: li5xi@ucmail.uc.edu (Xiang Li). binder jetting. Among the approaches, the powder bed fusion has been popularly developed since the products can be manufactured with high density and resolution [15, 1, 23, 9] . In the PBF process, the laser beam or electron beam is used as the energy source to melt the concerned area of the powder bed. In this way, the metallic bonding between different layers can be achieved. The directed energy deposition (DED) has been popularly adopted in AM, which generally achieves high accuracy in manufacturing and good quality in both mechanical and geometrical properties. In this paper, the DED method in AM is investigated.

While the AM technologies have been widely developed and promising results have been obtained, the mechanical quality of the manufactured parts generally can not be well guaranteed, and it is of great importance to monitor the AM process and inspect the part quality during manufacturing [17] . Basically, the part quality can be largely affected by the material microstructure, and the possible defects such as cracks, porosity, residual stress etc. generally result from some manufacturing factors, including laser scanning speed, hatch distance, laser power etc. Therefore, accurate monitoring of the manufacturing process is important in AM field in order to improve the part quality and reduce cost [9, 8] . However, this task is challenging in the real implementations due to the high complexity of the process and the difficulty in analysing the noisy condition monitoring data. In the current literature, most quality inspection studies are focused on the surface detection of the manufactured parts [21, 16] , and the process factor monitoring has received less attention. There are some important factors which affect the quality of final products [20] . Laser power and scan speed are among the most important parameters. If either of the two parameters locates on the outside of the acceptable range, it will deteriorate the quality of finish product. This study aims to bridge this gap and offers an effective solution for process monitoring using the images collected by the infrared camera during manufacturing [18] .

With respect to the condition monitoring images, deep learning is an effective technique for data processing, and has shown great capability in feature extraction [6, 12, 7, 10] . In general, deep learning is also known as deep neural network, which consists of multiple layers for representation learning. The highlevel features can be effectively and automatically learned, which facilitate the following tasks such as classification or regression. Meanwhile, little prior expertise on image processing and AM is required, that makes it easy for applications in the real industries.

In the past years, deep learning has been successfully applied in the industrial condition monitoring problems. For instance, the fault diagnosis problems of rolling bearing and gearboxes have been effectively addressed using the latest deep learning algorithms [22, 13] . Especially, the AM quality inspection problem has also largely benefited from the development of the deep neural network. Zhang et al. [21] proposed a deep convolutional neural network model for metal surface quality inspection in AM. The results show deep learning is well suited for the image processing problem in AM. Ye et al. [20] proposed a deep belief network for quality assessment in AM. The straightforward implementation flow benefits the real application in the industries. Grasso et al. [4] provided a comprehensive survey on monitoring methods in metal PBF. different classifications of defects and their main causes, are investigated. One of the main particular focuses is devoted to development of automated defect detection rules and study of process control strategies. Marrey et al. [14] proposed a framework for optimization a PBF process. The proposed framework characterizes the effect of process parameters. For the optimization of process parameters, they have exploited the Artificial Neural Network (ANN) model. Delli et al. [2] proposed a method in order to automatically evaluate the quality of the product in AM processes. They have utilized a machine learning method of Support Vector Machine to classify the products into two categories. This paper proposes a deep learning-based process monitoring method for directed energy deposition in AM. The thermal images collected during manufacturing are used to identify the process parameters. A deep convolutional neural network model is proposed for feature extraction and process classification. Experiments on a real DED thermal image dataset are carried out for validation. The results suggest the proposed method is able to accurately identify the AM process parameters, which is also easy for real implementations in the real industries.

The remainder of this paper starts with the proposed method in Section 2. The experiments are carried out for validation in Section 3. We close the paper with conclusions in Section 4.

In this study, the process monitoring problem in additive manufacturing is investigated based on the sequential thermal images collected during manufacturing. Specifically Since the process monitoring task is investigated and the parameters such as laser scanning speed are identified, a single image can not be used for pattern recognition. Therefore, the sample in this study is prepared as a series of multiple sequential images. Let x j k denote the k-th sample of the j-th experiment, and

where N L represents the number of sequential images in each sample.

In general, this study aims to develop a classifier f which establishes the relationship between the image sample and the process parameter label through explorations of the training dataset, i.e. y = f (x). The classifier is expected to accurately identify the AM process in the testing scenarios.

In the past years, deep learning has been attracting growing attention and achieved significant success in different applications, such as speech recognition, image processing, fault diagnosis etc. The basic multi-layer perceptron (MLP) architecture has been well developed. However, the convolutional neural network (CNN) structure has been more promising for feature extraction from high-dimensional signals and widely adopted in a number of industrial tasks [11, 22] .

Specifically, the convolutional operation convolves multiple CNN filters on the input data of the network, and the features can be obtained. In this study, since the images are processed, the 2-dimensional convolutional neural network is used. Let x denote the input data image sample. The convolutional operation is defined as a multiply calculation of the filter w, w ∈ R FL1×FL2 and a concatenated data x i:i+FL1−1, j: j+FL2−1 , where x i:i+FL1−1, j: j+FL2−1 denotes the pixel area on the image starting from the (i, j)-th pixel with dimensions of (F L1 , F L2 ). F L1 and F L2 represent the lengths of the convolutional filter. In this way, the convolutional operation can be expressed as,

where b and ϕ denote the bias term and non-linear activation function, respectively. The values of z i, j are the features learned from the CNN layer with respect to the filter. By sliding the filter over the input data, the feature map can be thus obtained, which can be used for further processing. Generally, the pooling operation is adopted after the convolutional layer, which reduces the dimension of the feature map and extract the most significant features. In most cases, the average-pooling and max-pooling operations are preferred, that aim to extract the average and maximum values of the neighboring data respectively. By stacking the convolutional and pooling layers, high-level features can be learned, and the down-stream tasks such as classification can be largely benefited.

In this paper, a deep learning framework is proposed to address the process monitoring problem in AM using the thermal images, and the overview of the proposed method is presented in Figure 1 .

First, the time-series thermal images are assumed to be collected by the cameras, and multiple DED process settings are considered in this study. Let N c denote the number of the classes of the processes. With respect to the sequential images, the samples are then prepared containing multiple consecutive images. The samples are fed into the deep neural network model afterwards for feature extraction and classification.

Specifically, the deep convolutional neural network architecture is adopted in this paper. First, the average-pooling layer is adopted to reduce the data dimension. Two convolutional layers are used next with filter numbers of 32 and 16, and filter size of 3×3. The average-pooling operation is also used between the convolutional layers to further compress the feature maps and extract the significant features. Afterwards, the obtained feature map is flattened and linked to a fully-connected layers with 64 neurons. At last, a fully-connected layer with N c neurons is adopted, where the neuron values represent the confidence of the input sample belonging to each class respectively. A softmax function is employed to interpret the confidence values to probabilities. Throughout the network model, the rectified linear unit (ReLU) activation function is adopted for each layer.

In this way, an end-to-end process monitoring model for DED is built, which takes the raw thermal images as inputs, and directly outputs the process inspection results. Therefore, the proposed method is easy for implementations in the real AM industries.

The proposed deep learning model consists of a large number of parameters, i.e. the weights and biases in each layer. Therefore, optimization of the network is of great importance. In this study, supervised learning paradigm is implemented for network training. Specifically, the model is supposed to be trained to minimize the empirical classification errors on the labeled training data. Corresponding with the softmax function, the popular cross-entropy loss function is adopted as the optimization objective L opt , which is defined as,

where n s denotes the number of the training samples, and x hi i, j represents the j-th element of the network output vector, taking the i-th training sample as the input. Through minimization of L opt , the prediction errors on the training data can be minimized, and the model is expected to achieve good process monitoring performance on the testing data.

In this study, the directed energy deposition process in additive manufacturing is investigated. As one of the pioneer researches in the literature on process monitoring using thermal images, a basic task is focused on, where a straight stick part is manufactured using DED. A co-axial system equipped with a 2 kW fiber laser system is adopted for the experiment for melting and depositing the 0Cr18Ni9 powders. Infrared cameras are employed to capture the temperature information during manufacturing. Figure 2 shows the overview of the experiments in this study.

Based on the expert knowledge of the AM engineers, the optimal scanning speed and laser power can be determined, and they are denoted as v * and p * , respectively. Specifically, the values for v * and p * are 1200 W and 3 mm/s, respectively. In order to evaluate the proposed method, different experiments are carried out where different process parameters are used. The thermal image examples in different processes in this study are presented in Figure 3 . It can be observed that different processes generally have similar patterns, which makes the automatic process monitoring task challenging with respect to the raw image data. Data overlapping is considered in the preparations of the samples. The detailed information of the experiments is presented in Table 1 .

The thermal images in this study are of high resolutions, which cover a wide range of the monitoring space. In order to reduce the noise introduced by the environments and decrease the computing burden, the center areas of the raw images are extracted as the region-of-interest (ROI), that focus on the manufactured parts specifically. As shown in Figure 4 , the image patches of 300 × 300 regions are used for sample preparation in this paper.

Based on the dataset information, different process monitoring tasks can be investigated in this study, and the task information is presented in Table 2 . The four tasks cover a wide range of the process monitoring problems and can well evaluate the proposed method. 3 Fig. 4 . The region-of-interest of the raw thermal images in this study.

In the model training, Xavier normal initializer is adopted for the initializations of the deep neural network model parameters. Back-propagation is employed for the optimization of the parameters. Specifically, the popular Adam optimization algorithm is adopted with mini-batches. Table 3 presents the detailed model parameters and experimental settings in the proposed method. 4 Table 2 . Information of the process monitoring tasks in this study. Task Pattern Training Data Testing Data Task Pattern Training Data Testing Data  T1  N  #2  #1  T2  N  #2  #1  LP  #3, #4  #5  LP  #3, #5  #4  LS  #6, #8  #7  LS  #6, #7  #8  HS  #10  #9  HS  #10  #9  T3  N  #1  #2  T4  N  #1  #2  LP  #3, #4  #5  LP  #3, #5  #4  LS  #6, #7  #8  LS  #6, #8  #7  HS  #9  #10 HS #10 #9 

In this section, the process monitoring performance of the proposed method is evaluated in different tasks. The reported experimental results are generally averaged by 5 runs to reduce the influence of model randomness. In order to show the superiority of the proposed deep convolutional neural network model, a basic deep neural network (DNN) structure is used for comparison. Specifically, only three fully-connected layers are included in the DNN model, where 512, 256 and 64 neurons are adopted respectively. The experimental setting is similar with the proposed method for evaluation. Figure 5 shows the testing accuracies in the four tasks. It can be observed that higher than 80% testing accuracies can be generally obtained in different tasks, which show the effectiveness of the proposed method in process monitoring. The proposed method also noticeably outperforms the DNN approach in different scenarios, that indicates the DCNN structure is more suitable for the thermal image processing problem in additive manufacturing.

The confusion matrices in the process classification are presented in Figures 6 and 7 . It is noted that promising classification performance is achieved in most AM settings, while the normal and high speed conditions have more errors. That is mostly due to the fact that fewer training data are generally included in that class compared with the other types, and more labeled samples are expected to further improve the process monitoring performance.

In the proposed method, the number of the images included in each sample is a key parameter, and Figure 8 shows its effects on the model performance on the task T1. It can be observed the best performance can be achieved with N L = 3. That indicates large N L may lead to model overfitting while the samples with small N L generally do not contain sufficient information for the process monitoring task.

To further validate the proposed method, the learned features at the last fully-connected layer are visualized using the t-SNE method for dimension reduction. The visualization results are presented in Figure 9 . It can be observed that using the proposed method, good clustering effect is obtained for different kinds of the processes. The discriminative features of different processes can be well extracted, and the they are well separated in the sub-space for classification. The training and testing data are also well aligned, that suggested the learned process monitoring knowledge from the training data can be well generalized to the testing data.

In this paper, a deep learning-based process monitoring method is proposed for directed energy deposition in additive manufacturing. The thermal images collected during manufacturing are used for pattern recognition, and a deep convolutional neural network model is proposed for processing the time-series images. Experiments on a real image dataset in DED are carried out for validation. The results indicate the proposed method is able to automatically and accurately identify the process variables such as scanning speed and laser power in AM. The estab-5 lished end-to-end process monitoring framework requires little prior expertise on signal processing and additive manufacturing, which largely facilitates applications in the real industries.

It should be pointed out that as a deep learning-based approach, the main limitation of the proposed method lies in the requirement of sufficient training data for model development, i.e. sufficient thermal images in different process patterns. Further research works will be carried out on building an effective process monitoring model in AM with less labeled training data. Technology 28, 044005.

[5] Hu, Z., Qin, X., Li, Y., Yuan, J., Wu, Q., 2019. Multi-bead overlapping model with varying cross-section profile for robotic GMAWbased additive manufacturing. Journal of Intelligent Manufacturing DOI:

Process monitoring and inspection systems in metal additive manufacturing: Status and applications

Automated process monitoring in 3D printing using supervised machine learning

Review of in-situ process monitoring and in-situ metrology for metal additive manufacturing

Process defects andin situmonitoring methods in metal powder bed fusion: a review. Measurement Science and Fig. 9. Visualizations of the learned features in the fully-connected layer by the proposed method in the task

Adaptive virtual metrology for semiconductor chemical mechanical planarization process using GMDH-type polynomial neural networks

A deviation based assessment methodology for multiple machine health patterns classification and fault detection

Assessment of data suitability for machine prognosis using maximum mean discrepancy

A deep neural network for classification of melt-pool images in metal additive manufacturing

A blockchain enabled cyber-physical system architecture for Industry 4.0 manufacturing systems

Understanding and improving deep learning-based rolling bearing fault diagnosis with attention mechanism

Diagnosing rotating machines with weakly supervised data using deep transfer learning

Deep learning-based machinery fault diagnostics with domain adaptation across sensors at different places

A framework for optimizing process parameters in powder bed fusion (PBF) process using artificial neural network (ANN)

Using machine learning to identify in-situ melt pool signatures indicative of flaw formation in a laser powder bed fusion additive manufacturing process

Acoustic emission for in situ quality monitoring in additive manufacturing using spectral convolutional neural networks

Deep learning for in situ and real-time quality monitoring in additive manufacturing using acoustic emission

Dynamics and vibrations of particle-sensing MEMS considering thermal and electrostatic actuation

A review on process monitoring and control in metal-based additive manufacturing

Defect detection in selective laser melting technology by acoustic signals with deep belief networks

Convolutional neural network-based inspection of metal additive manufacturing parts

Deep residual learning-based fault diagnosis method for rotating machinery

Extraction and evaluation of melt pool, plume and spatter information for powder-bed fusion am process monitoring