Image processing of weld pool and keyhole in Nd:YAG laser welding of stainless steel based on visual sensing

GAO Jin-qiang¹, QIN Guo-liang¹, YANG Jia-lin², HE Jian-guo², ZHANG Tao¹, WU Chuan-song¹

1. Key Laboratory for Liquid-Solid Structural Evolution and Processing of Materials (Ministry of Education)

Shandong University, Ji’nan 250061, China;

2. Institute of Machinery Manufacturing Technology, China Academy of Engineering Physics, Mianyang 621900, China

Received 22 February 2010; accepted 17 May 2010

Abstract: In order to obtain good welding quality, it is necessary to apply quality control because there are many influencing factors in laser welding process. The key to realize welding quality control is to obtain the quality information. Abundant weld quality information is contained in weld pool and keyhole. Aiming at Nd:YAG laser welding of stainless steel, a coaxial visual sensing system was constructed. The images of weld pool and keyhole were obtained. Based on the gray character of weld pool and keyhole in images, an image processing algorithm was designed. The search start point and search criteria of weld pool and keyhole edge were determined respectively.

Key words: laser welding; keyhole; weld pool; edge; image processing algorithm

1 Introduction

Since laser beam welding has the characteristics of high efficiency, high energy density, low heat input, large ratio of weld penetration to width, less thermal distortion and easy controlling parameters, it is widely used in industries. During the process of laser beam welding, many factors, such as laser-induced plasma, focus position, workpiece distortion and surface condition, influence the absorption rate of laser energy, and then the weld quality may be changed. In order to ensure good weld quality, the process control should be conducted during laser beam welding. The key to realize weld quality control is to obtain information characterizing weld quality in real time by effective sensors.

Many sensors are used to monitor laser beam welding[1-5]. Among them, visual sensor is the most promising sensor. MATSUNAWA et al[6] studied the dynamic behavior of keyhole in laser beam welding by means of paraxial visual sensing technology. They used high speed camera to acquire the weld pool image during laser beam welding process, and at the same time, the side shape of keyhole was detected by means of X-ray transmission aided techniques. The keyhole diameter and depth were obtained by image processing. But no relationship between both keyhole and weld pool geometry and weld penetration was established. CCD cameras were used to capture the weld pool and keyhole images from the topside of workpiece[7], and the relationship between the keyhole depth and the maximum image gray level was established. But it is still a doubt whether the maximum gray level is able to reflect the keyhole depth exactly. BEERSIEK[8] used a coaxial visual system to capture the weld pool and keyhole images, and analyzed qualitatively the relationship between the weld pool width, the keyhole width and the weld penetration. But the quantitative relationship was not established. QIN and LZN[9] extracted the radial dimension of weld pool and keyhole based on the coaxial visual sensing of Nd:YAG laser welding of carbon steel, but the relationship between the radial dimension of keyhole and its depth was still not obtained. Other researchers also used vision system to capture the images of weld pool and/or keyhole for obtaining some information on the keyhole depth and weld penetration[10-15]. However, they just relied on one or two geometry parameters of keyhole or weld pool to describe the weld penetration extent. To establish the correlation of the weld quality with the keyhole and weld pool geometry, it is necessary to employ as more geometric parameters of weld pool and keyhole as possible in laser beam welding. To this end, clear and complete images of weld pool and keyhole must be obtained firstly, and appropriate algorithms of image processing must be proposed. This study makes some efforts in this field. A coaxial visual sensing system is used to capture clear images of weld pool area. And the image processing algorithm is developed to get the information on the pool and keyhole boundary. It lays foundation for keyhole laser welding process control.

2 Coaxial visual sensing system

The sketch of the coaxial visual sensing system is shown in Fig.1. The main components are the computer, image grabber, CCD camera, zoom lens, laser output head with special optical mirrors, narrow-band filter and neutral filter. The camera is an ordinary color CCD camera with the resolution of 456?608 and the sampling rate of 25 frame per second. The center wavelength of the narrow-band filter is in the scope of 600-650 nm. The transmission of the neural filter is 10%. In the optical path, three neural filters and one narrow-band filter are installed.

The semi-permeable and semi-reflective mirror is placed in laser output head with an angle of 45° to the horizontal plane. The Nb:YAG laser with a wavelength of 1.064 nm can get through the mirror with high permeable rate. The other wavelength light ray from weld pool can be reflected to horizontal direction. When the light ray from weld pool is reflected by the mirror to horizontal direction, it passes through the neural and narrow-band filters, and then an image is captured by the camera. The video signal from the camera is digitalized by the image grabber and then is stored in the computer.

Nd:YAG laser beam welding experiments were conducted to capture the images of weld pool and keyhole. The workpiece was 200 mm in length, 50 mm in width and 2 mm in thickness. The workpiece material was 304 stainless steel. The defocusing distance of laser was 0 mm. The shielding gas was Ar gas and its flow rate was 15 L/s. Figure 2 shows a frame of typical image captured by the system.

3 Image processing

Generally speaking, the image processing algorithm includes filtering, edge enhancement, binaryzation, edge extracting and so on.

3.1 Image transformation

The image captured by the system adopts RGB color system, and its three primary colors are red (700 nm), green (546.1 nm) and blue (435.8 nm). In this study, the central wavelength of the narrow-band filter is in the scope of 600-650 nm, and this wave band belongs to red light. In order to increase the speed of image processing, only red primary color is processed. Thus, the color image is transformed to gray image according to red primary color. Figure 3 shows the gray image transformed from a color image. Two coordinate systems Oxy and Oij are setup in Fig.3, and Oij is the discrete coordinate system.

3.2 Image filtering

The noise is introduced in image grabbing and sampling process, so image filtering is needed. Here Gaussian function is used to reduce noise, which is expressed in Eq.(1). It can reserve the edge information well while reducing the noise.

          (1)

where s is Gaussian distribution parameter. In this study, s =1. Eq.(1) should be discretized as 5?5 model defined

Fig.1 Sketch of coaxial visual sensing system

as follows:

Convolving the image and the model, the Gaussian filtering is done. Figure 4 shows the image convolved by the model.

Fig.2 Image captured by system( P=1.8 kW, v=10 mm/s)

Fig.3 Gray image and two coordinate systems

Fig.4 Image convolved by Gaussian model

3.3 Edge enhancing

Generally, edges can be divided into two types, step type and slope type. In Fig.4, the gray value can be extracted along the A, B and C lines, and the variations of gray are shown in Figs.5, 6 and 7. The figures indicate that the weld pool edge and keyhole edge are slope type edges. In order to precisely extract the edges of weld pool and keyhole, the image must be enhanced. So the enhancing equation is designed as follows:

EnhanceGray(i, j)=k+FilterGray(i+3, j)+FilterGray(i+2, j)+FilterGray(i+1, j)-FilterGray(i-1, j)-FilterGray(i-2, j)-FilterGray(i-3, j)                           (2)

where FilterGray(i, j) is the gray level of point (i, j) after filtering by Gaussion function.

Fig.5 Gray distribution by filtering and by enhancing along  line A

Fig.6 Gray distribution by filtering and by enhancing along  line B

The function of constant k is to adjust the value into the gray level scope, so the enhancing result can be displayed in computer. Figures 5-7 show the filtering result and enhancing results of lines A, B and C, respectively. The edge research can be changed to extreme point research of curve. The left and right edges of weld pool and keyhole corrspond to the minimum point and maxmimum point, respectively. Also the upper edge and lower edge of keyhole corrspond to the minimum point and maxmimum point, respectively. Because of block of plasma, the upper edge of weld pool isn’t distinguished. Here the weld pool edge is estimated according to the poistion of maxmimum point.

Fig.7 Gray distribution by filtering and by enhancing along  line C

3.4 Weld pool edge extracting

Edge tracking is adopted in weld edge extracting. The accuracy of edge tracking is mainly determined by two factors. One is the start point selecting. The other is the criteria of edge tracking. So it is very important to determine the start point and criteria of edge tracking.

Many weld pool images are analyzed. It is found that the weld pool edge tracking can be done in two steps: weld pool rear edge tracking and weld pool front edge tracking.

3.4.1 Start point selecting

Because there are many points which have the similar character of edge point, which will affect the accuracy selection of edge point. From Fig.4, it can be seen that the weld pool edge approximates to a line in the middle of weld pool below line B. Making full use of this feature, the start point can be determined.

Select 21 lines in the middle of weld pool. Among these lines, the center line is denoted by j. The left edge point and right edge point are searched in each line according to the character of weld pool edge mentioned above. And the average value of the abscissa of all left edge point is regard as the abscissa of left edge of line j. Also the average value of the abscissa of all right edge point is regarded as the abscissa of right edge of line j. These two points are denoted as i_ls and i_rs, and are taken as start points.

3.4.2 Criteria of rear edge tracking

As metioned in section 3.3, the edge research is changed to extreme point research of curve.

1) Left edge must be the minimum extreme point, and right edge must be the maximum extreme point.

2) If there are more than one minimum extreme point, compare the most minimum extreme point and the second minimum extreme point. If the difference is less than 5, then take the one that is closer to upper line edge point as candidate point of left edge point in this line. In other situation, take the most minimum extreme point as candidate point.

3) If the difference between the abscissa of candidate point of left edge in this line and that in upper line is less than 3, the candidate point is taken as the left edge point, otherwise, add 1 to the abscissa of left edge point in upper line and take this value as abscissa of left edge point in this line.

4) If there are more than one maximum extreme point, compare the most maximum extreme point and the second maximum extreme point. If the difference is less than 5, then take the one that is closer to upper line edge point as candidate point of right edge point in this line. In other situation, take the most maximum extreme point as candidate point of the right edge in this line.

5) If the difference between the abscissa of candidate point of right edge in this line and that in upper line is less than 3, the candidate point is taken as right edge point, otherwise, subtract 1 from the abscissa of right edge point in upper line and take this value as abscissa of right edge point in this line.

3.4.3 Criteria of front edge tracking

1) Front edge must be the maximum extreme point.

2) If there are more than one maximum extreme point, compare the most maximum extreme point and the second maximum extreme point. If the difference is less than 5, then take the one that is closer to left column edge point as candidate point of rear edge point in this column. In other situation, take the most maximum extreme point as candidate point of the rear edge in this column.

In order to reduce the computation time, the search scope is limited. Here the vertical coordinate of the front edge determined in left column is denoted as j_r. The search scope is [j_r-30, j_r+30]. Considering the continuation of edge, mean filter is done. Figure 8 shows the edge searched by the algorithm.

3.5 Keyhole edge extracting

The keyhole tracking can be done in three steps: middle edge tracking, front edge tracking and rear edge tracking.

3.5.1 Start point selecting

The position of keyhole has close relationship with laser spot. The center of the keyhole varies little with the welding parameters. So, in one image, the keyhole center is found by hand and denoted as (i_c，j_c). This point is taken as keyhole center in all images. Here i_c=381 and j_c=115. The line number [95, 135] is defined as middle

Fig.8 Weld pool edge searched by image process algorithm

part of keyhole.

3.5.2 Criteria of middle edge tracking

1) The left edge must be the minimum extreme point, and the right edge must be the maximum extreme point.

2) The most minimum extreme point is taken as the left edge point. The most maximum extreme point is taken as right edge point.

3.5.3 Criteria of front and rear edge tracking

1) The front edge must be the minimum extreme point, and the rear edge must be the maximum extreme point.

2) The most minimum extreme point is taken as the front edge point. The most maximum extreme point is taken as the rear edge point.

The search strategy is searched line by line or column by column. The middle edge searching is conducted line by line and the rear or front edge searching is conducted column by column.

Because the size of keyhole has closer relationship with laser spot, the edge search scope can be determined according to center point. According to the maximum size estimated, the middle edge searching scope is defined as [311, 451]. Thus, the computation time is reduced and the accuracy is increased also.

After finishing the middle edge searching, the column that the left edge point in the 95th line locates is taken as the start column of front edge searching and the right edge point as the end column. The column that the left edge point in the 135th line locates is taken as the start column of rear edge searching and the right edge point as the end column.

Figure 9 shows the keyhole edge searched. Figure 10 shows the keyhole and weld pool edge searched under different welding parameters. It can be seen that the image process algorithm mentioned above can extract the edge well.

Fig.9 Keyhole edge searched by image process algorithm

Fig.10 Weld pool-keyhole edge searched by image process algorithm under different welding conditions: (a) P=1.8 kW, v=10 mm/s; (b) P=2 kW, v=13.3 mm/s; (c) P=1.8 kW, v=13.3 mm/s; (d) P=1.6 kW, v=13.3 mm/s

4 Conclusions

1) An coaxial visual sensing system is constructed, and the clear image of weld pool and keyhole is captured.

2) Based on the analysis of the image character of weld pool and keyhole, Gaussian filter algorithm, edge enhancing algorithm and edge tracking algorithm are designed. The start point and searching criteria of weld pool and keyhole edges are determind, and the weld pool and keyhole edges are well extracted.

References

[1] WANG C M, HU L J, HU X Y. The relationship between plasma optic signal and penetration depth during partial-penetration laser welding [J]. Laser Technology, 2006, 30(1): 23-26. (in Chinese)

[2] ZHANG X D, CHEN W Z, LIU C, GUO J. Coaxial monitoring and penetration control in CO₂ laser welding (3): Relationship between optical signal and penetration statuses for oblique-plate welding [J]. Transactions of the China Welding Institution, 2006, 27(1): 13-16. (in Chinese)

[3] DUAN A Q, CHEN L, WANG Y J, HU L J. Dynamic behavior of plasma in CO₂ laser welding of stainless steel [J]. Transactions of the China Welding Institution, 2005, 26(11): 17-20. (in Chinese)

[4] LI G H, CAI Y, WU Y X. Stability information in plasma image of high-power CO₂ laser welding [J]. Optics and Lasers in Engineering, 2009, 47(9): 990-994.

[5] QIN G L, LIN S Y. Characteristics of keyhole shape in Nd: YAG laser deep penetration welding[J]. Transaction of the China Welding Institution, 2006, 27(1): 81-84. (in Chinese)

[6] MATSUNAWA A, SETO N, KIN J D, MIZUTANI M, KATAYAMA S. Dynamics of keyhole and molten pool in high power CO₂ laser welding [C]//ROBERT A. Proceedings of SPIE-The International Society for Optical Engineering. USA: Society photo-Optical Instrumentation Engineers, 2000, 3888: 34-45.

[7] ABELS P, KAIERLE S, KRATZSCH C. Universial coaxial process control system for laser material processing[C]//BEYER P. ICALEO’99, Section E. Orlando, Fla: LIA, 1999: 99-108.

[8] BEERSIEK J. A CMOS camera as a tool for process analysis not only for laser beam welding [C]//BEYER P. ICALEO’ 2001, Section F. Orlando, Fla: LIA, 2001: 206-215.

[9] QIN G L, LIN S Y. Weld penetration monitoring in ND:YAG laser deep penetration welding based on coaxial visual sensing technology [J]. Chinese Journal of Mechanical Engineering, 2006, 42(8): 229-233. (in Chinese)

[10] KAMIMUKI K, INOUE T, YASUDA K. Development of monitoring method for YAG laser welding and its application: Study monitoring Technology for YAG laser welding [J]. Welding International, 2002, 16(3): 212-218.

[11] CHEN W Z, JIA L, ZHANG X D, PU Y K. Coaxial vision sensing system and detection of penetration status in CO₂ laser welding [J]. Applied Laser, 2004, 24(3): 130-134. (in Chinese)

[12] FANG J F, CHEN Y B, WANG W, WU L. Detection of weld dimension in open keyhole laser welding based on a CMOS coaxial visual sensing system [J]. Transactions of the China Welding Institution, 2006, 27(7): 57-60. (in Chinese)

[13] AALDERINK B J, AARTS R G K M, de LANGE D F, MEIJER J. Experimental observation of keyhole shapes in the laser welding of aluminum blanks [J]. Journal of Laser Applications, 2007, 19(4): 245-251.

[14] MATSUMOTO N, KAWAHITO Y, MIZUTANI M, KATAYAMA S. Laser absorption in high-power fiber laser welding of stainless steel and aluminum alloy[C]//BEYER P. ICALEO’08. Orlando, Fla: LIA, 2008: 387-391.

[15] KAWAHITO Y, MATSUMOTO N, ABE Y, KATAYAMA S. Laser absorption characteristics in high power fiber laser welding of stainless steel [J]. Quarterly Journal of the Japan Welding Society, 2009, 27(3): 183-188.

基于视觉的不锈钢Nd:YAG激光焊焊接熔池和小孔的图像处理

高进强¹, 秦国梁¹, 杨家林², 何建国², 张 涛¹, 武传松¹

1. 山东大学 材料液固结构演变与加工教育部重点实验室, 济南 250061;

2. 中国工程物理研究院 机械制造工艺研究所, 绵阳 621900

摘  要：在激光焊接过程中存在众多干扰因素，为保证焊接质量，需要对激光焊进行质量控制。实现焊接质量控制的关键是表征焊接质量信息的获取，而焊接熔池和小孔包含丰富的焊接质量信息。针对不锈钢Nd:YAG激光焊，构建了同轴视觉检测系统，获得了熔池和小孔图像。根据熔池和小孔图像的特点，设计了图像处理算法，并分别针对熔池和小孔确定了搜索起始点和搜索准则。

关键词：激光焊；小孔；熔池；边缘；图像处理算法

(Edited by YUAN Sai-qian)

Foundation item: Project (10776020) supported by the Joint Foundation of the National Natural Science Foundation of China and China Academy of Engineering Physics

Corresponding author: GAO Jin-qiang; Tel: +86-53l-88395987; E-mail: jqg@sdu.edu.cn

DOI: 10.1016/S1003-6326(11)60731-0