Abstract: This work presents a method of information fusion involving data captured by both a standard charge-coupled device(CCD) camera and a time-of-flight (ToF) camera to be used in the detection of the proximity between a manipulatorrobot and a human. Both cameras are assumed to be located above the work area of an industrial robot. The fusion ofcolour images and time-of-flight information makes it possible to know the 3D localization of objects with respect to aworld coordinate system. At the same time, this allows to know their colour information. Considering that ToFinformation given by the range camera contains innacuracies including distance error, border error, and pixelsaturation, some corrections over the ToF information are proposed and developed to improve the results. Theproposed fusion method uses the calibration parameters of both cameras to reproject 3D ToF points, expressed in acommon coordinate system for both cameras and a robot arm, in 2D colour images. In addition to this, using the 3Dinformation, the motion detection in a robot industrial environment is achieved, and the fusion of information isapplied to the foreground objects previously detected. This combination of information results in a matrix that linkscolour and 3D information, giving the possibility of characterising the object by its colour in addition to its 3Dlocalisation. Further development of these methods will make it possible to identify objects and their position in thereal world and to use this information to prevent possible collisions between the robot and such objects.

Authorship

Download reference