APPROACHES TO THE COMPUTER VISION SYSTEM PROPOSAL ON PURPOSES OF OBJECTS RECOGNITION WITHIN THE HUMAN-ROBOT SHARED WORKSPACE COLLABORATION APPROACHES TO THE COMPUTER VISION SYSTEM PROPOSAL ON PURPOSES OF OBJECTS RECOGNITION WITHIN THE HUMAN-ROBOT SHARED WORKSPACE COLLABORATION

]. We also focused on the sensors operating on the basic of these two principles and test them for purposes their utilization in the proposed computer vision system. Method of processing of three-dimensional images is given by the way how we can approach to the recognition of a human and of the human body significant parts in monitored environment. The methods frequently used for the object recognition are: Histogram of Oriented Gradients (HOG), Dominant Orientation Templates (DOT) [18], Viola-Jones method [19], selected methods of skeletonization [20], etc. The functions needed for application of the different methods of the image processing and the object


Introduction
The exploration of possibilities of different ways of a humanrobot safe cooperation within a shared workplace is strongly developed problem of robotics.Human-robot collaboration activities require utilization of the specially designed robot that is able to work within a protected workspace of a robotized workcell [1 -3].
These workplaces are usually equipped with the different types of safety elements, as follows: inductive sensors, optical sensors, laser sensors, pressure sensitive floors and also 3D camera systems [4 -7].Improving of the safety of a robotized workplace can be reached also by the appropriate choice of the additional subsystems on a robot (end effector, force-torque sensor) or peripheral devices that support the required safety features [8 -10].The computer vision technologies are used for both 3D image capturing and recognition of the objects moving within the monitored zone [11 -12].A computer vision is a complex of techniques intended to obtain, process, analyze and understand the complex multi-dimensional data obtained from the environment for scientific and technical research.The different types of camera are used for sensing the object within the monitored zone.A standard 2D camera doesn´t provide information about the depth of sensed technological scene.A stereo camera provides three-dimensional information about sensed scene but it requires a thorough calibration and utilization of more complex functions in the image processing.The features of 3D cameras best meet requirements of our basic research.Most of all of 3D sensing devices is based on one of three basic principles, as are: triangulation, time-of-fly and interferometry.These principles differ in the degree of effect of distance between the captured object and the camera on the measurement accuracy.More detailed information about these three principles is described in [13].The devices operating on the interferometry principle guarantee the high measurement accuracy but this principle is not appropriate for capturing fast moving big objects [9, 14 -15].The possibility of utilization of both triangulation and time-of-fly methods for capturing the human moving within monitored workspace was evaluated in several research papers [16 -17].We also focused on the sensors operating on the basic of these two principles and test them for purposes their utilization in the proposed computer vision system.Method of processing of three-dimensional images is given by the way how we can approach to the recognition of a human and of the human body significant parts in monitored environment.The methods frequently used for the object recognition are: Histogram of Oriented Gradients (HOG), Dominant Orientation Templates (DOT) [18], Viola-Jones method [19], selected methods of skeletonization [20], etc.The functions needed for application of the different methods of the image processing and the object The article deals with the present topic of solution of the tasks of a safe cooperation between human and robot within industrial applications.The attention is aimed at the utilization of state-in-art devices of computer vision and related methods of object recognition within the monitored zone of the laboratory of automated assembly.The article presents some steps of the computer vision system design with focus on the suitable sensors selection and experimental verification of their parameters for demands of monitoring the specified safety zones.The designed computer vision system is one of the elements of the complex safety system that is in the process of designing in the laboratory workplace conditions.
Keywords: Human-robot cooperation.Computer vision system.Microsoft Kinect.Open-source platform ROS.

R E V I E W
in the collaborative space.The sensor used for Safety Level 3 will monitor the presence of objects within its field of view and recognize them.The following key criteria were specified for selection of suitable sensors: • The existence of support from Robot Operating System (ROS) side • Availability of the technology • The effect of the sensor parameters on the system basic qualities, functionality and stability

Selection of sensors
The development of the safety system for solving the safe human-robot collaboration was based on utilization of the Robotic Operating System (ROS).The reason for this decision was its openness on area of development of the new robotic applications through integration of knowledge of a robot controlling, computer vision, collision states solutions, etc. Open-source platform ROS recognition are available in the complete free-access libraries PCL and OpenCV.PCL (Point Cloud Library) library processes the data about the three-dimensional space represented by the point cloud [21].The OpenCV library functions are used totally within all levels of the image processing [22].

Criteria on selection of suitable sensors
The computer vision system is created for the safety system whose functional structure was already proposed in [23], in laboratory conditions of Department of Automation and Production Systems.This safety system consists of four subsystems: sensing, decision making, controlling and executing.The selection of sensors follows the requirements of both decision making and controlling subsystems.The monitored technological scene was divided into three safety zones for three different levels of sensing.In the proposal of the computer vision system we consider to use three individual sensors.Each one of them will sense only one of three safety zones.A level of sensing was defined for each safety zone and identified by name: Safety Level 1, Safety Level 2 and Safety Level 3. The simplified functional scheme of the proposal of the safety system including three safety zones and related Safety Levels is shown in Fig. 1.The sensor used for Safety Level 1 will capture a position of an object, i. e. a human, when he is present in the specified collaborative space.The human body is converted to the skeleton with help of methods of an image processing.The skeleton consists of highlighted points representing the links between the main structural parts of the human body.Each highlighted point contains information about position and orientation of the related structural part of the human body in 3D space.The sensor used for Safety Level 2 will monitor the work surface of the complex settings and calibration of the system utilization of more methods of an image processing high demands on the hardware Motion capture system + high accuracy of a data acquisition + high frame rate -complex installation and setting of the system necessity of wearing clothing equipped with reflection markers or light emitting markers high purchase costs building the safety system proposed for testing in the laboratory conditions.Pagliari, D. and Pinto, L. dealt with comparison of accuracy of both versions of the sensing devices Kinect.The results of this research [29] confirmed that the sensing device Kinect One achieves qualitatively better results in comparison with its previous version.
The frame rate is a number of frames that the system is able to receive per time unit.Both tested sensing devices are able to achieve a frame rate equal to 30Hz.An achievement of this value is directly dependent on used computing hardware configuration, an instantaneous performance of the system and the way of the received data presentation.The proposal of the safety system considers three safety zones.Each one is monitored by separate sensing device and data received from single sensing devices are processed by different way.From this reason, the resulting frame rate can vary.

Verification of the devices parameters
Parameters of selected devices were tested and compared with a utilization computer, with parameters shown in Table 3. Experimental testing of the frame rate when all three sensing devices operate at the same time was beyond the performance possibilities of the used computation hardware and, therefore, the related results are missing in the charts.
Computing hardware configuration and used software The ROS 3D visualizer Rviz was used for visualization of data from the devices.Rviz enables to create a visual representation offers the database of partial packages that accelerate and simplify a design and creation of new custom robotic applications.It is also important to mention the open-source project ROS-Industrial that covers activities related to extending advanced capabilities of ROS for industrial applications, including area of industrial robotics [24 -25].
The first criterion of a suitable sensor selection is a support of ROS on selected sensor.This criterion eliminates a timeconsuming process of creating communication channels between the selected sensor and ROS.ROS offers the drivers on a wide scale of 3D cameras and motion-capture systems available on the market [26].Advantages and disadvantages of selected sensing technologies [27] are shown in Table 1.
The devices equipped with sensor PrimeSence are most of all used on development of new applications in robotics area.Thanks to the support of OpenNI community it is possible to create reliable applications effectively and in a short time.The devices supported by OpenNI are: Microsoft Kinect, Asus Xtion Pro and PrimeSence PSDK 5.0 [28].For a proposal of the computer vision system and its testing within the safety system we decided on Microsoft Kinect because of its low acquisition price and availability on the market.Two types of Microsoft Kinects were tested: Kinect Xbox 360 and Kinect One.These two devices differ in used sensing principle and technical parameters [29].Their specifications are shown in Table 2.

Determination of comparative parameters of tested devices
Operation principles of both devices (Table 2) are based on the different sensing methods, which is reflected in parameters of the accuracy and frame rate of multi-dimensional image.The important comparative parameter is also a bandwidth of the used communication bus.All these parameters are important for Technical parameters of selected devices

R E V I E W
for the device Kinect Xbox 360 does not require the depth data visualization and allows the achievement of the better frame rate value.The added depth map visualization is used only for better illustration of a scene (see Fig. 4a).The package for the device Kinect One requires imaging of point clouds that offers extensive visualization options (see Fig. 4b).Visible colored coordinate systems indicate current position and orientation of main parts of the human body.

Conclusion
The article presents the steps related to the proposal of the computer vision system for the purposes of verification of the structure of the safety system proposed for solving the selected of selected type of data that can be processed in the ROS environment.Each of defined safety levels uses a different representation of data.In the analysis, we took in account the resulting frame rate that is given by the environment of program Rviz.The average values of the frame rate are shown in Table 4.
The charts of frame rate values observed for individual tested devices are shown in Figs. 2 and 3.
The values of frame rates presented in Table 4 show that the Kinect Xbox 360 device is more suitable alternative for use in our continuing research.Safety Level 1 (SL1) package used for Kinect Xbox 360 allows achieving 30 fps at all times of its activity.Due to compatibility issues different type of SL1 package was used for Kinect One device.This change caused that frame rate for this device is almost three times lower.In the case of Safety Level 2 we are comparing the frame rate for Point Cloud (PC) and Depth Map (DM) imaging technique.In contrast to a depth map, values of point clouds are low and not suitable for use.The values obtained by both devices are very similar, but it should be noted that the parameters of Kinect One were set to standard resolution, comparable to Kinect Xbox 360.At high resolution parameters and given computing hardware the Kinect One device is unusable.Package for Safety Level 3 can be run only with the model Kinect Xbox 360.However, achieved values are not sufficient for our further research.multiple devices.Room lighting had significant impact on the obtained results.Variable values of daylight and also low value of illumination reduced the frame rate values or caused their significant fluctuation.For proper operation of the system will be necessary to provide stable and high value of illuminance.From the analysis it follows that for building controlling and deciding system packages it is better to use the Kinect Xbox 360 device which has a guaranteed compatibility and its parameters are sufficient for this purpose.For deployment in normal use it will be better to consider completion of support for a newer model.Next steps of our research will lead to an elaboration of the computer vision system design and its finalization.problems of a human-robot collaboration with utilization the open-source platform ROS.
The knowledge acquired during a preparation and performance and conduct of analysis have pointed to several shortcomings.One of the main shortcomings is the limited compatibility between the Kinect One device and a number of existing packages for object detection.One possibility is to edit an existing package or create a new package fully compatible with this device.For the next step in the development of computer vision system it is necessary to provide powerful computing hardware.This removes part of the problems with low values of the frame rate and ensures smooth operation during work of

APPROACHES
TO THE COMPUTER VISION SYSTEM PROPOSAL ON PURPOSES OF OBJECTS RECOGNITION WITHIN THE HUMAN-ROBOT SHARED WORKSPACE COLLABORATION APPROACHES TO THE COMPUTER VISION SYSTEM PROPOSAL ON PURPOSES OF OBJECTS RECOGNITION WITHIN THE HUMAN-ROBOT SHARED WORKSPACE COLLABORATION Alexander Rengevic -Darina Kumicakova -Ivan Kuric -Vladimír Tlach -Pawel Drozdziel*

Fig. 1
Fig. 1 Simplified functional structure of the safety system and safety levels

Fig. 4
Fig. 4 The visual representation of data emitted by the devices -SafetyLevel 1