ORDER PICKING USING AUGMENTED REALITY ORDER PICKING USING AUGMENTED REALITY

This article examines the problem of application of augmented reality in terms of picking goods in static systems with the deployment of a man (systems „goods to man“). Article describes a comprehensive methodology for the use of augmented reality in the process of picking starting from the preparation of input data, through optimizing the movement of workers in the warehouse, to actual worker navigation using augmented reality. The problem was solved on the authors´ workplace and the proposed solution has been verified in the experimental workplace built under the concept of ZIMS (Zilina Intelligent Manufacturing System) which is developed at the University of Zilina in coo-peration with the Central European Institute of Technology (CEIT).


Paperless picking systems
Picking process creates in the logistics chain an irreplaceable role associated with the preparation and processing of orders. With increasing pressure on the quality and timeliness of deliveries, and the increasing number and diversity of picking items, also new forms of organization are developed and new technologies are applied to the picking process [1].
The traditional approach is based on picking goods according to the documents. This principle is advantageous mainly in terms of its simplicity, financial cost and relative flexibility to change volumes of picking process. On the other hand, the need for repeated reading and checking the data in the documents leads to significant time losses and higher error rates due to human error.
Therefore, in practice, we should start to apply paperless picking systems. The advantage of these systems is mainly in replacing the traditional paper documents by picking orders electronically, which allows, in combination with other technologies, to simplify, accelerate and improve the process of searching items into the picking order. An example of such a method of picking is called "Pick-by" systems.
Types of "pick-by" systems are [1]: 1. System Pick-by-light (Pick-to-Light): This system is based on storage places equipped with special light panels which contain the lights indicating the storage location from which the goods need to be picked. The navigation system Pick-bylight is also used in reverse action, i.e. when saving assortment of items in the correct storage position. Then we talk about the so-called Put-to-light system. The system Pick-by-light is not suitable for large warehouses where there are large distances between picking items. 2. System Pick-by-voice: The picker is equipped with a headset (headphones + microphone) ensuring two-way communication between the picker and control system and portable terminal which is used for storing and radio transmission of information between the picker and control system. The system includes a radio transmission of information and voice recognition software. The picker receives from the control system gradually in the optimal sequence information necessary for picking individual order lines. After removing the item, the picker reports back to the managing systems the number of taken items, also confirms the completion of the processing line orders. The system Pick-by-voice puts higher demands on worker´s concentration (only voice information) and is not suitable for noisy operations. 3. System Pick-to-belt: Items are in the required quantity collected from a store and placed on the conveyor which transports them automatically to a packaging workplace.
In addition to above mentioned basic types of "pick-by" systems, manufacturers of storage and handling equipment also offer other types of paperless picking systems such as: e-pick, pickto-tote, pick-to-bucket, etc., which further modify and develop the fundamental principles of mentioned "pick-by" systems.
Among the new, progressive types of "pick-by" systems is currently classifiable the picking system known as Pick-by-vision [2] which represents the use of augmented reality for navigation worker during picking and which is currently the subject of intense research in terms of the technological solutions and suitability of its application in industrial practice.

The application of augmented reality in picking process
The proposed methodology for augmented reality applications in the process of picking the goods consists of the following basic steps ( Fig. 1): 1. Preparation and processing of input data; 2. Proposal of the optimal picking route; 3. Proposal for navigation system using AR; 4. Selection of software and hardware resources; 5. Transformation of picking routes into the process map (workflow) in software for augmented reality; 6. Realization of picking process.
A more detailed description of each step of the methodology is presented in the following chapters of this article.

Base of input data
To be able to pick items from warehouse by order using augmented reality, it is necessary to know some basic input data and information. A basic resource for obtaining input data is warehouse management system (WMS -Warehouse management system). Required information for picking is [6]: 1. Information about the item itself: name of the item, code (catalog number), the available quantity, the number of items on stock position, type of items (food, chemical product, etc.), specific handling (weight, shape, dimensions, etc.). 2. Information about the location of items, warehouse map: name of the center, name of the department, room, mark of rack or other device where the item is stored. 3. Other information that may be useful for warehouseman: information about the supplier (name, address, etc.), information about the income items (date, time, quantity, etc.), specific information about the items (time of consumption in food products, specific conditions for handling with chemical products, etc.), etc.

Augmented reality technology
Augmented Reality (AR -Augmented Reality) is one of the fast growing sectors of virtual reality. The basis of augmented reality is the ability to combine elements of the real and virtual worlds into a single view. Augmented reality is a technology supported by human visual perception. With appropriate combination of real and virtual objects it is possible to provide large amount of additional information. The condition is to preserve the link with the user and real environment [3].
In contrast to virtual reality, which is all modeled by computer, augmented reality does not replace the real world, but only adds selected virtual elements or objects into the real environment. The view can be realized through the monitor and the camera or using HMD (Head Mounted Display), i.e. equipment placed on the head [4].
In practice there are two basic types of systems for augmented reality [5]: 1. Systems using position sensors and transparent display: position sensor sends information about the position and direction of the user's perspective, the scene generator based on this information displays virtual objects located in the user's field of vision. Subsequently, virtual objects are projected on semi-transparent mirror through which the user sees the real scene. 2. Capturing real camera image for the registration of markers that indicate the position of virtual objects: a video camera captures the real scene and sends it to the computer, the software on the computer is looking for the marker. If the marker is found, the software identifies the type of tags, calculates the camera position to the marker and assigns to the marker image of the virtual object and then draws on display the resulting image of scenes -the real scene added by the virtual objects.  Choosing the most appropriate routes for workers´ movement in the warehouse during the picking largely depends on the use of appropriate methods. The most appropriate method seems to be the Traveling Salesman Problem (TSP) [8]. For solution of TSP we use various methods. In the view of the nature of the task and the restriction that the picking worker must move only along the aisles between the rows of shelves, the Nearest neighbor heuristic method (resp. Greedy algorithm) was chosen, which provides satisfactory results at low computational complexity of the algorithm. When applying the nearest neighbor method to the conditions for optimizing the picking route, the route always begins in a nodal point which represents the starting and final position of the process of picking.

Proposal of navigation system using AR
The main task of augmented reality in the process of picking items is to navigate the worker in the warehouse in such a way that the worker moves by the shortest route designated in the previous step and also, this system provides the worker with basic information necessary for elimination of errors in the picking process (designation of the correct item for picking, determination of correct amount of removed items).
When designing a suitable navigation system with the support of AR, we can build on existing variants of tracking systems used in AR [9]: marker tracking system, 3D extensible tracking systems, 2D markerless tracking system, 3D markerless tracking One of the basic conditions for interconnection the AR system with WMS is the ability to extract information from the package which contains information system, only those, which are further input to software support for augmented reality [7]. This information must be exported to a format that the software for augmented reality could use as input for workflow and other actions. One possible solution is export to the XML (extensible markup language) format which is currently used as a standard for data exchange between applications. Interconnection of information system with the software solution for augmented reality is shown in Fig. 2 and can be summarized into the following steps: 1. Finding the required data in WMS -the item name, item code, location in the store, the available number, specifications, etc., 2. Export data from WMS in a compatible format (.xml), 3. Import data in .xml format to the software for augmented reality, 4. Creating a scenario using AR tools -location of markers, position of displayed information, animation, motion, rotation, control of objects, etc.

Proposal of the optimal picking route
The process of picking path optimization includes the following steps: 1. Identifying the position of picking items in the warehouse (warehouse address position), 2. Assigning coordinates of storage position of each storage unit which includes searched item, determine the coordinates of the reference or zero point -the beginning of the coordinate system), After picking a particular item, its name will change from red to green.
Yellow arrow -yellow arrow is a directional arrow that indicates the direction of the nearest location of items in a row for picking. By following the directional arrow operator is passively navigated to the item.
Green arrow -green arrow shows to the specific storage position from which the item has to be picked.
Numerical information -number located above the box represents the number of items which have to be picked from a particular type of stored items.
Move for picking the next item in the list is done after checking the previous item has already been collected. Next step is displayed after pushing the confirmation button.

Selection of software and hardware resources
For navigation through markers and displaying navigation elements described in Chapter 3.3 following software and hardware are necessary: • Software solution -software Unifeye Design from Metaio is designed for displaying virtual information using augmented reality. It is also suitable for use in the process of navigation, it allows the generation of markers and includes Workflow tool which can create a scenario of displaying virtual objects and how to control virtual objects in real space. • Hardware solution: m portable computer which contains the Windows operating system and, at least, 3 USB ports and also allows the VGA connection to Vuzix glasses for augmented reality. m VUZIX glasses for augmented reality. m handcart to store computer and picking items.

Transformation of picking route into a scenario of workflow for AR
An essential part of applications of augmented reality into the picking process is the creation of a scenario. The scenario defines how virtual elements added in the real scene will behave (display and control objects in space and time). Chosen software solution Unifeye Design has its own tool for creating scenarios -Workflow tool [11]. Workflow allows you to create a scenario by organizing individual blocks and creating links between them (Fig. 4). The resulting scenario from Workflow tool is saved as XML code. system, geolocation tracking systems, gyroscopic tracking systems, combination of individual tracking systems.
In the case of the navigation system in the process of picking, the marker appears as the optimal tracking system that is based on the use of special signs (markers) in the real scene, which are used to determine the position of the worker and also for identification of virtual object which has to be added to the real scene. Marker tracking system works according to the principle 2 described in Chapter 2.
The main reasons for the selection of marker tracking system [10]: 1. easy way to navigate, 2. wide support from software solutions, 3. low cost, 4. wide range of generating markers.
For navigation during the picking process, following method of displaying information (Fig. 3) was chosen: a) picking order b) navigation symbols Fig. 3 The basic elements of the navigation system with the support of AR which direction he needs to move. The directional arrows are shown from two sides, from the left and the right side of rack, for worker's better orientation. In addition to the yellow direction arrows the worker can see a green arrow pointing to the box from which items need to be picked and number of pieces showenby numerical information. 4. After taking the item and confirmation the worker can see the table again where the first line changes from red to green (because it has already been picked). 5. After confirmation the system again displays arrows and navigates the worker to the nearest point of picking -to the next item in the order list. 6. This process is repeated until the last item is picked.

Conclusion
Picking process with the support of augmented reality helps to short the time of finding and identification of items, it reduces cognition burden of workers in the process of selection, it reduces picking errors, increases flexibility, reliability and productivity and ultimately, it reduces logistics costs for picking process [12]. Described picking system is suitable for customized environment characterized by permanent change of storage places of items. This methodology of picking with the support of AR. was verified by the experimental workplace, which was built in within the concept ZIMS (Zilina Intelligent Manufacturing System) [13]. The problem of AR applications in the processes of picking Basic activities that can be assigned to each block are divided into the following groups: • Camera -work with camera, camera work, its activation, calibration and settings. • Geometry -work with displayed objects such as animation, rotation, scale, visibility, timing, loading and removing the object. • Interaction -interaction of virtual objects (e.g., motion control or object rotation). • Monitoring -allows you to stop the current monitoring and load the new configuration tracking. • Special functions -these blocks allow the switch to full-screen mode, synchronization, insert music background, and more.
The scenario of picking process is always based on specific order picking which is transformed into an optimal picking route. This implies the ranking of picking items and also the position of each item in the warehouse which determines at each step of the picking process kinds of navigation symbols which are assigned to the markers.

Implementation of picking process with AR
The process of picking using augmented reality runs as follows (Fig. 5): 1. In the first step the worker takes: • the box for picked items • the truck where the computer and the box are placed, • glasses for AR (iWear Vuzix VR920) which is connected to the computer. 2. After loading the picking order the worker can see the table which is placed on marker. The table contains a list of picked items arranged in the right sequence. All items are marked in red color, progressively, as they are picked the color changes from red to green. 3. After the confirmation with the button on the keyboard, the yellow directional arrow appears to navigate the worker of linking the picking system with automatic identification systems (barcodes, RFID).
requires further research especially in terms of closer integration of AR and WMS systems, increasing comfort of hardware components (wireless glasses, mobile terminal) and the possibility