Humanoid Type Hand Moved by Shape Memory Alloy

The main contribution of this paper is a practical application of generalised neural networks for a dextrous hand moved by Shape Memory Alloys (SMA). Since SMA have highly non-linear characteristics and their parameters depend on the environment (mainly on temperature) so the robot hand is controlled by a generalised neural network, which can learn the actual non-linear characteristics of the robot hand. The experimental setup consists of a 20 degree of freedom hand moved by SMA string used as artificial muscle. A video camera is used to detect the position of joints. The position is then sent to the visual display computer via the Internet, which displays the hand in 3D using OpenGl.


Introduction
Telemanipulation present an process where the operator has some task done at the far environment where he/she cannot be present physically. Goertz developed the first modern masterslave system teleoperator at Argonne National Laboratory in 1945. Telemanipulation is divided into two strongly coupled processes. One process is the interaction between the operator and the master device, the other is the interaction between the slave device and the far environment contact. The master device represents the far environment at the operator site, and the slave device represents the operator at the remote site. The information flow between the operator and remote site can be seen in Fig. 1, where only three types of information are fed back: visual, audio and sense of touch. Human beings get six types of sensing from the environment surrounding them but only some of these sensings are used during telemanipulation.
Telemanipulation is the extension of human manipulation to a remote location. By providing appropriate feedback, the telemanipulation can be well utilised in dangerous or otherwise unreachable environment. Human operators perform their tasks mostly by hand, so the master and slave devices must fit the operator's hand. In the field of telemanipulation, many haptic devices have been developed.
The glove type master device allows the most complex and dextrous manipulation. The human hand is the most widely used tool. The final goal is to make a device with which the operator can feel as if the function of his whole hand would expanded. There are several commercial sensor gloves without force feedback on the market. The paper [7] proposed a sensor glove with force feedback to the all 20 joints of the operator's hand. If the human hand is covered with these commercial devices, some certain difficult tasks can be done at the remote site, but commercial hand type slave devices are not available. This paper proposes a hand type slave device for precise telemanipulation. Until this point no force feedback has been implemented in this system, thus the actual set up is more simplified than the general one in Fig. 1. The system used is shown in Fig. 2.

The artificial hand description
The hand is primarily used for manipulating activities requiring very fine movement incorporating a wide variety of hand and finger postures. Consequently, there is interplay between the wrist joint positions and efficiency of finger actions. The hand region

HUMANOID TYPE HAND MOVED BY SHAPE MEMORY ALLOY HUMANOID TYPE HAND MOVED BY SHAPE MEMORY ALLOY Péter Korondi -Péter Zsíros -Fetah Kolonic *
The main contribution of this paper is a practical application of generalised neural networks for a dextrous hand moved by Shape Memory Alloys (SMA). Since SMA have highly non-linear characteristics and their parameters depend on the environment (mainly on temperature) so the robot hand is controlled by a generalised neural network, which can learn the actual non-linear characteristics of the robot hand. The experimental setup consists of a 20 degree of freedom hand moved by SMA string used as artificial muscle. A video camera is used to detect the position of joints. The position is then sent to the visual display computer via the Internet, which displays the hand in 3D using OpenGl. has many stable, yet very mobile segments, with very complex muscle and joint actions.
A robotic hand is made similar to the human hand, theoretically it can realise all the twenty freedoms of movement, but this time only eight of them are used. We do not want to control the movement of the most upper knuckle (DIP -Distal Interphalangeal Predominant (arthritis)) separately, because that movement can not be well controlled on a human hand either. Only the upper knuckle is pulled, and across the rubber it will pull up the middle knuckle too. The structure of the hand is shown in Fig. 3.

Generalised neural network
A generalised neural network is used to control the hand. The difference between a classical and a generalised neuron is that real numbers are used for the weights of a classical neuron, but the weights are fuzzy functions in the case of generalised neurons. A simple neuron is shown in Fig. 4 and a multi-layer generalised neural network will be discussed in Fig 8. For each input of a neuron there is a B set, which can be fired by an antecedent set A. Each antecedent set corresponds to a consequent set B, which is a singleton set in our case, as shown in Fig.  5. If input X 0 arrives to the input X of the system, it will cut two or more antecedents (depending on the type of antecedent function, triangular sets are assumed here). The B values belonging to the antecedents are weighed and the mass point is then calculated.
The value of the mass-point will go into the summation part of the neuron, see in The antecedents are forming Ruspini partition, which means that the sum of the membership values at any X is equal to 1. This partition is shown in Fig. 7.

Training of the generalised neural network
A generalised neural network is shown in Fig. 8. Using the notation introduced in this figure, the outputs of the neural network can be calculated in the following way: where X i is the output i-th neuron of the 1-st layer, A (2) i,t is the t-th antecedent of i-th input in the second layer. Each antecedent set corresponds to all neurons. B (2) i,j,t is the consequent set of the j-th neuron fired by A (2) i,t .
The output Y i is calculated as where k is the number of inputs, t is the number of antecedents, P k is the k-th input.
The question is how to modify the B k,i,t -th weight, if the error is h?
The error can not be derived directly according to B k,i,t , and therefore the chain rule has to be used as in the back propagation algorithm. The error h can be derived according to the output Y, the output can be derived according to the input X, and finally the input can be derived according to the required B. Then, h 2 can be calculated as: In general, if the errors in the layer n are h (n) j , then the error of the neurons in layer nϪ1 can be calculated in the following way: The weight modification is given as:

Control of the hand
Pulse width modulation is applied to control the movement of the hand. Two different duty rations are used, one to move a certain part of the hand and one to hold it at a certain position (see Fig.  9 and 10). When the operator changes the reference position, the Fig. 8 The generalised neural network neural network calculates how long the voltage should be switched across the wire to move a finger into the new reference point as shown in Figs. 9 and 10. Since the condition (mainly the temperature) of the environment has a great influence on the characteristic of SMA wires, the neural network must be retuned continuously.
The experimental results show good repeatability performance, where desired finger tip angle control is achieved in region from 10 to 90 degrees, Fig. 11

Complexity reduction of the generalized algorithm
One of the main problems of applying fuzzy or neural techniques is calculation complexity. Engineers have to face this problem in complex systems or especially in the field of information retrieval, where extremely large information maps of whole libraries or Internet have to be processed at each user's request. These applications apply the generalized type neural networks. Let us have a brief introduction to the results of complexity theorem.

Lemma 1. The calculation complexity grows proportionally with the number of parallel layers and the neurons.
Omitting the computational effort of added operation but considering the product operation, the computational requirement is characterised as: ( 1 1 ) P indicates the calculation of the membership functions: where s indicates the calculation of one membership value of one antecedent set.
The main objectives of this section are to propose an algorithm that is capable of filtering out common linear combinations and reducing the number of antecedent sets based on the transformation of the weighting functions. One of the main advantages of the proposed method is that the effectiveness of the compression is controlled by the help of a given error threshold.

Theorem 2. Equation (4) can always be transformed into the following form:
where "r" denotes "reduced", further n r lϩ1 Յ n lϩ1 and ᭙i : m r l,i Յ m l,i .
The reduced form is represented as neural network in Fig 12. The further proofs of this technique are discussed in [9,10].

Feedback
For finding the joints in the picture of the camera by the computer, the joints were painted red. This colour is chosen because out of the three primary colours (red, green, blue), red colour is the most characteristic. Unfortunately, the hand is made of copper, which has a quite high red colour content too so the hand had to be painted green. Green has been chosen, because this is another primary colour, and it is used in movie techniques (in spite of the name of the technique, which is blue box, the blue colour had been used only until about 10 years ago, when it was changed to green, because that has better characteristics for cameras). Because there are a lot of red objects in the laboratory so a full green box around the hand is used. We use a Sony Handycam with a Genius Video Wonder II TV card. The computer is a Pentium-166 with 40 Mbyte RAM and with Windows 98 operating system. This is because the programming of the TV card is quite easy under windows, there are drivers for it from the producer, and we simply have to use the routines provided by them. The program is written in Borland Delphi 4. The video-capturing program (eac401) was downloaded from the Internet ("Delphi Superpage"), and modified it to fit the requirements.
The size of the picture is chosen to be 160 times 120 points, because this is the smallest standard size. Even in this case the required computation power can be quite high. In this program different camera options can be set, for example brightness, contrast or saturation. By adjusting these options we could get much better pictures. The phases of the correction of the picture are shown in Figs. 15-18.

Visual interface
The software was developed to give visual feedback of a general hand in telemanipulation. Visual feedback is aimed at giving a quite real representation of the environment, and although at the moment this environment is far from complete, the program enables the user to wander in full three-dimensional space.
The program was designed to run either locally in full simulation mode or over TCP/IP connection, such as Internet or Local Area Network. In simulation mode, the program can be used to represent all the motions of the human hand. To accomplish it, both anatomical and mathematical models were built up, and these models were implemented in OpenGL (Open Graphics Language). The mathematical deduction uses the Denavit-Hartenberg notation, because it can be quite well applied both for hands and in OpenGL. The model uses several basic assumptions: • The base coordinate frame is the same for all fingers. This is used for having a completely general case, where the fingertip coordinates are calculated with respect to a common frame, for instance to the wrist. This common frame is considered to be at joint 0. This joint is used in order to have the possibility to extend the model at a later time if needed. • One finger contains three joints, the first has two degrees of freedom, and the other two have only one, which means a planar rotation. The position of the first joint is determined by a variable l 0 , which is the distance between the first, common coordinate frame and the first joint. • Along one finger, the value of d i is 0, which means that the common normal line on the same line. • Along one finger the value of a i is the length of the link. • The twist angle is taken into account only at the base coordinate frame and at the first joint, since lateral movement of the finger is also considered.
It is important that coordinate frame 0 is the same for all fingers, and is a common reference to the whole hand. The individual i angles and l 0 variables determine the exact position and orientation of the next frame of the next joint in each finger. Between the two frames d i is just equal to the length of the link, l 0 . Since the joint 0 is of the revolute type, q 0 is 1 the relationship between the two joints is In (14) A 0 1 is the 4 ϫ 4-transformation matrix between the two adjacent joints and X s are position vectors in the respective coordinate frames. Detailed form of (14) is shown in (15) and (16). As it was stated, in this case ␣ is 0°, which means that its cosine is equal to 1 and its sine is 0. For the two joints d is taken to be 0 and a is the length of the link. Substituting these values simplifies the previous equation and yields to ΄ ΅ ϭ ΄ ΅΄΅ . ( 1 7 ) Now the position and the orientation of the coordinate frame at joint 1 is received, further transformations can be carried out. Now, there is a general formula for calculating the position of the fingertip with reference to the common base coordinate frame. (5) shows this, and it is very similar to (14). Matrix T contains the position and orientation of the tip in the base coordinate system. Ϫsin 1 cos␣ 1 cos 1 cos␣ 1 sin␣ 1 0 It is useful to state that joint 2 has two transformation matrices and frames because of the two possible axes of rotation P 1 ϭ T 1 X 15 , ( 1 9 ) where X 15 is the position vector of the fingertip in its local co-ordinate frame.
The position and orientation of all the fingertips with respect to the common base frame (the wrist), is given by (20)  (20) The computer-aided animation of the robot hand is shown in Fig. 19.

Conclusion
This paper demonstrates a dextrous hand as a slave device for telemanipulation. Using shape memory alloy (SMA) as an "artificial muscle", efficiently increases the degree of freedom, in robot mechanisms offering much more flexibility than widely adopted hands that are mostly driven by pneumatic or electric actuators. The SMA wires have much less weight and size than the conventionally used actuators. The construction is much simpler; also its price is comparable. However, applying SMA actuators leads to a very difficult control problem, as the feature of SMA is strongly non-linear. This paper demonstrated that generalised neural networks are promoting solutions for control problems caused by SMA actuators.