ENGINEERING PROJECTS HUB

TOPIC:

Hand Gesture Recognition Using Image Processing

TEAM MEMBERS:

D Srinivas Reddy

Viswanathan B

Lalkishen M

Monish Ram K

GUIDE:

Dr.Narasimhan K, SEEE, SASTRA Deemed University

ABSTRACT:

Mouse and keyboard play a significant role in the human-machine interaction. However, owing to the rapid development of hardware and software technology, new types of interaction methods are required. Gesture recognition is one of the growing fields of research. Being a significant part of non-verbal communication, hand gestures play a vital role in our daily life. Hand recognition system provides us with an innovative, natural and user-friendly way of interaction with the computer. It has a wide area of applications such as human-machine interaction, sign language, gaming technology etc.

Hand gesture recognition is a process in which the hand gestures of humans are identified and are used to control computer and other electronic appliances. The most contributing reason for the emerging gesture recognition is that they can create a simple communication path between human and computer called Human-Computer Interaction. By keeping in mind the similarities of human hand shape with four fingers and one thumb, this project aims to identify the number of fingers in the gesture and the output can be used to control various electronic appliances. The simulation will be carried out in MATLAB R2016b.

OBJECTIVE:

The essential objective if this project is to find the number of fingers from the input hand gesture. The result obtained will be used to control various applications. Thus this will help to form a better human-computer interaction. The following flowchart explains our procedure:

TOOLS  USED:

We will be using MATLAB version 2016b along with a couple of support packages to it. One such support package (version 18.1.1.0) is for the webcam so that it will be useful for input image capture. Another one (version 18.1.1.0) is for the Arduino and MATLAB interfacing.

METHODOLOGY:

Over the last few years, numbers of researches have been conducted on hand gesture recognition for human-computer interaction using MATLAB and OpenCV. Several performance comparisons are conducted to improve the technique. The overview of the hand gesture recognition is described in the following figure. First, the hand is detected using the background subtraction method and the result of hand detection is transformed into a binary image. Then, the fingers and palm are segmented so as to facilitate the finger recognition.

The hand gestures are taken by a web camera. Image transformations are carried out on the RGB image to convert into YCbCr image. The YCbCr image is transformed into a binary image. This algorithm needs a uniform and plain background. The features like centroid, peaks detection, and Euclidean distance are found. The algorithm is implemented in MATLAB. Following are the steps in our approach:

1. Capturing a real-time RGB image using Web-Camera.
2. Pre-processing the RGB image for good brightness and contrast to achieve uniform and plain pixels.
3. Foreground Extraction of the hand part by detecting skin pixels using YCbCr method.
4. Conversion of RGB image to YCbCr image.
5. Skin pixel detection and extraction from the YCbCr image.
6. Conversion of the detected image into a binary image.
7. Finding the palm radius by taking the centroid as a reference with Distance transform method.
8. Extracting the palm with the obtained palm radius.
9. Finding the finger count and passing the value to Arduino via Serial port.

SERIAL COMMUNICATION BETWEEN MATLAB AND ARDUINO:

Serial communication is the most common low-level protocol for communicating between two or more devices. Normally, one device is a computer, while the other device can be a microcontroller (Arduino, LPC2148, NodeMCU etc.). Communications between the two devices occur with the help of a serial port cable. In this project, we have interfaced the MATLAB and Arduino (Figure 3.6) with the help of serial communication using a serial port cable. The finger count obtained finally after the simulation of the hand gesture image is sent via the serial port of the computer with MATLAB to the Arduino COM port through the serial port cable. Depending on the finger count received by the Arduino a specific task is performed by the Arduino. The serial communication is established through a serial port object called “serial” in the MATLAB workspace.

OUTPUT: