Sign Language Detection with CNN

Project Description

This project implements a Convolutional Neural Network (CNN) to classify hand sign language gestures. The model is trained to recognize different sign language hand positions and is capable of accurately predicting sign language classes from image data. This is a computer vision project that demonstrates the application of deep learning for gesture recognition.

Key Features

• CNN Architecture: A custom-built convolutional neural network with two convolutional layers
• Dataset: Images of hand signs from the Kaggle Signs Detection Dataset
• Multi-class Classification: Classifies 6 different sign language classes
• Performance Evaluation: Includes accuracy metrics, confusion matrix, and classification reports
• Visualization: Training/validation curves and prediction visualizations

---

Dataset

The project uses the Signs Detection Dataset from Kaggle, which contains:

• Training Set: Pre-processed images of hand signs with corresponding labels
• Testing Set: Images for model evaluation
• Format: HDF5 (.h5) files for efficient data handling
• Image Size: 64×64 pixels in RGB format
• Classes: 6 different sign language gestures

Dataset Source: https://www.kaggle.com/datasets/maneesh99/signs-detection-dataset/

---

Project Structure

finger_sign_detection_with_CNN/
├── signs_cnn.ipynb              # Main Jupyter notebook with full project implementation
├── Signs_Data_Training.h5          # Training dataset (HDF5 format)
├── Signs_Data_Testing.h5           # Testing dataset (HDF5 format)
└── README.md                        # Project documentation

---

Installation & Setup

Prerequisites

• Python 3.7+
• Jupyter Notebook or JupyterLab
• Required libraries (see below)

Required Libraries

Install the required packages using pip:

pip install numpy pandas matplotlib h5py tensorflow scikit-learn seaborn

Project Steps

1. Data Loading & Preprocessing

• Load training and testing data from HDF5 files
• Normalize pixel values by dividing by 255 (scale to [0, 1] range)
• Convert labels to one-hot encoded format using to_categorical()
• Final shapes: Training (1080, 64, 64, 3) and Testing (120, 64, 64, 3)

2. Exploratory Data Analysis (EDA)

• Visualize sample images from the training set
• Display data shapes and characteristics
• Verify label distributions

3. Model Architecture

The CNN model consists of:

- Conv2D Layer 1: 32 filters, 3×3 kernel, ReLU activation
- MaxPooling Layer 1: 2×2 pool size, stride 2
- Conv2D Layer 2: 64 filters, 3×3 kernel, ReLU activation
- MaxPooling Layer 2: 2×2 pool size, stride 3
- Flatten Layer
- Dense Layer: 64 neurons, ReLU activation
- Dropout Layer: 0.1 dropout rate
- Output Layer: 6 neurons, Softmax activation (for 6 classes)

4. Model Compilation

• Loss Function: Categorical Crossentropy (for multi-class classification)
• Optimizer: Adam (adaptive learning rate)
• Metrics: Accuracy

5. Model Training

• Batch Size: 64 images per batch
• Epochs: 25 training cycles
• Validation: Evaluated on test set after each epoch
• Monitors both training and validation accuracy/loss

6. Model Evaluation

• Calculate test loss and test accuracy
• Generate classification report with precision, recall, and F1-scores
• Compute confusion matrix to analyze prediction patterns

7. Visualization

• Training Curves: Accuracy and loss plots showing training vs. validation performance
• Confusion Matrix: Heatmap showing classification results for each class
• Predictions: Visualization of 10 random test images with actual vs. predicted labels

8. Individual Predictions

• Make predictions on single images
• Display predicted probabilities for each class
• Compare with actual labels

---

How to Run

Using Jupyter Notebook

1. Clone or download the project

   cd /Users/esemb/Desktop/code/finger_sign_detection_with_CNN

2. Open Jupyter Notebook

   jupyter notebook

3. Open signs_cnn.ipynb and run all cells sequentially

Expected Output

• Training progress for 25 epochs
• Final test accuracy and loss values
• Classification report with per-class metrics
• Confusion matrix visualization
• Training/validation curves
• Sample predictions on test images

---

Model Performance

The trained model provides:

• Multi-class Classification: Accurately classifies among 6 sign language classes
• Performance Metrics: Accuracy, precision, recall, and F1-score for each class
• Visualization: Confusion matrix reveals which classes are commonly misclassified

---

Key Technologies

• TensorFlow/Keras: Deep learning framework for building and training the CNN
• NumPy & Pandas: Data manipulation and numerical operations
• Matplotlib & Seaborn: Data visualization and plotting
• Scikit-learn: Metrics calculation and evaluation
• HDF5: Efficient data storage and loading

---

Author & License

This project is an educational implementation for sign language gesture recognition using CNNs.

---

References

• Kaggle Signs Detection Dataset
• TensorFlow/Keras Documentation
• Convolutional Neural Networks