Kayaker Detection and Segmentation using Mask R-CNN

A computer vision project that detects and segments kayakers in images and videos using a custom-trained Mask R-CNN model based on the Matterport Mask R-CNN implementation.

🎯 Project Overview

This project implements instance segmentation for kayaker detection using:

• Architecture: Mask R-CNN with Feature Pyramid Network (FPN) and ResNet101 backbone
• Custom Dataset: 65 kayaker images with pixel-by-pixel polygon annotations
• Output: Bounding boxes and segmentation masks for kayaker instances
• Best Performance: 93.3% test mAP (Model M6 & M8)

Input	Output

📊 Model Performance

Model	Starting Weights	Training Layers	Epochs	Training mAP	Test mAP
M1	COCO	heads	5	0.844	0.800
M2	COCO	heads	10	0.824	0.867
M3	COCO	heads	15	0.844	0.867
M4	COCO	heads	20	0.844	0.800
M5	COCO	all	5	0.865	0.867
M6	COCO	all	10	0.885	0.933
M7	COCO	all	15	0.865	0.867
M8	COCO	all	20	0.885	0.933

🚀 Quick Start

Prerequisites

• Python 3.7
• pip and venv (recommended)

Installation

1. Clone the repository

   git clone https://github.com/ndakic/kayaker-detection
   cd kayaker-detection

2. Download YOLOv3 weights (237 MB)

   • Download from here
   • Place in the model/ folder

3. Install dependencies

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate
   pip install -r requirements.txt

4. Install Mask R-CNN library

   cd Mask_RCNN
   python setup.py install
   cd ..

Usage

The main application (application.py) provides several modes:

1. Train a new model (default mode):

   python application.py

2. Generate predictions on video:

   • Uncomment the prediction lines in application.py
   • Set your model path in MODEL_PATH
   • Run: python application.py

3. Evaluate model performance:

   • Uncomment the evaluation lines in application.py
   • Run: python application.py

📁 Project Structure

kayaker-detection/
├── application.py          # Main application entry point
├── config.py              # Training and inference configurations
├── model.py               # Mask R-CNN model wrapper
├── dataset.py             # Custom dataset loader
├── evaluate.py            # Model evaluation utilities
├── video.py               # Video processing functions
├── util.py                # Utility functions
├── requirements.txt       # Python dependencies
├── dataset/               # Training and test datasets
│   ├── train/            # Training images and annotations
│   └── test/             # Test images and annotations
├── model/                # Model weights
│   └── yolov3.weights    # Pre-trained weights (download required)
├── files/                # Input/output files
│   ├── gif/              # Demo GIFs
│   ├── input_videos/     # Input video files
│   └── output_videos/    # Generated output videos
└── Mask_RCNN/            # Mask R-CNN library (clone required)

🎯 Configuration

• Training Config: KayakerConfig in config.py

  • Image dimensions: 512x512
  • Classes: background + kayaker (2 total)
  • Training ROIs per image: 32
  • Steps per epoch: 100

• Inference Config: PredictionConfig in config.py

  • Detection confidence threshold: 0.95
  • Maximum detections per image: 1