Using Logistic Regression, Random Forest, and Linear SVC with and without Dimensionality Reduction (PCA, Isomap, LLE)
This project aims to detect ships from satellite images using classical machine learning algorithms. Different dimensionality reduction techniques — PCA, Isomap, and LLE (Locally Linear Embedding) — are compared to analyze their effects on model performance.
The models are trained on the ships dataset, which contains both ship and non-ship image patches, and the best-performing model is used to detect ships in full satellite scenes.
Dataset Link: Google Drive Folder
The dataset includes:
ships_dataset/
├── ship_data.npy # Image feature vectors
├── ship_labels.npy # Labels (1 = ship, 0 = no ship)
└── scenes/ # Full satellite images for final ship detection
Trained_saved_model/ # Pretrained model files (.joblib)
Test_saved_dataset/ # Saved test datasetsYou can directly download these resources and run the project locally.
After downloading the Trained_saved_model/ and Test_saved_dataset/ folders, make sure to move all files into the main project directory (where your Jupyter Notebook or Python scripts are located). This ensures that the scripts can correctly locate the datasets and pretrained models for testing or further predictions.
- Clone or download this repository.
- Create and activate a conda environment:
conda create -n shipdetect python=3.10
conda activate shipdetect- Install dependencies:
pip install -r requirements.txt- Run the Jupyter Notebook or Python scripts.
Three classifiers were trained and evaluated:
- Logistic Regression
- Random Forest Classifier
- Linear SVM (LinearSVC)
Each classifier was trained under three conditions:
- Without Dimensionality Reduction (Baseline)
- With PCA (Principal Component Analysis)
- With Manifold Learning Techniques
- Isomap
- Locally Linear Embedding (LLE)
All models were evaluated using:
- F1-Score
- Accuracy
- Training and Testing Time
| Classifier | Method | F1 Score | Train Time (s) | Test Time (s) |
|---|---|---|---|---|
| Logistic Regression | Without DMR | ... | ... | ... |
| Random Forest | PCA | ... | ... | ... |
| Random Forest | Isomap (Best) | Highest | Low | Fast |
| Linear SVC | LLE | ... | ... | ... |
Random Forest with Isomap
- Achieved the highest F1 score
- Balanced training time and accuracy
- Used for final ship detection in full satellite scenes
Manifold Visualization
- Isomap and LLE plots show clear separation between ship and non-ship regions in reduced 2D space.
Misclassified Samples
- Misclassified samples primarily occur where:
- Ships are partially visible or small.
- Background textures resemble ships (e.g., clouds, bright water patches).
- Future work can improve robustness with deep features and augmentation.
Scene-level Detection
Using the best trained model (Random Forest + Isomap), ship regions are highlighted in red when detected from full satellite scenes.
detect_ships_in_scene("ships_dataset/scenes/scene_1.jpg")The function overlays red heatmaps where ships are predicted.
- Make sure your image_dir and ships_dataset/scenes paths are set correctly.
- Load your best model:
best_model = joblib.load("best_RandomForest_isomap.joblib") - Run detection:
detect_ships_in_scene("ships_dataset/scenes/scene_1.jpg") - Output is saved automatically as:
imgs_dir/ship_detection_scene.png
project/
│
├── results_reports/
│ ├── model_comparison_results.csv
│ ├── classification_reports.csv
│ ├── confusion_matrix_LogisticRegression.png
│ ├── confusion_matrix_RandomForest.png
│ ├── confusion_matrix_SVM.png
│ └── ...
│
├── plot_images/
│ ├── comparison_test_accuracy_isomap.png
│ ├── LLE_2D_Visualization.png
│ ├── Isomap_2D_Visualization.png
│ ├── PCA_2D_Visualization.png
│ ├── misclassified_samples.png
│ ├── ship_detection_scene.png
│ └── ...
│
├── requirements.txt
└── README.md- Use deep learning–based feature extraction (e.g., ResNet, VGG) for better spatial understanding.
- Apply data augmentation (rotation, brightness, cropping) to improve generalization.
- Balance dataset if class imbalance exists to reduce model bias.
- Experiment with t-SNE or UMAP for nonlinear feature visualization.
- Further tune Isomap hyperparameters like n_neighbors for better manifold representation.
Satyabrata Das