MLP vs CNN for Image Classification on CIFAR-10

A comparative deep learning project that evaluates the performance of Multi-Layer Perceptrons (MLPs) and Convolutional Neural Networks (CNNs) on the CIFAR-10 dataset using PyTorch.

---

Overview

Image classification is one of the most fundamental tasks in computer vision. This project compares two popular neural network architectures:

• Multi-Layer Perceptron (MLP)
• Convolutional Neural Network (CNN)

Both models are trained and evaluated on the CIFAR-10 dataset to understand how architectural choices affect classification performance, feature learning, and model generalization.

---

Objectives

• Implement an MLP for image classification.
• Design and train a CNN using convolution and pooling layers.
• Compare the performance of both architectures.
• Study the effect of model capacity on classification accuracy.
• Visualize feature maps learned by convolutional layers.
• Analyze incorrectly classified images.

---

Dataset

• Dataset: CIFAR-10
• Training Images: 50,000
• Testing Images: 10,000
• Classes: 10
• Image Size: 32 × 32 RGB

---

Tech Stack

• Python
• PyTorch
• NumPy
• Matplotlib
• torchvision

---

Multi-Layer Perceptron (MLP)

A fully connected neural network was implemented by flattening each input image into a one-dimensional feature vector before passing it through multiple dense layers.

Training Performance

Observations

• Training accuracy increases steadily during training.
• Training loss consistently decreases.
• The model learns useful representations but remains limited because spatial information is discarded.

---

Model Capacity Analysis

Different hidden layer configurations were evaluated to understand the impact of model complexity on classification accuracy.

Observations

• Smaller models underfit the dataset.
• Medium-sized models achieved the highest accuracy.
• Increasing network size further provided only marginal improvement while increasing the risk of overfitting.

---

Convolutional Neural Network (CNN)

A CNN consisting of convolution, ReLU activation, max-pooling, and fully connected layers was implemented to preserve spatial information and learn hierarchical image representations.

Training Performance

Observations

• CNN converges faster than the MLP.
• Achieves significantly higher classification accuracy.
• Learns richer visual features through convolutional filters.

---

Feature Map Visualization

Intermediate feature maps were visualized to understand how the CNN processes visual information across different convolutional layers.

Observations

• Early layers capture edges, corners, and simple textures.
• Intermediate layers combine low-level patterns into meaningful visual structures.
• Deeper layers learn abstract semantic features that contribute to object recognition.

---

Misclassified Images

Analyzing incorrect predictions provides insight into the limitations of the trained model.

Observations

• Most incorrect predictions occur between visually similar classes.
• Ambiguous images and unusual viewpoints remain challenging.
• Error analysis helps identify opportunities for improving model performance.

---

Results

Model	Test Accuracy
Multi-Layer Perceptron (MLP)	54.6%
Convolutional Neural Network (CNN)	75.3%

---

Key Takeaways

• CNNs significantly outperform MLPs for image classification tasks.
• Preserving spatial information is essential for effective feature learning.
• Model capacity must be carefully balanced to avoid underfitting and overfitting.
• Feature map visualization demonstrates how CNNs progressively learn hierarchical image representations.
• Convolutional neural networks provide superior generalization compared to fully connected networks.

---

Repository Structure

mlp-cnn-cifar10/
│
├── images/
│   ├── sample_images.png
│   ├── mlp_accuracy.png
│   ├── mlp_loss.png
│   ├── capacity_analysis.png
│   ├── cnn_accuracy.png
│   ├── cnn_loss.png
│   ├── feature_maps_1.png
│   ├── feature_maps_2.png
│   ├── feature_maps_3.png
│   └── misclassified_images.png
│
├── notebook/
│   └── mlp_vs_cnn_cifar10.ipynb
│
├── README.md
├── requirements.txt
└── LICENSE

---

Installation

git clone https://github.com/AryanMeshram059/mlp-cnn-cifar10.git

cd mlp-cnn-cifar10

pip install -r requirements.txt

jupyter notebook notebook/mlp_vs_cnn_cifar10.ipynb

---

Notes

This repository is based on a deep learning coursework project and has been reorganized into a standalone portfolio project to showcase the implementation, experimentation, and analysis of neural network architectures for image classification using PyTorch.