Sentiment Analysis

Binary sentiment classification using NLP and Machine Learning.

Project Overview

This project implements a complete machine learning pipeline for binary sentiment classification of movie reviews. The solution includes both Data Science (DS) and Machine Learning Engineering (MLE) components, with full Docker support for reproducibility.

Dataset: 50,000 movie reviews (25,000 positive, 25,000 negative)
Task: Binary classification (positive/negative sentiment)
Target Metric: Accuracy >= 0.85

---

DS Part: Data Science Analysis

1. Exploratory Data Analysis (EDA)

Key Findings:

• Dataset is perfectly balanced (50% positive, 50% negative)
• No missing values in the dataset
• Review lengths vary significantly (10-2500 words)
• Common positive words: "great", "excellent", "amazing", "wonderful"
• Common negative words: "bad", "terrible", "worst", "awful"

Notebook: notebooks/eda-modelling.ipynb

2. Feature Engineering

I compared multiple preprocessing and vectorization approaches:

Preprocessing Methods

• Stemming: Reduces words to their root form (e.g., "running" → "run")
• Lemmatization: Reduces words to their dictionary form (e.g., "better" → "good")

Result: Lemmatization performed better, preserving semantic meaning while reducing dimensionality.

Vectorization Methods

• TF-IDF: Term Frequency-Inverse Document Frequency
• Count Vectorization: Simple word count vectors

Result: TF-IDF performed better, effectively weighting important words while reducing the impact of common terms.

Final Configuration: Lemmatization + TF-IDF with bigrams (1,2)

3. Model Training & Selection

I trained and compared 5 different models:

Model	Accuracy	Precision	Recall	F1-Score
Logistic Regression	0.8878	0.8881	0.8878	0.8877
Linear SVC	0.8804	0.8805	0.8804	0.8804
Multinomial Naive Bayes	0.8606	0.8612	0.8606	0.8606
Random Forest	0.8430	0.8430	0.8430	0.8430
Gradient Boosting	0.8094	0.8138	0.8094	0.8087

Selected Model: Logistic Regression (max_iter=1000)

Training Configuration:

• Training samples: 32,000
• Test samples: 8,000
• Features: 5,000 (TF-IDF with lemmatization)
• Training time: ~0.07 seconds
• Total pipeline time: ~42 seconds

Model Performance:

Metric	Score
Accuracy	0.8878
Precision	0.8881
Recall	0.8878
F1-Score	0.8877

Per-Class Performance:

Class	Precision	Recall	F1-Score	Support
Negative	0.90	0.87	0.89	4,000
Positive	0.88	0.90	0.89	4,000

Reasoning:

• Highest accuracy (0.8878) among all models, exceeding the 0.85 requirement ✓
• Fast training and inference (0.07s training time)
• Good interpretability (feature weights)
• Balanced performance across both classes
• Lower computational requirements compared to ensemble methods

4. Performance Evaluation

Test Set Results (8,000 samples):

• Accuracy: 0.8878 (88.78%)
• Precision: 0.8881 (88.81%)
• Recall: 0.8878 (88.78%)
• F1-Score: 0.8877 (88.77%)

Confusion Matrix Analysis:

• True Negatives: 3,480 (87% of negative samples correctly identified)
• True Positives: 3,620 (90% of positive samples correctly identified)
• False Positives: 520 (13% of negatives misclassified as positive)
• False Negatives: 380 (10% of positives misclassified as negative)

The model performs well on both classes with slightly better recall on positive sentiments.

5. Business Value & Applications

Potential Applications:

1. Movie Review Aggregation: Automatically classify user reviews on platforms like IMDb, Rotten Tomatoes
2. Customer Feedback Analysis: Analyze product reviews for e-commerce platforms
3. Social Media Monitoring: Track brand sentiment on Twitter, Facebook, etc.
4. Content Moderation: Flag negative content for manual review
5. Market Research: Understand customer sentiment towards products/services

Business Value:

• Cost Reduction: Automate manual review classification (saves 100+ hours/month)
• Real-time Insights: Instant sentiment analysis vs. manual processing
• Scalability: Process millions of reviews automatically
• Data-Driven Decisions: Quantify customer sentiment for product improvements

ROI Estimate: For a platform with 10,000 reviews/day, automation saves ~$50,000/year in manual labor costs while providing instant insights.

---

How to Run This Project

Follow this step-by-step roadmap to run the complete sentiment analysis pipeline.

---

Prerequisites

• Python 3.9+ (for local execution)
• Docker (for containerized execution)
• Conda (recommended for environment management)

---

Quick Start Roadmap

Step 1: Setup Environment

Option A: Using Conda (Recommended)

# Create and activate conda environment
conda create -n myenv python=3.9
conda activate myenv

# Install dependencies
pip install -r requirements.txt

# Download NLTK data
python -c "import nltk; nltk.download('punkt'); nltk.download('punkt_tab'); nltk.download('stopwords'); nltk.download('wordnet'); nltk.download('omw-1.4')"

Option B: Using venv

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Download NLTK data
python -c "import nltk; nltk.download('punkt'); nltk.download('punkt_tab'); nltk.download('stopwords'); nltk.download('wordnet'); nltk.download('omw-1.4')"

---

Step 2: Prepare Data

Ensure your data is in the correct location:

data/
├── raw/
│   ├── train.csv          # Training data (required)
│   └── inference.csv      # Inference data (required)

Data Format: CSV files with columns text and sentiment (0=negative, 1=positive)

---

Step 3: Run Training

Local Execution

# Basic training
python src/train/train.py

# With custom parameters
python src/train/train.py \
  --preprocessing lemmatization \
  --vectorization tfidf \
  --model logistic \
  --test-size 0.2

Docker Execution

# Build and run training container
docker compose --profile training up --build

Expected Output:

• Model saved to outputs/models/best_model.pkl
• Vectorizer saved to outputs/models/vectorizer.pkl
• Preprocessor saved to outputs/models/preprocessor.pkl
• Metrics saved to outputs/predictions/
• Confusion matrix saved to outputs/figures/
• Processed data saved to data/processed/

Training Time: ~1-2 minutes (depending on hardware)

---

Step 4: Run Inference

Local Execution

# Basic inference
python src/inference/run_inference.py

# With custom parameters
python src/inference/run_inference.py \
  --model-dir outputs/models \
  --data-file data/raw/inference.csv \
  --output-dir outputs

Docker Execution

# Build and run inference container
docker compose --profile inference up --build

Expected Output:

• Predictions saved to outputs/predictions/predictions.csv
• Inference metrics saved to outputs/predictions/inference_metrics.json
• Processed inference data saved to data/processed/

Inference Time: ~30 seconds for 10,000 samples

---

Step 5: View Results

Check Accuracy

# View training metrics
cat outputs/predictions/test_metrics.txt

# View inference metrics
cat outputs/predictions/inference_metrics.txt

View Predictions

# View first 10 predictions with confidence levels
head -n 11 outputs/predictions/predictions.csv

View Confusion Matrix

# Open confusion matrix visualization
open outputs/figures/confusion_matrix.png  # macOS
xdg-open outputs/figures/confusion_matrix.png  # Linux

---

Project Structure

final-project/
├── data/                          # Git ignored
│   ├── raw/
│   │   ├── train.csv              # Training data
│   │   └── inference.csv          # Inference data
│   └── processed/                 # Preprocessed data (auto-generated)
├── notebooks/
│   ├── eda.ipynb                  # Exploratory data analysis
│   └── modeling.ipynb             # Feature engineering & modeling
├── src/
│   ├── __init__.py
│   ├── data_loader.py             # Data loading utilities
│   ├── preprocessing.py           # Text preprocessing with humanization
│   ├── feature_engineering.py     # Feature engineering experiments
│   ├── utils.py                   # Evaluation & visualization
│   ├── train/
│   │   ├── train.py              # Training script
│   │   └── Dockerfile            # Training Docker config
│   └── inference/
│       ├── run_inference.py      # Inference script
│       └── Dockerfile            # Inference Docker config
├── outputs/                       # Git ignored (auto-generated)
│   ├── models/                    # Trained models
│   ├── predictions/               # Predictions and metrics
│   └── figures/                   # Visualizations
├── requirements.txt               # Python dependencies
├── docker-compose.yml             # Docker orchestration
├── .gitignore
└── README.md

---

Actual Performance Results

Training Metrics (Latest Run)

• Model: Logistic Regression
• Training Samples: 32,000
• Test Samples: 8,000
• Features: 5,000 (TF-IDF + Lemmatization)
• Accuracy: 88.78%
• Precision: 88.81%
• Recall: 88.78%
• F1-Score: 88.77%
• Training Time: 0.07 seconds
• Total Pipeline Time: 42 seconds

Per-Class Performance

• Negative Class: Precision 90%, Recall 87%, F1 89%
• Positive Class: Precision 88%, Recall 90%, F1 89%

Inference Metrics

• Expected Accuracy: ~88%
• Expected Average Confidence: ~90%

---

Troubleshooting

Issue: NLTK Data Not Found

# Re-download NLTK data
python -c "import nltk; nltk.download('punkt'); nltk.download('punkt_tab'); nltk.download('stopwords'); nltk.download('wordnet'); nltk.download('omw-1.4')"

Issue: Module Not Found

# Ensure you're in the correct environment
conda activate myenv  # or source venv/bin/activate

# Reinstall dependencies
pip install -r requirements.txt

Issue: Docker Build Fails

# Clean Docker cache and rebuild
docker system prune -a
docker compose --profile training build --no-cache

Issue: Out of Memory

# Reduce batch size or use smaller dataset
# Or increase Docker memory limit in Docker Desktop settings

---

Clean Run from Scratch

To verify the project runs completely from scratch:

# 1. Clean all outputs
rm -rf outputs data/processed & rm -rf outputs data/raw
mkdir -p outputs data/raw & mkdir data/processed

# 2. Run training
conda run -n myenv python src/train/train.py

# 3. Run inference
conda run -n myenv python src/inference/run_inference.py

# 4. Verify outputs
ls -la outputs/models/
ls -la outputs/predictions/
ls -la data/processed/