Linear Regression from Scratch

This project implements Multiple Linear Regression using the Ordinary Least Squares (OLS) method from scratch. Instead of relying on libraries like scikit-learn for the core mathematical logic, this implementation uses custom Python functions for matrix operations and regression analysis.

🧠 Mathematical Foundation

The core of this project is the manual implementation of matrix-based Linear Regression. Key functions implemented include:

• Matrix Operations: transposeMatrix, multiple2Matrixs, inverse (using Gauss-Jordan elimination with an augmented matrix), and norm.
• OLS Solver: $\theta = (A^T A)^{-1} A^T y$
• Evaluation: Custom calculateRMSE (Root Mean Square Error) function.

---

🍇 Problem A: Wine Quality Prediction

File: Problem_A.ipynb | Dataset: wine.csv (1,599 samples)

Predicting wine quality scores (0-10) using 11 chemical features.

🔢 Evolution of Accuracy:

• Baseline (All Features, Linear): RMSE 22.09
• Best Single Predictor (Alcohol): RSS 24.18
• Final Optimized Model (Interaction + Squared Terms): RSS 19.13
  • Result: ~13% improvement in fit by capturing complex chemical interactions.

💡 Key Finding: Alcohol content is the single most powerful predictor of wine quality in this dataset.

---

🏥 Problem B: Diabetes Risk Prediction

File: Problem_B.ipynb | Dataset: NHANES (Age Prediction Subset)

Predicting diabetes indicators using physiological and lab metrics.

📊 Model Performance Comparison:

• Standard Linear Model (7 Features): RSS 8.8139
• Top Single Predictor (Insulin): RSS 8.8389
• Quadratic Optimization (7 Features Squared): RSS 8.8102
  • Result: Achieved the most precise fit using non-linear metabolic relationships.

💡 Key Finding: Insulin (LBXIN) and Age (RIDAGEYR) were identified as the primary drivers of diabetes risk markers.

---

🏆 Key Outcomes & Contributions

This project successfully demonstrates the effectiveness of custom-built linear regression models across two distinct domains.

📝 Key Contributions

• Custom Linear Algebra Library: Developed a fully functional Python toolkit for matrix operations (Multiplication, Transposition, and Gauss-Jordan Inversion) capable of solving OLS problems without high-level framework abstractions.
• Comparative Modeling: Conducted a systematic comparison between Simple Linear, Quadratic, and Interaction-based models, quantifying the importance of non-linear feature mapping.
• Predictor Identification: Successfully identified critical biological and chemical predictors (Alcohol for wine; Insulin and Age for diabetes risk).

📊 Project Outcomes

• Problem A (Wine Quality): Optimized the model using interaction terms and squared features to achieve a final RSS of 19.13.
• Problem B (Diabetes Prediction): Achieved the most accurate fit using a 7-feature Quadratic model, resulting in a minimum RSS of 8.81.

---

🛠️ Tools Used

• Python: Core programming language.
• Pandas: Data manipulation and dataset loading.
• Matplotlib & Seaborn: Data visualization and correlation heatmaps.

🚀 How to Run

1. Ensure you have the required datasets in the same directory:
   • wine.csv
   • NHANES_age_prediction.csv
2. Install dependencies: pip install pandas matplotlib seaborn
3. Open and run the Jupyter Notebooks: Problem_A.ipynb or Problem_B.ipynb.