📝 SUDHAAR: Nepali Grammar Error Correction (GEC) System

SUDHAAR is a state-of-the-art Grammatical Error Correction system for the Nepali language. It moves beyond traditional sequence-to-sequence approaches, employing a novel 4-model orchestration pipeline to achieve high precision in low-resource settings.

TRY THE DEMO HERE!

---

🎯 Overview

Grammatical Error Correction (GEC) for morphologically rich, low-resource languages like Nepali is challenging. Traditional NMT models often suffer from logical "hallucinations" due to data scarcity. SUDHAAR addresses this with a Divide-and-Conquer strategy, decomposing correction into four discrete stages:

1. Detection: Sequence-Level Grammatical Error Detection (GED)
2. Localization: Binary Token Classification
3. Diagnosis: Semantic Error Type Classification (tagging)
4. Correction: Masked Language Model (MLM) based suggestion

Key Innovations

• Reliability-Aware Tagging: Distinguishes between "Reliable" (e.g., REPLACE) and "Unreliable" (e.g., DELETE) operations to prevent false corrections.
• Threshold Optimization: Empirically determined logic (e.g., $\tau=0.42$ for detection) that maximizes F1 scores for Nepali.
• Iterative Verification Loop: A "feedback loop" where the detector re-evaluates corrections up to 10 times to ensure validity.

---

🧠 Theoretical Framework

Problem Formulation

SUDHAAR breaks the GEC task mapping $\mathcal{F}{GEC}: \mathcal{S}{err} \rightarrow \mathcal{S}_{corr}$ into four sub-functions:

1. $f_1: S \rightarrow {0, 1}$ (GED: Detection)
2. $f_2: S \rightarrow {0, 1}^n$ (Token Localization)
3. $f_3: S \rightarrow \mathcal{T}^n$ (Error Type Classification, where $\mathcal{T} = {\text{DELETE}, \text{REPLACE}, \dots}$)
4. $f_4: S, \mathcal{M} \rightarrow S'$ (MLM Correction)

Loss Functions

• Stages 1 & 2 (Binary): Minimize Binary Cross-Entropy (BCE) loss. $$ \mathcal{L}{BCE} = -\frac{1}{N}\sum{i=1}^{N}\left[y_i \log(\hat{y}_i) + (1-y_i)\log(1-\hat{y}_i)\right] $$
• Stage 3 (Multi-Class): Minimize Focal Loss to handle class imbalance (e.g., rare errors vs common inflection errors). $$ \mathcal{L}_{focal} = -\alpha_t(1-p_t)^\gamma \log(p_t) $$

---

🏗️ System Architecture

SUDHAAR employs a "Fail-Fast" pipeline. If a sentence is detected as correct, it bypasses expensive correction stages.

graph TD
    Input[Input Sentence S] --> GED[Stage 1: GED Model]
    GED -- "Logits < 0.5" --> Output(Output Unchanged)
    GED -- "Logits > 0.5" --> Loc[Stage 2: Localization]
    
    Loc -- "Error Indices" --> Type[Stage 3: Error Type]
    Type --> Check{Reliable Tag?}
    
    Check -- "Yes (High Conf)" --> Det[Deterministic Fix]
    Check -- "No (Low Conf)" --> MLM[Stage 4: MLM]
    
    Det --> Candidate[Candidate S']
    MLM --> Candidate
    
    Candidate -.-> Loop{Verification Loop}
    Loop -- "Re-check" --> GED
    Loop -- "Verified / Max Iter" --> Final(Final Corrected Output)
    
    style Input fill:#f9f,stroke:#333,stroke-width:2px
    style GED fill:#bbf,stroke:#333,stroke-width:2px
    style Final fill:#bfb,stroke:#333,stroke-width:2px
    style Loop stroke-dasharray: 5 5

Loading

The 4 Fine-Tuned Models

All models are fine-tuned versions of IRIIS-RESEARCH/RoBERTa_Nepali_125M, chosen for its superior performance on NepGLUE (95.60 score).

Stage	Model	Purpose	Key Metric
1	GED	Sequence Classification	92.51% Accuracy
2	Binary Token	Token-Level Detection	0.877 F1 (@ $\tau=0.42$)
3	Error Type	7-Class Tagging	Reliability-weighted execution
4	MLM	Contextual Infilling	0.4285 Val Loss

---

🛠️ Data Preprocessing: Multi-Pass Priority Alignment

To create the training data from raw parallel corpora, we developed the Multi-Pass Priority Alignment Algorithm. Regular Levenshtein distance fails on Nepali's free word order.

1. Pass 1: Long-Range Swap Detection: Scans for tokens displaced across the sentence (SOV structure violations). Tags them as $SWAP_NEXT / $SWAP_PREV.
2. Pass 2: Morphological Merge: Identifies split/merge errors (e.g., "खाना खायो" vs "खानाखायो"). Tags: $MERGE.
3. Pass 3: Opcode-Guided Fallback: Standard diff for remaining errors.
   • Insertions $\rightarrow$ $APPEND
   • Replacements $\rightarrow$ $REPLACE (triggers MLM)

---

📊 Experiments & Results

Detailed evaluation on the Aryal et al. (2024) test set.

1. Sequence-Level GED Performance

SUDHAAR outperforms larger baseline models.

System	Backbone	Params	Accuracy
Baseline 1	NepBERTa	110M	81.73%
Baseline 2	MuRIL	237M	91.15%
SUDHAAR	IRIIS-RoBERTa	125M	92.51%

2. Threshold Optimization

Standard classification thresholds (0.5) are suboptimal for error detection due to class imbalance. We performed a grid search to find $\tau^*$.

• Optimal Threshold: 0.42
• Result: maximizes F1 score to 0.877.
• Impact: Balances Precision (0.92) with Recall (0.83).

3. Confusion Matrix (Token Level)

At $\tau=0.42$:

	Pred Correct	Pred Error
Actual Correct	11,221,962 (TN)	46,708 (FP)
Actual Error	104,651 (FN)	539,599 (TP)

4. Reliability-Aware Performance

We categorize error tags based on their individual F1 scores:

Error Type	Precision	Reliability Classification	Action
REPLACE	0.92	✅ High	Trust Model C
APPEND	0.88	✅ High	Trust Model C
SWAP	0.90	✅ High	Trust Model C
DELETE	0.65	❌ Low	Conservative / Skip
MERGE	0.55	❌ Low	Conservative / Skip

---

🚀 Installation

Prerequisites

• Python 3.8+
• (Optional) CUDA GPU for faster inference (0.3s/sent vs 1.2s/sent on CPU)

Setup

# Clone the repository
git clone https://github.com/DipeshChaudhary/SUDHAAR.git
cd SUDHAAR

# Create virtual environment
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Download Models

You can manually download the models or let the script handle it on first run.

from huggingface_hub import snapshot_download

# Models
snapshot_download(repo_id="DipeshChaudhary/roberta-nepali-sequence-ged")
snapshot_download(repo_id="DipeshChaudhary/nepali-gec-binary-detector")
snapshot_download(repo_id="DipeshChaudhary/nepali-gec-error-type-classifier")
snapshot_download(repo_id="DipeshChaudhary/nepali-mlm-guesser-finetuned-model-1")

---

🎮 Usage

Streamlit Web App

streamlit run app.py

Visit http://localhost:8501.

Python API

from src.model_loader import load_models
from src.inference import NepaliGECEngine

# 1. Initialize Engine
models = load_models()
engine = NepaliGECEngine(models)

# 2. Correct Sentence
text = "नाम मेरो दिपेश हो ।"  # Incorrect Order
result = engine.correct_sentence(text)

print(f"Original: {text}")
print(f"Corrected: {result['corrected']}")
# Output: "मेरो नाम दिपेश हो ।"

---

🔮 Future Work: Self-Synthetic Generation

Current limitations stem from rule-based synthetic training data. Our future roadmap includes Self-Synthetic Generation:

• Fine-tuning a Generative LLM (e.g., LLaMA-2) to "write like a learner".
• Generating organic, semantic errors that rule-based systems miss.
• Creating a balanced dataset to improve "Unreliable" tags like DELETE/MERGE.

---

📚 References & Acknowledgments

This work builds upon:

1. Aryal & Jaiswal (2024): For the 8.1M sentence parallel corpus.
2. IRIIS-RESEARCH: For the RoBERTa foundation model.
3. GECToR Paradigm: For the tag-based correction philosophy.

cite:

• Thapa et al. (2025). Development of Pre-Trained Transformer-based Models for the Nepali Language.
• Aryal, S., Jaiswal, A. (2024). BERT-Based Nepali Grammatical Error Detection and Correction.

---

📄 License

MIT License. See LICENSE for details.