Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
247 lines (191 loc) · 6.76 KB

CONTRIBUTING.md

File metadata and controls

247 lines (191 loc) · 6.76 KB
Raw

Contributing to Synergistic Self-Correction (S2C)

We welcome contributions to the S2C framework! This guide will help you get started with contributing to our research project.

🚀 Quick Start

  1. Fork the repository
  2. Clone your fork: git clone https://github.com/your-username/Self-Correcting-LLM-Research.git
  3. Create a branch: git checkout -b feature/your-feature-name
  4. Make your changes
  5. Submit a pull request

🧪 Development Setup

Prerequisites

  • Python 3.8+
  • CUDA-compatible GPU (recommended for training)
  • 16GB+ RAM

Installation

# Clone the repository
git clone https://github.com/your-username/Self-Correcting-LLM-Research.git
cd Self-Correcting-LLM-Research

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Install development dependencies
pip install -r requirements-dev.txt

# Install pre-commit hooks
pre-commit install

📝 Contribution Types

🐛 Bug Reports

  • Use the bug report template
  • Include system information (OS, Python version, GPU)
  • Provide minimal reproducible example
  • Include error logs and stack traces

✨ Feature Requests

  • Use the feature request template
  • Explain the motivation and use case
  • Provide implementation suggestions if possible
  • Consider backward compatibility

🔬 Research Improvements

  • New training methodologies
  • Novel evaluation metrics
  • Dataset improvements
  • Architecture modifications

📖 Documentation

  • API documentation improvements
  • Tutorial and example additions
  • README enhancements
  • Research methodology clarifications

🛠️ Development Guidelines

Code Style

We use black, isort, and flake8 for code formatting:

# Format code
black src/ experiments/ scripts/
isort src/ experiments/ scripts/

# Check linting
flake8 src/ experiments/ scripts/

Testing

# Run all tests
pytest tests/

# Run specific test file
pytest tests/test_s2c_model.py

# Run with coverage
pytest --cov=src tests/

Documentation

  • Use Google-style docstrings
  • Include type hints for all functions
  • Add examples for complex functions
  • Update README.md for new features

Commit Messages

Follow conventional commits:

feat: add hierarchical attention mechanism
fix: resolve memory leak in training loop
docs: update installation instructions
test: add unit tests for critic module
refactor: simplify reward model architecture

🧪 Experimental Guidelines

Adding New Experiments

  1. Create experiment file in experiments/
  2. Use consistent configuration management
  3. Log results with wandb/tensorboard
  4. Include ablation studies where appropriate
  5. Document hyperparameters and setup

Dataset Contributions

  1. Follow existing data processing patterns
  2. Include data validation and statistics
  3. Ensure proper train/val/test splits
  4. Document data sources and preprocessing

Model Improvements

  1. Maintain backward compatibility when possible
  2. Include theoretical justification
  3. Benchmark against existing methods
  4. Provide clear performance comparisons

📊 Research Standards

Reproducibility

  • Set random seeds everywhere
  • Document all hyperparameters
  • Include system/hardware specifications
  • Provide detailed setup instructions

Evaluation

  • Use established benchmarks (GSM8K, MATH, etc.)
  • Include statistical significance testing
  • Report confidence intervals
  • Compare against relevant baselines

Documentation

  • Include mathematical formulations
  • Explain algorithmic choices
  • Provide intuitive explanations
  • Reference related work appropriately

🎯 Priority Areas

We're particularly interested in contributions in these areas:

High Priority

  • Scaling studies: Behavior with larger models (70B+)
  • Domain transfer: Extension to other reasoning domains
  • Efficiency improvements: Faster inference methods
  • Error analysis: Better understanding of failure modes

Medium Priority

  • Architecture variants: Alternative self-correction designs
  • Training improvements: Better reward modeling approaches
  • Evaluation metrics: Novel assessment methods
  • Visualization tools: Better analysis and debugging tools

Research Collaborations

  • Multi-modal reasoning: Extending S2C to vision-language tasks
  • Human-in-the-loop: Interactive correction systems
  • Theoretical analysis: Formal guarantees and bounds
  • Real-world applications: Domain-specific adaptations

📋 Pull Request Process

Before Submitting

  • Code follows style guidelines
  • Tests pass locally
  • Documentation is updated
  • Commit messages are descriptive
  • Branch is up-to-date with main

PR Template

Please include:

  • Description: What does this PR do?
  • Motivation: Why is this change needed?
  • Testing: How was this tested?
  • Checklist: Complete the PR checklist
  • Breaking Changes: Any backward compatibility issues?

Review Process

  1. Automated checks: CI/CD pipeline runs
  2. Code review: Maintainers review changes
  3. Testing: Reviewers test functionality
  4. Discussion: Address feedback and iterate
  5. Merge: Approved PRs are merged

🔍 Code Review Guidelines

For Contributors

  • Be responsive to feedback
  • Keep PRs focused and small
  • Write clear commit messages
  • Test your changes thoroughly

For Reviewers

  • Be constructive and specific
  • Focus on code quality and research validity
  • Check for reproducibility
  • Consider performance implications

📞 Getting Help

Communication Channels

  • GitHub Issues: Bug reports and feature requests
  • GitHub Discussions: General questions and ideas
  • Email: patel292@gannon.edu for research collaboration

Resources

  • Paper: Read our arXiv preprint for background
  • Documentation: Check the Wiki for detailed guides
  • Examples: Look at existing experiments for patterns
  • Issues: Search existing issues before creating new ones

🏆 Recognition

Contributors

All contributors will be acknowledged in:

  • Repository contributors list
  • Future paper acknowledgments (for significant contributions)
  • Release notes for their contributions

Types of Contributions Recognized

  • Code contributions
  • Documentation improvements
  • Bug reports and testing
  • Research ideas and discussions
  • Community support and help

📜 License

By contributing, you agree that your contributions will be licensed under the MIT License.

🙏 Thank You

Thank you for contributing to the advancement of self-correcting AI systems! Your contributions help make LLMs more reliable and trustworthy.

For questions about this guide or the contribution process, please open an issue or contact the maintainers directly.