Add comprehensive Python testing infrastructure with Poetry

[llbbl]

About UnitSeeker

Hi! This PR is part of the UnitSeeker project, a human-guided initiative to help Python repositories establish testing infrastructure.

Key points:

• Human-approved: Every PR is manually approved before work begins
• Semi-automated with oversight: Created and controlled via a homegrown wrapper around Claude Code with human quality control
• Infrastructure only: This PR intentionally contains only the testing setup without actual unit tests
• Your repository, your rules: Feel free to modify, reject, or request changes - all constructive feedback is welcome
• Follow-up support: All responses and discussions are personally written, not automated

Learn more about the project and see the stats on our progress at https://unitseeker.llbbl.com/

---

Summary

This PR adds a comprehensive testing infrastructure to the AI Scientist project, providing a solid foundation for writing and running tests. The setup uses Poetry for dependency management and pytest as the testing framework.

Changes Made

Package Management

• Poetry Configuration: Created pyproject.toml with Poetry setup
  • Migrated all dependencies from requirements.txt to Poetry format
  • Configured Python 3.11+ requirement
  • Organized dependencies into production and development groups

Testing Dependencies

Added the following testing tools as development dependencies:

• pytest (v8.0.0+) - Main testing framework
• pytest-cov (v6.0.0+) - Coverage reporting
• pytest-mock (v3.14.0+) - Enhanced mocking utilities

Directory Structure

Created organized test directory layout:

tests/
├── __init__.py
├── conftest.py           # Shared fixtures
├── test_infrastructure.py  # Infrastructure validation tests
├── unit/
│   └── __init__.py
└── integration/
    └── __init__.py

Testing Configuration

Configured pytest in pyproject.toml with:

• Test Discovery: Automatic detection of test files (test_*.py, *_test.py)
• Custom Markers:
  • @pytest.mark.unit - Unit tests for isolated components
  • @pytest.mark.integration - Integration tests for multiple components
  • @pytest.mark.slow - Tests with longer execution times
• Coverage Settings:
  • HTML and XML report generation
  • 80% coverage target (commented out until tests are written)
  • Intelligent exclusion of test files and common patterns
  • Terminal output showing missing coverage

Shared Fixtures

Created comprehensive shared fixtures in tests/conftest.py:

• temp_dir - Temporary directory for test files
• temp_file - Temporary file creation
• mock_config - Mock configuration dictionaries
• sample_data_dir - Sample data directory with test files
• mock_env_vars - Environment variable mocking
• capture_logs - Log capture for testing
• reset_environment - Automatic cleanup between tests

Validation Tests

Added tests/test_infrastructure.py with tests to verify:

• Python version requirements
• Pytest functionality and markers
• Directory structure
• Fixture functionality
• pytest-mock integration

Configuration Updates

• Poetry Scripts: Added test and tests commands for convenience
• .gitignore: Added testing artifacts (.pytest_cache/, .coverage, htmlcov/, coverage.xml) and Claude Code files

Running Tests

Install Dependencies

poetry install

Run All Tests

poetry run test
# or
poetry run tests

Run with Specific Options

# Run only unit tests
poetry run pytest -m unit

# Run with verbose output
poetry run pytest -v

# Run specific test file
poetry run pytest tests/test_infrastructure.py

# Run without coverage
poetry run pytest --no-cov

# Generate coverage report
poetry run pytest --cov=ai_scientist --cov-report=html

View Coverage Report

After running tests with coverage, open htmlcov/index.html in your browser to see detailed coverage information.

Notes

Coverage Threshold

The 80% coverage threshold is currently commented out in the configuration to allow the infrastructure to be validated without existing tests. Once unit tests are written, uncomment line 97 in pyproject.toml:

# fail_under = 80

Poetry Lock File

The poetry.lock file has been generated and should be committed to ensure reproducible builds across all environments.

Next Steps

This PR provides the testing infrastructure only. Developers can now:

1. Write unit tests in tests/unit/
2. Write integration tests in tests/integration/
3. Use the provided fixtures from conftest.py
4. Run tests with poetry run test
5. Monitor coverage with the generated reports

Validation

All infrastructure validation tests pass successfully:

• ✅ 15 tests passing
• ✅ All fixtures working correctly
• ✅ pytest-mock integration verified
• ✅ Coverage reporting functional
• ✅ Both test and tests commands operational

---

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]