Skip to content

Conversation

haroondilshad
Copy link

Fix .devpodignore Pattern Matching Bug

Problem

The .devpodignore file was not properly excluding files during workspace uploads. Despite having patterns like **/node_modules or *.log, the upload size remained unexpectedly large because the exclusion logic in pkg/extract/compress.go was using simple strings.HasPrefix matching instead of proper glob pattern matching.

Root Cause

The isExcluded function in pkg/extract/compress.go was performing basic string prefix checks rather than using the github.com/moby/patternmatcher library that properly handles .devpodignore syntax (which follows .dockerignore conventions).

Solution

  • Enhanced Archiver struct: Added patternMatcher field to store compiled patterns
  • Updated NewArchiver: Initialize pattern matcher when exclusion patterns are provided
  • Fixed isExcluded function:
    • Use patternMatcher.MatchesOrParentMatches() for proper glob pattern evaluation
    • Handle cross-platform path normalization with filepath.ToSlash
    • Maintain backwards compatibility with fallback to old prefix matching behavior

What This Enables

  • Recursive wildcards: **/node_modules correctly ignores nested node_modules directories
  • File extensions: *.log, *.tmp properly exclude files by extension
  • Complex patterns: **/.git, **/dist, *.{png,jpg} work as expected
  • Monorepo support: Patterns handle nested project structures correctly
  • Character classes: [!.]*/ and other advanced patterns are supported

Testing

Added comprehensive unit tests in pkg/extract/compress_test.go following the project's testing methodology:

  • TestArchiver_isExcluded: Table-driven tests for various pattern scenarios
  • TestWriteTarExclude_PatternMatching: End-to-end tar exclusion verification
  • TestWriteTarExclude_BackwardsCompatibility: Ensures existing behavior is preserved
  • TestNewArchiver_PatternMatcherCreation: Tests pattern matcher initialization

Impact

This fix should significantly reduce workspace upload times and sizes for users with properly configured .devpodignore files, especially in monorepo setups with patterns like **/node_modules, **/dist, etc.

Backwards Compatibility

The implementation includes a fallback mechanism that preserves the old strings.HasPrefix behavior if pattern matching fails, ensuring no regressions for existing setups.

Replace simple string prefix matching with github.com/moby/patternmatcher
in pkg/extract/compress.go to enable proper glob pattern support in
.devpodignore files.

Fixes:
- Recursive wildcards (e.g., **/node_modules) now work correctly
- File extension patterns (e.g., *.log) properly exclude files
- Complex patterns and character classes are supported
- Monorepo support with nested directory exclusions

Maintains backwards compatibility through fallback mechanism.

Adds comprehensive unit tests following project testing methodology.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant