fix: implement proper pattern matching for .devpodignore files #1916
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix .devpodignore Pattern Matching Bug
Problem
The
.devpodignore
file was not properly excluding files during workspace uploads. Despite having patterns like**/node_modules
or*.log
, the upload size remained unexpectedly large because the exclusion logic inpkg/extract/compress.go
was using simplestrings.HasPrefix
matching instead of proper glob pattern matching.Root Cause
The
isExcluded
function inpkg/extract/compress.go
was performing basic string prefix checks rather than using thegithub.com/moby/patternmatcher
library that properly handles.devpodignore
syntax (which follows.dockerignore
conventions).Solution
Archiver
struct: AddedpatternMatcher
field to store compiled patternsNewArchiver
: Initialize pattern matcher when exclusion patterns are providedisExcluded
function:patternMatcher.MatchesOrParentMatches()
for proper glob pattern evaluationfilepath.ToSlash
What This Enables
**/node_modules
correctly ignores nestednode_modules
directories*.log
,*.tmp
properly exclude files by extension**/.git
,**/dist
,*.{png,jpg}
work as expected[!.]*/
and other advanced patterns are supportedTesting
Added comprehensive unit tests in
pkg/extract/compress_test.go
following the project's testing methodology:TestArchiver_isExcluded
: Table-driven tests for various pattern scenariosTestWriteTarExclude_PatternMatching
: End-to-end tar exclusion verificationTestWriteTarExclude_BackwardsCompatibility
: Ensures existing behavior is preservedTestNewArchiver_PatternMatcherCreation
: Tests pattern matcher initializationImpact
This fix should significantly reduce workspace upload times and sizes for users with properly configured
.devpodignore
files, especially in monorepo setups with patterns like**/node_modules
,**/dist
, etc.Backwards Compatibility
The implementation includes a fallback mechanism that preserves the old
strings.HasPrefix
behavior if pattern matching fails, ensuring no regressions for existing setups.