Skip to content

Commit

Permalink
Merge pull request #78 from paketo-buildpacks/compression-support
Browse files Browse the repository at this point in the history 
Implements Paketo Utilities RFC 0001 to add compression support for native image executables
  • Loading branch information
Daniel Mikusa committed Jul 9, 2021
2 parents 52f6ae6 + 5b4096c commit 906db7b
Show file tree
Hide file tree
Showing 6 changed files with 390 additions and 20 deletions.
19 changes: 15 additions & 4 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# `gcr.io/paketo-buildpacks/native-image`

The Paketo Native Image Buildpack is a Cloud Native Buildpack that uses the [GraalVM Native Image builder][native-image] (`native-image`) to compile a standalone executable from an executable JAR.

Most users should not use this component buildpack directly and should instead use the [Paketo Java Native Image][bp/java-native-image], which provides the full set of buildpacks required to build a native image application.

## Behavior

This buildpack will participate if one the following conditions are met:

* `$BP_NATIVE_IMAGE` is set.
Expand All @@ -12,13 +14,22 @@ This buildpack will participate if one the following conditions are met:
The buildpack will do the following:

* Requests that the Native Image builder be installed by requiring `native-image-builder` in the build plan.
* If `$BP_BINARY_COMPRESSION_METHOD` is set to `upx`, requests that UPX be installed by requiring `upx` in the buildplan.
* Uses `native-image` a to build a GraalVM native image and removes existing bytecode.
* Uses `$BP_BINARY_COMPRESSION_METHOD` if set to `upx` or `gzexe` to compress the native image.

## Configuration
| Environment Variable | Description
| -------------------- | -----------
| `$BP_NATIVE_IMAGE` | Whether to build a native image from the application. Defaults to false.
| `$BP_NATIVE_IMAGE_BUILD_ARGUMENTS` | Arguments to pass to the `native-image` command.
| Environment Variable | Description |
| ---------------------------------- | --------------------------------------------------------------------------------------------- |
| `$BP_NATIVE_IMAGE` | Whether to build a native image from the application. Defaults to false. |
| `$BP_NATIVE_IMAGE_BUILD_ARGUMENTS` | Arguments to pass to the `native-image` command. |
| `$BP_BINARY_COMPRESSION_METHOD` | Compression mechanism used to reduce binary size. Options: `none` (default), `upx` or `gzexe` |

### Compression Caveats

1. Using `gzexe` if you intend to run your application on the Paketo Tiny image is not currently supported. The `gzexe` utility will compress your executable into what is a shell script, which executes and extracts the actual binary to a temp location. This process requires `/bin/sh` and that is not in the Tiny image. If you try using `gzexe` with the Tiny stack, it'll build OK but fail to run saying a file is missing.

2. Using `upx` will create a compressed executable that fails to run on M1 Macs. There is at the time of writing a bug in the emulation layer used by Docker on M1 Macs that is triggered when you try to run amd64 executable that has been compressed using `upx`. This is a known issue and will hopefully be patched in a future release.

## License
This buildpack is released under version 2.0 of the [Apache License][a].
Expand Down
15 changes: 14 additions & 1 deletion native/build.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ import (
const (
ConfigNativeImageArgs = "BP_NATIVE_IMAGE_BUILD_ARGUMENTS"
DeprecatedConfigNativeImageArgs = "BP_BOOT_NATIVE_IMAGE_BUILD_ARGUMENTS"
CompressorUpx = "upx"
CompressorGzexe = "gzexe"
CompressorNone = "none"
)

type Build struct {
Expand Down Expand Up @@ -66,7 +69,17 @@ func (b Build) Build(context libcnb.BuildContext) (libcnb.BuildResult, error) {
}
}

n, err := NewNativeImage(context.Application.Path, args, manifest, context.StackID)
compressor, ok := cr.Resolve(BinaryCompressionMethod)
if !ok {
compressor = CompressorNone
} else if ok {
if compressor != CompressorUpx && compressor != CompressorGzexe && compressor != CompressorNone {
b.warn(fmt.Sprintf("Requested compression method [%s] is unknown, no compression will be performed", compressor))
compressor = CompressorNone
}
}

n, err := NewNativeImage(context.Application.Path, args, compressor, manifest, context.StackID)
if err != nil {
return libcnb.BuildResult{}, fmt.Errorf("unable to create native image layer\n%w", err)
}
Expand Down
35 changes: 25 additions & 10 deletions native/detect.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,13 @@ import (
const (
ConfigNativeImage = "BP_NATIVE_IMAGE"
DeprecatedConfigNativeImage = "BP_BOOT_NATIVE_IMAGE"
BinaryCompressionMethod = "BP_BINARY_COMPRESSION_METHOD"

PlatEntryNativeImage = "native-image-application"
PlanEntryNativeImage = "native-image-application"
PlanEntryNativeImageBuilder = "native-image-builder"
PlanEntryJVMApplication = "jvm-application"
PlanEntrySpringBoot = "spring-boot"
PlanEntryUpx = "upx"
)

type Detect struct{}
Expand All @@ -48,7 +50,7 @@ func (d Detect) Detect(context libcnb.DetectContext) (libcnb.DetectResult, error
{
Provides: []libcnb.BuildPlanProvide{
{
Name: PlatEntryNativeImage,
Name: PlanEntryNativeImage,
},
},
Requires: []libcnb.BuildPlanRequire{
Expand All @@ -68,7 +70,7 @@ func (d Detect) Detect(context libcnb.DetectContext) (libcnb.DetectResult, error
{
Provides: []libcnb.BuildPlanProvide{
{
Name: PlatEntryNativeImage,
Name: PlanEntryNativeImage,
},
},
Requires: []libcnb.BuildPlanRequire{
Expand All @@ -86,20 +88,33 @@ func (d Detect) Detect(context libcnb.DetectContext) (libcnb.DetectResult, error

if ok, err := d.nativeImageEnabled(cr); err != nil {
return libcnb.DetectResult{}, err
} else if !ok {
// still participates if a downstream buildpack requires native-image-applications
return result, nil
} else if ok {
for i := range result.Plans {
result.Plans[i].Requires = append(result.Plans[i].Requires, libcnb.BuildPlanRequire{
Name: PlanEntryNativeImage,
})
}
}

for i := range result.Plans {
result.Plans[i].Requires = append(result.Plans[i].Requires, libcnb.BuildPlanRequire{
Name: PlatEntryNativeImage,
})
if d.upxCompressionEnabled(cr) {
for i := range result.Plans {
result.Plans[i].Requires = append(result.Plans[i].Requires, libcnb.BuildPlanRequire{
Name: PlanEntryUpx,
})
}
}

// still participates if a downstream buildpack requires native-image-applications or upx
return result, nil
}

func (d Detect) upxCompressionEnabled(cr libpak.ConfigurationResolver) bool {
if val, ok := cr.Resolve(BinaryCompressionMethod); ok {
return val == CompressorUpx
}
return false
}

func (d Detect) nativeImageEnabled(cr libpak.ConfigurationResolver) (bool, error) {
if val, ok := cr.Resolve(ConfigNativeImage); ok {
enable, err := strconv.ParseBool(val)
Expand Down
240 changes: 240 additions & 0 deletions native/detect_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,246 @@ func testDetect(t *testing.T, context spec.G, it spec.S) {
})
})

context("$BP_BINARY_COMPRESSION_METHOD", func() {
it.Before(func() {
Expect(os.Setenv("BP_NATIVE_IMAGE", "true")).To(Succeed())
})

it.After(func() {
Expect(os.Unsetenv("BP_NATIVE_IMAGE")).To(Succeed())
})

context("upx", func() {
it.Before(func() {
Expect(os.Setenv("BP_BINARY_COMPRESSION_METHOD", "upx")).To(Succeed())
})

it.After(func() {
Expect(os.Unsetenv("BP_BINARY_COMPRESSION_METHOD")).To(Succeed())
})

it("requires upx", func() {
Expect(detect.Detect(ctx)).To(Equal(libcnb.DetectResult{
Pass: true,
Plans: []libcnb.BuildPlan{
{
Provides: []libcnb.BuildPlanProvide{
{Name: "native-image-application"},
},
Requires: []libcnb.BuildPlanRequire{
{
Name: "native-image-builder",
},
{
Name: "jvm-application",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "spring-boot",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "native-image-application",
},
{
Name: "upx",
},
},
},
{
Provides: []libcnb.BuildPlanProvide{
{Name: "native-image-application"},
},
Requires: []libcnb.BuildPlanRequire{
{
Name: "native-image-builder",
},
{
Name: "jvm-application",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "native-image-application",
},
{
Name: "upx",
},
},
},
},
}))
})
})

context("gzexe", func() {
it.Before(func() {
Expect(os.Setenv("BP_BINARY_COMPRESSION_METHOD", "gzexe")).To(Succeed())
})

it.After(func() {
Expect(os.Unsetenv("BP_BINARY_COMPRESSION_METHOD")).To(Succeed())
})

it("no additional provides or requires", func() {
Expect(detect.Detect(ctx)).To(Equal(libcnb.DetectResult{
Pass: true,
Plans: []libcnb.BuildPlan{
{
Provides: []libcnb.BuildPlanProvide{
{Name: "native-image-application"},
},
Requires: []libcnb.BuildPlanRequire{
{
Name: "native-image-builder",
},
{
Name: "jvm-application",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "spring-boot",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "native-image-application",
},
},
},
{
Provides: []libcnb.BuildPlanProvide{
{Name: "native-image-application"},
},
Requires: []libcnb.BuildPlanRequire{
{
Name: "native-image-builder",
},
{
Name: "jvm-application",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "native-image-application",
},
},
},
},
}))
})
})

context("none", func() {
it.Before(func() {
Expect(os.Setenv("BP_BINARY_COMPRESSION_METHOD", "none")).To(Succeed())
})

it.After(func() {
Expect(os.Unsetenv("BP_BINARY_COMPRESSION_METHOD")).To(Succeed())
})

it("no additional provides or requires", func() {
Expect(detect.Detect(ctx)).To(Equal(libcnb.DetectResult{
Pass: true,
Plans: []libcnb.BuildPlan{
{
Provides: []libcnb.BuildPlanProvide{
{Name: "native-image-application"},
},
Requires: []libcnb.BuildPlanRequire{
{
Name: "native-image-builder",
},
{
Name: "jvm-application",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "spring-boot",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "native-image-application",
},
},
},
{
Provides: []libcnb.BuildPlanProvide{
{Name: "native-image-application"},
},
Requires: []libcnb.BuildPlanRequire{
{
Name: "native-image-builder",
},
{
Name: "jvm-application",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "native-image-application",
},
},
},
},
}))
})
})

context("not a supported method", func() {
it.Before(func() {
Expect(os.Setenv("BP_BINARY_COMPRESSION_METHOD", "foo")).To(Succeed())
})

it.After(func() {
Expect(os.Unsetenv("BP_BINARY_COMPRESSION_METHOD")).To(Succeed())
})

it("ignore and no additional provides or requires", func() {
Expect(detect.Detect(ctx)).To(Equal(libcnb.DetectResult{
Pass: true,
Plans: []libcnb.BuildPlan{
{
Provides: []libcnb.BuildPlanProvide{
{Name: "native-image-application"},
},
Requires: []libcnb.BuildPlanRequire{
{
Name: "native-image-builder",
},
{
Name: "jvm-application",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "spring-boot",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "native-image-application",
},
},
},
{
Provides: []libcnb.BuildPlanProvide{
{Name: "native-image-application"},
},
Requires: []libcnb.BuildPlanRequire{
{
Name: "native-image-builder",
},
{
Name: "jvm-application",
Metadata: map[string]interface{}{"native-image": true},
},
{
Name: "native-image-application",
},
},
},
},
}))
})
})
})

context("$BP_BOOT_NATIVE_IMAGE", func() {
it.Before(func() {
Expect(os.Setenv("BP_BOOT_NATIVE_IMAGE", "true")).To(Succeed())
Expand Down
Loading

0 comments on commit 906db7b

Please sign in to comment.