Scripts: Add guestagent and trivy into wsl-distro tarball instead of installing at runtime #7145

mook-as · 2024-07-04T23:55:14Z

Convert the go utility building to Dependencys, and implement crude dependency tracking so that they can depend on each other.
- Note that GoDependency can take an array as the first argument, so new GoDependency(['networking', 'cmd', 'network']) is valid.
Embed the guest agent and trivy into the WSL distro. Note that we do not change trivy for Lima-based platforms for now.
Mark some files as not needing to be installed (this was from when I was ignoring specific files; I've changed to put the files in a new staging directory instead so not packaging them is easier).

This allows dependencies to depend on other dependencies, including across platforms; we intend to allow WSLDistro to depend on Linux bits. Signed-off-by: Mark Yen <[email protected]>

This makes the go utilities act like dependencies, so we can use them as dependencies of other dependencies. Signed-off-by: Mark Yen <[email protected]>

This embeds rancher-desktop-guestagent as part of the WSL distro on post- install, so that we do not need to copy it into the VM at run time. Signed-off-by: Mark Yen <[email protected]>

Signed-off-by: Mark Yen <[email protected]>

- We don't actually support localization - There is no point in including source maps, we don't ship the sources - Support merging file lists Signed-off-by: Mark Yen <[email protected]>

Nino-K · 2024-07-05T18:02:12Z

scripts/dependencies/go-source.ts

+    const outFile = this.outFile(context);
+
+    console.log(`Building go utility \x1B[1;33;40m${ this.name }\x1B[0m from ${ sourceDir } to ${ outFile }...`);
+    await simpleSpawn('go', ['build', '-ldflags', '-s -w', '-o', outFile, '.'], {


Do we need to handle any potential errors here?

Any errors that return a non-zero exit code would throw an exception here, so it should be fine.

Nino-K · 2024-07-05T18:19:13Z

scripts/dependencies/tar-archives.ts

+
+  async download(context: DownloadContext): Promise<void> {
+    // Build the extension proxy image.
+    const workDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'rd-build-rdx-pf-'));


Does the fs.promise methods also need error handling, I am just confirming.

No, because they throw exceptions (which will fall over correctly).

Nino-K · 2024-07-08T15:46:27Z

scripts/dependencies/tar-archives.ts

+      const { size } = await fs.promises.stat(fromPath);
+      const inputFile = fs.createReadStream(fromPath);
+
+      console.log(`WSL Distro: Adding ${ fromPath } to ${ name }...`);


nit; I noticed on line 172 it is spelled WSLDistro and here is WSL Distro, we should probably just use one of them to unify the logs.

Nino-K · 2024-07-08T15:48:52Z

scripts/dependencies/go-source.ts

+
+  override environment(context: DownloadContext) {
+    return {
+      ...super.environment(context),


Curious if we need GOOS here?

super.environment() (line 33) sets GOOS.

Nino-K · 2024-07-08T15:56:51Z

scripts/package.ts

@@ -127,6 +131,23 @@ class Builder {
      break;
    }

+    // If we have files (resp. extraFiles / extraResources) both at the top


Would it be clearer to change this to

// When there are files (e.g., extraFiles or extraResources) specified at both // the top-level and platform-specific levels, we need to combine them // and place the combined list at the top level. This approach enables us to have // platform-specific exclusions, since the two lists are initially processed // separately and then merged together afterward.

Thanks, I'll just grab your text wholesale.

This rewords the comment describing how we need to merge the files lists for electron-builder because they way they do merging is a union of the two resultant sets instead of calculating a set based on a union of the rules. Signed-off-by: Mark Yen <[email protected]>

This should make it easier to handle custom paths / subcommands. For example, we may want: new goUtils.GoDependency('networking/cmd/vm', { outputPath: 'staging/vm-switch'}) Signed-off-by: Mark Yen <[email protected]>

mook-as

I updated GoDependency to better support multiple commands in one package; see the last commit message for details.

mook-as · 2024-07-08T17:01:25Z

scripts/dependencies/go-source.ts

+    const outFile = this.outFile(context);
+
+    console.log(`Building go utility \x1B[1;33;40m${ this.name }\x1B[0m from ${ sourceDir } to ${ outFile }...`);
+    await simpleSpawn('go', ['build', '-ldflags', '-s -w', '-o', outFile, '.'], {


Any errors that return a non-zero exit code would throw an exception here, so it should be fine.

mook-as · 2024-07-08T17:02:01Z

scripts/dependencies/go-source.ts

+
+  override environment(context: DownloadContext) {
+    return {
+      ...super.environment(context),


super.environment() (line 33) sets GOOS.

mook-as · 2024-07-08T17:02:30Z

scripts/dependencies/tar-archives.ts

+
+  async download(context: DownloadContext): Promise<void> {
+    // Build the extension proxy image.
+    const workDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'rd-build-rdx-pf-'));


No, because they throw exceptions (which will fall over correctly).

mook-as · 2024-07-08T17:06:51Z

scripts/package.ts

@@ -127,6 +131,23 @@ class Builder {
      break;
    }

+    // If we have files (resp. extraFiles / extraResources) both at the top


Thanks, I'll just grab your text wholesale.

mook-as added 5 commits July 4, 2024 14:14

Scripts: Implement transitive dependencies

22d36cf

This allows dependencies to depend on other dependencies, including across platforms; we intend to allow WSLDistro to depend on Linux bits. Signed-off-by: Mark Yen <[email protected]>

Postinstall: Build go utilities as a dependency

c5b03d0

This makes the go utilities act like dependencies, so we can use them as dependencies of other dependencies. Signed-off-by: Mark Yen <[email protected]>

Postinstall: embed guest agent in WSL distro

69c9752

This embeds rancher-desktop-guestagent as part of the WSL distro on post- install, so that we do not need to copy it into the VM at run time. Signed-off-by: Mark Yen <[email protected]>

Postinstall: Also embed trivy in wsl-distro

fe76137

Signed-off-by: Mark Yen <[email protected]>

Installer: Remove unneeded files

759acc1

- We don't actually support localization - There is no point in including source maps, we don't ship the sources - Support merging file lists Signed-off-by: Mark Yen <[email protected]>

gunamata requested a review from Nino-K July 5, 2024 16:51

Nino-K reviewed Jul 8, 2024

View reviewed changes

mook-as added 2 commits July 8, 2024 10:13

postinstall: GoDependency: refactor

0c3e88b

This should make it easier to handle custom paths / subcommands. For example, we may want: new goUtils.GoDependency('networking/cmd/vm', { outputPath: 'staging/vm-switch'}) Signed-off-by: Mark Yen <[email protected]>

mook-as commented Jul 8, 2024

View reviewed changes

mook-as requested a review from Nino-K July 8, 2024 21:14

Nino-K approved these changes Jul 11, 2024

View reviewed changes

Nino-K merged commit e07b22e into rancher-sandbox:main Jul 11, 2024
15 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scripts: Add guestagent and trivy into wsl-distro tarball instead of installing at runtime #7145

Scripts: Add guestagent and trivy into wsl-distro tarball instead of installing at runtime #7145

mook-as commented Jul 4, 2024

Nino-K Jul 5, 2024

mook-as Jul 8, 2024

Nino-K Jul 5, 2024

mook-as Jul 8, 2024

Nino-K Jul 8, 2024

Nino-K Jul 8, 2024

mook-as Jul 8, 2024

Nino-K Jul 8, 2024

mook-as Jul 8, 2024

mook-as left a comment

mook-as Jul 8, 2024

mook-as Jul 8, 2024

mook-as Jul 8, 2024

mook-as Jul 8, 2024

Scripts: Add guestagent and trivy into wsl-distro tarball instead of installing at runtime #7145

Scripts: Add guestagent and trivy into wsl-distro tarball instead of installing at runtime #7145

Conversation

mook-as commented Jul 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mook-as left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment