Skip to content

pkg/architecture: Add architecture support validation#1783

Open
DaliborKr wants to merge 6 commits intocontainers:mainfrom
DaliborKr:pr04-arch-validation
Open

pkg/architecture: Add architecture support validation#1783
DaliborKr wants to merge 6 commits intocontainers:mainfrom
DaliborKr:pr04-arch-validation

Conversation

@DaliborKr
Copy link
Copy Markdown
Collaborator

This is PR 4/10 in a series adding cross-architecture container support using QEMU and binfmt_misc.

Depends on: #1782 (cmd/create, pkg/utils: Add spinner helpers and distro image matching)
Please review #1782 first. The new commit in this PR is:

  • pkg/architecture: Add architecture support validation

Summary

Before creating or initializing a cross-architecture container, the system must be verified to have the required QEMU emulator and binfmt_misc registration in place, and provide a user with an appropriate message. Without these checks, container creation would silently fail or produce broken containers.

Add:

  • Host-side validation (IsArchSupportedOnCreation()) that locates a statically linked QEMU binary via exec.LookPath and verifies that a matching binfmt_misc registration exists. Returns the path to the QEMU binary for use during container initialization

  • Container-side validation (IsArchSupportedOnInitialization()) that verifies QEMU support from inside the container using the interpreter path passed from the host, with fallback to the standard host-mounted locations under /run/host/usr/bin/

  • ELF binary inspection (isStaticallyLinkedELF()) that confirms the QEMU binary is statically linked, because a dynamically linked one would cause the kernel to attempt to resolve its host-native shared libraries (such as libc.so) within the container, resulting in an immediate crash.

The existing RunContextWithExitCode() wraps all errors from
exec.Command in generic "failed to invoke" messages, which prevents
callers from distinguishing between actual error types.

Add RunContextWithExitCode2() and RunWithExitCode2() that return
errors with their original types intact, including for ExitError.
This allows callers to use errors.Is() and errors.As() to handle
specific failure modes. For example, detecting a missing skopeo binary
(exec.ErrNotFound) or an ENOEXEC error from inside non native
containers, when an emulator is not set correctly.

These new functions are meant to replace its original versions in the
future.

containers#1780

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
In /src/cmd/create.go, the same pattern of spinner creation and
nil-safe stopping is repeated.

Extract this into startSpinner() and stopSpinner() helper functions so
that future callers can use spinners without duplicating the code.
Replace the existing inline spinner code in createContainer() and
pullImage() with calls to these new helpers.

containers#1781

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
…atching

Add IsSupportedDistroImage(), which iterates over all supported distros
and checks if the image basename matches any of them. This will be used
by the architecture resolution code to decide whether to parse
architecture suffixes from image tags, as this should be done only for
natively supported images [1].

[1] Toolbx supported distributions: https://containertoolbx.org/distros/

containers#1781

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
@DaliborKr DaliborKr requested a review from debarshiray as a code owner April 23, 2026 12:39
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new architecture package to manage cross-architecture support, including binfmt_misc registration and QEMU emulation checks. It also refactors the spinner logic into reusable helper functions and adds host architecture detection to the root command. Feedback focuses on improving maintainability by avoiding generic function suffixes like '2', ensuring consistent logging by using logrus instead of direct stderr writes, and adding validation for architecture IDs to prevent confusing error messages.

Comment thread src/pkg/shell/shell.go
Comment on lines +84 to +129
func RunContextWithExitCode2(ctx context.Context,
name string,
stdin io.Reader,
stdout, stderr io.Writer,
arg ...string) (int, error) {

logLevel := logrus.GetLevel()
if stderr == nil && logLevel >= logrus.DebugLevel {
stderr = os.Stderr
}

cmd := exec.CommandContext(ctx, name, arg...)
cmd.Stdin = stdin
cmd.Stdout = stdout
cmd.Stderr = stderr

if err := cmd.Run(); err != nil {
exitCode := 1

if ctxErr := ctx.Err(); ctxErr != nil {
return 1, ctxErr
}

var exitErr *exec.ExitError
if errors.As(err, &exitErr) {
exitCode = exitErr.ExitCode()
return exitCode, err
}

return exitCode, err
}

return 0, nil
}

func RunWithExitCode(name string, stdin io.Reader, stdout, stderr io.Writer, arg ...string) (int, error) {
ctx := context.Background()
exitCode, err := RunContextWithExitCode(ctx, name, stdin, stdout, stderr, arg...)
return exitCode, err
}

func RunWithExitCode2(name string, stdin io.Reader, stdout, stderr io.Writer, arg ...string) (int, error) {
ctx := context.Background()
exitCode, err := RunContextWithExitCode2(ctx, name, stdin, stdout, stderr, arg...)
return exitCode, err
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Adding new functions with a 2 suffix (e.g., RunContextWithExitCode2 and RunWithExitCode2) is poor practice for maintainability. It's unclear how they differ from the original versions without inspecting the implementation.

If a different error handling behavior is required, consider using a more descriptive name (e.g., RunContextWithExitError) or refactoring the existing functions to be more flexible. Furthermore, these functions appear to be unused in the current PR and should be removed if not strictly necessary.


// Fallback: check standard locations on the host
logrus.Debugf("Interpreter at %s not found or not statically linked, checking fallback locations in '/run/host/usr/bin/'", interpreterPath)
fmt.Fprintf(os.Stderr, "Warning: QEMU emulator not found at expected path '%s', using fallback at '/run/host/usr/bin/'\n", interpreterPath)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Bypassing the logging system by writing directly to os.Stderr is inconsistent with the rest of the codebase. Using logrus.Warnf ensures that the message follows the configured log format and level.

Suggested change
fmt.Fprintf(os.Stderr, "Warning: QEMU emulator not found at expected path '%s', using fallback at '/run/host/usr/bin/'\n", interpreterPath)
logrus.Warnf("QEMU emulator not found at expected path '%s', using fallback at '/run/host/usr/bin/'", interpreterPath)

Comment on lines +159 to +160
func IsArchSupportedOnCreation(archID int) (string, error) {
archName := getArchNameBinfmt(archID)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If archID is NotSpecified, this function proceeds to look for a binary named qemu-arch_not_specified-static, which will fail and result in a confusing error message for the user. It would be better to validate that archID is a supported architecture at the beginning of the function.

DaliborKr added a commit to DaliborKr/toolbox that referenced this pull request Apr 23, 2026
Before creating or initializing a cross-architecture container, the
system must be checked for the required QEMU emulator and binfmt_misc
registration. This prevents users from creating or running non-native
containers when their host system doesn't meet the requirements, and
provides users with an informative error message referring to the
problem.

Add IsArchSupportedOnCreation(), which searches for a statically linked
QEMU binary on the host using exec.LookPath() and verifies that a
matching binfmt_misc registration exists. It returns the path to the
QEMU binary for use during container creation, which is meant to be
passed to the init-container and registered through sandboxed
binfmt_misc within the container.

Add IsArchSupportedOnInitialization() which performs similar checks
from inside the container, looking at the interpreter path passed from
the host and falling back to standard host-mounted locations under
/run/host/usr/bin/.

Add isStaticallyLinkedELF() helper that uses debug/elf to verify a
binary is statically linked. Only a statically linked QEMU interpreter
can be used, because a dynamically linked one would cause the kernel
to attempt to resolve its host-native shared libraries (such as libc.so)
within the container, resulting in an immediate crash.

Add validateBinfmtRegistration(), which checks for the presence of
qemu-<arch> entries in binfmt_misc (or qemu-<arch>-static, since it can
differ based on the system).

containers#1783

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
@DaliborKr DaliborKr force-pushed the pr04-arch-validation branch from ac0712e to 8374177 Compare April 23, 2026 12:46
@softwarefactory-project-zuul
Copy link
Copy Markdown

DaliborKr added a commit to DaliborKr/toolbox that referenced this pull request Apr 24, 2026
Add the --arch flag to the 'create' command, allowing users to create
Toolbx containers for architectures different from the host
(e.g., 'toolbox create --arch arm64').

Utilize the architecture resolution pipeline in create() by using
resolveArchitectureID() (added in [1]) to determine the target
architecture from the --arch flag and image tags.

Validate host support via IsArchSupportedOnCreation() (added in [2]),
which checks for the required QEMU emulator and binfmt_misc
registration.

Pass architecture ID to resolveContainerAndImageNames() (updated in [1])
so that non-native containers get architecture-suffixed names.

Update pullImage() to handle cross-architecture image pulling: when the
target architecture is non-native, use skopeo.CopyOverrideArch()
(added in [3]) instead of podman.Pull(), since Podman does not support
pulling foreign architecture images into locally addressable names.
The need for this is explained in a discussion in [4].

Add a 'toolbox-arch' label to created containers to record the target
architecture in OCI format.

Extract the image pull error formatting into createErrorImagePull() in
utils.go to avoid duplication between the native and cross-arch pull
paths.

Update the createContainer() call in run.go to pass the default
architecture config via GetArchConfigDefault(), maintaining the
existing native-architecture behavior.

[1] containers#1786
[2] containers#1783
[3] containers#1784
[4] containers/podman#27780

containers#1787

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
DaliborKr added a commit to DaliborKr/toolbox that referenced this pull request Apr 24, 2026
Add the --arch and --arch-emulator-path flags to the init-container
command, passed from the create command when creating a
cross-architecture container. The --arch flag defaults to the host
architecture ID so that existing native containers continue to work
without changes.

When the container's architecture differs from the host, the
init-container entry point configures QEMU emulation inside the
container before any foreign-architecture binaries can run:

1. Validate QEMU emulation by running the 'true' command, which fails
with ENOEXEC if the host's binfmt_misc registration is not working
(detected via RunWithExitCode2() added in [1]), because it is necessary
to have host emulation working to emulate the binfmt_misc registration
in the following step.

2. Mount a fresh binfmt_misc filesystem inside the container via
MountBinfmtMisc() (added in [2]) to create a sandboxed binfmt_misc
registration with the C flag.

3. Validate architecture support via IsArchSupportedOnInitialization()
(added in [3]), which verifies the QEMU interpreter at the host-mounted
path under /run/host.

4. Register the QEMU interpreter with the C flag via
RegisterBinfmtMisc() (added and explained in [2])

The binfmt_misc registration is performed inside the container rather
than relying on the host's registration, as explained in [2].

Update showEntryPointLog() in run.go to propagate lines prefixed with
'Warning:' to stderr on the host, instead of treating them as errors.
This is needed because the cross-architecture initialization may emit
warnings that should be visible to the user but are not fatal.

[1] containers#1780
[2] containers#1782
[3] containers#1783

containers#1788

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
DaliborKr added a commit to DaliborKr/toolbox that referenced this pull request Apr 24, 2026
Before creating or initializing a cross-architecture container, the
system must be checked for the required QEMU emulator and binfmt_misc
registration. This prevents users from creating or running non-native
containers when their host system doesn't meet the requirements, and
provides users with an informative error message referring to the
problem.

Add IsArchSupportedOnCreation(), which searches for a statically linked
QEMU binary on the host using exec.LookPath() and verifies that a
matching binfmt_misc registration exists. It returns the path to the
QEMU binary for use during container creation, which is meant to be
passed to the init-container and registered through sandboxed
binfmt_misc within the container.

Add IsArchSupportedOnInitialization() which performs similar checks
from inside the container, looking at the interpreter path passed from
the host and falling back to standard host-mounted locations under
/run/host/usr/bin/.

Add isStaticallyLinkedELF() helper that uses debug/elf to verify a
binary is statically linked. Only a statically linked QEMU interpreter
can be used, because a dynamically linked one would cause the kernel
to attempt to resolve its host-native shared libraries (such as libc.so)
within the container, resulting in an immediate crash.

Add validateBinfmtRegistration(), which checks for the presence of
qemu-<arch> entries in binfmt_misc (or qemu-<arch>-static, since it can
differ based on the system).

containers#1783

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
@DaliborKr DaliborKr force-pushed the pr04-arch-validation branch from 8374177 to 0b97666 Compare April 24, 2026 14:44
DaliborKr added a commit to DaliborKr/toolbox that referenced this pull request Apr 24, 2026
Before creating or initializing a cross-architecture container, the
system must be checked for the required QEMU emulator and binfmt_misc
registration. This prevents users from creating or running non-native
containers when their host system doesn't meet the requirements, and
provides users with an informative error message referring to the
problem.

Add IsArchSupportedOnCreation(), which searches for a statically linked
QEMU binary on the host using exec.LookPath() and verifies that a
matching binfmt_misc registration exists. It returns the path to the
QEMU binary for use during container creation, which is meant to be
passed to the init-container and registered through sandboxed
binfmt_misc within the container.

Add IsArchSupportedOnInitialization() which performs similar checks
from inside the container, looking at the interpreter path passed from
the host and falling back to standard host-mounted locations under
/run/host/usr/bin/.

Add isStaticallyLinkedELF() helper that uses debug/elf to verify a
binary is statically linked. Only a statically linked QEMU interpreter
can be used, because a dynamically linked one would cause the kernel
to attempt to resolve its host-native shared libraries (such as libc.so)
within the container, resulting in an immediate crash.

Add validateBinfmtRegistration(), which checks for the presence of
qemu-<arch> entries in binfmt_misc (or qemu-<arch>-static, since it can
differ based on the system).

containers#1783

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
@DaliborKr DaliborKr force-pushed the pr04-arch-validation branch from 0b97666 to 46abf95 Compare April 24, 2026 15:50
@softwarefactory-project-zuul
Copy link
Copy Markdown

DaliborKr added a commit to DaliborKr/toolbox that referenced this pull request Apr 24, 2026
Add the --arch flag to the 'create' command, allowing users to create
Toolbx containers for architectures different from the host
(e.g., 'toolbox create --arch arm64').

Utilize the architecture resolution pipeline in create() by using
resolveArchitectureID() (added in [1]) to determine the target
architecture from the --arch flag and image tags.

Validate host support via IsArchSupportedOnCreation() (added in [2]),
which checks for the required QEMU emulator and binfmt_misc
registration.

Pass architecture ID to resolveContainerAndImageNames() (updated in [1])
so that non-native containers get architecture-suffixed names.

Update pullImage() to handle cross-architecture image pulling: when the
target architecture is non-native, use skopeo.CopyOverrideArch()
(added in [3]) instead of podman.Pull(), since Podman does not support
pulling foreign architecture images into locally addressable names.
The need for this is explained in a discussion in [4].

Add a 'toolbox-arch' label to created containers to record the target
architecture in OCI format.

Extract the image pull error formatting into createErrorImagePull() in
utils.go to avoid duplication between the native and cross-arch pull
paths.

Update the createContainer() call in run.go to pass the default
architecture config via GetArchConfigDefault(), maintaining the
existing native-architecture behavior.

[1] containers#1786
[2] containers#1783
[3] containers#1784
[4] containers/podman#27780

containers#1787

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
DaliborKr added a commit to DaliborKr/toolbox that referenced this pull request Apr 24, 2026
Add the --arch and --arch-emulator-path flags to the init-container
command, passed from the create command when creating a
cross-architecture container. The --arch flag defaults to the host
architecture ID so that existing native containers continue to work
without changes.

When the container's architecture differs from the host, the
init-container entry point configures QEMU emulation inside the
container before any foreign-architecture binaries can run:

1. Validate QEMU emulation by running the 'true' command, which fails
with ENOEXEC if the host's binfmt_misc registration is not working
(detected via RunWithExitCode2() added in [1]), because it is necessary
to have host emulation working to emulate the binfmt_misc registration
in the following step.

2. Mount a fresh binfmt_misc filesystem inside the container via
MountBinfmtMisc() (added in [2]) to create a sandboxed binfmt_misc
registration with the C flag.

3. Validate architecture support via IsArchSupportedOnInitialization()
(added in [3]), which verifies the QEMU interpreter at the host-mounted
path under /run/host.

4. Register the QEMU interpreter with the C flag via
RegisterBinfmtMisc() (added and explained in [2])

The binfmt_misc registration is performed inside the container rather
than relying on the host's registration, as explained in [2].

Update showEntryPointLog() in run.go to propagate lines prefixed with
'Warning:' to stderr on the host, instead of treating them as errors.
This is needed because the cross-architecture initialization may emit
warnings that should be visible to the user but are not fatal.

[1] containers#1780
[2] containers#1782
[3] containers#1783

containers#1788

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
Introduce the architecture package that represents the core of the
Toolbx cross-architecture support, which is based on user-mode emulation
using QEMU and binfmt_misc.

The Architecture struct collects all per-architecture data (ELF
magic/mask, OCI and binfmt naming, aliases, binfmt registration
parameters) into a single map. Architectures present in the
supportedArchitectures map represent the set of supported
architectures within Toolbx.

Define architecture ID constants NotSpecified, Aarch64, Ppc64le, and
X86_64, along with their supportedArchitectures entries.

Add core query functions:
- ParseArgArchValue() for resolving user-supplied architecture strings

- GetArchNameBinfmt() and GetArchNameOCI() for name
  lookups (one architecture can have multiple valid names [1])

- HasContainerNativeArch() for comparing against the host

- ImageReferenceGetArchFromTag() for extracting architecture
  from image tag suffixes like "42-aarch64" for architecture detection

Expose the HostArchID package variable, which is set in the init()
function, so the variable can be accessed in the early init() state
from every inheritor that utilizes the architecture package (HostArchID
serves as a default value for initContainer --arch flag), and the Config
struct for preserving the architecture ID and the QEMU emulator path,
through the call chain.

[1] https://itsfoss.com/arm-aarch64-x86_64/

containers#1782

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
Cross-architecture containers need QEMU binfmt_misc handlers registered
within the container so that non-native architecture binaries can be
executed via the host's kernel.

Add the Registration struct that models a binfmt_misc registration
entry, including name, magic type, offset, ELF magic/mask bytes,
interpreter path, and flags.

Add functions:
- MountBinfmtMisc() to mount the sanboxed binfmt_misc filesystem inside
  a container, which enables setting the C flag in binfmt_misc
  registration without affecting the host system. The C flag presents a
  threat of privilege escalation when registered on the host, that why
  we want to have it isolated [1]

- getDefaultRegistration() to fill a Registration struct containing all
  necessary binfmt_misc information taken from the
  architecture.supportedArchitectures data

- RegisterBinfmtMisc() to write the registration string to
  /proc/sys/fs/binfmt_misc/register, which makes the non-native binary
  perception active

- bytesToEscapedString() helper that formats byte slices into the
  \xHH-escaped string format required by the binfmt_misc register
  interface

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=21ca59b365c0

containers#1782

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
Before creating or initializing a cross-architecture container, the
system must be checked for the required QEMU emulator and binfmt_misc
registration. This prevents users from creating or running non-native
containers when their host system doesn't meet the requirements, and
provides users with an informative error message referring to the
problem.

Add IsArchSupportedOnCreation(), which searches for a statically linked
QEMU binary on the host using exec.LookPath() and verifies that a
matching binfmt_misc registration exists. It returns the path to the
QEMU binary for use during container creation, which is meant to be
passed to the init-container and registered through sandboxed
binfmt_misc within the container.

Add IsArchSupportedOnInitialization() which performs similar checks
from inside the container, looking at the interpreter path passed from
the host and falling back to standard host-mounted locations under
/run/host/usr/bin/.

Add isStaticallyLinkedELF() helper that uses debug/elf to verify a
binary is statically linked. Only a statically linked QEMU interpreter
can be used, because a dynamically linked one would cause the kernel
to attempt to resolve its host-native shared libraries (such as libc.so)
within the container, resulting in an immediate crash.

Add validateBinfmtRegistration(), which checks for the presence of
qemu-<arch> entries in binfmt_misc (or qemu-<arch>-static, since it can
differ based on the system).

containers#1783

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
@DaliborKr DaliborKr force-pushed the pr04-arch-validation branch from 46abf95 to 76658c0 Compare April 24, 2026 21:49
DaliborKr added a commit to DaliborKr/toolbox that referenced this pull request Apr 24, 2026
Add the --arch flag to the 'create' command, allowing users to create
Toolbx containers for architectures different from the host
(e.g., 'toolbox create --arch arm64').

Utilize the architecture resolution pipeline in create() by using
resolveArchitectureID() (added in [1]) to determine the target
architecture from the --arch flag and image tags.

Validate host support via IsArchSupportedOnCreation() (added in [2]),
which checks for the required QEMU emulator and binfmt_misc
registration.

Pass architecture ID to resolveContainerAndImageNames() (updated in [1])
so that non-native containers get architecture-suffixed names.

Update pullImage() to handle cross-architecture image pulling: when the
target architecture is non-native, use skopeo.CopyOverrideArch()
(added in [3]) instead of podman.Pull(), since Podman does not support
pulling foreign architecture images into locally addressable names.
The need for this is explained in a discussion in [4].

Add a 'toolbox-arch' label to created containers to record the target
architecture in OCI format.

Extract the image pull error formatting into createErrorImagePull() in
utils.go to avoid duplication between the native and cross-arch pull
paths.

Update the createContainer() call in run.go to pass the default
architecture config via GetArchConfigDefault(), maintaining the
existing native-architecture behavior.

[1] containers#1786
[2] containers#1783
[3] containers#1784
[4] containers/podman#27780

containers#1787

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
DaliborKr added a commit to DaliborKr/toolbox that referenced this pull request Apr 24, 2026
Add the --arch and --arch-emulator-path flags to the init-container
command, passed from the create command when creating a
cross-architecture container. The --arch flag defaults to the host
architecture ID so that existing native containers continue to work
without changes.

When the container's architecture differs from the host, the
init-container entry point configures QEMU emulation inside the
container before any foreign-architecture binaries can run:

1. Validate QEMU emulation by running the 'true' command, which fails
with ENOEXEC if the host's binfmt_misc registration is not working
(detected via RunWithExitCode2() added in [1]), because it is necessary
to have host emulation working to emulate the binfmt_misc registration
in the following step.

2. Mount a fresh binfmt_misc filesystem inside the container via
MountBinfmtMisc() (added in [2]) to create a sandboxed binfmt_misc
registration with the C flag.

3. Validate architecture support via IsArchSupportedOnInitialization()
(added in [3]), which verifies the QEMU interpreter at the host-mounted
path under /run/host.

4. Register the QEMU interpreter with the C flag via
RegisterBinfmtMisc() (added and explained in [2])

The binfmt_misc registration is performed inside the container rather
than relying on the host's registration, as explained in [2].

Update showEntryPointLog() in run.go to propagate lines prefixed with
'Warning:' to stderr on the host, instead of treating them as errors.
This is needed because the cross-architecture initialization may emit
warnings that should be visible to the user but are not fatal.

[1] containers#1780
[2] containers#1782
[3] containers#1783

containers#1788

Signed-off-by: Dalibor Kricka <dalidalk@seznam.cz>
@softwarefactory-project-zuul
Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant