Skip to content

Spike: C interop + external native linking via CcInfo (the OpenSSL case) #1687

Description

@cgruber

Child of #1682. Type: spike / research. See anchor Section 4b and Section 5 P4.

Goal. Build a native binary that links a real C library (the canonical case is OpenSSL
into a linux_x64 binary
) hermetically and cross-target. This is the difference between
toy native rules and ones that build real software, and it's the one genuinely unsolved
piece
of native linking.

How Kotlin/Native does it (two steps).

  1. Binding generation (cinterop). The K/N-bundled cinterop tool reads a .def file (C
    headers to import, package name, linkerOpts/compilerOpts) and emits a bindings klib
    (target-specific, like any klib) which konanc consumes as an ordinary dependency. Note
    cinterop runs libclang to parse headers, and libclang ships in the heavy konan
    deps
    (the same LLVM bundle P3 binary-linking pulls), not the lightweight prebuilt dist
    that plain klib compilation (P2) needs, so this inherits P3's hermetic-provisioning work
    (hence it sequences after P3).
  2. Final link. konanc drives the link itself (bundled cross-toolchain for Linux
    targets, host Xcode/ld for Apple targets
    ), and the actual native library
    (.a/.so/.dylib) plus search paths must reach that link via -linker-options (or the
    .def's linkerOpts).

The research question: where headers + the compiled lib come from.

  • Interim: point the .def at a sysroot or prebuilt (system OpenSSL, or a vendored
    .a). Quick, but less hermetic and doesn't cross-compile cleanly: a macOS exec host can't
    satisfy a linux_x64 OpenSSL from its own system.
  • Final (Bazel-idiomatic): source both headers (for cinterop) and the static lib (for
    the link) from a Bazel cc_library / CcInfo, threaded into konanc's link. Hermetic
    and cross-target-correct, but konanc's self-driven link is an impedance mismatch with
    Bazel's cc_common linking
    , and the hard part is cross-target availability: the C
    library must exist for the target (a linux_x64 OpenSSL built or imported when the exec
    host is macOS), so the dependency itself must cross-compile or be a per-target imported
    prebuilt.

Spike deliverable. A kt_native_cinterop rule (.def + CcInfo deps → bindings klib)
plus wiring CcInfo libraries into the kt_native_binary link; a worked OpenSSL example for
linux_x64. Interim sysroot/prebuilt acceptable as a first milestone, with the CcInfo path
as the research goal.

Success criteria.

  • A linux_x64 binary calling an OpenSSL function builds and runs (in a bare container, no
    JVM), first via the interim path.
  • Then: the same via a Bazel cc_library/CcInfo dependency, with the OpenSSL build/import
    resolved for the target while the exec host is macOS (the cross-target proof).
  • Hermetic: builds with --sandbox_default_allow_network=false and an empty disk cache.

Decision riding on it. Whether the CcInfo-bridged model is the target dependency story
for native C libs (vs. a sysroot/prebuilt-only stance, or something else). My lean: make
the CcInfo-bridged form work, for coherence with the rest of the build, on one firm
condition: it has to consume cc_common/CcInfo as they stand. If bridging into cinterop
and the konanc link turns out to need changes to rules_cc itself, that's the signal it's
the wrong approach and we stay on the interim. I'd rather pick a direction early because it
shapes the kt_native_cinterop/kt_native_binary provider contract.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions