How to *not* wrap headers?

[maleadt]

I'm trying to wrap a header depending on another (and using its definitions), but I don't want to include any of its definitions as I'll have wrapped that header separately. I used to do so with the header_wrapped attribute, https://github.com/JuliaGPU/CUDA.jl/blob/6dbcedcfc33d3e5087ff82cea37792eee8c341d1/res/wrap/wrap.jl#L24-L29, but there doesn't seem to be an equivalent in the new API.

While looking into this, it even seems that dependent headers get wrapper entirely, even though the toml documentation states that is_local_header_only (defaulting to true) should only include dependent headers' declarations when they are actually used. I'm probably doing something wrong though 🙂

So concretely, wrap.jl invokes Clang.jl on sycl.h, which includes ze_api.h as found in the Level Zero Headers JLL:

function wrap(name, headers...; library="lib$name", defines=[], include_dirs=[])
    options =  load_options(joinpath(@__DIR__, "wrap.toml"))
    options["general"]["library_name"] = library
    options["general"]["output_file_path"] = "lib$(name).jl"

    args = get_default_args()
    for include_dir in include_dirs
        push!(args, "-I$include_dir")
    end
    ctx = create_context([headers...], args, options)

    build!(ctx)

    return options["general"]["output_file_path"]
end

    process("sycl", joinpath(dirname(@__DIR__), "deps", "sycl.h");
            include_dirs=[dirname(dirname(oneAPI_Level_Zero_Headers_jll.ze_api))],
            library="liboneapilib", modname="sycl")

sycl.h is really simple:

#pragma once

#include <stddef.h>

#include <level_zero/ze_api.h>

#ifdef __cplusplus
extern "C" {
#endif

typedef struct syclPlatform_st *syclPlatform_t;
int syclPlatformCreate(syclPlatform_t *obj, ze_driver_handle_t driver);
int syclPlatformDestroy(syclPlatform_t obj);

...

#ifdef __cplusplus
}
#endif

wrap.toml is empty, so I understand this sets is_local_header_only to true. This gives the following output:

[ Info: Found dependent header: /home/tim/.julia/artifacts/36af6e708eaf0f487d1d9f505891a5c9a6586cfe/include/level_zero/ze_api.h
[ Info: Parsing headers...
[ Info: Processing header: /home/tim/.julia/dev/oneAPI/deps/sycl.h
[ Info: Building the DAG...
[ Info: Emit Julia expressions...
[ Info: [ProloguePrinter]: print to libsycl.jl
[ Info: [GeneralPrinter]: print to libsycl.jl
[ Info: [EpiloguePrinter]: print to libsycl.jl
[ Info: Done!

so ze_api.h is correctly flagged as a dependent header, yet libsycl.jl contains all its definitions:

using CEnum

const ze_bool_t = UInt8

mutable struct _ze_driver_handle_t end

# also all method definitions

# at the very end, there's the wrappers for sycl.h

mutable struct syclPlatform_st end

const syclPlatform_t = Ptr{syclPlatform_st}

function syclPlatformCreate(obj, driver)
    ccall((:syclPlatformCreate, liboneapilib), Cint,
          (Ptr{syclPlatform_t}, ze_driver_handle_t),
          obj, driver)
end

function syclPlatformDestroy(obj)
    ccall((:syclPlatformDestroy, liboneapilib), Cint,
          (syclPlatform_t,),
          obj)
end

So summarized:

• despite is_local_header_only=true, all declarations from a dependent header are getting picked up
• is it possible to completely omit a dependent header's declaration, even the used ones?

[Gnimuc]

With is_local_header_only=false, all decls in a translation unit will be picked up, including all decls in the current header and dependent headers(both system headers and non-system headers).

To generate self-contained and light-weighted Julia code, the default behavior(with is_local_header_only=true) of Clang.jl is only to pick up those decls used in the system headers. Any header can be treated as a system header if we pass the include directory flag as -isystem$include_dir instead of -I include_dir .

But there are cases that can not be handled with the -isystem hack, for example,

https://github.com/maleadt/LLVM.jl/blob/9155a8c30d6743197728e63c167d078167f34725/res/wrap_llvmextra.jl#L23-L43

LLVM.jl needs to ignore everything in a dependent header. In this case, we need to loop through all of the nodes and manually mark and ignore those not required nodes.

[Gnimuc]

The new API exposes the whole DAG of the TUs, so users can apply any custom passes after the codegen stage.

[maleadt]

Sweet, that works: -isystem for the ze_api.h includes, and rewriting the Exprs for local includes.

It does still include the "necessary" struct defs though, e.g.:

# these come from `ze_api.h` -- and are unwanted
mutable struct _ze_driver_handle_t end
const ze_driver_handle_t = Ptr{_ze_driver_handle_t}

# these from `sycl.h` -- and are wanted
mutable struct syclPlatform_st end
const syclPlatform_t = Ptr{syclPlatform_st}

function syclPlatformCreate(obj, driver)
    @ccall liboneapilib.syclPlatformCreate(obj::Ptr{syclPlatform_t},
                                           driver::ze_driver_handle_t)::Cint
end

Is it possible to leave those out, since they are defined already by a separate wrapping of ze_apil.h (for a different module)? I don't want to duplicate them, because I have unsafe_convert methods set-up for generating those pointers.

[maleadt]

Actually, scratch that. Mistake on my part, it works perfectly!