Generator ideas

[pthom]

Generate bindings for struct member of type "fixed size array"

The generator already generates accessors for numeric and bool fixed size arrays. However it does not generate accessors for user-defined types in an array.

With this code:

enum ImGuiCol_
{
    ImGuiCol_Text,
    ....
    ImGuiCol_COUNT
};    
    
struct ImGuiStyle
{
   ...
    ImVec4      Colors[ImGuiCol_COUNT];
};

The ideas would be to generate bindings like this:

class ImGuiStyle:
    ...
    def get_colors(idx): # if possible, find a way to make this a singular form
          """Returns a modifiable element from Colors[ImGuiCol_COUNT]
          (modifiable means that modifs done in python propagate automatically to C++, since C++ returned a ImVec4&)
          """
          pass
    def set_colors(idx):
          """modifies an entry in Colors[ImGuiCol_COUNT]""". # convenience method

[dcnieho]

Generate bindings for struct member of type "fixed size array"

The generator already generates accessors for numeric and bool fixed size arrays. However it does not generate accessors for user-defined types in an array.

With this code:

enum ImGuiCol_
{
    ImGuiCol_Text,
    ....
    ImGuiCol_COUNT
};    
    
struct ImGuiStyle
{
   ...
    ImVec4      Colors[ImGuiCol_COUNT];
};

The ideas would be to generate bindings like this:

class ImGuiStyle:
    ...
    def get_colors(idx): # if possible, find a way to make this a singular form
          """Returns a modifiable element from Colors[ImGuiCol_COUNT]
          (modifiable means that modifs done in python propagate automatically to C++, since C++ returned a ImVec4&)
          """
          pass
    def set_colors(idx):
          """modifies an entry in Colors[ImGuiCol_COUNT]""". # convenience method

As per #29, would it be possible to expose the actual array instead of getter and setter functions? That would be the closest to the original API (as well as pyimgui's) and is therefore easiest to discover/port from C++.

[pthom]

struct ImGuiStyle
{
   ....
    ImVec4      Colors[ImGuiCol_COUNT];
};

C arrays like int v[10] are liars: their type is int *, not int [10].
The only thing the compiler does to help is to reserve 10 slots in the memory at build time, and then it happily drops the size information into oblivion.
As far as pybind11 is concerned, Colors is a ImVec4* with no information whatsoever about its size.

Porting it to a python would require

1/ either, patching ImGui like this:

struct ImGuiStyle
{
   ....
    ImVector<ImGuiCol> Colors;
};

But then the color info would be on the heap and performance would drop.

2/ Or write hacks to add a kind-of translation

[dcnieho]

Copying discussion from the closed issue to here:

But the generator already handles many other fixed-size arrays well, such as (random example) imgui.get_io().mouse_clicked_count. More generally, if we look at a section of ImGuiIO and see how its wrapped:

struct ImGuiIO
{
   ....
    ImGuiKeyData KeysData[ImGuiKey_KeysData_SIZE];  // Key state for all known keys. Use IsKeyXXX() functions to access this.
    bool        WantCaptureMouseUnlessPopupClose;   // Alternative to WantCaptureMouse: (WantCaptureMouse == true && WantCaptureMouseUnlessPopupClose == false) when a click over void is expected to close a popup.
    ImVec2      MousePosPrev;                       // Previous mouse position (note that MouseDelta is not necessary == MousePos-MousePosPrev, in case either position is invalid)
    ImVec2      MouseClickedPos[5];                 // Position at time of clicking
    double      MouseClickedTime[5];                // Time of last click (used to figure out double-click)
    bool        MouseClicked[5];                    // Mouse button went from !Down to Down (same as MouseClickedCount[x] != 0)
    bool        MouseDoubleClicked[5];              // Has mouse button been double-clicked? (same as MouseClickedCount[x] == 2)
    ImU16       MouseClickedCount[5];               // == 0 (not clicked), == 1 (same as MouseClicked[]), == 2 (double-clicked), == 3 (triple-clicked) etc. when going from !Down to Down
    ImU16       MouseClickedLastCount[5];           // Count successive number of clicks. Stays valid after mouse release. Reset after another click is done.
    bool        MouseReleased[5];                   // Mouse button went from Down to !Down
    bool        MouseDownOwned[5];                  // Track if button was clicked inside a dear imgui window or over void blocked by a popup. We don't request mouse capture from the application if click started outside ImGui bounds.
    bool        MouseDownOwnedUnlessPopupClose[5];  // Track if button was clicked inside a dear imgui window.
    float       MouseDownDuration[5];               // Duration the mouse button has been down (0.0f == just clicked)
    float       MouseDownDurationPrev[5];           // Previous time the mouse button has been down
    ImVec2      MouseDragMaxDistanceAbs[5];         // Maximum distance, absolute, on each axis, of how much mouse has traveled from the clicking point
    float       MouseDragMaxDistanceSqr[5];         // Squared maximum distance of how much mouse has traveled from the clicking point (used for moving thresholds)
    float       NavInputsDownDuration[ImGuiNavInput_COUNT];
    float       NavInputsDownDurationPrev[ImGuiNavInput_COUNT];
    float       PenPressure;                        // Touch/Pen pressure (0.0f to 1.0f, should be >0.0f only when MouseDown[0] == true). Helper storage currently unused by Dear ImGui.
    .....
    ImVector<ImWchar> InputQueueCharacters;
};

we see that all fixed size array that:

1. have an integer to denote their size, and
2. are declared as a simple/basic type such as int or float

are wrapped and work. So float MouseDownDuration[5]; yes, ImVec2 MouseDragMaxDistanceAbs[5]; and float NavInputsDownDuration[ImGuiNavInput_COUNT]; no. The ImVec2 case would work fine (at least on the python side, no complaints about np.array([imgui.ImVec2(1,2),imgui.ImVec2(3,4)]), and i guess the generator has to be taught to recognize preprocessor macros. ImVec4 Colors[ImGuiCol_COUNT]; is a combination of the two ignored above, but should be equally possible. I would say that, in the long run, spending effort on getting these two things fixed maybe a better place to put your energy (or tell me if i can help!) than manual patches such as you did for getting and setting colors now. There would be a bunch more of those to needed to, for instance, wrap a whole bunch of items in ImGuiIO. Of course i do not oversee the complexity, and also am not saying this should be done now. But I hope you would consider it at some point.

[pthom]

But I hope you would consider it at some point.

This is the back of my head. Not easy though. I'll tell you more about the generator some day if you are willing to help.

[dcnieho]

I would be interested in learning about this. Partially also because as i've told you i have a C++ project (not published on github yet) that i would like to build a Python wrapper for, but would really not want to do that manually. I have a simpler other project that is on github (but no need for you to check it out) that i could try it on first. I have a manual pybind wrapper there, but it is out of date.

I am assuming that the generator must consist of at least two stages, one for parsing the C++ and one for outputting the wrapper. Both my projects also have a manually built matlab wrapper, and it would be great if i could automate that too by writing a matlab frontend to the generator. Way more hairy still as nothing like pybind exists for matlab (perhaps i should build that for as far as i need it myself, i have a bunch of utils for data exchange in both directions with matlab already). Anyway, another motivation to learn more about this.

[dcnieho]

__getitem__ and __setitem__ generation

See discussion here:
1d98595#r92730234

Currently this is generated:

    # double  operator[] (size_t idx) const { return (&x)[idx]; }    /* original C++ signature */
    def __getitem__(self, idx: int) -> float:
        pass
    # double& operator[] (size_t idx)       { return (&x)[idx]; }    /* original C++ signature */
    def __getitem__(self, idx: int) -> float:
        pass

Perhaps the reference-returning version should be used to trigger generation of this instead (note that it doesn't return anything)

    # double& operator[] (size_t idx)       { return (&x)[idx]; }    /* original C++ signature */
    def __setitem__(self, idx: int, val: float):
        pass

[pthom]

Yes, the solution is somewhere in the generator around https://github.com/pthom/srcmlcpp/blob/main/packages/litgen/internal/adapted_types/operators.py

Beware, the generator (litgen) is a giant beast, and I did not make effort to make it usable outside (I don't have time now, this will be 2023 thing. I still am pondering what its license will be. It will be open-source, but may be not MIT)

[dcnieho]

Licensing i am not so familiar with, in general i suppose it is good if many people can use it freely (freedom). Big corporations I feel ambivalent about, but perhaps with luck they would at least contribute back what they change.

Yeah, that looks like the place, perhaps a special case is needed there too for the [] operator. If a reference is returned then both getitem and setitem should be emitted, if a value is returned then only getitem is the logic i guess.