cffi: update to use libyang v4.2.2 #160

steweg · 2025-11-03T12:03:56Z

This PR introduces several commits needed for proper functionality with libyang v4.1.0

steweg · 2025-11-03T12:12:45Z

This PR apparently needs to wait for official release of libyang v4.1.0. If you would like to test it by yourself in the mean time, use libyang devel branch

choppsv1 · 2025-11-07T01:41:01Z

What will happen if the underlying system hasn't upgraded to libyang v4?

steweg · 2025-11-07T06:56:01Z

What will happen if the underlying system hasn't upgraded to libyang v4?

It won't work. As libyang v3 vs v4 makes backward incompatible changes, the similar will happen with libyang-python. I tried to keep the libyang-python API unchanged as much as possible, but some options are simply gone and AFAIK there is no other way around. I believe it is similar to when libyang was upgraded from v2 to v3.

choppsv1 · 2025-11-08T16:53:47Z

What will happen if the underlying system hasn't upgraded to libyang v4?

It won't work. As libyang v3 vs v4 makes backward incompatible changes, the similar will happen with libyang-python. I tried to keep the libyang-python API unchanged as much as possible, but some options are simply gone and AFAIK there is no other way around. I believe it is similar to when libyang was upgraded from v2 to v3.

I think for our use we won't be able to require our users upgrade to 4. So I think the main problem is going to be making sure they get a working version of the python bindings. It might make sense to create a libyang3 stable branch to track critical fixes. Also not sure how to handle 2 versions of the bindings with pypi. Does pypi let you update multiple versions that vary by major?

steweg · 2025-11-08T19:41:42Z

What will happen if the underlying system hasn't upgraded to libyang v4?

It won't work. As libyang v3 vs v4 makes backward incompatible changes, the similar will happen with libyang-python. I tried to keep the libyang-python API unchanged as much as possible, but some options are simply gone and AFAIK there is no other way around. I believe it is similar to when libyang was upgraded from v2 to v3.

I think for our use we won't be able to require our users upgrade to 4. So I think the main problem is going to be making sure they get a working version of the python bindings. It might make sense to create a libyang3 stable branch to track critical fixes. Also not sure how to handle 2 versions of the bindings with pypi. Does pypi let you update multiple versions that vary by major?

Frankly speaking, the requirement of having specific version or higher of libyang is kind of dependency that I would expect to be there since the first version of libyang-python, including some checks during first installation and also during upgrade process. The simple reason is that even new APIs are introduced in libyang within the same major version and if you want to introduce that within libyang-python, you still need to make sure about the version of libyang. Otherwise this will never work correctly. As I am not maintainer of this repo, I never looked in specific how this is being achieved besides just putting version restriction within cffi/source.c file. There is also one more important note. I was told by libyang maintainer that bugfixes are not always backported to older versions (see this example: CESNET/libyang#2428 (comment)). So at some point the upgrade of underlying libyang becomes inevitable.

Regarding the pypi versioning. In general updates are possible and can support multiple major versions, but it depends on maintainers, how many stable versions they want to support and all the staff along with it (backporting bugfixes....).

rjarry · 2025-11-11T11:21:33Z

My time for maintaining libyang-python is very limited. I will not be able to support maintenance/stable branches. When libyang 4.x is released, I will accept this PR and we can bump the python project version to v4.0.0 to follow libyang version scheme.

rjarry

Could you squash all in a single commit? I want the git history to remain bisectable.

Also, could you use imperative mood in your commit message?

libyang-python/README.rst

Lines 258 to 261 in ddb7e99

    
              -  Describe your changes in imperative mood, e.g. *"make xyzzy do frotz"* 
        
                 instead of *"[This patch] makes xyzzy do frotz"* or *"[I] changed xyzzy to 
        
                 do frotz"*, as if you are giving orders to the codebase to change its 
        
                 behaviour.

steweg · 2025-11-11T12:36:06Z

Could you squash all in a single commit? I want the git history to remain bisectable.

Also, could you use imperative mood in your commit message?

libyang-python/README.rst

Lines 258 to 261 in ddb7e99

- Describe your changes in imperative mood, e.g. *"make xyzzy do frotz"*

instead of *"[This patch] makes xyzzy do frotz"* or *"[I] changed xyzzy to

do frotz"*, as if you are giving orders to the codebase to change its

behaviour.

Done

choppsv1 · 2025-11-11T14:00:24Z

My time for maintaining libyang-python is very limited. I will not be able to support maintenance/stable branches. When libyang 4.x is released, I will accept this PR and we can bump the python project version to v4.0.0 to follow libyang version scheme.

Perhaps I could offer to help? I'd prefer not to have to fork and do this alone, so a collaboration would be preferable. Maybe you could create a v3 stable branch and grant permissions to that, and I (for a time) could maintain it, doing the back-ports required (in FRR we use mergifyio to make this as painless as possible). That way v3 would still be updated and fetchable the same way from pypi.

libyang/extension.py

rjarry · 2025-11-24T20:28:32Z

Perhaps I could offer to help? I'd prefer not to have to fork and do this alone, so a collaboration would be preferable. Maybe you could create a v3 stable branch and grant permissions to that, and I (for a time) could maintain it, doing the back-ports required (in FRR we use mergifyio to make this as painless as possible). That way v3 would still be updated and fetchable the same way from pypi.

Oh that would be much appreciated 👍

Sorry, I had missed your message and only see it now. I have invited you as a member of the project with write/push access. I have created a b_3.x branch which should be ready to work.

master...b_3.x

In order to create new releases on pypi.org, all you need to do is push a signed/annotated tag on that branch following the pattern v3.x.y where x and y are positive integers. Try to use semantic versioning where it makes sense. The CI will take care of the rest.

libyang-python/.github/workflows/ci.yml

Lines 94 to 114 in 3815807

    
           deploy: 
        
             needs: [lint, test] 
        
             if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v') 
        
             runs-on: ubuntu-24.04 
        
             steps: 
        
               - uses: actions/checkout@v3 
        
               - uses: actions/setup-python@v4 
        
                 with: 
        
                   python-version: 3.x 
        
               - uses: actions/cache@v3 
        
                 with: 
        
                   path: ~/.cache/pip 
        
                   key: pip 
        
                   restore-keys: pip 
        
               - run: python -m pip install --upgrade pip setuptools wheel 
        
               - run: python -m pip install --upgrade pep517 twine 
        
               - run: python -m pep517.build --out-dir dist/ --source . 
        
               - uses: pypa/gh-action-pypi-publish@release/v1 
        
                 with: 
        
                   user: ${{ secrets.pypi_user }} 
        
                   password: ${{ secrets.pypi_password }}

Thanks for stepping up ❤️

choppsv1 · 2025-11-24T21:52:32Z

In order to create new releases on pypi.org, all you need to do is push a signed/annotated tag on that branch following the pattern v3.x.y where x and y are positive integers. Try to use semantic versioning where it makes sense. The CI will take care of the rest.

Got it.

Might need to do something with tox.ini/tox-install.sh to have it fetch the correct v3 of libyang too. I'll take a look at that.

Thanks for stepping up ❤️

Thanks for the invitation. :)

Krisscut · 2025-12-08T10:20:39Z

Looks like libyang released last week: https://github.com/CESNET/libyang/releases/tag/v4.2.2

steweg · 2025-12-11T13:19:29Z

Looks like libyang released last week: https://github.com/CESNET/libyang/releases/tag/v4.2.2

I adjusted the patch to work with v4.2.2. Unfortunately there is still one test failing due to very strange issue, for which I have asked help of libyang maintainers (CESNET/libyang#2457).

This patch adjust cffi definitions and also all associated functions based on libyang v4.2.2. Signed-off-by: Stefan Gula <[email protected]>

jeremie6wind · 2025-12-11T16:43:33Z

Hi,
Do you need to do something like this?

------------------------- tests/yang/yang-library.json -------------------------
index 3dbf2568a99f..f9664845d6d8 100644
@@ -1,16 +1,31 @@
 {
   "ietf-yang-library:modules-state": {
     "module-set-id": "e595da11ace92c0d881995fa7e56bbe86f1f48e9",
     "module": [
       {
         "name": "yolo-system",
         "revision": "1999-04-01",
         "namespace": "urn:yang:yolo:system",
         "conformance-type": "implement"
       }
     ]
   },
   "ietf-yang-library:yang-library": {
+    "module-set": [
+      {
+        "name": "complete",
+        "module": [
+          {
+            "name": "yang",
+            "revision": "2025-01-29",
+            "namespace": "urn:ietf:params:xml:ns:yang:1",
+            "location": [
+              "file://@internal"
+            ]
+          }
+        ]
+      }
+    ],
     "content-id": "321566"
   }
 }

steweg · 2025-12-11T17:17:46Z

Hi, Do you need to do something like this?

------------------------- tests/yang/yang-library.json -------------------------
index 3dbf2568a99f..f9664845d6d8 100644
@@ -1,16 +1,31 @@
 {
   "ietf-yang-library:modules-state": {
     "module-set-id": "e595da11ace92c0d881995fa7e56bbe86f1f48e9",
     "module": [
       {
         "name": "yolo-system",
         "revision": "1999-04-01",
         "namespace": "urn:yang:yolo:system",
         "conformance-type": "implement"
       }
     ]
   },
   "ietf-yang-library:yang-library": {
+    "module-set": [
+      {
+        "name": "complete",
+        "module": [
+          {
+            "name": "yang",
+            "revision": "2025-01-29",
+            "namespace": "urn:ietf:params:xml:ns:yang:1",
+            "location": [
+              "file://@internal"
+            ]
+          }
+        ]
+      }
+    ],
     "content-id": "321566"
   }
 }

Unfortunately this is little bit outside of my expertise as I don't use this feature of libyang, I was just trying to fix the tests. The issue was actually taken care of directly by libyang maintainer. It is already fixed, but it will be in the next release. The question now would be that, which option should we go through:

adjust the test as was proposed (I am assuming that it works) - side effect could be that if there is a user which uses exactly the same approach as was used in the original test, needs to adapt
remove the test for good - according to indication from libyang maintainer it can be that is completely wrong use-case. I cannot judge this. And it has the same side effect as in previous point.
wait for fixed version of libyang and keep the test as-is

My approach was to minimize the number of backward incompatible changes to bare minimum. But adding one more is not that big of deal. So I will leave it up to maintainers of this repo to decide

samuel-gauthier · 2026-01-05T16:16:13Z

I merged the commit, including the patch that Jeremie wrote to unlock the tests. Thanks guys!

Krisscut · 2026-01-07T15:19:09Z

Hi !

Sorry to hijack this issue, let me know if you want me to create a new ticket.

I'm having a weird issue following the upgrade to the new libyang as well as libyang-python / sysrepo-python libs.

I described it on sysrepo initially since I though it was related to this, but in the end when using the raw C apis it seems fine.
See sysrepo/sysrepo#3719

I'm setting a string at some point, but I have a libyang error about an invalid boolean value:

2026-01-07 14:55:29.448 [ ERROR] [util.py:56] - log_callback() - Invalid boolean value "" for leaf "/simple:test_container/stringConfigFalse". (path "/simple:test_container/stringConfigFalse")
2026-01-07 14:55:29.448 [ ERROR] [util.py:56] - log_callback() - Invalid boolean value "" for leaf "/simple:test_container/stringConfigFalse".

Does this ring a bell to you guys, knowing in the model the leaf is a string ?

EDIT:

It seems to be pointing to the yang:orig-default metadata but I'm not so familiar with this (I added a few debug traces). Is it ok to have the yang:orig-default created with NULL ?

DEBUG lyd_create_meta: Creating metadata 'yang:operation'
DEBUG lyd_create_meta: Value: 'replace'
DEBUG lyd_new_meta: Called with:

name: yang:orig-value

val_str: 'temp2.1'

module: NULL

parent: stringConfigFalse (schema: stringConfigFalse)

parent schema module: simple

parent schema nodetype: 4

options: 0x0
DEBUG lyd_create_meta: Creating metadata 'yang:orig-value'
DEBUG lyd_create_meta: Value: 'temp2.1'
DEBUG lyd_new_meta: Called with:

name: yang:orig-default

val_str: 'NULL'

module: NULL

parent: stringConfigFalse (schema: stringConfigFalse)

parent schema module: simple

parent schema nodetype: 4

options: 0x0
DEBUG lyd_create_meta: Creating metadata 'yang:orig-default'
DEBUG lyd_create_meta: Value: ''

=== BOOLEAN VALIDATION ERROR STACK TRACE ===
Invalid boolean value "" for node: /simple:test_container/stringConfigFalse
Context node info:

Node name: stringConfigFalse

Node type: 4

Node flags: 0x806

Module: simple

Module namespace: s

Parent: test_container
Value info:

Format: 4

Value size: 0

Options: 0x0

Hints: 0x3f3

Type: 0x55a70d6f8660
[0] /opt/app/lib64/libyang.so.4(+0x16b29d) [0x7f52c782029d]
[1] /opt/app/lib64/libyang.so.4(+0x4463b) [0x7f52c76f963b]
[2] /opt/app/lib64/libyang.so.4(+0x38bd4) [0x7f52c76edbd4]
[3] /opt/app/lib64/libyang.so.4(lyd_new_meta+0x7ef) [0x7f52c77037ec]
[4] /opt/app/lib64/libsysrepo.so.8(+0x353ce) [0x7f52c75273ce]
[5] /opt/app/lib64/libsysrepo.so.8(+0x4c2c4) [0x7f52c753e2c4]
[6] /opt/app/lib64/libsysrepo.so.8(+0x4c95b) [0x7f52c753e95b]
[7] /opt/app/lib64/libsysrepo.so.8(+0x4cd15) [0x7f52c753ed15]
[8] /opt/app/lib64/libsysrepo.so.8(+0x50d9c) [0x7f52c7542d9c]
[9] /opt/app/lib64/libyang.so.4(+0x3cbad) [0x7f52c76f1bad]
=== END STACK TRACE ===

2026-01-07 16:40:41.190 [�[31m�[1m ERROR�[0m] [util.py:56] - log_callback() - Invalid boolean value "" for leaf "/simple:test_container/stringConfigFalse". (path "/simple:test_container/stringConfigFalse")
2026-01-07 16:40:41.191 [�[31m�[1m ERROR�[0m] [util.py:56] - log_callback() - Invalid boolean value "" for leaf "/simple:test_container/stringConfigFalse".

steweg · 2026-01-07T17:05:27Z

Hi !

Sorry to hijack this issue, let me know if you want me to create a new ticket.

I'm having a weird issue following the upgrade to the new libyang as well as libyang-python / sysrepo-python libs.

I described it on sysrepo initially since I though it was related to this, but in the end when using the raw C apis it seems fine. See sysrepo/sysrepo#3719

I'm setting a string at some point, but I have a libyang error about an invalid boolean value:

2026-01-07 14:55:29.448 [ ERROR] [util.py:56] - log_callback() - Invalid boolean value "" for leaf "/simple:test_container/stringConfigFalse". (path "/simple:test_container/stringConfigFalse")
2026-01-07 14:55:29.448 [ ERROR] [util.py:56] - log_callback() - Invalid boolean value "" for leaf "/simple:test_container/stringConfigFalse".

Does this ring a bell to you guys, knowing in the model the leaf is a string ?

EDIT:

It seems to be pointing to the yang:orig-default metadata but I'm not so familiar with this (I added a few debug traces). Is it ok to have the yang:orig-default created with NULL ?

DEBUG lyd_create_meta: Creating metadata 'yang:operation'
DEBUG lyd_create_meta: Value: 'replace'
DEBUG lyd_new_meta: Called with:

name: yang:orig-value

val_str: 'temp2.1'

module: NULL

parent: stringConfigFalse (schema: stringConfigFalse)

parent schema module: simple

parent schema nodetype: 4

options: 0x0
DEBUG lyd_create_meta: Creating metadata 'yang:orig-value'
DEBUG lyd_create_meta: Value: 'temp2.1'
DEBUG lyd_new_meta: Called with:

name: yang:orig-default

val_str: 'NULL'

module: NULL

parent: stringConfigFalse (schema: stringConfigFalse)

parent schema module: simple

parent schema nodetype: 4

options: 0x0
DEBUG lyd_create_meta: Creating metadata 'yang:orig-default'
DEBUG lyd_create_meta: Value: ''

=== BOOLEAN VALIDATION ERROR STACK TRACE ===
Invalid boolean value "" for node: /simple:test_container/stringConfigFalse
Context node info:

Node name: stringConfigFalse

Node type: 4

Node flags: 0x806

Module: simple

Module namespace: s

Parent: test_container
Value info:

Format: 4

Value size: 0

Options: 0x0

Hints: 0x3f3

Type: 0x55a70d6f8660
[0] /opt/app/lib64/libyang.so.4(+0x16b29d) [0x7f52c782029d]
[1] /opt/app/lib64/libyang.so.4(+0x4463b) [0x7f52c76f963b]
[2] /opt/app/lib64/libyang.so.4(+0x38bd4) [0x7f52c76edbd4]
[3] /opt/app/lib64/libyang.so.4(lyd_new_meta+0x7ef) [0x7f52c77037ec]
[4] /opt/app/lib64/libsysrepo.so.8(+0x353ce) [0x7f52c75273ce]
[5] /opt/app/lib64/libsysrepo.so.8(+0x4c2c4) [0x7f52c753e2c4]
[6] /opt/app/lib64/libsysrepo.so.8(+0x4c95b) [0x7f52c753e95b]
[7] /opt/app/lib64/libsysrepo.so.8(+0x4cd15) [0x7f52c753ed15]
[8] /opt/app/lib64/libsysrepo.so.8(+0x50d9c) [0x7f52c7542d9c]
[9] /opt/app/lib64/libyang.so.4(+0x3cbad) [0x7f52c76f1bad]
=== END STACK TRACE ===

2026-01-07 16:40:41.190 [�[31m�[1m ERROR�[0m] [util.py:56] - log_callback() - Invalid boolean value "" for leaf "/simple:test_container/stringConfigFalse". (path "/simple:test_container/stringConfigFalse")
2026-01-07 16:40:41.191 [�[31m�[1m ERROR�[0m] [util.py:56] - log_callback() - Invalid boolean value "" for leaf "/simple:test_container/stringConfigFalse".

Hi, I am not familiar with the issue that you are facing, but by simply reading your original post in sysrepo, my best guess that there is something wrong with underlying pointers directly within libyang itself. There were multiple issue in near past like that, which were having similar symptoms as you are mentioning (unexpected results, wrong values...). Therefore I would propose just to limit your scenario to bare minimum (e.g. remove all not needed leafs...) and check your scenario with valgrind and look for invalid reads/writes.

Krisscut · 2026-01-08T13:43:27Z

Thanks @steweg for the reply !

I might try this indeed, for now I was trying to understand the bool error and it seems to be linked with a "yang:orig-default" metadata which is added to my leaf, but with a NULL value. (and a bool is expected, hence the error)

Just to confirm I tried to modify the string value to "false" when I detect this case and it seems fine with this.

DEBUG lyd_new_meta: Called with:

name: yang:orig-default

val_str: 'NULL'

module: NULL

parent: stringConfigFalse (schema: stringConfigFalse)

parent schema module: simple

parent schema nodetype: 4

options: 0x0

Modified val_str to: 'false'
DEBUG lyd_create_meta: Creating metadata 'yang:orig-default'
DEBUG lyd_create_meta: Value: 'false'

So now I just need to understand who create this metadata and why it configures it with a NULL instead of true/false

Krisscut · 2026-01-08T14:46:16Z

It seems to be an issue in sysrepo, I added details on the other repo thread here: sysrepo/sysrepo#3719 (comment)

rjarry requested changes Nov 11, 2025

View reviewed changes

steweg force-pushed the feature/libyang_v4 branch from 2f56b6a to f2852c7 Compare November 11, 2025 12:35

jeremie6wind reviewed Nov 24, 2025

View reviewed changes

libyang/extension.py Show resolved Hide resolved

steweg force-pushed the feature/libyang_v4 branch from f2852c7 to 4ae29fc Compare November 24, 2025 19:35

jeremie6wind mentioned this pull request Dec 8, 2025

port to libyang 4 sysrepo 4 sysrepo/sysrepo-python#92

Merged

steweg force-pushed the feature/libyang_v4 branch 2 times, most recently from d71a3a4 to d6faad4 Compare December 11, 2025 13:17

steweg changed the title ~~cffi: update to use libyang v4.1.0~~ cffi: update to use libyang v4.2.2 Dec 11, 2025

cffi: allows to usage of libyang v4.2.2

8c24656

This patch adjust cffi definitions and also all associated functions based on libyang v4.2.2. Signed-off-by: Stefan Gula <[email protected]>

steweg force-pushed the feature/libyang_v4 branch from d6faad4 to 8c24656 Compare December 11, 2025 13:22

samuel-gauthier closed this in 0a4b09f Jan 5, 2026

	- Describe your changes in imperative mood, e.g. "make xyzzy do frotz"
	instead of "[This patch] makes xyzzy do frotz" or *"[I] changed xyzzy to
	do frotz"*, as if you are giving orders to the codebase to change its
	behaviour.

cffi: update to use libyang v4.2.2 #160

cffi: update to use libyang v4.2.2 #160

Uh oh!

Conversation

steweg commented Nov 3, 2025

Uh oh!

steweg commented Nov 3, 2025

Uh oh!

choppsv1 commented Nov 7, 2025

Uh oh!

steweg commented Nov 7, 2025

Uh oh!

choppsv1 commented Nov 8, 2025

Uh oh!

steweg commented Nov 8, 2025

Uh oh!

rjarry commented Nov 11, 2025

Uh oh!

rjarry left a comment

Choose a reason for hiding this comment

Uh oh!

steweg commented Nov 11, 2025

Uh oh!

choppsv1 commented Nov 11, 2025

Uh oh!

Uh oh!

rjarry commented Nov 24, 2025

Uh oh!

choppsv1 commented Nov 24, 2025

Uh oh!

Krisscut commented Dec 8, 2025

Uh oh!

steweg commented Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jeremie6wind commented Dec 11, 2025

Uh oh!

steweg commented Dec 11, 2025

Uh oh!

samuel-gauthier commented Jan 5, 2026

Uh oh!

Krisscut commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steweg commented Jan 7, 2026

Uh oh!

Krisscut commented Jan 8, 2026

Uh oh!

Krisscut commented Jan 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

steweg commented Dec 11, 2025 •

edited

Loading

Krisscut commented Jan 7, 2026 •

edited

Loading