Bug: libyang3-py3 fails to run tests on Trixie, crashing with SIGSEGV

[saiarcot895]

Is it platform specific

generic

Importance or Severity

Medium

Description of the bug

When building libyang3-py3 for Trixie, the tests.test_schema.IfFeatureTest.test_iffeature_state test case fails to run, instead crashing with SIGSEGV. It appears that there is a read-after-free happening somewhere in libyang.

Steps to Reproduce

1. Use the trixie development branch at https://github.com/saiarcot895/sonic-buildimage/tree/trixie
2. Configure with make NOBUSTER=1 NOBULLSEYE=1 NOBOOKWORM=0 NOTRIXIE=0 configure PLATFORM=vs
3. Build with make NOBUSTER=1 NOBULLSEYE=1 NOBOOKWORM=1 NOTRIXIE=0 target/debs/trixie/python3-libyang_3.1.0-1_amd64.deb

Actual Behavior and Expected Behavior

When run under valgrind, this is the output:

sarcot@481a586d7f50:/sonic/src/libyang3-py3/libyang-python/.pybuild/cpython3_3.13/build$ valgrind python3 -m unittest -k tests.
test_schema.IfFeatureTest.test_iffeature_state
==20833== Memcheck, a memory error detector
==20833== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
==20833== Using Valgrind-3.24.0 and LibVEX; rerun with -h for copyright info
==20833== Command: python3 -m unittest -k tests.test_schema.IfFeatureTest.test_iffeature_state
==20833==
==20833== Invalid read of size 8
==20833==    at 0x6CE4475: ??? (in /usr/lib/python3/dist-packages/_cffi_backend.cpython-313-x86_64-linux-gnu.so)
==20833==    by 0x549EF1: PyObject_GetAttr (object.c:1261)
==20833==    by 0x55D542: _PyEval_EvalFrameDefault (generated_cases.c.h:3770)
==20833==    by 0x5F941A: UnknownInlinedFun (pycore_ceval.h:119)
==20833==    by 0x5F941A: UnknownInlinedFun (genobject.c:229)
==20833==    by 0x5F941A: UnknownInlinedFun (genobject.c:589)
==20833==    by 0x5F941A: builtin_next.lto_priv.0 (bltinmodule.c:1545)
==20833==    by 0x587CBD: cfunction_vectorcall_FASTCALL.lto_priv.0 (methodobject.c:425)
==20833==    by 0x5499A1: UnknownInlinedFun (pycore_call.h:168)
==20833==    by 0x5499A1: PyObject_Vectorcall (call.c:327)
==20833==    by 0x55F626: _PyEval_EvalFrameDefault (generated_cases.c.h:813)
==20833==    by 0x5C30D8: UnknownInlinedFun (pycore_call.h:168)
==20833==    by 0x5C30D8: method_vectorcall.lto_priv.0 (classobject.c:92)
==20833==    by 0x560FED: UnknownInlinedFun (call.c:373)
==20833==    by 0x560FED: _PyEval_EvalFrameDefault (generated_cases.c.h:1355)
==20833==    by 0x64B89E: UnknownInlinedFun (pycore_ceval.h:119)
==20833==    by 0x64B89E: UnknownInlinedFun (ceval.c:1816)
==20833==    by 0x64B89E: UnknownInlinedFun (call.c:413)
==20833==    by 0x64B89E: _PyObject_VectorcallDictTstate (call.c:135)
==20833==    by 0x69EC9F: UnknownInlinedFun (call.c:504)
==20833==    by 0x69EC9F: slot_tp_call.lto_priv.0 (typeobject.c:9556)
==20833==    by 0x543ACA: _PyObject_MakeTpCall (call.c:242)
==20833==  Address 0x5b12da8 is 8 bytes inside a block of size 120 free'd
==20833==    at 0x484787F: free (vg_replace_malloc.c:989)
==20833==    by 0x6B3164F: lysc_node_container_free (tree_schema_free.c:997)
==20833==    by 0x6B31AAA: lysc_node_free_ (tree_schema_free.c:1160)
==20833==    by 0x6B3213E: lysc_module_free (tree_schema_free.c:1284)
==20833==    by 0x6B0C8C9: lys_compile_depset_r (schema_compile.c:1539)
==20833==    by 0x6B0CA3D: lys_compile_depset_all (schema_compile.c:1611)
==20833==    by 0x6B2A479: lys_set_implemented (tree_schema.c:1195)
==20833==    by 0x6A8DD9C: _cffi_f_lys_set_implemented (_libyang.c:7942)
==20833==    by 0x583ACF: cfunction_call.lto_priv.0 (methodobject.c:551)
==20833==    by 0x543ACA: _PyObject_MakeTpCall (call.c:242)
==20833==    by 0x55F626: _PyEval_EvalFrameDefault (generated_cases.c.h:813)
==20833==    by 0x5C30D8: UnknownInlinedFun (pycore_call.h:168)
==20833==    by 0x5C30D8: method_vectorcall.lto_priv.0 (classobject.c:92)
==20833==  Block was alloc'd at
==20833==    at 0x484BBA3: calloc (vg_replace_malloc.c:1675)
==20833==    by 0x6B1588E: lys_compile_node (schema_compile_node.c:4303)
==20833==    by 0x6B160D9: lys_compile_uses_children (schema_compile_node.c:3971)
==20833==    by 0x6B16788: lys_compile_uses (schema_compile_node.c:4086)
==20833==    by 0x6B19346: lys_compile_node (schema_compile_node.c:4351)
==20833==    by 0x6B19346: lys_compile_node_container (schema_compile_node.c:2945)
==20833==    by 0x6B15268: lys_compile_node_ (schema_compile_node.c:2770)
==20833==    by 0x6B15820: lys_compile_node (schema_compile_node.c:4361)
==20833==    by 0x6B09A46: lys_compile (schema_compile.c:1746)
==20833==    by 0x6B0C8DE: lys_compile_depset_r (schema_compile.c:1543)
==20833==    by 0x6B0CA3D: lys_compile_depset_all (schema_compile.c:1611)
==20833==    by 0x6B2A479: lys_set_implemented (tree_schema.c:1195)
==20833==    by 0x6A8DD9C: _cffi_f_lys_set_implemented (_libyang.c:7942)

This is the Python backtrace when run under pytest (python -m unittest didn't give a backtrace):

Current thread 0x0000703a74e0e100 (most recent call first):
  File "/sonic/src/libyang3-py3/libyang-python/.pybuild/cpython3_3.13/build/libyang/schema.py", line 105 in features
  File "/sonic/src/libyang3-py3/libyang-python/.pybuild/cpython3_3.13/build/libyang/schema.py", line 1381 in if_features
  File "/sonic/src/libyang3-py3/libyang-python/.pybuild/cpython3_3.13/build/tests/test_schema.py", line 226 in test_iffeature_state
  File "/usr/lib/python3.13/unittest/case.py", line 606 in _callTestMethod
  File "/usr/lib/python3.13/unittest/case.py", line 651 in run
  File "/usr/lib/python3.13/unittest/case.py", line 707 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/unittest.py", line 351 in runtest
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 174 in pytest_runtest_call
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 242 in <lambda>
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 341 in from_call
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 241 in call_and_report
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 132 in runtestprotocol
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 362 in pytest_runtestloop
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 337 in _main
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 283 in wrap_session
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 330 in pytest_cmdline_main
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 175 in main
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 201 in console_main
  File "/usr/bin/pytest", line 8 in <module>

Relevant log output

Output of show version, show techsupport

Attach files (if any)

No response

[saiarcot895]

@bradh352 FYI

[bhouse-nexthop]

Ooph, this looks like a use-after-free condition so likely visible under valgrind on other platforms too. I'll spin up a Trixie VM and see if it is easily reproducible there without taking in the sonic stuff. That said, the patches I applied to the code don't affect these areas, so this seems to be an upstream bug.

[bradh352]

FYI, this is confirmed as an upstream issue using a fresh debian trixie rc2 Virtual Machine with no sonic-specific libyang patches:

CESNET/libyang-python#140

I've done some evaluation in that upstream ticket. As far as I can tell an internal pointer is invalidated. The way libyang is designed you have to be aware of when an internal pointer retrieved may be invalidated by other calls, and this appears to be the test case itself isn't aware of this.

I'll see if I can submit a PR upstream to address this. I don't think its an overall code issue, just the test case.

[bradh352]

PR #23229 submitted to resolve this issue

[saiarcot895]

I've done some evaluation in that upstream ticket. As far as I can tell an internal pointer is invalidated. The way libyang is designed you have to be aware of when an internal pointer retrieved may be invalidated by other calls, and this appears to be the test case itself isn't aware of this.

That, IMO, is unfortunate that someone has to basically do object validity tracking in Python when using libyang.

At any rate, I've confirmed locally that your patch works fine when building a Trixie-based SONiC image.