swift_retain error when running smart-light on ESP32-C3

[cheif]

Hi!

I wanna start by saying thanks for the huge amount of work that has gone into making this whole swift-embedded/matter thing happen, I'm so excited to start writing my own firmware using this.

However, when I tried to get the smart-light example working on a board I've got at home (Wemos C3 mini v2.1.0, which is based on ESP32-C3FH4) I stumbled into some problems. Building and flashing works as expected, but when I run the code I get this crash:

Crash

ESP-ROM:esp32c3-api1-20210207
Build:Feb  7 2021
rst:0xc (RTC_SW_CPU_RST),boot:0xd (SPI_FAST_FLASH_BOOT)
Saved PC:0x40380684
0x40380684: esp_restart_noos at ~/esp/esp-idf/components/esp_system/port/soc/esp32c3/system_internal.c:111

SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd5820,len:0x1738
load:0x403cc710,len:0xb9c
load:0x403ce710,len:0x2e40
entry 0x403cc71a
I (24) boot: ESP-IDF v5.2.1 2nd stage bootloader
I (24) boot: compile time Jun 29 2024 13:09:23
I (24) boot: chip revision: v0.3
I (27) boot.esp32c3: SPI Speed      : 80MHz
I (32) boot.esp32c3: SPI Mode       : DIO
I (36) boot.esp32c3: SPI Flash Size : 4MB
I (46) boot: Partition Table:
I (50) boot: ## Label            Usage          Type ST Offset   Length
I (57) boot:  0 esp_secure_cert  unknown          3f 06 0000d000 00002000
I (65) boot:  1 nvs              WiFi data        01 02 00010000 0000c000
I (72) boot:  2 nvs_keys         NVS keys         01 04 0001c000 00001000
I (80) boot:  3 otadata          OTA data         01 00 0001d000 00002000
I (87) boot:  4 phy_init         RF data          01 01 0001f000 00001000
I (95) boot:  5 ota_0            OTA app          00 10 00020000 001e0000
I (102) boot:  6 ota_1            OTA app          00 11 00200000 001e0000
I (110) boot:  7 fctry            WiFi data        01 02 003e0000 00006000
I (117) boot:  8 coredump         Unknown data     01 03 003e6000 00010000
I (125) boot: End of partition table
I (129) esp_image: segment 0: paddr=00020020 vaddr=3c120020 size=39cc8h (236744) map
I (175) esp_image: segment 1: paddr=00059cf0 vaddr=3fc95400 size=04c18h ( 19480) load
I (179) esp_image: segment 2: paddr=0005e910 vaddr=40380000 size=01708h (  5896) load
I (181) esp_image: segment 3: paddr=00060020 vaddr=42000020 size=1118d4h (1120468) map
I (364) esp_image: segment 4: paddr=001718fc vaddr=40381708 size=13c00h ( 80896) load
I (386) boot: Loaded app from partition at offset 0x20000
I (387) boot: Disabling RNG early entropy source...
I (398) cpu_start: Unicore app
I (407) cpu_start: Pro cpu start user code
I (407) cpu_start: cpu freq: 160000000 Hz
I (407) cpu_start: Application information:
I (410) cpu_start: Project name:     light
I (415) cpu_start: App version:      1.0
I (419) cpu_start: Compile time:     Jun 29 2024 13:09:16
I (425) cpu_start: ELF file SHA256:  829cd3c71...
I (431) cpu_start: ESP-IDF:          v5.2.1
I (436) cpu_start: Min chip rev:     v0.3
I (440) cpu_start: Max chip rev:     v1.99
I (445) cpu_start: Chip rev:         v0.3
I (450) heap_init: Initializing. RAM available for dynamic allocation:
I (457) heap_init: At 3FCAAE90 len 00015170 (84 KiB): RAM
I (463) heap_init: At 3FCC0000 len 0001C710 (113 KiB): Retention RAM
I (470) heap_init: At 3FCDC710 len 00002950 (10 KiB): Retention RAM
I (477) heap_init: At 50000010 len 00001FD8 (7 KiB): RTCRAM
I (484) spi_flash: detected chip: generic
I (488) spi_flash: flash io: dio
W (493) rmt(legacy): legacy driver is deprecated, please migrate to `driver/rmt_tx.h` and/or `driver/rmt_rx.h`
I (503) sleep: Configure to isolate all GPIO pins in sleep state
I (509) sleep: Enable automatic switching of GPIO sleep configuration
I (517) esp_core_dump_flash: Init core dump to flash
I (522) esp_core_dump_flash: Found partition 'coredump' @ 3e6000 65536 bytes
I (532) esp_core_dump_flash: Core dump data checksum is correct
I (536) esp_core_dump_flash: Found core dump 2948 bytes in flash @ 0x3e6000
I (544) coexist: coex firmware version: 77cd7f8
I (550) coexist: coexist rom version 9387209
I (555) main_task: Started on CPU0
I (555) main_task: Calling app_main()
🏎️    Hello, Embedded Swift! (Smart Light)
I (565) led_driver_ws2812: Initializing light driver
I (565) led_driver_ws2812: led set r:0, g:0, b:0
Guru Meditation Error: Core  0 panic'ed (Illegal instruction). Exception was unhandled.

Core  0 register dump:
Stack dump detected
MEPC    : 0x42009e48  RA      : 0x42009598  SP      : 0x3fcae0a0  GP      : 0x3fc95c00
0x42009e48: $ss12swift_retain6objectBpBp_tF at ??:?
0x42009598: $s10__idf_main6MatterO4NodeCAEycfc at ??:?

TP      : 0x3fc76a84  T0      : 0x4005890e  T1      : 0x4038ac08  T2      : 0xffffffff
0x4005890e: memset in ROM
0x4038ac08: xQueueCreateMutex at ~/esp/esp-idf/components/freertos/FreeRTOS-Kernel/queue.c:673

S0/FP   : 0x3fcaed08  S1      : 0x3fcaed08  A0      : 0x3fcaed08  A1      : 0x3fcaed0c
A2      : 0x00000001  A3      : 0x00000001  A4      : 0x00000000  A5      : 0x600c2194
A6      : 0x000000ff  A7      : 0x000000ff  S2      : 0x3fcaeae8  S3      : 0x00000000
S4      : 0x00000000  S5      : 0x00000000  S6      : 0x00000000  S7      : 0x00000000
S8      : 0x00000000  S9      : 0x00000000  S10     : 0x00000000  S11     : 0x00000000
T3      : 0x3fca9000  T4      : 0x00000001  T5      : 0x3fca9000  T6      : 0x00000001
MSTATUS : 0x00001881  MTVEC   : 0x40380001  MCAUSE  : 0x00000002  MTVAL   : 0x00c5a5af
0x40380001: _vector_table at ??:?

MHARTID : 0x00000000


Backtrace:


0x42009e48 in $ss12swift_retain6objectBpBp_tF ()
#0  0x42009e48 in $ss12swift_retain6objectBpBp_tF ()
#1  0x42009598 in $s10__idf_main6MatterO4NodeCAEycfc ()
Backtrace stopped: frame did not save the PC
ELF file SHA256: 829cd3c71

I (940) esp_core_dump_flash: Save core dump to flash...
I (946) esp_core_dump_flash: Erase flash 4096 bytes @ 0x3e6000

My thinking is that the IllegalInstruction followed by swift_retain means that swift_retain for some reason isn't linked correctly, but I can't say I understand this good enough to know if that's a possibility.

Following the above, I've managed to understand that this has something to do with _ = Unmanaged.passRetained(self) that happens in a bunch of places, e.g. Matter.Node.init, commenting this out let's the program continue a bit longer, but still crashes later on.

A more minimal example that also crashes

@_cdecl("app_main")
func app_main() {
  Retaining()

  while true {
    sleep(1)
  }
}

class Retaining {
    init() {
        print("Retained") // If this is removed the code doesn't crash, for some reason
        _ = Unmanaged.passRetained(self)
        print("Retained")
    }
}

I've tried verifying that swift_retain is actually linked, by running nm -gm build/light.elf | grep swift (Am I doing it right?), which lists swift_retain at least:

42009e38 T $ss12swift_retain6objectBpBp_tF
42009e4e T $ss13swift_release6objectyBp_tF
42009db4 T $ss17swift_allocObject8metadata12requiredSize0E13AlignmentMaskBpBp_S2itF
42009db6 T $ss17swift_allocObject8metadata12requiredSize0E13AlignmentMaskSpys04HeapC0VGSpys13ClassMetadataVG_S2itF
42009e1a T $ss21swift_initStackObject8metadata6objectBpBp_BptF
3fc958b4 D $ss23_swiftEmptyArrayStorageSi_S3itvp
42009e8c T $ss24swift_deletedMethodErrors5NeverOyF
42009e28 T $ss26swift_deallocClassInstance6object13allocatedSize0F9AlignMaskyBp_S2itF
42009e7c T $ss41swift_isUniquelyReferenced_nonNull_native6objectSbBp_tF
4200901e T swift_allocObject
42009ad6 T swift_deallocClassInstance
42009e8a T swift_deletedMethodError
42009a2e T swift_initStackObject
42009e96 T swift_isUniquelyReferenced_nonNull_native
420095c6 T swift_release
42009a1e T swift_retain

I'd be happy to continue investigating/fixing this, but it feels like I've gotten as far as I can without external support by now 😅

[cheif]

Note: I can run the led-blink example on the same board without crashes (although the onboard led doesn't blink), by just adding the same lines as in the smart-light example to CMakeLists.txt

+    elseif("${IDF_TARGET}" STREQUAL "esp32c3")
+        set(ENV{ESP_MATTER_DEVICE_PATH} $ENV{ESP_MATTER_PATH}/device_hal/device/esp32c3_devkit_m)

[haoliver]

I had a similar crash when compiling with -Onone option. Looking at the GitHub swift-embedded-examples CMakeLits.txt, I noticed that the matter examples CMakeLists.txt are missing options below for ESP32-C3
-Xcc -march=rv32imc_zicsr_zifencei
-Xcc -mabi=ilp32

Adding those fixed the crash for me

[cheif]

@haoliver Thank you so much for finding this out, this did solve my problem, and I opened a PR so that hopefully more people will benefit. 🙏

[jgranie]

Hi,
On the led-blink with the flags mentionned above I still get errors (no crash) :
The gap between .flash.appdesc and .flash.rodata must not exist to produce the final bin image
The gap between .flash.rodata and .eh_frame_hdr must not exist to produce the final bin image
Did you perform modifications other than adding Xcc flags and the esp32c3 else if ? (for example in the partitions.csv)

[cheif]

@jgranie I'm not sure that this is the same problem we're seeing, I had no issues at all with the led-blink example, just with the smart-light one. And that one compiled fine (AKA no compile errors), I just had runtime crashes.

[jgranie]

Thanks for your reply. Yes it seems not correlated. I will investigate a bit further and post a new issue.