Memory growth issue while using daq mode

[huangbinghao]

Dear author!
I encountered a memory growth issue while using daq mode. Specifically, when I enable synchronization:
self._xcp_master.startStopSynch(0x01) # Command Code=0xDD start synch
the memory of the process will increase over time:
(my own log file)
...
Time: 2024-09-24 16:27:22, PID: 454658, RSS: 154.58 MB, VMS: 1857.73 MB, System memory: 2962.98 MB
Time: 2024-09-24 16:27:25, PID: 454658, RSS: 155.61 MB, VMS: 1858.98 MB, System memory: 2961.78 MB
...
Time: 2024-09-24 16:27:49, PID: 454658, RSS: 164.38 MB, VMS: 1867.23 MB, System memory: 2952.36 MB
Time: 2024-09-24 16:27:52, PID: 454658, RSS: 165.41 MB, VMS: 1868.23 MB, System memory: 2954.08 MB
...
My software version is: 0.21.11

Look forward to your reply！

[huangbinghao]

I noticed that the self. daqQueue=deque() in the transport/base.py seems to be growing. Is there any suitable release mechanism for this data structure

[christoph2]

Yes, your're right, the daqQueue should be consumed, otherwise the memory usage grows unbounded...

The handling of received frames could be handled by so called policies; here is a basic example to record raw XCP frames to a proprietary binary file format.

It looks like you are running DAQ lists and here are the good news:
DAQ list setup and recording are built-in features in the newest releases.

Please refer to run_daq.py which was tested with CANape's XCPsim and XCPlite.
Please note: The XCPlite addresses are certainly not working on your system, so make sure to adapt them -- on the other hand you may start directly with your on DAQ list setup.

There are three further examples ex_mdf, ex_arrow, ex_sqlite.
demonstrating what could be done with the recorded DAQ data.

P.S.: I'm currently working hard on documentation (Installation on Linux + tutorial).
Please also consider reading this discussion where I've written about some details.