Skip to content

[feat] log with rate limit#821

Merged
mag1c-h merged 1 commit intoModelEngine-Group:developfrom
Lijiachen1018:dev_logger_fix
Mar 30, 2026
Merged

[feat] log with rate limit#821
mag1c-h merged 1 commit intoModelEngine-Group:developfrom
Lijiachen1018:dev_logger_fix

Conversation

@Lijiachen1018
Copy link
Copy Markdown
Contributor

@Lijiachen1018 Lijiachen1018 commented Mar 16, 2026

Purpose

to implement a rate limit logger that suppress duplicate messages in a short time

Modifications

add a rate limit filter to the self-defined spdlog sink

Test

Usage of rate limit interface on python side

def test_log(test: str):
    logger.info_once("test log info once")
    logger.info_limit("test log info limit %s", test)

for i in range(5):
    test_log(f"test{i}")
image

will print when last log time has reached its time to live which set to 10s in testing and 60s on production

    import time
    for i in range(3):
        if i == 2:
            time.sleep(11)
        logger.info_limit("test message %d", i)
image

Usage of rate limit interface on CPP side

        for (int i = 0;i < 5;i++) {
            UC_INFO_LIMIT("test message {}", i);
        }
image

performance test

We run two performance tests on comparing the latency of duplicate filtering log interface and the plain log interface, by logging same messages / randomly generated message 2000 times.
image

@Lijiachen1018 Lijiachen1018 force-pushed the dev_logger_fix branch 2 times, most recently from 19ce951 to 5a04c8f Compare March 16, 2026 07:54
@Lijiachen1018 Lijiachen1018 force-pushed the dev_logger_fix branch 2 times, most recently from baee6c6 to 2cb6342 Compare March 18, 2026 07:08
@Lijiachen1018 Lijiachen1018 force-pushed the dev_logger_fix branch 3 times, most recently from 8ae429e to 50bb6b7 Compare March 18, 2026 12:39
qyh111
qyh111 previously approved these changes Mar 19, 2026
@Lijiachen1018 Lijiachen1018 force-pushed the dev_logger_fix branch 4 times, most recently from 6a5d4e0 to 9026009 Compare March 20, 2026 09:06
@Lijiachen1018 Lijiachen1018 force-pushed the dev_logger_fix branch 3 times, most recently from 1619a87 to 18f8f06 Compare March 21, 2026 08:47
@Lijiachen1018 Lijiachen1018 force-pushed the dev_logger_fix branch 3 times, most recently from 9b8abdc to 858f1ee Compare March 23, 2026 09:32
@Lijiachen1018 Lijiachen1018 force-pushed the dev_logger_fix branch 4 times, most recently from 40871c0 to e9b1b50 Compare March 24, 2026 08:32
@Lijiachen1018 Lijiachen1018 force-pushed the dev_logger_fix branch 3 times, most recently from a93e557 to eb7eaa1 Compare March 27, 2026 01:33
@mag1c-h mag1c-h merged commit 478d98b into ModelEngine-Group:develop Mar 30, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants