Skip to content

bugfix: reconcile max_seqs_per_batch with max_concurrent_requests. - #2016

Merged
Kang-Meng merged 1 commit into
xLLM-AI:mainfrom
Kang-Meng:bugfix_singleblock
Jul 24, 2026
Merged

bugfix: reconcile max_seqs_per_batch with max_concurrent_requests.#2016
Kang-Meng merged 1 commit into
xLLM-AI:mainfrom
Kang-Meng:bugfix_singleblock

Conversation

@Kang-Meng

@Kang-Meng Kang-Meng commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator
  • Align the scheduler batch cap and the service admission cap at
    startup in initialize_configs(): min when both > 0, the non-zero
    one when one is 0, LOG(FATAL) when both are 0. Downstream users
    (Master, Engine, BlockManagerPool, RateLimiter, ProfileManager)
    now see the same effective cap on both fields.
  • Drop the scheduler default from 1024 to 200 so it matches
    ServiceConfig::max_concurrent_requests out of the box (keeps the
    gflag DEFINE_* and the PROPERTY in-class default in sync per
    custom-code-style §10).
  • Fix VLM and Rec engines silently dropping max_seqs_per_batch
    when building BlockManagerPool::Options -- the SINGLE block pool
    was falling back to a 2-slot capacity (1 padding + 1 usable),
    effectively capping concurrent sequences at 1 on those paths.
  • Tighten a few unit tests that relied on the pool default being
    0 by setting .max_seqs_per_batch(0) explicitly, so the intent to
    drive the SINGLE pool from num_single_blocks alone stays local
    and does not depend on Options' default value.
image

Comment thread tests/core/framework/block/block_manager_test.cpp Outdated
Comment thread xllm/xllm.cpp
@yingxudeng

Copy link
Copy Markdown
Collaborator

目前 cicd npu 失败,请补充本地 npu 通过所有单测的截图

@Kang-Meng
Kang-Meng force-pushed the bugfix_singleblock branch from ccf9e1f to 62c356a Compare July 24, 2026 02:42
@Kang-Meng
Kang-Meng merged commit 3e6917f into xLLM-AI:main Jul 24, 2026
10 of 26 checks passed
@Kang-Meng
Kang-Meng deleted the bugfix_singleblock branch July 24, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants