Skip to content

Conversation

@bytecii
Copy link
Collaborator

@bytecii bytecii commented Jan 22, 2026

Description

Validate the model before the task starts and shows the errors if the basic validation fails.

Examples:

image image

Fixes #1010
Fixes #1034

What is the purpose of this pull request?

  • Bug fix
  • New Feature
  • Documentation update
  • Other

bytecraftii added 2 commits January 21, 2026 17:57
@bytecii bytecii self-assigned this Jan 22, 2026
@LuoPengcheng12138
Copy link
Collaborator

some questions:

  1. In backend/app/service/error_handler.py, it seems difficult to ensure that ModelProcessingError is only raised when item.action == Action.new_task_state (the task may directly raise Exception, or the agent may also raise ModelProcessingError for other actions).
  2. In backend/app/service/error_handler.py, does the should_stop_task function guarantee that for all BYOK models, the returned error codes are exactly "invalid_api_key", "insufficient_quota", and "model_not_found"?
    Perhaps a better approach is to perform a pre-check on the API key before the task starts, rather than handling the thrown errors during runtime.
    cc @fengju0213 @a7m-1st

@bytecii
Copy link
Collaborator Author

bytecii commented Jan 23, 2026

@LuoPengcheng12138 Updated with model validation at first.

@LuoPengcheng12138
Copy link
Collaborator

@LuoPengcheng12138 Updated with model validation at first.

hi, @bytecraftii Merge is good to go now. One more thing: we need to remove the Python pre-commit hook, as Wendong pointed out that we’re making a large number of updates to the project right now, and adding changes like linting rules will make PR reviews much more difficult.

@bytecii
Copy link
Collaborator Author

bytecii commented Jan 23, 2026

@LuoPengcheng12138 Updated with model validation at first.

hi, @bytecraftii Merge is good to go now. One more thing: we need to remove the Python pre-commit hook, as Wendong pointed out that we’re making a large number of updates to the project right now, and adding changes like linting rules will make PR reviews much more difficult.

For now the pre-commit will not be automatically run, need to be trigger manually. Also I do think try to make the code cleaner will help to make the code review life easier in the long run.

@Wendong-Fan Wendong-Fan added this to the Sprint 13 milestone Feb 1, 2026
@Wendong-Fan Wendong-Fan requested a review from 4pmtong February 2, 2026 12:41
@4pmtong
Copy link
Collaborator

4pmtong commented Feb 6, 2026

Hi @bytecii how about reuse the validation logic in #1155 ?

Comment on lines +709 to +710
# Make a simple test call in executor to avoid blocking
loop = asyncio.get_event_loop()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change to get_running_loop()? Since get_event_loop() is about deprecation

Comment on lines +167 to +168
mock_agent.step = Mock(
side_effect=Exception("Error code: 429 - Rate limit exceeded")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we treat rate limit as invalid signal? I think rate limit is temporal

Comment on lines +711 to +712
await loop.run_in_executor(None, lambda: agent.step("test"))

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also add a timeout in external? like:

try:
      await asyncio.wait_for(
          loop.run_in_executor(None, lambda: agent.step("test")),
          timeout=30.0
      )
  except asyncio.TimeoutError:
      return False, "timed out"

Copy link
Contributor

@Wendong-Fan Wendong-Fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @bytecii ! I think this validation may be doing duplicate work. We already validate the model configuration via /model/validate on the settings page before users start tasks, so adding this would introduce extra latency to every task.

the root cause is in question_confirm when an API error occurs, the exception handler does return True instead of propagating the error. This silently swallows the failure and allows the task to continue running.

A much simpler fix would be to change that return True to raise. This lets the error bubble up to the main loop and be sent back to the frontend via SSE, no additional validation step required, WDYT?

cc @nitpicker55555 @4pmtong @LuoPengcheng12138

@Wendong-Fan
Copy link
Contributor

added enhance PR #1170 based on review, feel free to check

@bytecii
Copy link
Collaborator Author

bytecii commented Feb 6, 2026

@Wendong-Fan Ok sounds good

@bytecii bytecii closed this Feb 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

style: add Python pre-commit [Feature Request] Abnormal frontend behavior when BYOK model key becomes invalid

5 participants