Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error handling on thread level #29

Open
dkropachev opened this issue Oct 28, 2024 · 5 comments
Open

Error handling on thread level #29

dkropachev opened this issue Oct 28, 2024 · 5 comments
Assignees
Labels
enhancement New feature or request

Comments

@dkropachev
Copy link
Collaborator

When the test starts c-s spins threads to generate a load, you can control the number of threads by -rate threads=X.
Unless -errors ignore is provided, the thread is going to be killed after reaching a number of errors.

The problem is that c-s does not stop the test when thread is killed, even if all of them are killed.

Recently we have merged #26, which has changed c-s behavior to the following:

  1. If -errors fail-fast is provided it would fail whole run after first thread is killed
  2. Otherwise it would keep waiting, unless all the threads are killed.

Now, in my book there are only three correct behaviors in regards to the threads handling:

  1. Fail the test when the first thread is killed. Currently works only if -errors fail-fast
  2. Not to kill a thread when error is encountered. keep it running. Currently works only if -errors ignore is provided
  3. Fail all thread at once when they reach error state, but keep them running untill that point. Not implemented currently

Anything in between I see as unwanted behavior that can lead to unexpected, unpredictable or inconsitant results.
On this issue I want to start discussion to decide on what is the correct behavior` and implement it eventually.

@dkropachev
Copy link
Collaborator Author

@CodeLieutenant , @Bouncheck , @fruch, please take a look

@CodeLieutenant CodeLieutenant self-assigned this Oct 29, 2024
@CodeLieutenant CodeLieutenant added the enhancement New feature or request label Oct 29, 2024
@CodeLieutenant
Copy link
Collaborator

3. Fail all thread at once when they reach error state, but keep them running untill that point. Not implemented currently

I don't really understand this point, this looks like to me a fail-fast. Kill all other threads when on thread reaches error state. Am I getting it correctly, or run and kill threads one by one, as they encounter errors.

We also need to decide what should be the default behaviour when -errors is not passed in, and whats the current state of the system

@dkropachev
Copy link
Collaborator Author

  1. Fail all thread at once when they reach error state, but keep them running untill that point. Not implemented currently

I don't really understand this point, this looks like to me a fail-fast. Kill all other threads when on thread reaches error state. Am I getting it correctly, or run and kill threads one by one, as they encounter errors.

We also need to decide what should be the default behaviour when -errors is not passed in, and whats the current state of the system

It is like fail-fast but it does no fail fast, it ensures that all threads are in failed state before failing stress command, which suppose to give more stability.

@CodeLieutenant
Copy link
Collaborator

It is like fail-fast but it does no fail fast, it ensures that all threads are in failed state before failing stress command, which suppose to give more stability.

So, fail threads one by one with no restart?

@dkropachev
Copy link
Collaborator Author

It is like fail-fast but it does no fail fast, it ensures that all threads are in failed state before failing stress command, which suppose to give more stability.

So, fail threads one by one with no restart?

No, fail them all at once, only when all threads can't execute a single query, otherwise all threads continue trying executing queries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants