Skip to content

Query sharding proposal #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

moadz
Copy link
Owner

@moadz moadz commented May 9, 2022

For comments

@fpetkovski fpetkovski force-pushed the query-sharding-proposal branch 4 times, most recently from ca390ad to 3a42501 Compare May 10, 2022 11:02
heylongdacoder and others added 18 commits May 10, 2022 10:41
Add metrics about the hot and main caches[0].
* Number of bytes in each cache.
* Number of items in each cache.
* Counter of evictions from each cache.

[0]: https://pkg.go.dev/github.com/vimeo/galaxycache#CacheStats

Signed-off-by: SuperQ <[email protected]>
…s-io#5348)

* test: Added Alert compatibilty test.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Tmp.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Update.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* update.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* update.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* e2e: Refactored service helpers for newest e2e version.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Removed alert combatibiltiy test for now.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Fixed lint.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Fixed lint2.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Fixed nginx service.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Fixes.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Fix.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Fix.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* fix.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Refactored ruler.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Fixed test.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* fixes.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Fix.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Fixed compactor.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Fix.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* What about now?

Signed-off-by: Bartlomiej Plotka <[email protected]>
Use https://github.com/julienschmidt/httprouter#catch-all-parameters for
the groupcache route otherwise slashes in the cache's key gets
interpreted by the router and thus groupcache's function never gets
invoked, and all clients get 404.

Remove test regarding cache hit because now Thanos Store during test
constantly generates cache hits due to 1s delay between block
information refreshes.

Signed-off-by: Giedrius Statkevičius <[email protected]>
* Adds more info about the formatting part. Closes thanos-io#5282

Signed-off-by: Wiard van Rij <[email protected]>

* adds extra newline

Signed-off-by: Wiard van Rij <[email protected]>
…hanos-io#5358)

* e2e: Refactored Receive Builder to be consistent with other helpers.

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Addressed comments.

Signed-off-by: Bartlomiej Plotka <[email protected]>
Signed-off-by: GitHub <[email protected]>

Co-authored-by: yeya24 <[email protected]>
…o#5360)

* Enforce memcached concurrency limit with unbatched requests

This ensures that requests that are _not_ split into batches still count
towards the concurrency limit that the client enforces.

This fixes an issue introduced in thanos-io#5301

Signed-off-by: Nick Pillitteri <[email protected]>

* Lint fix

Signed-off-by: Nick Pillitteri <[email protected]>
I think I've found a replacement for the dead link.

Signed-off-by: Giedrius Statkevičius <[email protected]>
Add a unsafe codec which uses the given byte slices directly to avoid
copying - we are doing ioutil.ReadAll() either way so there is no need
to copy anything.

Signed-off-by: Giedrius Statkevičius <[email protected]>
…#5239)

* feat: a simple stacktrace utility

Signed-off-by: Bisakh Mondal <[email protected]>

* feat: custom errors package with new, errorf, wrapping, unwrapping and stacktrace

Signed-off-by: Bisakh Mondal <[email protected]>

* chore: update existing errors import (small subset)

Signed-off-by: Bisakh Mondal <[email protected]>

* chore: update comments

Signed-off-by: Bisakh Mondal <[email protected]>

* add errors into skip-files linter config

Signed-off-by: Bisakh Mondal <[email protected]>

* intoduce UnwrapTillCause to suffice the limitation of Unwrap

Signed-off-by: Bisakh Mondal <[email protected]>

* Revert "chore: update existing errors import (small subset)"

This reverts commit d27f017.

Signed-off-by: Bisakh Mondal <[email protected]>

* revert makefile && golangcilint file

Signed-off-by: Bisakh Mondal <[email protected]>

* apply PR feedbacks

Signed-off-by: Bisakh Mondal <[email protected]>

* stacktrace and errors test

Signed-off-by: Bisakh Mondal <[email protected]>

* fix typo

Signed-off-by: Bisakh Mondal <[email protected]>

* update stacktrace testing regex

Signed-off-by: Bisakh Mondal <[email protected]>

* add lint ignore for standard errors import inside errors pkg

Signed-off-by: Bisakh Mondal <[email protected]>

* [test files] add copyright headers

Signed-off-by: Bisakh Mondal <[email protected]>

* add no lint to avoid false misspell detection of keyword Tast

Signed-off-by: Bisakh Mondal <[email protected]>

* update stacktrace output test line number with regex pattern

Signed-off-by: Bisakh Mondal <[email protected]>

* return pc slice with reduced capacity

Signed-off-by: Bisakh Mondal <[email protected]>

* segregate formatted vs non formatted methods

Signed-off-by: Bisakh Mondal <[email protected]>

* update with only f functions

Signed-off-by: Bisakh Mondal <[email protected]>
…os-io#5356)

* Group memcached keys based on server when performing batch gets

Order and group keys during batch get operations based on the memcached
server they will be sharded to. This reduces the number of connections
that must be made within each batch of get operations.

Fixes thanos-io#5353

Signed-off-by: Nick Pillitteri <[email protected]>

* Code review changes

Signed-off-by: Nick Pillitteri <[email protected]>

* Fix error in testutil method added

Signed-off-by: Nick Pillitteri <[email protected]>

* Code review: comments for selector interface

Signed-off-by: Nick Pillitteri <[email protected]>
* pre compile regexp

Signed-off-by: Jin Dong <[email protected]>

* rename oppattern to labelvaluespattern

Signed-off-by: Jin Dong <[email protected]>
@fpetkovski fpetkovski force-pushed the query-sharding-proposal branch 5 times, most recently from 2cda8b0 to a37cf55 Compare May 31, 2022 07:37
* support series relabeling on Thanos receiver

Signed-off-by: Ben Ye <[email protected]>

* add changelog

Signed-off-by: Ben Ye <[email protected]>

* fix lint

Signed-off-by: Ben Ye <[email protected]>

* update lint

Signed-off-by: Ben Ye <[email protected]>

* fix e2e test

Signed-off-by: Ben Ye <[email protected]>

* fix relabel config pass

Signed-off-by: Ben Ye <[email protected]>

* cleanup white space

Signed-off-by: Ben Ye <[email protected]>

* address review comments

Signed-off-by: Ben Ye <[email protected]>

* address comments

Signed-off-by: Ben Ye <[email protected]>

* update comment

Signed-off-by: Ben Ye <[email protected]>
@fpetkovski fpetkovski force-pushed the query-sharding-proposal branch from a37cf55 to 295ca9b Compare May 31, 2022 18:16
@fpetkovski fpetkovski force-pushed the query-sharding-proposal branch from 295ca9b to 3b00514 Compare June 6, 2022 06:14
Jéssica Lins and others added 3 commits June 7, 2022 16:19
…5405)

* Error out earlier when building alertmanager config

Signed-off-by: Jéssica Lins <[email protected]>

* Add test case for empty host

Signed-off-by: Jéssica Lins <[email protected]>
…hanos-io#5392)

* add updated bingo .gitignore

Signed-off-by: B0go <[email protected]>

* update bingo minio version to commit 91130e884b5df59d66a45a0aad4f48db88f5ca63

Signed-off-by: B0go <[email protected]>

* trigger CI

Signed-off-by: B0go <[email protected]>
@fpetkovski fpetkovski force-pushed the query-sharding-proposal branch from 6ec3924 to 78edeea Compare June 8, 2022 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.