Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support of the new data structure HyperLogLog #1307

Open
PragmaTwice opened this issue Mar 9, 2023 · 16 comments
Open

Add support of the new data structure HyperLogLog #1307

PragmaTwice opened this issue Mar 9, 2023 · 16 comments
Assignees
Labels
feature type new feature help wanted Good for newcomers

Comments

@PragmaTwice
Copy link
Member

PragmaTwice commented Mar 9, 2023

As a plan in Kvrocks 2023 Roadmap, we are eager to explore the ability to implement HyperLogLog commands in Redis for kvrocks.

Everyone is welcome to ask questions, or contribute your ideas/PRs.

@PragmaTwice PragmaTwice added help wanted Good for newcomers feature type new feature labels Mar 9, 2023
@dongdongwcpp
Copy link
Contributor

dongdongwcpp commented Mar 12, 2023

I am working on this,just assign to me

@PragmaTwice
Copy link
Member Author

Glad to hear that. Assigned.

@PragmaTwice
Copy link
Member Author

Hi @dongdongwcpp, how is it going now?

@infdahai
Copy link
Contributor

infdahai commented Jun 17, 2023

I'd like to design the part in kvrocks. And I'm considering the storage about hyperloglog.

@infdahai
Copy link
Contributor

In a few days, I will present a concrete idea.

@PragmaTwice
Copy link
Member Author

@dongdongwcpp Released for long time nonresponse. Feel free to participate the community via other tasks : )

@PragmaTwice
Copy link
Member Author

In a few days, I will present a concrete idea.

Thanks! Assigned.

@PragmaTwice PragmaTwice assigned infdahai and unassigned dongdongwcpp Jul 3, 2023
@infdahai
Copy link
Contributor

infdahai commented Jul 3, 2023

In a few days, I will present a concrete idea.

Thanks! Assigned.

Some questions I haven't thought through, and time is limited, so it's going to take some time.

@PragmaTwice
Copy link
Member Author

@infdahai How is it going now? Feel free to share your problems with us.

@Yangsx-1
Copy link
Contributor

I'd like to take over this task. @PragmaTwice

@PragmaTwice
Copy link
Member Author

I'd like to take over this task. @PragmaTwice

Great! Assigned.

@PragmaTwice PragmaTwice assigned Yangsx-1 and unassigned infdahai Nov 21, 2023
@qujihan
Copy link

qujihan commented Jan 26, 2024

Hello, I have observed that this issue has not progressed for a long time.. Are you still doing it now? If not, I want to try it. @Yangsx-1

@git-hulk
Copy link
Member

@qujihan Sure, I have changed the assigned. Thank you!

@mapleFU
Copy link
Member

mapleFU commented Jan 26, 2024

You can first draft a format in discussion, and then move forward to implement it.

Some database uses HLL to optimize the ndv:

  1. https://github.com/facebookincubator/velox/tree/main/velox/common/hyperloglog
  2. https://github.com/ClickHouse/ClickHouse/blob/92e63bf45c3cd36163131e42d4bd003e543031f4/src/Common/HyperLogLogCounter.h

You can choose them or using similar like HLL in Redis

@Yangsx-1
Copy link
Contributor

Hello, I have observed that this issue has not progressed for a long time.. Are you still doing it now? If not, I want to try it. @Yangsx-1

Just do it! I'm very busy these days, you can take this task.

@tutububug
Copy link
Contributor

@PragmaTwice I create a PR #2142 for support hyperloglog.
The core algorithm code is transplanted from redis, and may have optimization room on disk storage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature type new feature help wanted Good for newcomers
Projects
Status: In Progress
Development

No branches or pull requests

8 participants