Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let users specify number of chunks in the config. #9

Closed
codetalker7 opened this issue Jul 12, 2024 · 1 comment
Closed

Let users specify number of chunks in the config. #9

codetalker7 opened this issue Jul 12, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@codetalker7
Copy link
Member

The indexer saves the index in chunks (see the index function for reference). However, the number of chunks into which the data is stored is figured out from the size of the collection and the chunksize from the get_chunksize function:

https://github.com/codetalker7/ColBERT.jl/blob/6e0f89f61abf421755f5201d39538d15f36d2321/src/indexing/collection_indexer.jl#L151

It will be good to let users control what chunksize they want to use; this will also be helpful for testing.

@codetalker7 codetalker7 added the enhancement New feature or request label Jul 12, 2024
@codetalker7 codetalker7 self-assigned this Jul 12, 2024
@codetalker7
Copy link
Member Author

Update: was solved in #27.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant