You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The indexer saves the index in chunks (see the index function for reference). However, the number of chunks into which the data is stored is figured out from the size of the collection and the chunksize from the get_chunksize function:
The indexer saves the index in chunks (see the
index
function for reference). However, the number of chunks into which the data is stored is figured out from the size of the collection and the chunksize from theget_chunksize
function:https://github.com/codetalker7/ColBERT.jl/blob/6e0f89f61abf421755f5201d39538d15f36d2321/src/indexing/collection_indexer.jl#L151
It will be good to let users control what chunksize they want to use; this will also be helpful for testing.
The text was updated successfully, but these errors were encountered: