Skip to content

Issues: embeddings-benchmark/mteb

MMTEB Overview
#360 opened Apr 13, 2024 by KennethEnevoldsen
Open 4
Language Overview
#366 opened Apr 14, 2024 by KennethEnevoldsen
Open
Finalizing MMTEB
#784 opened May 22, 2024 by KennethEnevoldsen
Open 6
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

84 Open 229 Closed
84 Open 229 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

LLM2Vec inference error
#997 opened Jun 27, 2024 by AlexeyVatolin
calculate_metadata_metrics on MSMARCOv2 goes OOM
#992 opened Jun 27, 2024 by isaac-chung
Adding a BM25 baseline? enhancement New feature or request
#990 opened Jun 26, 2024 by malteos
2
Option to remove cached dataset files on large runs enhancement New feature or request
#984 opened Jun 25, 2024 by isaac-chung
2
Incorrect logic when retrieval top-k question Further information is requested
#978 opened Jun 23, 2024 by thangylvp
1
Some retrieval datasets don't calculate metadata correctly bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed
#964 opened Jun 19, 2024 by orionw
@KennethEnevoldsen @isaac-chung
4
Be able to cache embeddings and load them enhancement New feature or request good first issue Good for newcomers
#946 opened Jun 17, 2024 by orionw
5
Allow for model specific similarity in Bitext Mining and Retrieval enhancement New feature or request
#943 opened Jun 17, 2024 by KennethEnevoldsen
Avoid using global seeds enhancement New feature or request
#942 opened Jun 17, 2024 by KennethEnevoldsen
Standardising matrix arrays (adopt Array API spec) enhancement New feature or request
#941 opened Jun 17, 2024 by KennethEnevoldsen
Remove CrosslingualTask & MultilingualTask? enhancement New feature or request good first issue Good for newcomers
#933 opened Jun 15, 2024 by Muennighoff
1
End-to-end integration / testing with leaderboard
#932 opened Jun 15, 2024 by Muennighoff
4
issue on trust remote code enhancement New feature or request
#930 opened Jun 15, 2024 by westonli-thu
17
Add Voyage multilingual datasets good first issue Good for newcomers new-dataset
#920 opened Jun 14, 2024 by Muennighoff
1
Integrate InstructIR with MTEB
#905 opened Jun 12, 2024 by henilp105
10
paper writing;: table overview of tasks
#897 opened Jun 10, 2024 by KennethEnevoldsen
5
Paper Writing: An overview issue
#896 opened Jun 10, 2024 by KennethEnevoldsen
1 of 29 tasks
6
Model which we need to implement
#879 opened Jun 4, 2024 by KennethEnevoldsen
10 of 13 tasks
12
Integrate RAR-b good first issue Good for newcomers new-dataset
#859 opened May 31, 2024 by Muennighoff
About Evaluation Scripts
#857 opened May 31, 2024 by twadada
1
Versioning for AbsTasks
#851 opened May 30, 2024 by x-tabdeveloping
1
Paper: Multilevel clustering and Multi clf
#849 opened May 30, 2024 by x-tabdeveloping
@x-tabdeveloping
2
Problems with mteb_meta for german evaluation bug Something isn't working
#847 opened May 29, 2024 by achibb
6
running baseline models
#840 opened May 28, 2024 by KennethEnevoldsen
@Muennighoff
6
Paper segment: Score aggregation
#839 opened May 28, 2024 by KennethEnevoldsen
@KennethEnevoldsen @vaibhavad
9
ProTip! Mix and match filters to narrow down what you’re looking for.