3BBBQM$T~2%g+kRKf2BsH$C}=
zSmQ>O@6k1t31B^B*Fh9um;%u@1HsEeeDkMf{TN)NNqiMp6H8wWazgLm5BXx)V$$z`&i9f$-^Rv`A0`o#%Exx9BR=eQj3wZL>NupWtzk#KpT
zIu#`+;N^yGTVOa9xsJz{R$H7dVBO^9hx47yz4gDUaPH6DZt>yVaVUuU%_K*P~tJ@dy=$+XJh7i>tcW#B3`f7$q{&*l1UF!9eTQ-_wVv6ss7L60jh
z$anoS`;Cnze#fY>YdO5vjuL;lty55QlB;PtZQVAq1F}!fy>{y)GE!o;gP}T$);gZq
z0kJd~hLgbb>Z0@*ClGQ4JO2!7Zw`rt;*s
zmYlWD{IgS@O{ZswGBwCL@9PLrzkDX+Xiyax(K#ZY0uSEf|k_
zs!R}>1L1rJ!uC8+UL;m$e1Oo&JnbniI-(Z2sz$%F_67GW-W);rwE-m4peGv22i(0H
zw}9s)zPT9w4_SfjUgddPk-ij{SRMyq>|T-civAxJ<}mV-=Ea3nxTFDWkr8NtXf@+Z
z?4pow!gCpDM7T*g=lfK{a|=`?xz~&sRJq&=6h3w-j~z;dbFo=(+vVc`@eu(o+Pe;=
zNr1SLI{IVzK-VEaK;fK|2z$>2{EPiNhl-*1L~80Ey%f$R=e_ZrjRT~{{?W*t=c02y
zRnM*G;v_)oe-X;Rt$$oo=ac91_aVhGzfAU(&vX$$ldXF*{dvE`L|~7mw{TIUq1QRv
z8-88kibRmt-pR7I_PU%`Kk2*8Ch%o!@Tddv4YJcy7V)3Z!u~%*3lH=E`6VW2zx5Nj
z`$D5W9OP_(Ly%mV5gxqQ1f1s)T<-R^Hg5RnM}(qSb5sNWaQA&iz+NX`Ypa9eu=p^3MiOSk~;SA_ZD+Nc-*K3)z?P~
za_Mrft&94E2*E?f(@Rja@a#8BmEGRhq2bnuC2#Mj5zVnQJy*6G%Gg&O$Cby_!xTrG
zd`F{{pf*#bT7&A(kx695-yjqLQ6Rs;>Ot^9Zn!}B0{VZmgDk5s*x!Myf^b4`gJj_r
zG^)da2jFT#;e%94468=N2jhlWd)!j1MbSN`v5t&13X6~6;BwH1)JL@iWQ;hZdExya
z#6-6l`Go-DwGIi=2DRTm!Gj5-i;qdLX;ong#s_lK6hfHneInEZ!4AV|jjPDCthE$?
zJ76vcZWZFX%3h#H)>MjcV)ZDu
+
+The following rankers were used:
+
+- **Diversity Ranker:** The Diversity Ranker enhances the diversity of the paragraphs selected for the context window.
+
+- **Lost In The Middle Ranker:** The Lost In The Middle Ranker optimizes the layout of the selected documents in the LLM’s context window.
+
+- **Transformers Similarity Ranker:** The Transformers Similarity Ranker ranks Documents based on how similar they are to the query. It uses a pre-trained cross-encoder model to embed both the query and the Documents. It then compares the embeddings to determine how similar they are.
+
+**Dense Retrieval:**
+
+For Dense retrieval, `INSTRUCTOR-XL` and `all-mpnet-base-v2` models were employed.
+
+
+
+**Hybrid Retrieval:**
+
+BM25 retrieval was used for Sparse retrieval in the Hybrid pipelines. The `bge-reranker-large` model was used in the Similarity Ranker, and `ms-marco-MiniLM-L-12-v2` for the Diversity Ranker.
+
+**Reciprocal Rank Fusion** (RRF) was used to combine the results for Hybrid retrieval.
+
+
+
+## Usage
+
+To run the pipelines, you will need to clone this repository and install the required libraries.
+
+1. Install the `rrf` package:
+
+```bash
+git clone https://github.com/avnlp/rrf
+cd rrf
+pip install -e .
+```
+
+2. To add the data to an index in Pinecone using the INSTRUCTOR-XL embedding model:
+
+```python
+cd src/rrf/indexing_pipeline/fiqa
+python pinecone_instructor_index.py
+```
+
+3. To run a specific pipeline you will have to go that file path and then run the file.
+For example, running the pipeline that uses dense retrieval with a combination of Diversity Ranker, Lost In The Middle Ranker and Similarity Ranker:
+
+```python
+cd src/rrf/pointwise/instructor_xl/fiqa/
+python dense_similarity_diversity_litm.py
+```
+
+## License
+
+The source files are distributed under the [MIT License](https://github.com/avnlp/rrf/blob/main/LICENSE).
diff --git a/paper/rankers_rrf.pdf b/paper/rankers_rrf.pdf
new file mode 100644
index 0000000000000000000000000000000000000000..45b3739f7f9f705af44e16ba38f45c02d80f9bec
GIT binary patch
literal 10430760
zcmeFZWpE_DwytT$Hnf>tW@ct)<~B1kGgF(HnVGrG%nWU2ZZosTJ>Nce&W<@Taev+U
zvn!%9L(