Multimodel RAG

[Aisuko]

Note: I combined two topics into one discussion in here.

RAG ranking

We will redesign the RAG pipeline add reranking(It is an optional feature in lancedb which we are using in voyager). Here, I'd like to introduce you a mechanism with name "multiple projects support each others". It's can be s ecosystem. For example, if we found stable useful features in project A and we will also migrate these features to project B.

https://artgor.medium.com/paper-review-rankrag-unifying-context-ranking-with-retrieval-augmented-generation-in-llms-7e5400ea74f2

Multiple-modal RAG

We already support load image through UI and we have did experiment with Transformer.js on image-to-text, the result is unacceptable also the audio-to-text.

Based on: https://arxiv.org/pdf/2103.00020