You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently our LakeFSLoader loads document without metadata object tags and only adds the logical path tag as source key to Document. We ignore any user set object metadata.
So What?
This means that the LakeFSLoader is not helpful the moment I need to use object metadata for embedding, retrieval etc.
For example, very common case in RAG of time weighted retriever.
To use the object tags I would have to implement custom logic for this listing all documents and getting the tags, bottom line it means I don't have a use in LakeFSLoader since I would have to load everything anyway myself.
Suggestion
When calling LakeFSLoader constructor in Langchain, add additional optional param to indicate id metadata tags should be added to the Document.
Problem
Currently our LakeFSLoader loads document without metadata object tags and only adds the logical path tag as source key to Document. We ignore any user set object metadata.
So What?
This means that the LakeFSLoader is not helpful the moment I need to use object metadata for embedding, retrieval etc.
For example, very common case in RAG of time weighted retriever.
To use the object tags I would have to implement custom logic for this listing all documents and getting the tags, bottom line it means I don't have a use in LakeFSLoader since I would have to load everything anyway myself.
Suggestion
When calling LakeFSLoader constructor in Langchain, add additional optional param to indicate id metadata tags should be added to the Document.
The text was updated successfully, but these errors were encountered: