How does Ragflow "unlearn" a document?

[ralienpp]

How does ragflow handle the deletion or update of a document? Can I be certain that any references to information extracted from the removed or old document will not influence the system's response in any way?

My limited understanding is that when a document is ingested, the vector database contains fragments of data from the document. If the document were to be removed, how are all the relevant pieces scattered across the vector database removed along with it?

[dosubot[bot]]

Hi @ralienpp! I'm Dosu and I’m helping the ragflow team.

RAGFlow "unlearns" a document by systematically removing all fragments (chunks) of that document from the vector database and associated storage. When you delete a document, the system uses the remove_document method in the DocumentService class to orchestrate this process. This method searches for all chunks associated with the document, deletes them from the vector database (Elasticsearch), removes any related files from storage, and updates the knowledge graph to eliminate references to the document. It also decrements the relevant counters in the knowledge base to reflect the removal of tokens and chunks. This ensures that no fragments of the deleted document remain in the retrieval system, so information from the removed document cannot influence future responses in any way see implementation.

For document updates, RAGFlow uses similar mechanisms: it can update or remove specific chunks using the update and delete methods in the vector database connection layer (DocStoreConnection). Chunk-level operations are also exposed via API endpoints, allowing for granular control over which pieces of a document are updated or removed see chunk operations.

You can be confident that, once a document is deleted, all its fragments are removed from the vector database and storage, and the knowledge graph is updated accordingly. This prevents any references to information from the removed or outdated document from influencing the system’s responses details. The reliability of this process depends on the underlying vector database and storage connectors, but the orchestration logic in RAGFlow is designed to ensure complete removal.

To reply, just mention my name @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}