You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
embed_and_store() computes the dedup key as sha256(f"{url}|{i}|{chunk[:60]}") - only the first 60 characters of each chunk are hashed, not the full text.
If a scraped blog post is edited but the first 60 characters of a chunk stay the same, the new content is wrongly treated as a duplicate and skipped.
Result: the stale embedding stays in ChromaDB indefinitely, and your RAG pipeline analyzes outdated competitor data without you knowing.