feat: Add LangChain vector store adapter for CrateDB #6011
Conversation
dosubot
bot
added
size:L
dosubot
bot
added
size:XL
amotl
changed the title
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
|
Hey @amotl I was not able to run Langflow because the icon you provided is not valid. Could you take a look at it, please? |
There was a problem hiding this comment.
Hi @ogabrielluiz, thanks for your reply. I hope we can resolve the icon/logo issue soon. I've provided a few bits of information below, within a PR discussion thread attached to the cratedb.svg file.
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughA new CrateDB vector store component was added, including backend logic, documentation, integration tests, and frontend icon integration. The backend supports ingesting and searching documents in CrateDB, while the frontend introduces a CrateDB icon. The dependency Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CrateDBVectorStoreComponent
participant CrateDBVectorStore
participant CrateDB
User->>CrateDBVectorStoreComponent: Provide configuration (URL, collection, embedding, etc.)
CrateDBVectorStoreComponent->>CrateDBVectorStore: Build or load vector store
CrateDBVectorStore->>CrateDB: Connect and initialize collection/table
User->>CrateDBVectorStoreComponent: Ingest documents
CrateDBVectorStoreComponent->>CrateDBVectorStore: Add documents
CrateDBVectorStore->>CrateDB: Store embeddings
User->>CrateDBVectorStoreComponent: Search query
CrateDBVectorStoreComponent->>CrateDBVectorStore: Perform similarity/MMR search
CrateDBVectorStore->>CrateDB: Query embeddings
CrateDB-->>CrateDBVectorStore: Return search results
CrateDBVectorStore->>CrateDBVectorStoreComponent: Return results
CrateDBVectorStoreComponent-->>User: Return search results as Data objects
Suggested labels
Suggested reviewers
✨ Finishing Touches🧪 Generate Unit Tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (3)
src/frontend/src/icons/CrateDB/index.tsx (1)
1-9: Refine the props type to avoid using{}.
Biome flags{}as a banned type. UseReact.SVGProps<SVGSVGElement>to accurately type SVG props:-export const CrateDBIcon = forwardRef< - SVGSVGElement, - React.PropsWithChildren<{}> -> ((props, ref) => { +export const CrateDBIcon = forwardRef< + SVGSVGElement, + React.SVGProps<SVGSVGElement> +>((props, ref) => { return <SvgCrateDBIcon ref={ref} {...props} />; });src/frontend/src/icons/CrateDB/CrateDB.jsx (1)
3-8: Consider making size theme-awareHard-coding
width="600"/height="600"forces a 600×600 icon unless callers override both props.
Omitting them (or using1em) lets consumers size the icon with CSS like every other icon in the set.src/backend/tests/integration/components/cratedb/test_cratedb_component.py (1)
66-69: Dimension initialisation likely too small
_init_models_with_dimensionality(3)configures vectors of length 3, whereas OpenAI embeddings are 1536-D.
Using an arbitrary small value may mask real-world errors. Prefervector_store._init_models_with_dimensionality(1536)or compute dynamically from the embedding.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
src/frontend/src/icons/CrateDB/cratedb.svgis excluded by!**/*.svguv.lockis excluded by!**/*.lock
📒 Files selected for processing (9)
docs/docs/Components/components-vector-stores.md(1 hunks)pyproject.toml(1 hunks)src/backend/base/langflow/components/vectorstores/__init__.py(2 hunks)src/backend/base/langflow/components/vectorstores/cratedb.py(1 hunks)src/backend/tests/integration/components/cratedb/test_cratedb_component.py(1 hunks)src/frontend/src/icons/CrateDB/CrateDB.jsx(1 hunks)src/frontend/src/icons/CrateDB/index.tsx(1 hunks)src/frontend/src/icons/eagerIconImports.ts(2 hunks)src/frontend/src/icons/lazyIconImports.ts(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (5)
src/backend/base/langflow/components/vectorstores/__init__.py (1)
src/backend/base/langflow/components/vectorstores/cratedb.py (1)
CrateDBVectorStoreComponent(11-70)
src/frontend/src/icons/eagerIconImports.ts (1)
src/frontend/src/icons/CrateDB/index.tsx (1)
CrateDBIcon(4-9)
src/frontend/src/icons/CrateDB/index.tsx (1)
src/frontend/src/icons/CrateDB/CrateDB.jsx (1)
SvgCrateDBIcon(1-74)
src/backend/base/langflow/components/vectorstores/cratedb.py (3)
src/backend/base/langflow/base/vectorstores/model.py (2)
LCVectorStoreComponent(41-193)check_cached_vector_store(17-38)src/backend/base/langflow/helpers/data.py (1)
docs_to_data(14-23)src/backend/base/langflow/schema/data.py (2)
Data(23-275)to_lc_document(139-149)
src/backend/tests/integration/components/cratedb/test_cratedb_component.py (2)
src/backend/base/langflow/components/vectorstores/cratedb.py (4)
CrateDBVectorStoreComponent(11-70)cratedb_collection_to_data(73-90)build_vector_store(32-56)search_documents(58-70)src/backend/base/langflow/schema/data.py (1)
Data(23-275)
🪛 Biome (1.9.4)
src/frontend/src/icons/CrateDB/index.tsx
[error] 6-6: Don't use '{}' as a type.
Prefer explicitly define the object shape. '{}' means "any non-nullable value".
(lint/complexity/noBannedTypes)
🪛 Pylint (3.3.7)
src/backend/tests/integration/components/cratedb/test_cratedb_component.py
[error] 15-15: No name 'components' in module 'langflow'
(E0611)
[error] 16-16: No name 'schema' in module 'langflow'
(E0611)
[error] 42-42: No name 'components' in module 'langflow'
(E0611)
🔇 Additional comments (8)
pyproject.toml (1)
100-100: Verify the new package dependency constraint.
Ensure thatlangchain-cratedb<0.2exists on PyPI and aligns with the compatibility requirements. Consider specifying a minimum tested version (e.g.,>=0.1.0,<0.2) to avoid untested or breaking releases.Run this script to list available versions:
#!/bin/bash # Verify available versions of langchain-cratedb on PyPI curl -s https://pypi.org/pypi/langchain-cratedb/json | jq '.releases | keys'src/frontend/src/icons/eagerIconImports.ts (2)
25-25: Add CrateDBIcon import to eager icon set.
The import path and symbol name are consistent with other icon definitions.
143-143: Register CrateDB in eagerIconsMapping.
Using the key"CrateDB"matches the component name, and its alphabetical placement afterCouchbaseis correct.src/backend/base/langflow/components/vectorstores/__init__.py (2)
8-8: Expose CrateDBVectorStoreComponent.
The import from.cratedbaligns with the module structure and mirrors other vector store components.
35-35: Export CrateDBVectorStoreComponent in all.
Including the new component in__all__makes it publicly accessible via the vectorstores package.src/frontend/src/icons/lazyIconImports.ts (1)
67-68: Add lazy-loaded CrateDB icon mapping.
The async import path is correct and returns theCrateDBIconcomponent as default.docs/docs/Components/components-vector-stores.md (1)
431-438: Document all supported search types & correct defaultsThe component accepts
"similarity_score_threshold"(see tests) and defaultsnumber_of_resultsto4, not10.Please update the search_type row and the default column to match the implementation so users don’t configure invalid values.
src/backend/tests/integration/components/cratedb/test_cratedb_component.py (1)
24-30: Skip tests when CrateDB isn’t reachableThe fixture always drops tables; on a machine without a running CrateDB instance this raises a connection error and fails the whole suite.
Recommend:
try: engine = sa.create_engine(CRATEDB_SQLALCHEMY_URL) with engine.connect() as connection: ... except sa.exc.OperationalError: pytest.skip("CrateDB instance not available")
| inputs = [ | ||
| SecretStrInput(name="server_url", display_name="CrateDB SQLAlchemy URL", required=True), | ||
| StrInput(name="collection_name", display_name="Table", required=True), | ||
| *LCVectorStoreComponent.inputs, | ||
| HandleInput(name="embedding", display_name="Embedding", input_types=["Embeddings"], required=True), | ||
| IntInput( | ||
| name="number_of_results", | ||
| display_name="Number of Results", | ||
| info="Number of results to return.", | ||
| value=4, | ||
| advanced=True, | ||
| ), | ||
| ] |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Missing search_type input
LCVectorStoreComponent’s base helpers expect self.search_type, and tests set it, yet the input isn’t declared here.
Add a StrInput with choices ("Similarity", "MMR", "similarity_score_threshold") to prevent AttributeError at runtime.
🤖 Prompt for AI Agents
In src/backend/base/langflow/components/vectorstores/cratedb.py around lines 17
to 29, the inputs list is missing a declaration for the search_type input, which
is expected by LCVectorStoreComponent and used in tests. Add a StrInput named
"search_type" with display_name "Search Type" and choices set to ["Similarity",
"MMR", "similarity_score_threshold"] to the inputs list to ensure the attribute
exists and prevent runtime AttributeError.
| data_dict = { | ||
| "id": doc.id, | ||
| "text": doc.document, | ||
| } | ||
| data_dict.update(doc.cmetadata) | ||
| data.append(Data(**data_dict)) |
There was a problem hiding this comment.
Guard against None metadata
doc.cmetadata can be None, causing dict.update(None) → TypeError.
- data_dict.update(doc.cmetadata)
+ if doc.cmetadata:
+ data_dict.update(doc.cmetadata)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| data_dict = { | |
| "id": doc.id, | |
| "text": doc.document, | |
| } | |
| data_dict.update(doc.cmetadata) | |
| data.append(Data(**data_dict)) | |
| data_dict = { | |
| "id": doc.id, | |
| "text": doc.document, | |
| } | |
| if doc.cmetadata: | |
| data_dict.update(doc.cmetadata) | |
| data.append(Data(**data_dict)) |
🤖 Prompt for AI Agents
In src/backend/base/langflow/components/vectorstores/cratedb.py around lines 84
to 89, the code calls dict.update(doc.cmetadata) without checking if
doc.cmetadata is None, which causes a TypeError. Fix this by adding a guard to
ensure doc.cmetadata is not None before calling update, for example by using a
conditional check or defaulting to an empty dictionary when doc.cmetadata is
None.
| from langflow.base.vectorstores.model import LCVectorStoreComponent, check_cached_vector_store | ||
| from langflow.helpers import docs_to_data | ||
| from langflow.io import HandleInput, IntInput, SecretStrInput, StrInput | ||
| from langflow.schema import Data |
There was a problem hiding this comment.
Incorrect helper import path
docs_to_data lives in langflow.helpers.data, not the package root.
-from langflow.helpers import docs_to_data
+from langflow.helpers.data import docs_to_dataThis avoids ImportError in environments where langflow.helpers.__init__ doesn’t re-export the symbol.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| from langflow.base.vectorstores.model import LCVectorStoreComponent, check_cached_vector_store | |
| from langflow.helpers import docs_to_data | |
| from langflow.io import HandleInput, IntInput, SecretStrInput, StrInput | |
| from langflow.schema import Data | |
| from langflow.base.vectorstores.model import LCVectorStoreComponent, check_cached_vector_store | |
| from langflow.helpers.data import docs_to_data | |
| from langflow.io import HandleInput, IntInput, SecretStrInput, StrInput | |
| from langflow.schema import Data |
🤖 Prompt for AI Agents
In src/backend/base/langflow/components/vectorstores/cratedb.py around lines 5
to 8, the import statement for docs_to_data is incorrect as it imports from
langflow.helpers instead of langflow.helpers.data. Update the import path to
import docs_to_data from langflow.helpers.data to prevent ImportError in
environments where langflow.helpers.__init__ does not re-export docs_to_data.
| def search_documents(self) -> list[Data]: | ||
| vector_store = self.build_vector_store() | ||
|
|
||
| if self.search_query and isinstance(self.search_query, str) and self.search_query.strip(): | ||
| docs = vector_store.similarity_search( | ||
| query=self.search_query, | ||
| k=self.number_of_results, | ||
| ) | ||
|
|
||
| data = docs_to_data(docs) | ||
| self.status = data | ||
| return data | ||
| return [] |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Duplicated search logic bypasses MMR & score-threshold paths
This override always calls similarity_search, ignoring search_type and duplicates logic already present in the base class.
Consider deleting the method entirely and relying on LCVectorStoreComponent.search_documents, or delegate:
- def search_documents(self) -> list[Data]:
- vector_store = self.build_vector_store()
- ...
- docs = vector_store.similarity_search(
- query=self.search_query,
- k=self.number_of_results,
- )
+ # Remove this override; the base implementation already handles
+ # caching and dispatches to vector_store.search with the chosen
+ # search_type.This instantly enables MMR and score-threshold searches without extra code.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def search_documents(self) -> list[Data]: | |
| vector_store = self.build_vector_store() | |
| if self.search_query and isinstance(self.search_query, str) and self.search_query.strip(): | |
| docs = vector_store.similarity_search( | |
| query=self.search_query, | |
| k=self.number_of_results, | |
| ) | |
| data = docs_to_data(docs) | |
| self.status = data | |
| return data | |
| return [] | |
| # Remove this override; the base implementation already handles | |
| # caching and dispatches to vector_store.search with the chosen | |
| # search_type. |
🤖 Prompt for AI Agents
In src/backend/base/langflow/components/vectorstores/cratedb.py around lines 58
to 70, the search_documents method duplicates logic from the base class and
always calls similarity_search, ignoring search_type and bypassing MMR and
score-threshold functionality. To fix this, remove this method entirely so the
class inherits the base class's search_documents method, which handles all
search types correctly including MMR and score-threshold, thereby avoiding code
duplication and enabling full search functionality without extra code.
|
Hi! I'm I would like to apply some automated changes to this pull request, but it looks like I don't have the necessary permissions to do so. To get this pull request into a mergeable state, please do one of the following two things:
|
autofix.ci2 participants
Hi there,
thanks a stack for conceiving Langflow. Hereby, we are adding another vector store adapter for CrateDB, which is effectively the same like the one for PostgreSQL/PGVector, because CrateDB is compatible with PostgreSQL.
With kind regards,
Andreas.
Summary by CodeRabbit
New Features
Documentation
Chores
Tests