Replies: 1 comment
-
Hey @dhoubhad! I'm here to help you with any bugs, questions, or contributions you have in mind. Let's tackle this issue together! No, Here is the relevant code snippet for """Simple keyword-table based index.
Similar to KeywordTableIndex, but uses a simpler keyword extraction
technique that doesn't involve GPT - just uses regex.
"""
from typing import Any, Set, Union
from llama_index.core.base.base_retriever import BaseRetriever
from llama_index.core.indices.keyword_table.base import (
BaseKeywordTableIndex,
KeywordTableRetrieverMode,
)
from llama_index.core.indices.keyword_table.utils import simple_extract_keywords
from llama_index.core.prompts.default_prompts import (
DEFAULT_QUERY_KEYWORD_EXTRACT_TEMPLATE,
)
DQKET = DEFAULT_QUERY_KEYWORD_EXTRACT_TEMPLATE
class SimpleKeywordTableIndex(BaseKeywordTableIndex):
"""Simple Keyword Table Index.
This index uses a simple regex extractor to extract keywords from the text.
"""
def _extract_keywords(self, text: str) -> Set[str]:
"""Extract keywords from text."""
return simple_extract_keywords(text, self.max_keywords_per_chunk)
def as_retriever(
self,
retriever_mode: Union[
str, KeywordTableRetrieverMode
] = KeywordTableRetrieverMode.SIMPLE,
**kwargs: Any,
) -> BaseRetriever:
return super().as_retriever(retriever_mode=retriever_mode, **kwargs)
# legacy
GPTSimpleKeywordTableIndex = SimpleKeywordTableIndex For hybrid search, you might want to consider using other classes specifically designed for that purpose, such as |
Beta Was this translation helpful? Give feedback.
-
I'm using SimpleKeywordTableIndex for hybrid search retrieval along with VectorStoreIndex.
The SimpleKeywordTableIndex class inherits from BaseKeywordTableIndex class which requires OpenAI API key or a LLM or else it throws an error.
Is there a reason why SimpleKeywordTableIndex (a simple regex extractor) should require a LLM? Should I be using an alternative class for Hybrid search?
Beta Was this translation helpful? Give feedback.
All reactions