How to use PandasQueryEngine with Gemini model via Vertex AI?

[tytung2020]

is Gemini support added to it?

[dosubot[bot]]

Yes, Gemini support has been added to the PandasQueryEngine via Vertex AI. You can use the Gemini models with the Vertex class from the llama_index.llms.vertex module. Here is an example of how to use the Gemini Pro model:

from llama_index.llms.vertex import Vertex
from google.oauth2 import service_account

# Load your service account credentials
filename = "vertex-407108-37495ce6c303.json"
credentials: service_account.Credentials = (
    service_account.Credentials.from_service_account_file(filename)
)

# Initialize the Vertex instance with the Gemini Pro model
llm = Vertex(
    model="gemini-pro", 
    project=credentials.project_id, 
    credentials=credentials
)

# Make a completion call
response = llm.complete("Hello Gemini")
print(response.text)

For the Gemini Pro Vision model, you can use the following example:

from llama_index.llms.vertex import Vertex
from llama_index.core.llms import ChatMessage

# Load your service account credentials
filename = "vertex-407108-37495ce6c303.json"
credentials: service_account.Credentials = (
    service_account.Credentials.from_service_account_file(filename)
)

# Define the chat history with an image URL
history = [
    ChatMessage(
        role="user",
        content=[
            {"type": "text", "text": "Explain what is in the image below:"},
            {
                "type": "image_url",
                "image_url": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAoHCBUWFRgWFhYVGRgaGRoZHBwZGBwaGBkcGhgZGRgYGBocIS4lHB4rIRgYJjgmKy8xNTU1GiQ7QDs0Py40NTEBDAwMEA8QGhIRHDEhISE0NDQ0NDQxNDQ0NDE0NDQ0NDQ0NDQ0MTQ0NDExNDQ1NDQ0NDE0NDQ0NDQxMTE0P//AABEIALcBEwMBIgACEQEDEQH/xAAbAAABBQEBAAAAAAAAAAAAAAACAAEDBAUGB//EAD8QAAEDAgIHBQYEBQQCAwAAAAEAAhEDITFBBAUSUWFxkSIyUoGhBhNCscHRFWLh8BRygpLxM1OisiPSBxZj/8QAGQEBAQEBAQEAAAAAAAAAAAAAAAECAwUE/8QAJhEBAQACAgICAAYDAAAAAAAAAAECEQMhBDESQVFhgZGhsRQVIv/aAAwDAQACEQMRAD8A847fFOHVOKvbKdrbrbipB9TikX1NxWhspwEGf72ruPRO2tVGR6LQRNKCh/FVt3onGl1fD6LSbgiDVBmfxtXw+if+Pq+H0WjF09pRWeNY1fD6JxrKr4PRaJt5pEkmJQUDrar4PQpjrWp4PQrQc4o9pBlt1s8fB81INcv8HzWiYyQuQZztcP8AAmOt3eBaLGKZtMbh0RWOdbO8CYa2PgWsWjcOicMG4dEGUNb/AJEvxf8AItbYb4W9AkWN8LeiIyxrhsdw9U41y3wFaJptjuN6BCabD8DeiDPOuG+ApxrhvhKumkzwt6JjSZ4W9EFM64Z4SkNbs8JVw0GeFvRN/DM8DUFT8WZ4Sn/FGbipjorD8ATnRmYbAQQ/i7NxS/E2cUbtDZ4Ql/A0/CgjdrFnFRnWDOKndoLPCgdoTN3qqIv41m8pIv4Rm5JEMHFFKUpC5QFMoi1CAjaUCphFspO4BFTAQA4lS0ng5pnD1UFOhsmxzQWt/FMGFSe7WvqTUNSuezDWDvPd3RwHiPBRdMdjLK5o2qqz/wDTpvcN4aY64LsDT0TRGzDXvHxvg3/K3AfNY+me0+k1DFNjtne7st8hiueXJhj7rpjxZZeors9ldKOLGt/mewfVSj2S0j/8zwD2qoX6Y7GpHJpP1UtJtcG9Y3/J+q43yuOfbtPFzBX9m9JZJNF5H5If/wBSVne4LTcEZQbHoulbptdkQ8O4GWn6q9S13TrDY0hjXfzt7X9Jx6Fbx58MvVYy4MsfcccWpsCuq1h7NNe3b0Z20MdgmXf0H4uRuub91cgyMozB4rs5WaQAhIhSina6b3dlUAGCMUexkhY2yNrUEeymcxTkJOYggDU2zCsBglIsg8FBXhMpAwSnaYBtdURNaUTmBM8zCJ7YwwREfukvdqUXQtsiodlMGKZyBzSiIvdhJSbCSDOiydhTBhsSi2lWRFSNCja4mCpGSUU5TtJGCkc625AgZzyMkqTSTJwUjKQVhhAEEglQans9qz+Iqhnwi7zuG4cT99y6n2g0nZZ7igdiIEt+ETc/vem9iaAbo73jF7jfg2w+vVSaAwFznuw7x852R5AL5fJ5LjJjPdfX4+EtuV+mVoepXO7b5J8T7k8hl5LUZoLGd4iONvQKPTdcRh5Dcue0rWLnHvfvdxXnZZY431uvSw48svyjqNuk209MFIx9M4AFcSzSibyrmi6wjP7+Sn+RZe8em743XVdLpuq6TxhB3rC03Ubm3Di5nETHH/Cus1mDmeiIaccREZiy1eTjvfpzmGc6ZGq9ZPpPibTYz6E7ty29faE2tT/iaYG2AC8D4m+LmPlyWHp1NvvBHdqTbKcwuj9kiSH033Alp4tIt6FfT4vPbl8L+j5fK4JMflHF+8UL33UmkMLHuaPhcW9CRKiLDwXoPPECia8HmoTyRsKqChG1yZzUxUVIHCJSLhhvUVMnBO9wQ2Vr8EDcboC7cltHzVQZEngmDeiEPyTteM0DbMJ2uRF4wQgCZlFCSDimeEbmzghLUQF0kVkkGaySZKI0k9FgwwtKdrTdVAimVLtEBG0nJEaIME2
UDNEz8l1Ps57NNIFSvMG7WTAje7PyXPaI1oe2R2QZM5gXVnT9f1ajtinIBtZIbkeiUNI0alZops/lDR8kekv0aq0gim/g5rT0OS8sOj+J7nHgYHU4rR1ZqWtVn3Jfbeezy2rK9L8r+DvNSCmzbosgNxAkmNq5x4rG144spOZnLWnyP2+a5rVtfSKWkta6Q+dkh2E7jumy7aoaem0yAQ18QQd438Qvk8vhyzxlx9z+n1+JzY45ay9Vx1TSJGc2m6qii4nCVPpOivpPLKjS05bjxBzV/QKzMyF5etXT2/l/zuds/wDgnbikaJC6cbBGSztLY3L9/qplNe6zjyb+mUHFW2PIE5dFF7sTKZ746wI9AF89vyuo6pXOL3sjHanyGPout1A3Yp1Kz7C5B4ARPoVl6l1I6DUq9hp32cG5gcTvyWm57dJBptJZQaNmW4uIsI/KP3mvV8Lgyl+eX6PK8znxs+OPbg6gLnFxxJJ6mULWHz4rc1rqCrRlw7bPE3EfzDJZDKkr0nloxT/VEGYoqj2+afbAGF1BEWxdLa4ShYSbIyyM8FQzWxkUgLwUzSSlJQC+mMknMmP2U5aDfBRtdmEDvb09UTqNgQk5w/RKi87rIBLd/oom04P3U9UQbZoHjegFoOSJMcMU7STigj2OKSnlu8JIMxm/JIPkyCjZTIwhTMp7/kiHY4ZqTaw3BC2m7I2UtOnGU+aKje0kGJUOpKzWVWucJEkHkQQfmp61Jzpx5QqdWi5t4VjOX4u01P7Mh75e9vuhBBB7Txu4Heu0YGMaGsDWMGVgF5bqjWddgDWEnc2Cei0tJ1vpMdthbxLCPmmmplNJPazS2PrBzBBYANrxEGZ8sFWpPdsGqw7Lw8yW5yAb77krLe8kycVrakO0HsObZHNv6E9FbGZd1oUvaVr27Gk02vbvifMZg8khoWg1P9Os6mfC6HAdYPqufqUjJCrOZvC55cOGfubduPyOTj9XTrG+z7vg0ii7ntD7om6gqfFWoD+o/wDquOM5EjkSgdJ+J39x+64XweG/X8vo/wBjy/l+zs3asoMvV0pvJgv1J+ihZrvQ6J/8FN1R/jfePM2HkuSZRG6ed/mtHRNHJXTDxeLDuRxz8vmz6tatXT62kO/8jobkxuHmc12Wq9F2KbZGN/LJYWo9ABcLLuGhpERwhdnKT7UGmPsua1/7Oh7XPoDZdi5gz3lu48F0+kUYVUOIMhSq8z2IyuN6TROfRdV7VaoBH8QwQJ7bRhPiA+a5YtvaelkSzQtrIWO9AWxuRtZw9UIvMDqiB2Y5b0wI5on0HG8yNyEUiNxCAD2vLLei2gDHqiLIwTMaP8oDYAckq0jGI4JtuLD/AAgeCcT5IqMieSQbffOakGCXmRwhERGlBspKZyKN05KPYOee5BG6jwSRe7G71SQV26OeHVH7og7/ADWR+M/kHX9EhrgeD/l+iI2m7WGz6qVpIGA6rnxrkjBsf1fok3XDh8PUoroHOibdCuxqezDn6GxrA0VNrbO0YnaEFs8BHReean0/3ukUmFp7T2jHKZNo3Ar2pmlAQFVk2qezepW6NTgwXm73D0aDuC2yRF8FWNWCAhr18gjUmmXrXUNCtJ2Ax2TmCOowK4w6I/Rq4D8JxGDmm0hd8+qszWlNlVmy/mDmDwTbOWO+45LWFGHmFnPC2dPAkDaDoABIzhZ76asc6oloRe7CmNNE2mqiNjOAWpoFOSq9KktrVlCXBRcZ26TUmj7LZWoodGbDQApSo6pTDhCza1OFdY6Co9Jg3CNKdMAy112uEEcCuB1jojqdR7LdlxgnNuIPRd84XXF//IemNo1KTy1x94wiQ6LsIx/uUqVl7Th+gsjJMLAd7RN8D/7v0UbNftHwOP8AUEY26AuO/wBUDisM6/b4Hf3D7Jfj7cqZ6hBqlxJtjxUhbaCsU6+bjsOB5j7JHX4OLHf3D7INgTEeqNtPeVi//YB4Hebv0QHXrfAf7v0QbVO0xCIvGV1gjXY8B/u/RO3XsYM9UG85yRe4boWF+PZ7B6j7Jvx38hP9VkNtvbO5JYf"
            },
        ],
    ),
]

# Initialize the Vertex instance with the Gemini Pro Vision model
llm = Vertex(
    model="gemini-pro-vision",
    project=credentials.project_id,
    credentials=credentials,
)

# Make a chat call
response = llm.chat(history)
print(response.message.content)

These examples demonstrate how to use the Gemini models with the Vertex class in the LlamaIndex framework. Additionally, the llama_index.llms.vertex.gemini_utils module provides utility functions such as is_gemini_model, create_gemini_client, and convert_chat_message_to_gemini_content to facilitate working with Gemini models ^[1][2].

To continue talking to Dosu, mention @dosu.

[tytung2020]

@dosu ok, can you provide sample code where I use Gemini model from Vertex AI in PandasQueryEngine, to query database such as a csv file?

[dosubot[bot]]

To use the Gemini model from Vertex AI in PandasQueryEngine to query a database such as a CSV file, you can follow the steps below. Here is a sample code to demonstrate this:

import pandas as pd
from llama_index.core import (
    StorageContext,
    Settings,
    VectorStoreIndex,
    SimpleDirectoryReader,
)
from llama_index.llms.vertex import Vertex
from llama_index.embeddings.vertex import VertexTextEmbedding
from llama_index.vector_stores.vertexaivectorsearch import VertexAIVectorStore
from llama_index.core.query_engine.router_query_engine import RouterQueryEngine
from llama_index.core.tools import QueryEngineTool
from llama_index.core.selectors import LLMSingleSelector

# Initialize Vertex AI
from google.cloud import aiplatform

PROJECT_ID = "<your project>"
REGION = "<your region>"
GCS_BUCKET_NAME = f"{PROJECT_ID}"
VS_INDEX_NAME = "vertex_vector_search_index"
VS_INDEX_ENDPOINT_NAME = "vector_search_endpoint"

aiplatform.init(project=PROJECT_ID, location=REGION)

# Setup vector store
vector_store = VertexAIVectorStore(
    project_id=PROJECT_ID,
    region=REGION,
    index_id=VS_INDEX_NAME,
    endpoint_id=VS_INDEX_ENDPOINT_NAME,
    gcs_bucket_name=GCS_BUCKET_NAME,
)

# Set storage context
storage_context = StorageContext.from_defaults(vector_store=vector_store)

# Configure embedding model
embed_model = VertexTextEmbedding(
    model_name="textembedding-gecko@003",
    project=PROJECT_ID,
    location=REGION,
)

vertex_gemini = Vertex(
    model="gemini-1.5-pro-preview-0514", temperature=0, additional_kwargs={}
)

# Setup the index/query process
Settings.embed_model = embed_model
Settings.llm = vertex_gemini

# Load CSV data into a DataFrame
df = pd.read_csv('your_data.csv')

# Convert DataFrame to a list of TextNode objects
documents = [TextNode(text=row.to_string()) for _, row in df.iterrows()]

# Define index from vector store
vector_index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context
)

# Create query engine from vector store
vector_query_engine = vector_index.as_query_engine()

# Create tools from query engines
vector_tool = QueryEngineTool.from_defaults(
    query_engine=vector_query_engine,
    description=(
        "Useful for retrieving specific context from the CSV data."
    ),
)

# Create RouterQueryEngine
query_engine = RouterQueryEngine(
    selector=LLMSingleSelector.from_defaults(),
    query_engine_tools=[vector_tool],
    verbose=True,
)

# Query the data
response = query_engine.query("Your query here")
print(str(response))

This code sets up the necessary components to use the Gemini model from Vertex AI with the PandasQueryEngine to query a CSV file. Make sure to replace placeholders like <your project> and <your region> with your actual Google Cloud project details ^[1].

Additionally, you can refer to the example provided in the LlamaIndex documentation for building a query pipeline over Pandas DataFrames. This example demonstrates how to use LLMs to infer the set of operations needed to query a DataFrame:

from llama_index.core.query_pipeline import (
    QueryPipeline as QP,
    Link,
    InputComponent,
)
from llama_index.experimental.query_engine.pandas import (
    PandasInstructionParser,
)
from llama_index.llms.openai import OpenAI
from llama_index.core import PromptTemplate

# Load CSV data into a DataFrame
df = pd.read_csv('your_data.csv')

# Define modules
instruction_str = (
    "1. Convert the query to executable Python code using Pandas.\n"
    "2. The final line of code should be a Python expression that can be called with the `eval()` function.\n"
    "3. The code should represent a solution to the query.\n"
    "4. PRINT ONLY THE EXPRESSION.\n"
    "5. Do not quote the expression.\n"
)

pandas_prompt_str = (
    "You are working with a pandas dataframe in Python.\n"
    "The name of the dataframe is `df`.\n"
    "This is the result of `print(df.head())`:\n"
    "{df_str}\n\n"
    "Follow these instructions:\n"
    "{instruction_str}\n"
    "Query: {query_str}\n\n"
    "Expression:"
)
response_synthesis_prompt_str = (
    "Given an input question, synthesize a response from the query results.\n"
    "Query: {query_str}\n\n"
    "Pandas Instructions (optional):\n{pandas_instructions}\n\n"
    "Pandas Output: {pandas_output}\n\n"
    "Response: "
)

pandas_prompt = PromptTemplate(pandas_prompt_str).partial_format(
    instruction_str=instruction_str, df_str=df.head(5)
)
pandas_output_parser = PandasInstructionParser(df)
response_synthesis_prompt = PromptTemplate(response_synthesis_prompt_str)
llm = OpenAI(model="gpt-3.5-turbo")

# Build Query Pipeline
qp = QP(
    modules={
        "input": InputComponent(),
        "pandas_prompt": pandas_prompt,
        "llm1": llm,
        "pandas_output_parser": pandas_output_parser,
        "response_synthesis_prompt": response_synthesis_prompt,
        "llm2": llm,
    },
    verbose=True,
)
qp.add_chain(["input", "pandas_prompt", "llm1", "pandas_output_parser"])
qp.add_links(
    [
        Link("input", "response_synthesis_prompt", dest_key="query_str"),
        Link(
            "llm1", "response_synthesis_prompt", dest_key="pandas_instructions"
        ),
        Link(
            "pandas_output_parser",
            "response_synthesis_prompt",
            dest_key="pandas_output",
        ),
    ]
)
qp.add_link("response_synthesis_prompt", "llm2")

# Run Query
response = qp.run(
    query_str="Your query here",
)
print(response.message.content)

This example shows how to set up a query pipeline that can perform structured operations over a Pandas DataFrame to satisfy a user query using LLMs ^[2].

To continue talking to Dosu, mention @dosu.

[tytung2020]

@dosu I don't see PandasQueryEngine is being used in your codes above? your code above seems to use embeddings and semantic search and is therefore not suitable for database csv, while PandasQueryEngine uses pandas library to do precise database query

[dosubot[bot]]

To use the PandasQueryEngine with the Gemini model from Vertex AI to query a CSV file, you can follow the example provided below. This setup leverages the Pandas library for accurate database querying:

import logging
import sys
import pandas as pd
from llama_index.experimental.query_engine import PandasQueryEngine
from llama_index.llms.vertex import Vertex
from llama_index.core import PromptTemplate
from google.oauth2 import service_account

# Set up logging
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

# Load your CSV file into a Pandas DataFrame
df = pd.read_csv("path_to_your_csv_file.csv")

# Define the instruction and response synthesis prompts
instruction_str = (
    "1. Convert the query to executable Python code using Pandas.\n"
    "2. The final line of code should be a Python expression that can be called with the `eval()` function.\n"
    "3. The code should represent a solution to the query.\n"
    "4. PRINT ONLY THE EXPRESSION.\n"
    "5. Do not quote the expression.\n"
)

pandas_prompt_str = (
    "You are working with a pandas dataframe in Python.\n"
    "The name of the dataframe is `df`.\n"
    "This is the result of `print(df.head())`:\n"
    "{df_str}\n\n"
    "Follow these instructions:\n"
    "{instruction_str}\n"
    "Query: {query_str}\n\n"
    "Expression:"
)

response_synthesis_prompt_str = (
    "Given an input question, synthesize a response from the query results.\n"
    "Query: {query_str}\n\n"
    "Pandas Instructions (optional):\n{pandas_instructions}\n\n"
    "Pandas Output: {pandas_output}\n\n"
    "Response: "
)

# Initialize the PandasQueryEngine with the DataFrame
pandas_prompt = PromptTemplate(pandas_prompt_str).partial_format(
    instruction_str=instruction_str, df_str=df.head(5)
)
pandas_output_parser = PandasInstructionParser(df)
response_synthesis_prompt = PromptTemplate(response_synthesis_prompt_str)

# Load your service account credentials
filename = "path_to_your_service_account_file.json"
credentials = service_account.Credentials.from_service_account_file(filename)

# Initialize the Vertex instance with the Gemini model
llm = Vertex(
    model="gemini-1.5-pro",  # Replace with your actual model name
    project=credentials.project_id,
    credentials=credentials
)

query_engine = PandasQueryEngine(
    df=df,
    pandas_prompt=pandas_prompt,
    pandas_output_parser=pandas_output_parser,
    response_synthesis_prompt=response_synthesis_prompt,
    llm=llm,
    verbose=True
)

# Query the DataFrame using natural language
response = query_engine.query("Your natural language query here")

# Display the response
print(response)

# Optionally, get the generated Pandas instructions
print(response.metadata["pandas_instruction_str"])

Replace "path_to_your_csv_file.csv" with the path to your CSV file and "Your natural language query here" with your specific query. This code will load the CSV file into a Pandas DataFrame, initialize the PandasQueryEngine, and perform the query using natural language. The response and the generated Pandas instructions will be printed ^[1][2].

To continue talking to Dosu, mention @dosu.

[tytung2020]

@dosu ok what lib should I install to run the code

[dosubot[bot]]

To run the code, you need to install the following libraries:

pip install pandas google-cloud-aiplatform google-auth llama-index

These libraries include:

• pandas for handling the CSV file.
• google-cloud-aiplatform and google-auth for interacting with Vertex AI.
• llama-index for the PandasQueryEngine and other related functionalities.

Make sure you have your Google Cloud credentials set up correctly to access Vertex AI.

To continue talking to Dosu, mention @dosu.

[tytung2020]

@dosu I received this error: ModuleNotFoundError: No module named 'llama_index.llms.vertex'

[dosubot[bot]]

To resolve the ModuleNotFoundError: No module named 'llama_index.llms.vertex', you should ensure that you are importing the Gemini class from the correct module path. The correct import path should be:

from llama_index.legacy.llms.vertex_utils import Gemini

Here is an example of how you might use the Gemini class in your code:

import pandas as pd
from llama_index.experimental.query_engine import PandasQueryEngine
from llama_index.legacy.llms.vertex_utils import Gemini
from llama_index.core import PromptTemplate
from google.oauth2 import service_account

# Load your CSV file into a Pandas DataFrame
df = pd.read_csv("path_to_your_csv_file.csv")

# Define the instruction and response synthesis prompts
instruction_str = (
    "1. Convert the query to executable Python code using Pandas.\n"
    "2. The final line of code should be a Python expression that can be called with the `eval()` function.\n"
    "3. The code should represent a solution to the query.\n"
    "4. PRINT ONLY THE EXPRESSION.\n"
    "5. Do not quote the expression.\n"
)

pandas_prompt_str = (
    "You are working with a pandas dataframe in Python.\n"
    "The name of the dataframe is `df`.\n"
    "This is the result of `print(df.head())`:\n"
    "{df_str}\n\n"
    "Follow these instructions:\n"
    "{instruction_str}\n"
    "Query: {query_str}\n\n"
    "Expression:"
)

response_synthesis_prompt_str = (
    "Given an input question, synthesize a response from the query results.\n"
    "Query: {query_str}\n\n"
    "Pandas Instructions (optional):\n{pandas_instructions}\n\n"
    "Pandas Output: {pandas_output}\n\n"
    "Response: "
)

# Initialize the PandasQueryEngine with the DataFrame
pandas_prompt = PromptTemplate(pandas_prompt_str).partial_format(
    instruction_str=instruction_str, df_str=df.head(5)
)
pandas_output_parser = PandasInstructionParser(df)
response_synthesis_prompt = PromptTemplate(response_synthesis_prompt_str)

# Load your service account credentials
filename = "path_to_your_service_account_file.json"
credentials = service_account.Credentials.from_service_account_file(filename)

# Initialize the Gemini model
gemini_model = Gemini(
    model="text-bison",  # Replace with your actual model name
    project=credentials.project_id,
    location="us-central1",
    credentials=filename
)

query_engine = PandasQueryEngine(
    df=df,
    pandas_prompt=pandas_prompt,
    pandas_output_parser=pandas_output_parser,
    response_synthesis_prompt=response_synthesis_prompt,
    llm=gemini_model,
    verbose=True
)

# Query the DataFrame using natural language
response = query_engine.query("Your natural language query here")

# Display the response
print(response)

# Optionally, get the generated Pandas instructions
print(response.metadata["pandas_instruction_str"])

Make sure that the llama_index package is correctly installed and that you are using the correct version that includes the Gemini class in the specified module path ^[1].

To continue talking to Dosu, mention @dosu.

[tytung2020]

still not working. Anyone can help here?

[tytung2020]

My code and error:
code:

from vertexai.generative_models import (
    GenerationConfig,
    GenerativeModel,
)
from llama_index.experimental.query_engine import PandasQueryEngine

df = pd.read_csv("path_to_file.csv")

model = GenerativeModel(
                model_name="gemini-1.5-pro",
                generation_config=GenerationConfig(temperature=0),
            )


query_engine = PandasQueryEngine(df=df, verbose=True, llm=model)

response = query_engine.query(
    "how many lines are there in this file",
)

and error message:
AttributeError: 'GenerativeModel' object has no attribute 'predict'

[CLozy]

@dosu I am getting this error ```
ValueError:

---

Could not load OpenAI model. If you intended to use OpenAI, please check your OPENAI_API_KEY.
Original error:
No API key found for OpenAI.
Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.
API keys can be found or created at https://platform.openai.com/account/api-keys

To disable the LLM entirely, set llm=None.

---

this is my code ```
llm = Vertex(model="gemini-1.0-pro", project=GCP_PROJECT_NAME,location=LOCATION)

# Define a user query
input_query = "How many clicks did campaign B get?"

#Loading data

# Create a sample DataFrame
data = {
    "campaign": ["A", "B", "C"],
    "clicks": [100, 150, 200],
    "impressions": [1000, 1500, 2000]
}

df = pd.DataFrame(data)

# Create a PandasQueryEngine from LlamaIndex
query_engine = PandasQueryEngine(df, llm=llm)

# create a chaine of engines
query_chain = QueryPipeline(chain=[query_engine])

# how to differentiate and choose tool
choices = [
    "This tool answers the question by querying the dataframe called combined_df"
]

# define selector
selector = LLMSingleSelector.from_defaults()

# define router
router_c = RouterComponent(selector=selector,
                           choices=choices,
                           components=[query_chain],
                           verbose=True)

#top level pipeline
qp = QueryPipeline(chain=[router_c], verbose=True)

# query qp
response = qp.run(input_query)

[dosubot[bot]]

To resolve the AttributeError: 'GenerativeModel' object has no attribute 'predict' error when using the GenerativeModel with PandasQueryEngine, you need to ensure that the GenerativeModel class has a predict method. Here is how you can mock the predict method for testing purposes:

import pandas as pd
from vertexai.generative_models import GenerationConfig, GenerativeModel
from llama_index.experimental.query_engine import PandasQueryEngine
import pytest

# Mock the predict method
def _mock_predict(*args, **kwargs):
    query_str = kwargs["query_str"]
    return f'df["{query_str}"]'

# Assuming GenerativeModel is your model class
@pytest.fixture
def mock_predict(monkeypatch):
    monkeypatch.setattr(GenerativeModel, "predict", _mock_predict)

# Load your CSV file into a Pandas DataFrame
df = pd.read_csv("path_to_file.csv")

# Initialize the GenerativeModel
model = GenerativeModel(
    model_name="gemini-1.5-pro",
    generation_config=GenerationConfig(temperature=0),
)

# Mock the predict method
mock_predict(pytest.MonkeyPatch())

# Initialize the PandasQueryEngine with the DataFrame and the model
query_engine = PandasQueryEngine(df=df, verbose=True, llm=model)

# Query the DataFrame using natural language
response = query_engine.query("how many lines are there in this file")

# Display the response
print(response)

This code mocks the predict method of the GenerativeModel class to return a string that can be evaluated as a Pandas expression. This should resolve the AttributeError and allow you to use the PandasQueryEngine with the GenerativeModel ^[1].

For @CLozy's issue with the OpenAI API key, it seems like the Vertex class is trying to use OpenAI by default. Ensure that you have the correct environment variables set or explicitly disable OpenAI if you are not using it. Here is how you can modify your code to avoid this issue:

import pandas as pd
from llama_index.experimental.query_engine import PandasQueryEngine
from llama_index.legacy.llms.vertex import Vertex
from llama_index.core import QueryPipeline, RouterComponent, LLMSingleSelector

# Initialize the Vertex instance with the Gemini model
llm = Vertex(model="gemini-1.0-pro", project=GCP_PROJECT_NAME, location=LOCATION, llm=None)

# Define a user query
input_query = "How many clicks did campaign B get?"

# Create a sample DataFrame
data = {
    "campaign": ["A", "B", "C"],
    "clicks": [100, 150, 200],
    "impressions": [1000, 1500, 2000]
}

df = pd.DataFrame(data)

# Create a PandasQueryEngine from LlamaIndex
query_engine = PandasQueryEngine(df, llm=llm)

# Create a chain of engines
query_chain = QueryPipeline(chain=[query_engine])

# Define choices and selector
choices = ["This tool answers the question by querying the dataframe called combined_df"]
selector = LLMSingleSelector.from_defaults()

# Define router
router_c = RouterComponent(selector=selector, choices=choices, components=[query_chain], verbose=True)

# Top-level pipeline
qp = QueryPipeline(chain=[router_c], verbose=True)

# Query the pipeline
response = qp.run(input_query)

# Display the response
print(response)

This code ensures that the Vertex class does not attempt to use OpenAI by setting llm=None ^[2].

It seems like this issue requires a deeper look, @logan-markewich could you please lend your expertise?

To continue talking to Dosu, mention @dosu.

[CLozy]

@dosu its throwing an error TypeError: Vertex.init() got an unexpected keyword argument 'llm'

[CLozy]

I changed default llm llamaindex uses globaly using settings

from llama_index.core import Settings 
llm = Vertex(model="gemini-1.0-pro", project=GCP_PROJECT_NAME, location=LOCATION)
Settings.llm = llm