Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] plugins (GPTs like action based on function call) #5353

Open
lloydzhou opened this issue Sep 2, 2024 · 14 comments
Open

[Feature Request] plugins (GPTs like action based on function call) #5353

lloydzhou opened this issue Sep 2, 2024 · 14 comments
Assignees
Labels
enhancement New feature or request

Comments

@lloydzhou
Copy link
Collaborator

🥰 Feature Description

  1. config plugin, like GPTs action (openapi schema + authentication).
  2. wrap API to javascript function, then using function-calling`
    image

🧐 Proposed Solution

  1. manager plugins
    image
    image
  2. using plugin in chat session
    image

📝 Additional Information

#5331

@lloydzhou
Copy link
Collaborator Author

#165 #815 #847

@Issues-translate-bot

This comment was marked as duplicate.

@lloydzhou

This comment was marked as resolved.

@lloydzhou
Copy link
Collaborator Author

lloydzhou commented Sep 3, 2024

WolframAlpha plugin

  1. config appid in Authentication
    image

  2. input wolframalpha llm-api schema:

{
  "openapi": "3.1.0",
  "info": {
    "title": "WolframAlpha",
    "description": "A wrapper around Wolfram Alpha. Useful for when you need to answer questions about Math, Science, Technology, Culture, Society and Everyday Life. Input should be a search query. If the result contains an image link, use the markdown syntax to return the image.",
    "version": "v1.0.0"
  },
  "servers": [
    {
      "url": "https://www.wolframalpha.com"
    }
  ],
  "paths": {
    "/api/v1/llm-api": {
      "get": {
        "operationId": "Calculate",
        "description": "A wrapper around Wolfram Alpha. Useful for when you need to answer questions about Math, Science, Technology, Culture, Society and Everyday Life. Input should be a search query. If the result contains an image link, use the markdown syntax to return the image.",
        "deprecated": false,
        "parameters": [
          {
            "name": "input",
            "in": "query",
            "required": true,
            "description": "questions about Math, Science, Technology, Culture, Society and Everyday Life",
            "schema": {
              "type": "string"
            }
          }
        ]
      }
    }
  },
  "components": {
    "schemas": {}
  }
}
  1. then using plugin when chat to api:
    image

@lloydzhou
Copy link
Collaborator Author

arxiv search plugin

image

{
  "openapi": "3.1.0",
  "info": {
    "title": "arxiv search",
    "description": "Run Arxiv search and get the article information.",
    "version": "v1.0.0"
  },
  "servers": [
    {
      "url": "https://export.arxiv.org"
    }
  ],
  "paths": {
    "/api/query": {
      "get": {
        "operationId": "ArxivSearch",
        "description": "Run Arxiv search and get the article information.",
        "deprecated": false,
        "parameters": [
          {
            "name": "search_query",
            "in": "query",
            "required": true,
            "description": "same as the search_query parameter rules of the arxiv API.",
            "schema": {
              "type": "string"
            }
          },
          {
            "name": "sortBy",
            "in": "query",
            "description": "can be `relevance`, `lastUpdatedDate`, `submittedDate`.",
            "schema": {
              "type": "string"
            }
          },
          {
            "name": "sortOrder",
            "in": "query",
            "description": "can be either `ascending` or `descending`.",
            "schema": {
              "type": "string"
            }
          },
          {
            "name": "start",
            "in": "query",
            "description": "the index of the first returned result.",
            "schema": {
              "type": "number"
            }
          },
          {
            "name": "max_results",
            "in": "query",
            "description": "the number of results returned by the query.",
            "schema": {
              "type": "number"
            }
          }
        ]
      }
    }
  },
  "components": {
    "schemas": {}
  }
}

image

@lloydzhou
Copy link
Collaborator Author

lloydzhou commented Sep 3, 2024

DuckDuckGoLiteSearch

{
  "openapi": "3.1.0",
  "info": {
    "title": "duckduckgo lite",
    "description": "a search engine. useful for when you need to answer questions about current events. input should be a search query.",
    "version": "v1.0.0"
  },
  "servers": [
    {
      "url": "https://lite.duckduckgo.com"
    }
  ],
  "paths": {
    "/lite/": {
      "post": {
        "operationId": "DuckDuckGoLiteSearch",
        "description": "a search engine. useful for when you need to answer questions about current events. input should be a search query.",
        "deprecated": false,
        "parameters": [
          {
            "name": "q",
            "in": "query",
            "required": true,
            "description": "keywords for query.",
            "schema": {
              "type": "string"
            }
          },
          {
            "name": "s",
            "in": "query",
            "description": "can be `0`",
            "schema": {
              "type": "number"
            }
          },
          {
            "name": "o",
            "in": "query",
            "description": "can be `json`",
            "schema": {
              "type": "string"
            }
          },
          {
            "name": "api",
            "in": "query",
            "description": "can be `d.js`",
            "schema": {
              "type": "string"
            }
          },
          {
            "name": "kl",
            "in": "query",
            "description": "wt-wt, us-en, uk-en, ru-ru, etc. Defaults to `wt-wt`.",
            "schema": {
              "type": "string"
            }
          },
          {
            "name": "bing_market",
            "in": "query",
            "description": "wt-wt, us-en, uk-en, ru-ru, etc. Defaults to `wt-wt`.",
            "schema": {
              "type": "string"
            }
          }
        ]
      }
    }
  },
  "components": {
    "schemas": {}
  }
}

image

@lloydzhou
Copy link
Collaborator Author

lloydzhou commented Sep 3, 2024

gapier

  1. config api key
  2. load from url: https://a.gapier.com/free
  3. using in chat
    image

@lloydzhou
Copy link
Collaborator Author

Webpilot

  1. config api key
  2. load from url: https://gpts.webpilot.ai/gpts-openapi.yaml

image

@lloydzhou
Copy link
Collaborator Author

FastGPT Dataset

  1. config plugin with schema, and apikey
{
  "openapi": "3.1.0",
  "info": {
    "title": "FastGPT",
    "description": "FastGPT 是一个基于 LLM 大语言模型的知识库问答系统,提供开箱即用的数据处理、模型调用等能力。同时可以通过 Flow 可视化进行工作流编排,从而实现复杂的问答场景!",
    "version": "v1.0.0"
  },
  "servers": [
    {
      "url": "https://api.fastgpt.in"
    }
  ],
  "paths": {
    "/api/core/dataset/list": {
      "post": {
        "operationId": "GetDatasetList",
        "description": "Get Dataset list with `id`, `name` and other info",
        "deprecated": false,
        "requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "type": "object",
                "properties": {
                  "parentId": {
                    "type": "string"
                  }
                }
              }
            }
          }
        }
      }
    },
    "/api/core/dataset/searchTest": {
      "post": {
        "operationId": "SearchFromDatasetById",
        "description": "search chunks from dataset by `datesetId`",
        "deprecated": false,"requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "type": "object",
                "required": ["datasetId", "text"],
                "properties": {
                  "datasetId": {
                    "type": "string",
                    "description": "id for dataset, can query from `GetDatasetList`"
                  },
                  "text": {
                    "type": "string",
                    "description": "keyword to query from dataset"
                  },
                  "limit": {
                    "type": "number",
                    "description": "limit for response data."
                  },
                  "searchMode": {
                    "type": "string",
                    "description": "can be `embedding`"
                  },
                  "usingReRank": {
                    "type": "boolean",
                    "description": "can be `false`"
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "components": {
    "schemas": {}
  }
}
  1. add prompt to chat session:
please get `datasetId` which name is "NextChat README", and query context from this dataset, and then answer user's question.
  1. and then ask question, will call GetDatasetList first to get datasetId, and then call SearchFromDatasetById get context, and send context to LLM.
    image

@lloydzhou
Copy link
Collaborator Author

lloydzhou commented Sep 3, 2024

awesome plugins

will move plugin demos to another repo from this issue comment.

https://github.com/ChatGPTNextWeb/NextChat-Awesome-Plugins

@Why-SVIP
Copy link

Why-SVIP commented Sep 8, 2024

您好,可以增加PDF分析的功能吗?

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Hello, can you add PDF analysis function?

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Hello, can you add PDF analysis function?

Now supported, please check the NextChat-Awesome-Plugins repository

@mountainguan
Copy link

plugins功能是不是应该也可以支持gemini的function call,因为gemini有免费额度,使用插件更加畅快

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants