-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
6 changed files
with
195 additions
and
22 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
MIT License | ||
|
||
Copyright (c) 2023 Oleh Astappiev, Learnweb Team and contributors | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all | ||
copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
SOFTWARE. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
<p style="text-align: center"><img src="./.github/logo.svg" width="200" alt="Interweb Logo"/></p> | ||
|
||
# Interweb - Unified API for Information Retrieval | ||
|
||
<p style="text-align: center"> | ||
<a href="https://opensource.org/licenses/MIT" alt="License: MIT"> | ||
<img src="https://img.shields.io/badge/License-MIT-yellow.svg"/></a> | ||
<a href="https://github.com/l3s-learnweb/interweb/tags" alt="Releases"> | ||
<img src="https://img.shields.io/github/v/tag/l3s-learnweb/interweb"/></a> | ||
<a href="https://github.com/l3s-learnweb/interweb/pkgs/container/interweb" alt="Container"> | ||
<img alt="Static Badge" src="https://img.shields.io/badge/docker-container_image-blue"></a> | ||
<a href="https://github.com/l3s-learnweb/learnweb/packages/1951023" alt="Container"> | ||
<img alt="Static Badge" src="https://img.shields.io/badge/maven-java_client-orange"></a> | ||
</p> | ||
|
||
Interweb is a powerful and versatile API that consolidates multiple data providers into one unified interface, simplifying the process of searching and retrieving information. | ||
Whether you're building a chatbot, a data analysis tool, or any application that requires access to various data sources, Interweb can help streamline your development process. | ||
|
||
## Features | ||
|
||
- **Unified search**: Interweb consolidates multiple search APIs into one, making it easy to search for information from a variety of sources. | ||
|
||
- **Unified API**: Interweb offers a single API endpoint to access multiple data providers, reducing the complexity of managing multiple APIs in your project. | ||
|
||
- **Preserved History**: When using LLM (ChatGPT), Interweb preserves the conversation history, provides a simple interface for building and maintaining conversational interactions with AI models. | ||
|
||
- **Blazingly fast**: Interweb is built on top of Quarkus Reactive, compiled into native executable, and can handle thousands of requests per second. | ||
|
||
- **Save quota**: Interweb caches responses from data providers, reducing the number of API calls and saving your quota. | ||
|
||
## Use Cases | ||
|
||
Interweb can be used in a wide range of applications, but was designed with the following use cases in mind: | ||
|
||
- **Chatbots**: Build intelligent chatbots that can answer questions, provide recommendations, and engage in natural conversations with users. Everything you can do with ChatGPT, but conversation history is included. | ||
|
||
- **Information Retrieval**: Quickly search and retrieve information from multiple data providers without the hassle of integrating each API individually. | ||
|
||
## Implemented Providers (Connectors) | ||
|
||
Interweb currently supports the following data providers: | ||
|
||
1. **Bing**: | ||
- Search: the web for images, videos, news, and more. | ||
- Suggest: get related queries to enhance user search experience. | ||
2. **Flickr**: | ||
- Search: for photos and images. | ||
- Describe: a media resource by url. | ||
3. **Giphy**: | ||
- Search: for variety of gifs in one of the largest gif libraries. | ||
4. **Google**: | ||
- Suggest: access related search queries from one of the world's leading search engines. | ||
5. **Ipernity**: | ||
- Search: discover photos and images within one of the largest non-commercial clubs. | ||
- Describe: obtain photo information using its url. | ||
6. **OpenAI**: | ||
- Interact with OpenAI's ChatGPT for natural language understanding and generation. | ||
7. **SlideShare**: | ||
- Search: find presentations and documents for various topics. | ||
8. **Vimeo**: | ||
- Search: locate videos created by creative content creators. | ||
- Describe: obtain video information using its url. | ||
9. **YouTube**: | ||
- Search: for videos in the largest video hosting platform. | ||
- Describe: obtain detailed information about a video using its url. | ||
|
||
## How to Use Interweb | ||
|
||
### Using the Docker Image | ||
|
||
Interweb is available as a Docker image, making it easy to deploy and use in your projects. Follow these steps: | ||
|
||
1. **Pull the Docker Image**: | ||
|
||
```shell | ||
docker pull ghcr.io/l3s-learnweb/interweb:latest | ||
``` | ||
|
||
2. **Run the Docker Container**: | ||
|
||
```shell | ||
docker run -p 8080:8080 --env-file ./path/to/.env ghcr.io/l3s-learnweb/interweb:latest | ||
``` | ||
|
||
Replace `./path/to/.env` with the path to your environment file. | ||
Make sure your configuration file contains the necessary API keys and settings for the supported connectors. | ||
An example can be found in the [example.env](./interweb-server/example.env) file in the interweb-server subproject. | ||
|
||
3. **Access the API spec**: | ||
|
||
You can now read the API specification at [http://localhost:8080/](http://localhost:8080), or you can now make a requests to Interweb using e.g.: | ||
|
||
``` | ||
GET http://localhost:8080/suggest?q=hello+world | ||
``` | ||
|
||
For more advanced usage, you need to create an access token and use it in your requests. | ||
|
||
## Contributing | ||
|
||
We welcome contributions from the community. If you have ideas for new providers, features, or improvements, please open an issue or submit a pull request. | ||
|
||
## License | ||
|
||
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
QUARKUS_HTTP_PORT=8080 | ||
|
||
# Database is required for Interweb operation | ||
# If you run quarkus in dev mode, you can skip it and dev services will run devcontainers with mariadb automatically | ||
QUARKUS_DATASOURCE_REACTIVE_URL=mariadb://localhost:3306/interweb | ||
QUARKUS_DATASOURCE_USERNAME=interweb | ||
QUARKUS_DATASOURCE_PASSWORD= | ||
|
||
# It would be good to use independent JWT keys for each environment, check the application.properties for more info | ||
JWT_ISSUER= | ||
JWT_KEY= | ||
|
||
# Most of the connectors require API keys, you need to obtain the independently and provide them here | ||
CONNECTOR_BING_APIKEY= | ||
CONNECTOR_FLICKR_APIKEY= | ||
CONNECTOR_GIPHY_APIKEY= | ||
CONNECTOR_IPERNITY_APIKEY= | ||
CONNECTOR_VIMEO_APIKEY= | ||
CONNECTOR_YOUTUBE_APIKEY= | ||
|
||
CONNECTOR_SLIDESHARE_APIKEY= | ||
CONNECTOR_SLIDESHARE_SECRET= | ||
|
||
CONNECTOR_OPENAI_URL=https://....openai.azure.com | ||
CONNECTOR_OPENAI_APIKEY= |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters