Skip to content

Latest commit

 

History

History
188 lines (146 loc) · 5.59 KB

CONFIG.md

File metadata and controls

188 lines (146 loc) · 5.59 KB

Configuration Guide

Standalone, high-performance image resolution and persistence microservice.

Ready-to-use configuration examples for each provider are available in the examples/ directory:


1. Authentication & Separate REST API Tokens ([auth])

The service provides separate REST API tokens for LRU-cache operations and persistent DB-storage operations, plus optional web browser basic authentication.

[auth]
# Web UI authentication (for browser access to /upload and /explore)
web_auth_enabled = false
username = "admin"
password = "supersecret"
protect_upload = false
protect_explore = false

# Separate REST API Service Tokens:
cache_token = "my-cache-secret"       # Required to post into LRU cache
storage_token = "my-storage-secret"   # Required to post into DB storage

Passing API Tokens in Requests

Clients can authenticate using any of the following methods:

  1. Standard Authorization Header:
    Authorization: Bearer my-cache-secret
  2. Dedicated Token Headers:
    X-Cache-Token: my-cache-secret
    X-Storage-Token: my-storage-secret
    X-API-Token: my-token
    X-API-Key: my-token
  3. Query Parameter:
    POST /api/cache?token=my-cache-secret
    
  • Cache Access: Validates against cache_token (also allows storage_token).
  • Storage Access: Validates against storage_token.
  • If a token is omitted or empty in configuration, authentication for that scope is unrestricted.

2. Storage & Database Providers ([database])

The service runs standalone with zero coupling to external bot databases:

[database]
# Options: "postgres", "mongodb", "s3", "hetzner_s3", "hetzner_storage_box", "cache_only"
provider = "postgres"
url = "postgresql://postgres:password@localhost:5432/image_db"
table_name = "images"

PostgreSQL (provider = "postgres")

Standalone PostgreSQL image storage:

  • Stores binary image payloads directly as BYTEA
  • Automatic deduplication on primary key hash
  • Automatic createdAt timestamp and indexing
CREATE TABLE IF NOT EXISTS "images" (
    hash TEXT PRIMARY KEY,
    "mimeType" TEXT NOT NULL,
    "sizeBytes" INT NOT NULL,
    "imageData" BYTEA NOT NULL,
    "metaName" TEXT,
    "createdAt" TIMESTAMP DEFAULT NOW()
);

MongoDB (provider = "mongodb")

Optimal NoSQL document-based image storage. Stores images directly as BSON binary with unique hash indexing.

[database]
provider = "mongodb"

[mongodb]
uri = "mongodb://localhost:27017"
database = "image_db"
collection = "images"

AWS S3 & S3-Compatible Storage (provider = "s3")

Works with Amazon S3, MinIO, Cloudflare R2, Wasabi, Backblaze B2, and Ceph.

[database]
provider = "s3"

[s3]
endpoint = "https://s3.amazonaws.com"
region = "us-east-1"
bucket = "my-image-bucket"
access_key_id = "YOUR_ACCESS_KEY_ID"
secret_access_key = "YOUR_SECRET_ACCESS_KEY"
prefix = "images/"
force_path_style = false

Hetzner Storage: Object Storage vs. Storage Box

1. Hetzner Storage Box (provider = "hetzner_storage_box")

Hetzner's bulk backup box (uXXXXX.your-storagebox.de). Operates natively via WebDAV over HTTPS (HTTP PUT/GET/MKCOL with Basic Auth).

[database]
provider = "hetzner_storage_box"

[hetzner.storage_box]
endpoint = "https://u123456.your-storagebox.de"
username = "u123456"
password = "your_storage_box_password"
base_path = "/images"

2. Hetzner Object Storage (provider = "hetzner_s3")

Hetzner's S3-compatible bucket service in fsn1, hel1, nbg1.

[database]
provider = "hetzner_s3"

[hetzner.object_storage]
region = "fsn1"
bucket = "my-hetzner-bucket"
access_key_id = "YOUR_HETZNER_ACCESS_KEY"
secret_access_key = "YOUR_HETZNER_SECRET_KEY"
prefix = "images/"

Cache Only / In-Memory (provider = "cache_only")

Runs purely in memory using the LRU cache without persisting to an external database.

[database]
provider = "cache_only"

3. Semi-Cached DB Storage Architecture

To achieve the best of both worlds (persistent storage + sub-millisecond retrieval speeds), DB storage is semi-cached:

  1. When storing into DB storage, the image is written to the persistent provider and automatically warmed in the in-memory LRU cache.
  2. Read requests (GET /storage/{hash} or GET /md5/{hash} or GET /image/{hash}) check the LRU cache first. If present, the image is served directly from RAM with 0 database or S3 latency.
  3. On a cache miss, the service loads from the database/S3, populates the LRU cache, and returns the response.

4. Clever Image Compression & Storage Saving ([compression])

[compression]
enabled = true
format = "webp" # "webp", "jpeg", "png", or "original"
quality = 80 # 1-100 (80 is the optimal perceptual quality ratio)
max_width = 2048 # Max width (proportional downscale)
max_height = 2048 # Max height (proportional downscale)
strip_metadata = true # Strips EXIF/GPS/device metadata
only_if_smaller = true # Guarantee file size never increases
min_bytes_threshold = 1024 # Skip compression for tiny icons (< 1 KB)