Enhance README with usage and configuration details

Expanded the README to include detailed usage instructions, environment variables, configuration examples, and use cases for the MCP Web URL Reader.

Author: clsferguson

README.md

@@ -1,28 +1,170 @@
-# mcp-web-url-reader
-
-An MCP server that exposes a single tool, `read_web_url`, which fetches absolute URLs with `curl -sL` after prepending a configurable `CUSTOM_PREFIX` on the server side. This server uses the MCP TypeScript/JavaScript SDK and serves a Streamable HTTP endpoint at `/mcp` for compatibility across many AI applications.
+# MCP Web URL Reader
+An MCP server that exposes a single tool, `read_web_url`, which fetches absolute URLs with `curl -sL` after prepending a configurable `CUSTOM_PREFIX` on the server side. This server uses the MCP TypeScript/JavaScript SDK and serves a Streamable HTTP endpoint at `/mcp` for compatibility across many AI applications. Perfect for routing AI web requests through proxies, caching layers, or custom gateways.
 
 ## Features
 
-- Streamable HTTP transport with session management on `/mcp` (POST for JSON-RPC requests, GET for SSE notifications, DELETE to end sessions).
-- CORS enabled with `Mcp-Session-Id` exposed for browser-based clients, as recommended.
-- Tool: `read_web_url` with input `{ url: string }`, which runs `curl -sL CUSTOM_PREFIX/<url>` and returns the response text.
-- Tiny Docker image based on `node:20-alpine` with `curl` installed via `apk add --no-cache curl`.
+- 🌐 Simple web URL reading via curl
+- 🔧 Configurable URL prefix for proxying/routing
+- 🐳 Tiny Docker image based on Node.js
+- 🚀 Easy deployment with Docker Compose
+- 🛠️ Compatible with any MCP-enabled AI application
+
+## How It Works
+
+The AI thinks it's reading a normal URL:
+```
+curl -sL https://www.example.com
+```
+
+But the server actually executes:
+```
+curl -sL CUSTOM_PREFIX/https://www.example.com
+```
+
+This allows you to:
+- Route requests through a caching proxy
+- Add authentication layers
+- Use custom gateways
+- Bypass rate limits with your own infrastructure
+- Monitor/log AI web access
+
+## Quick Start
+
+### Using Docker Compose
+
+See `docker-compose.yml` for a working example service mapping `8080:8080` and injecting both environment variables.
+
+The MCP server will be available at `http://localhost:8080`
+
+### Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `CUSTOM_PREFIX` | URL prefix to prepend to all requests | `""` (empty) |
+| `INTERNAL_PORT` | Internal server port | `80800` |
+
+### Configuration Examples
+
+#### Direct Mode (No Prefix)
+```yaml
+environment:
+  - CUSTOM_PREFIX=
+  - PORT=3000
+```
+
+#### Proxy Mode
+```yaml
+environment:
+  - CUSTOM_PREFIX=https://proxy.myserver.com
+  - INTERNAL_PORT=8080
+```
+
+#### Cache Gateway
+```yaml
+environment:
+  - CUSTOM_PREFIX=http://cache-server:8000/fetch
+  - INTERNAL_PORT=8080
+```
+
+## Using with AI Applications, MCP Client Integration
+
+The server exposes tools over Streamable HTTP: initialize a session with a POST to `/mcp`, reuse `Mcp-Session-Id` from the response, then call `tools/call` with `{ name: "read_web_url", arguments: { url: "https://example.com" } }`. The server will run `curl -sL CUSTOM_PREFIX/https://example.com` and return text content. Ensure the `Mcp-Session-Id` header is included in subsequent requests and allowed via CORS.
 
-## Environment Variables
+### Claude Desktop (MCP)
 
-- `INTERNAL_PORT` (default: `8080`): Port the HTTP server listens on inside the container. [web:1]
-- `CUSTOM_PREFIX` (required for prefixing): The string to prepend before the requested absolute URL, e.g., `https://your-proxy/forward`.
+Add to your `claude_desktop_config.json`:
 
-### Run
+```json
+{
+  "mcpServers": {
+    "web-url-reader": {
+      "url": "http://localhost:8080/mcp"
+    }
+  }
+}
+```
 
+### n8n
+
+Use the MCP Server node and point it to `http://localhost:8080/mcp`
+
+### Custom Applications
+
+Connect to the MCP endpoint at `http://localhost:8080/mcp`
+
+## Tool Schema
+
+The server exposes one tool:
+
+**`read_web_url`**
+- Description: Read and return the contents of a web page
+- Parameter: `url` (string, required) - The complete URL to fetch
+- Returns: Text content of the web page
+
+Example tool call:
+```json
+{
+  "name": "read_web_url",
+  "arguments": {
+    "url": "https://www.example.com"
+  }
+}
+```
+
+## Use Cases
+
+### Caching Layer
+Route all AI web requests through your caching server to reduce external calls:
+```
+CUSTOM_PREFIX=http://cache-server/proxy
+```
+
+### Authentication Gateway
+Add authentication to web requests:
+```
+CUSTOM_PREFIX=http://auth-gateway/fetch
+```
+
+### Rate Limit Management
+Route through your own infrastructure to manage rate limits:
+```
+CUSTOM_PREFIX=https://ratelimit-proxy.myserver.com
+```
+
+### Privacy/Monitoring
+Log and monitor all AI web access:
+```
+CUSTOM_PREFIX=http://monitoring-gateway/fetch
+```
+
+## Run
 - Open `http://localhost:8080/` for a basic health probe and connect your MCP client to `http://localhost:8080/mcp`.
 - The MCP endpoint is on `http://localhost:8080/mcp` with Streamable HTTP semantics for POST/GET/DELETE and CORS exposure for `Mcp-Session-Id`.
 
-## Docker Compose
+## Troubleshooting
 
-See `docker-compose.yml` for a working example service mapping `8080:8080` and injecting both environment variables.
+### Server won't start
+- Check that port 3000 is not in use
+- Verify environment variables are set correctly
+- Check Docker logs: `docker-compose logs -f`
+
+### Requests failing
+- Verify CUSTOM_PREFIX is accessible from container
+- Test curl manually: `docker exec -it mcp-web-reader curl -sL YOUR_URL`
+- Check timeout settings (default 30s)
+
+### GitHub Actions not building
+- Ensure workflow permissions are enabled
+- Check that GITHUB_TOKEN has package write access
+- Verify Dockerfile syntax
+
+## License
+
+MIT License - Feel free to use and modify as needed.
 
-## MCP Client Integration
+## Contributing
 
-The server exposes tools over Streamable HTTP: initialize a session with a POST to `/mcp`, reuse `Mcp-Session-Id` from the response, then call `tools/call` with `{ name: "read_web_url", arguments: { url: "https://example.com" } }`. The server will run `curl -sL CUSTOM_PREFIX/https://example.com` and return text content. Ensure the `Mcp-Session-Id` header is included in subsequent requests and allowed via CORS.
+Pull requests welcome! Please ensure:
+- Docker image builds successfully
+- README is updated for new features
+- GitHub Actions workflow passes