Add SFTP data plane microservice with API key authentication and scoped permissions

apps/gameservers-service/go.mod

@@ -5,7 +5,6 @@ go 1.25
 require (
 	connectrpc.com/connect v1.19.1
 	github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
-	github.com/docker/go-connections v0.6.0
 	github.com/joho/godotenv v1.5.1
 	github.com/moby/moby/api v1.52.0
 	github.com/moby/moby/client v0.2.1
@@ -23,6 +22,7 @@ require (
 	github.com/containerd/errdefs/pkg v0.3.0 // indirect
 	github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
 	github.com/distribution/reference v0.6.0 // indirect
+	github.com/docker/go-connections v0.6.0 // indirect
 	github.com/docker/go-units v0.5.0 // indirect
 	github.com/felixge/httpsnoop v1.0.4 // indirect
 	github.com/go-logr/logr v1.4.3 // indirect

apps/sftp-service/Dockerfile

@@ -0,0 +1,48 @@
+# Build stage for SFTP Service
+FROM golang:1.25-alpine AS go-builder
+
+# Accept build arguments for multi-arch support
+ARG TARGETARCH=amd64
+ARG TARGETOS=linux
+
+# Install build dependencies
+# Use --no-scripts to disable triggers and avoid QEMU emulation issues
+RUN apk add --no-cache --no-scripts git make
+
+# Set working directory
+WORKDIR /build
+
+# Enable Go build cache and module cache
+ENV GOCACHE=/root/.cache/go-build
+ENV GOMODCACHE=/root/go/pkg/mod
+
+RUN --mount=type=bind,source=.,target=/src,rw \
+    --mount=type=cache,target=/root/go/pkg/mod \
+    --mount=type=cache,target=/root/.cache/go-build \
+    sh -c "export GOWORK=off && \
+           cd /src/apps/shared && go mod download && \
+           cd /src/apps/sftp-service && go mod download && \
+           CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} \
+           GOWORK=off go build -trimpath -ldflags=\"-w -s -extldflags '-static'\" \
+           -o /build/sftp-service /src/apps/sftp-service"
+
+# Final stage
+FROM alpine:latest
+
+# Install CA certificates, curl, and netcat for health checks
+# Use --no-scripts to disable triggers and avoid QEMU emulation issues
+RUN apk update && apk --no-cache --no-scripts add ca-certificates curl netcat-openbsd
+
+WORKDIR /app
+
+# Copy binary from builder
+COPY --from=go-builder /build/sftp-service .
+
+# Create directory for SFTP files and host key
+RUN mkdir -p /var/lib/sftp
+
+# Expose SFTP port (2222) and HTTP health port (3020)
+EXPOSE 2222 3020
+
+# Run the service
+CMD ["./sftp-service"]

apps/sftp-service/IMPLEMENTATION.md

@@ -0,0 +1,196 @@
+# SFTP Service Implementation Summary
+
+## Overview
+
+Successfully implemented a complete SFTP data plane microservice for the Obiente Cloud platform. The service provides secure SFTP file transfer capabilities with API key authentication, scoped permissions, comprehensive audit logging, and organization-based isolation.
+
+## Key Features
+
+### 1. Security
+- **API Key Authentication**: No password-based auth; all authentication via API keys
+- **SHA-256 Hashing**: API keys are hashed before storage and comparison
+- **Scoped Permissions**: Granular control with `sftp:read`, `sftp:write`, `sftp:*`, and `sftp` scopes
+- **Organization Isolation**: Files organized by org/user, preventing cross-org access
+- **Path Traversal Protection**: All paths validated to stay within user's directory
+- **No Symlinks**: Symlink operations disabled for security
+
+### 2. Architecture
+- **Microservice Pattern**: Follows existing patterns (audit-service, auth-service, etc.)
+- **Graceful Shutdown**: Proper cleanup of SFTP and HTTP servers
+- **Health Checks**: HTTP endpoint on port 3020 for monitoring
+- **Database Integration**: PostgreSQL for API key storage
+- **Audit Logging**: TimescaleDB for operation logs
+
+### 3. Operations
+- **Read Operations**: Download, list, stat (requires `sftp:read`)
+- **Write Operations**: Upload, delete, mkdir, rename (requires `sftp:write`)
+- **Wildcard Scopes**: `sftp:*` and `sftp` grant both read and write
+- **Audit Trail**: All operations logged with user, org, path, and result
+
+## Files Created
+
+### Core Package (`apps/shared/pkg/sftp/`)
+- `server.go`: SFTP server implementation with SSH integration
+- `handler.go`: File operation handlers with permission checking
+
+### Microservice (`apps/sftp-service/`)
+- `main.go`: Service entry point with HTTP and SFTP servers
+- `internal/service/auth.go`: API key validator and audit logger
+- `Dockerfile`: Multi-stage build following existing patterns
+- `README.md`: Comprehensive documentation
+- `go.mod`: Dependency management
+
+### Database (`apps/shared/pkg/database/`)
+- `api_keys.go`: APIKey model with GORM annotations
+
+### Infrastructure
+- `migrations/001_create_api_keys_table.sql`: Database schema
+- `scripts/create-api-key.sh`: Utility for creating API keys
+- Updated `docker-compose.yml`: Local development config
+- Updated `docker-compose.swarm.yml`: Production swarm config
+- Updated `go.work`: Workspace configuration
+
+## Configuration
+
+### Environment Variables
+- `SFTP_PORT`: SFTP server port (default: 2222)
+- `SFTP_BASE_PATH`: Base directory for files (default: /var/lib/sftp)
+- `SFTP_HOST_KEY_PATH`: SSH host key location (default: /var/lib/sftp/host_key)
+- `PORT`: HTTP health check port (default: 3020)
+- Standard database and auth variables from existing services
+
+### Docker
+- **Port 2222**: SFTP access
+- **Port 3020**: HTTP health checks
+- **Volume**: `sftp-data` for persistent storage
+- **Networks**: `obiente-network` overlay
+
+## API Key Scopes
+
+| Scope | Read | Write | Description |
+|-------|------|-------|-------------|
+| `sftp:read` | ✅ | ❌ | Download and list files |
+| `sftp:write` | ❌ | ✅ | Upload, delete, modify files |
+| `sftp:*` | ✅ | ✅ | Full access |
+| `sftp` | ✅ | ✅ | Full access |
+
+## Directory Structure
+
+Files are organized as:
+```
+/var/lib/sftp/
+  ├── org-123/
+  │   ├── user-456/
+  │   │   ├── file1.txt
+  │   │   └── subdir/
+  │   └── user-789/
+  │       └── file2.txt
+  └── org-abc/
+      └── user-def/
+          └── file3.txt
+```
+
+## Usage Examples
+
+### Creating an API Key
+```bash
+./apps/sftp-service/scripts/create-api-key.sh \
+  "My SFTP Key" \
+  user-123 \
+  org-456 \
+  "sftp:read,sftp:write"
+```
+
+### Connecting via SFTP
+```bash
+# Using command-line client
+sftp -P 2222 user@hostname
+# When prompted for password, enter your API key
+
+# Using FileZilla
+Host: sftp://hostname
+Port: 2222
+User: any_username
+Password: your-api-key
+```
+
+### Testing Health
+```bash
+curl http://localhost:3020/health
+```
+
+## Audit Logging
+
+All operations are logged to TimescaleDB with:
+- User ID and Organization ID
+- Operation type (upload, download, delete, etc.)
+- File path
+- Success/failure status
+- Bytes transferred
+- Timestamp
+
+## Deployment
+
+### Docker Compose (Development)
+```bash
+docker compose up -d sftp-service
+```
+
+### Docker Swarm (Production)
+```bash
+docker stack deploy -c docker-compose.swarm.yml obiente
+```
+
+## Security Considerations
+
+1. **API Keys**: 
+   - Stored as SHA-256 hashes
+   - Never logged in plain text
+   - Include expiration and revocation support
+
+2. **File Isolation**:
+   - Each user restricted to their directory
+   - Path traversal attempts blocked
+   - No symlink support
+
+3. **Audit Trail**:
+   - All operations logged
+   - Failed attempts recorded
+   - User and organization tracked
+
+4. **Network Security**:
+   - SSH protocol for transport encryption
+   - Host key verification
+   - No password fallback
+
+## Code Review Fixes
+
+1. ✅ Replaced custom string splitting with `strings.Split`
+2. ✅ Implemented SHA-256 hashing for API keys
+3. ✅ Fixed wildcard scope permissions (`sftp:*` and `sftp` now grant both read/write)
+
+## Testing Checklist
+
+- [x] Service builds successfully
+- [x] Code review completed and issues fixed
+- [x] Security scan passed (no CodeQL issues)
+- [ ] Manual testing with SFTP client
+- [ ] API key authentication verification
+- [ ] Permission scope enforcement testing
+- [ ] Audit log verification
+- [ ] Cross-org isolation testing
+
+## Future Enhancements
+
+1. **Rate Limiting**: Implement per-user/org rate limits
+2. **Quota Management**: Enforce storage quotas per user/org
+3. **Web UI**: Admin interface for API key management
+4. **Metrics**: Prometheus metrics for SFTP operations
+5. **WebDAV**: Add WebDAV support for web-based file access
+
+## References
+
+- SFTP Protocol: RFC 4251-4254
+- Go SSH Package: golang.org/x/crypto/ssh
+- Go SFTP Package: github.com/pkg/sftp
+- Existing Microservices: audit-service, auth-service, etc.

apps/sftp-service/README.md

@@ -0,0 +1,137 @@
+# SFTP Service
+
+A secure SFTP data plane microservice with API key authentication, permission scoping, and comprehensive audit logging.
+
+## Features
+
+- **API Key Authentication**: Authenticate using API keys instead of passwords
+- **Permission Scoping**: Separate read and write permissions (sftp:read, sftp:write)
+- **User Isolation**: Each user has their own isolated directory (organized by org/user)
+- **Audit Logging**: All operations are logged to the audit service
+- **Secure by Design**: No symlinks, path traversal protection, permission enforcement
+
+## Configuration
+
+Environment variables:
+
+- `SFTP_PORT`: SFTP server port (default: 2222)
+- `SFTP_BASE_PATH`: Base directory for SFTP files (default: /var/lib/sftp)
+- `SFTP_HOST_KEY_PATH`: Path to SSH host key (default: /var/lib/sftp/host_key)
+- `PORT`: HTTP health check port (default: 3020)
+- `DATABASE_URL`: PostgreSQL database URL
+- `METRICS_DATABASE_URL`: TimescaleDB URL for audit logs
+- `LOG_LEVEL`: Logging level (debug, info, warn, error)
+
+## API Key Scopes
+
+API keys must have one or more of these scopes:
+
+- `sftp:read` - Allow reading/downloading files and listing directories
+- `sftp:write` - Allow uploading, deleting, and modifying files
+- `sftp:*` or `sftp` - Grant both read and write permissions
+
+## Usage
+
+### Creating an API Key
+
+API keys must be created through the auth service or admin interface with the appropriate SFTP scopes.
+
+Example:
+```
+Scopes: sftp:read,sftp:write
+```
+
+### Connecting via SFTP
+
+```bash
+# Using command-line SFTP client
+sftp -P 2222 -o User=any_username -o IdentityFile=/path/to/key user@hostname
+
+# When prompted for password, enter your API key
+
+# Using FileZilla or other GUI clients
+Host: sftp://hostname
+Port: 2222
+User: any_username (username doesn't matter, authentication is via API key)
+Password: your-api-key
+```
+
+### Directory Structure
+
+Files are organized by organization and user:
+
+```
+/var/lib/sftp/
+  ├── org-123/
+  │   ├── user-456/
+  │   │   ├── file1.txt
+  │   │   └── subdir/
+  │   └── user-789/
+  │       └── file2.txt
+  └── org-abc/
+      └── user-def/
+          └── file3.txt
+```
+
+Each user can only access their own directory within their organization.
+
+## Operations
+
+All operations are audited and logged:
+
+- **upload**: Upload a file
+- **download**: Download a file
+- **delete**: Delete a file or directory
+- **mkdir**: Create a directory
+- **rename**: Rename/move a file
+- **list**: List directory contents
+- **stat**: Get file information
+
+## Security
+
+- **Path Traversal Protection**: Users cannot escape their directory
+- **No Symlinks**: Symlink creation and reading is disabled
+- **Permission Enforcement**: Operations are checked against API key scopes
+- **Audit Trail**: All operations are logged with user, org, and result
+- **API Key Tracking**: Last used timestamp is updated on each connection
+
+## Health Check
+
+HTTP endpoint available at `http://localhost:3020/health`
+
+Returns:
+```json
+{
+  "status": "healthy",
+  "service": "sftp-service",
+  "timestamp": "2024-01-17T20:00:00Z",
+  "details": {
+    "sftp_address": "0.0.0.0:2222",
+    "base_path": "/var/lib/sftp"
+  }
+}
+```
+
+## Development
+
+```bash
+# Build
+go build -o sftp-service
+
+# Run
+./sftp-service
+
+# Test connection
+sftp -P 2222 test@localhost
+# Enter your API key when prompted for password
+```
+
+## Architecture
+
+The service consists of three main components:
+
+1. **SFTP Server** (`pkg/sftp/server.go`): Handles SSH/SFTP protocol
+2. **Auth Validator** (`internal/service/auth.go`): Validates API keys against database
+3. **Audit Logger** (`internal/service/auth.go`): Logs operations to TimescaleDB
+
+The server uses the standard Go SSH and SFTP libraries with custom handlers for permission checking and audit logging.