Add RustShare Object Spaces for public S3-compatible artifact storage

[senolcolak]

Summary

Add a new RustShare product capability called Object Spaces: managed S3-compatible storage areas that RustShare users/workspace admins can expose through a shared public S3 endpoint.

This is not intended to give every RustShare user a separate S3 endpoint. Instead, RustShare should expose one controlled public S3 endpoint, for example:

https://s3.rustshare.io

RustShare users should then be able to create multiple scoped S3 identities/access keys for different use cases:

release-ci-writer
website-cdn-uploader
apt-repo-publisher
registry-storage-writer
readonly-customer-access

Each identity should be scoped to a specific Object Space, bucket, and/or prefix.

---

Product goal

Turn RustShare from only a file-sharing and workspace-memory product into a stronger technical storage platform for teams.

Primary use cases:

• CDN/static website assets
• CI/CD build artifacts
• release binaries
• Debian/Ubuntu package repository files
• generic public/private S3-compatible object storage
• future OCI/Docker registry backend storage

This feature should be positioned as:

RustShare Object Spaces — secure S3-compatible storage for technical teams to publish artifacts, CDN assets, package repositories, and registry-backed storage.

---

Important product boundary

Object Spaces must be separate from normal RustShare files, notes, kanban attachments, and workspace documents.

Do not allow external S3 clients to write directly into the internal RustShare file/blob/metadata prefixes.

Reason:

• normal RustShare files depend on RustShare metadata, permissions, versioning, audit, and indexing
• direct S3 mutation could create metadata/blob drift
• out-of-band changes must not silently corrupt RustShare expectations

Recommended boundary:

Internal RustShare storage:
  apps/rustshare/...
  blobs/...
  meta/...

Object Spaces:
  object-spaces/{tenant_id}/{space_id}/...
  or dedicated buckets per Object Space

---

Proposed architecture

RustShare Web UI / API
  - Object Space management
  - S3 identity management
  - quotas / audit / public access rules
        |
        v
RustFS Admin / IAM Adapter
  - create bucket or prefix
  - create S3 identity/access key
  - attach least-privilege policy
  - revoke / rotate keys
        |
        v
Public endpoints:
  https://s3.rustshare.io        authenticated S3 API
  https://cdn.rustshare.io       optional public HTTP/CDN access
  https://packages.rustshare.io  future package repository frontend
  https://registry.rustshare.io  future OCI registry frontend
        |
        v
RustFS object storage

RustShare should manage the control plane. Large object uploads/downloads should go directly to RustFS through the public S3 endpoint or through the dedicated public HTTP/CDN frontend.

---

Public vs private terminology

The UI must clearly distinguish these two concepts:

Public S3 endpoint

The S3 API endpoint is reachable from the internet but still requires credentials.

Example:

https://s3.rustshare.io

Public object access

Objects can be anonymously downloaded over HTTP/CDN if explicitly enabled.

Example:

https://cdn.rustshare.io/kubedo/releases/app.tar.gz

The UI should show this distinction clearly:

S3 API Access: authenticated endpoint
Public Web Access: disabled / enabled

---

Recommended data model

Add domain entities similar to:

object_spaces
  id
  tenant_id
  workspace_id
  name
  bucket
  base_prefix
  visibility: private | public | cdn
  purpose: generic | cdn | artifacts | packages | registry
  quota_bytes
  created_by
  created_at
  disabled_at

s3_identities
  id
  tenant_id
  object_space_id
  name
  access_key_id
  secret_hash_or_encrypted_secret_ref
  permissions
  allowed_prefix
  expires_at
  last_used_at
  revoked_at
  created_by

s3_access_audit_events
  id
  tenant_id
  object_space_id
  identity_id
  operation
  object_key
  source_ip
  user_agent
  status
  bytes
  timestamp

object_publication_rules
  id
  object_space_id
  public_base_url
  cache_control
  allowed_content_types
  directory_listing_enabled

Secrets must never be logged. Secret values should be shown only once. If storage is required for automation, encrypt them using the existing RustShare secret encryption mechanism.

---

Permission modes

Start with simple user-facing modes:

• Read-only
• Write-only
• Read/write
• Read/write/delete
• Publisher
• Registry backend

Map these to S3 permissions such as:

s3:GetObject
s3:PutObject
s3:DeleteObject
s3:ListBucket
s3:AbortMultipartUpload
s3:ListMultipartUploadParts

Default safety rules:

• no access outside the assigned Object Space or prefix
• no access to internal RustShare prefixes
• no DeleteObject unless explicitly enabled
• no anonymous listing by default
• no root RustFS credentials exposed
• access keys can expire
• access keys can be revoked
• all security-sensitive actions are audited

---

MVP scope

Implement the first version as a generic Object Spaces feature.

MVP requirements

• Create Object Space from RustShare UI/API
• Choose purpose: generic, CDN/static assets, artifacts
• Create scoped S3 identities/access keys
• Show endpoint, bucket, prefix, access key, secret key once
• Revoke access key
• Rotate access key
• Enforce bucket/prefix scoped permissions
• Enforce quota per Object Space
• Track storage usage
• Basic audit log for key creation, revocation, public access changes, and object activity where available
• Provide examples for aws s3, s3cmd, and rclone

Example output shown to user:

Endpoint: https://s3.rustshare.io
Bucket: rs-kubedo-artifacts
Prefix: releases/
Access Key: RS3...
Secret Key: shown once
Region: us-east-1
Path-style: enabled

Example CLI:

aws --endpoint-url https://s3.rustshare.io s3 cp ./release.tar.gz s3://rs-kubedo-artifacts/releases/v1.0/

---

Phase 2: CDN/public HTTP publishing

Add a public HTTP layer separate from the S3 API.

Requirements:

• public-read toggle per Object Space or prefix
• cdn.rustshare.io public URL mapping
• correct Content-Type
• configurable Cache-Control
• immutable asset support
• optional custom domain later
• optional purge/invalidation hook later
• bandwidth metrics
• no directory listing unless explicitly allowed

---

Phase 3: Debian/Ubuntu package repository publishing

S3 alone does not make a Debian/Ubuntu repository. RustShare should provide or integrate a package publishing layer.

Requirements:

• upload .deb files
• generate package metadata
• generate Packages, Release, and InRelease
• sign repository metadata
• publish atomically
• keep previous snapshots
• rollback support
• public package repository URL, for example:

https://packages.rustshare.io/kubedo

The object store should hold the final static repository tree, but RustShare should manage the publisher workflow.

---

Phase 4: OCI/Docker registry integration

Do not expose raw S3 as the Docker image interface.

Docker/OCI clients need a registry API. The correct model is:

docker push registry.rustshare.io/kubedo/app:1.0
        |
        v
OCI Registry / CNCF Distribution
        |
        v
RustFS S3 backend

RustShare should eventually provision:

• registry namespace
• registry storage bucket/prefix
• registry storage credentials
• registry auth integration
• image retention rules
• garbage collection policy
• usage/audit visibility

---

Security and operational requirements

• Public S3 must go through a controlled reverse proxy/load balancer, not raw node exposure
• TLS required
• RustFS console must not be publicly exposed
• RustFS root credentials must never be shown or used by end users
• all created S3 identities must be least-privilege
• tenant isolation must be contract-tested
• quota enforcement must exist at the RustShare layer and, where possible, storage layer
• audit logs must not leak secrets or sensitive object names across tenants
• object names and access keys must be sanitized from application logs
• public object access must be explicit, visible, and revocable

---

Acceptance criteria

• RustShare has an Object Spaces UI/API surface
• A workspace admin can create an Object Space
• A workspace admin can create multiple S3 identities for one Object Space
• Each S3 identity is scoped to one bucket/prefix
• The S3 endpoint is shared globally, not per user
• Secret key is shown only once
• Keys can be revoked and rotated
• Access outside the assigned scope is denied
• Object Spaces cannot access internal RustShare file/blob/metadata prefixes
• Public web access is separate from authenticated S3 access
• Object Space storage usage is visible
• Object Space quota is enforced or clearly marked as pending if not yet supported by RustFS
• Security-sensitive actions produce audit events
• aws s3 compatibility example works
• rclone compatibility example works
• Contract tests cover tenant isolation, prefix isolation, revocation, and public access behavior

---

Non-goals for first implementation

• full Docker registry implementation
• full Debian/Ubuntu repo publisher
• custom CDN domain management
• billing integration
• multi-region active/active object storage
• allowing S3 writes into normal RustShare file metadata areas

---

Why this matters

This feature strongly fits RustShare's technical-team positioning. It creates a bridge between file sharing, team memory, and practical infrastructure storage.

It also supports real engineering workflows:

• CI publishes build artifacts
• websites publish static assets
• teams host release downloads
• package repositories can be served from controlled object storage
• future OCI registry can use RustFS as backend storage

This makes RustShare more than a Dropbox-like file app. It becomes technical storage infrastructure for teams.