Skip to content

Conversation

corneliusludmann
Copy link
Contributor

@corneliusludmann corneliusludmann commented Oct 17, 2025

Problem

The Classic PAYG sunset implementation in #21100 only added blocking checks to the new gRPC API (WorkspaceServiceAPI - gitpod.v1.WorkspaceService) but missed the legacy websocket API (GitpodServerImpl). This created multiple bypass routes that allowed blocked users to continue creating and starting workspaces.

Architecture Overview

There are THREE separate workspace APIs in Gitpod:

  1. Websocket API (OLD) - GitpodServerImpl in components/server

    • Protocol: JSON-RPC over WebSocket
    • Metrics: gitpod_server_api_calls_total{method="startWorkspace"}
    • Sunset Check: ❌ NO (on main)
  2. New gRPC API - WorkspaceServiceAPI in components/server

  3. Experimental API - public-api-server (separate Go service)

    • Service: gitpod.experimental.v1.WorkspacesService
    • Protocol: gRPC/Connect → proxies to Websocket API (API 1)
    • Metrics: connect_server_handled_seconds{package="gitpod.experimental.v1.WorkspacesService"}
    • Sunset Check: ❌ NO (proxies to API 1 which has no check)

Bypass Routes Identified

Users can bypass the sunset blocking through:

  1. Gitpod CLI (gitpod workspace create/start)

    • Uses gitpod.experimental.v1.WorkspacesService (API 3)
    • Routes through: CLI → public-api-server → websocket → GitpodServerImpl
    • Primary user of experimental/v1 API
  2. Gitpod Local App (Desktop application)

    • Same as CLI, uses experimental/v1 API
    • Routes through: Local App → public-api-server → websocket → GitpodServerImpl
  3. JetBrains Gateway

    • Connects directly to websocket API (API 1)
    • Routes through: Gateway → websocket → GitpodServerImpl
  4. External integrations with Personal Access Tokens

    • Can use experimental/v1 API
    • Routes through: External client → public-api-server → websocket → GitpodServerImpl
  5. Dashboard fallback

    • When dashboard_public_api_workspace_enabled feature flag is disabled
    • Routes through: Dashboard → websocket → GitpodServerImpl

All these routes ultimately call GitpodServerImpl.startWorkspace() or GitpodServerImpl.createWorkspace() which had no sunset checks.

Solution

Added the sunset check to both methods in GitpodServerImpl:

  • startWorkspace() - Checks before starting existing workspace
  • createWorkspace() - Checks before creating and starting new workspace

Uses the same isWorkspaceStartBlockedBySunset() function already implemented for the gRPC API, ensuring consistent behavior across all entry points.

Why This Works

The fix is applied at the websocket API layer (GitpodServerImpl), which is the common backend for:

  • Direct websocket connections (JetBrains, Dashboard fallback)
  • Proxied connections from public-api-server (CLI, Local App, external clients)

This ensures ALL workspace creation/start requests are blocked, regardless of which API surface they use.

Testing

  • ✅ Code compiles successfully
  • ✅ Follows exact same pattern as workspace-service-api.ts
  • ✅ Handles both organizationId sources (workspace object and request options)

Recommended Testing

  1. Enable sunset feature flag in ConfigCat
  2. Test CLI: gitpod workspace create <url> should be blocked
  3. Test Local App: Creating workspace should be blocked
  4. Test JetBrains Gateway: Creating workspace should be blocked
  5. Verify exempted orgs still work
  6. Verify dedicated installations are not affected

Expected Metrics After Deployment

For blocked users, you should see:

# Websocket API (backend)
gitpod_server_api_calls_total{method="startWorkspace", statusCode="403"}
gitpod_server_api_calls_total{method="createWorkspace", statusCode="403"}

# Experimental API (CLI/Local App)
connect_server_handled_seconds{package="gitpod.experimental.v1.WorkspacesService", code="permission_denied"}

Security Impact

This closes a critical security hole that allowed users to bypass the Classic PAYG sunset restrictions. Without this fix, the sunset blocking is ineffective for:

  • All CLI users (gitpod workspace create/start)
  • All Local App users
  • JetBrains Gateway users
  • External integrations using Personal Access Tokens
  • Dashboard users when the new gRPC API is disabled

The fix ensures sunset blocking works consistently across all API surfaces.

The original sunset implementation only added checks to the new gRPC API
(WorkspaceServiceAPI) but missed the legacy websocket API (GitpodServerImpl).
This allowed users to bypass the sunset blocking through:

- Gitpod CLI/Local App (uses experimental/v1 API)
- JetBrains Gateway (uses websocket API directly)
- Public API with Personal Access Tokens
- Dashboard when feature flag is disabled

This fix adds the sunset check to both startWorkspace() and createWorkspace()
methods in GitpodServerImpl, using the same isWorkspaceStartBlockedBySunset()
function that's already used in WorkspaceServiceAPI.

The check:
- Blocks installation-owned users (no organizationId)
- Blocks users in non-exempted organizations
- Exempts dedicated installations
- Exempts organizations in the exemptedOrganizations list

Co-authored-by: Ona <[email protected]>
@roboquat roboquat merged commit 7421edc into main Oct 17, 2025
61 of 62 checks passed
@roboquat roboquat deleted the fix/classic-payg-sunset-websocket-bypass branch October 17, 2025 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants