vlt serverless registry (vsr)

vsr aims to be a minimal "npm-compatible" registry which replicates the core features found in registry.npmjs.org as well as adding net-new capabilities.

Table of Contents:

• Quick Starts
• Requirements
• API
• Compatibility
• Comparisons
• Roadmap
• License
• Testing

Quick Starts

Local

You can quickly get started by installing/executing vsr with the following command:

npx -y vltpkg/vsr

Production

You can deploy vsr to Cloudflare in under 5 minutes, for free, with a single click (coming soon).

Alternatively, you can deploy to production using wrangler after following the Development quick start steps.

Development

# clone the repo
git clone https://github.com/vltpkg/vsr.git

# navigate to the repository directory
cd ./vsr

# install the project's dependencies
npm install

# run the development script
npm run dev

For detailed information on development workflow, testing, and contributing to the project, please see our CONTRIBUTING.md guide.

Requirements

Production

• Cloudflare (free account at minimum)
  • Workers (free: 100k requests /day)
  • D1 Database (free: 100k writes, 5M reads /day & 5GB Storage /mo)
  • R2 Bucket (free: 1M writes, 10M reads & 10GB /mo)

Note: all usage numbers & pricing documented is as of October 24th, 2024. Plans & metering is subject to change at Cloudflare's discretion.

Development

• git
• node
• npm

Granular Access Tokens

All tokens are considered "granular access tokens" (GATs). Token entries in the database consist of 3 parts:

• token the unique token value
• uuid associative value representing a single user/scope
• scope value representing the granular access/privileges

scope as a JSON Object

A scope contains an array of privileges that define both the type(s) of & access value(s) for a token.

Note

Tokens can be associated with multiple "types" of access

• type(s):
  • pkg:read read associated packages
  • pkg:read+write write associated packages (requires read access)
  • user:read read associated user
  • user:read+write write associated user (requires read access)
• value(s):
  • * an ANY selector for user: or pkg: access types
  • ~<user> user selector for the user: access type
  • @<scope>/<pkg> package specific selector for the pkg: access type
  • @<scope>/* glob scope selector for pkg: access types

Note

• user/org/team management via @<scope> is not supported at the moment

Granular Access Examples

End-user/Subscriber Persona

• specific package read access
• individual user read+write access

[
  {
    "values": ["@organization/package-name"],
    "types": {
      "pkg": {
        "read": true,
      }
    }
  },
  {
    "values": ["~johnsmith"],
    "types": {
      "user": {
        "read": true,
        "write": true,
      }
    }
  }
]

Team Member/Maintainer Persona

• scoped package read+write access
• individual user read+write access

[
  {
    "values": ["@organization/*"],
    "types": {
      "pkg": {
        "read": true
      }
    }
  },
  {
    "values": ["~johnsmith"],
    "types": {
      "user": {
        "read": true,
        "write": true,
      }
    }
  }
]

Package Publish CI Persona

• organization scoped packages read+write access
• individual user read+write access

[
  {
    "values": ["@organization/package-name"],
    "types": {
      "pkg": {
        "read": true
      }
    }
  },
  {
    "values": ["~johnsmith"],
    "types": {
      "user": {
        "read": true,
        "write": true,
      }
    }
  }
]

Organization Admin Persona

• organization scoped package read+write access
• organization users read+write access

[
  {
    "values": ["@company/*"],
    "types": {
      "pkg": {
        "read": true,
        "write": true
      },
      "user": {
        "read": true,
        "write": true
      }
    }
  }
]

Registry Owner/Admin Persona

[
  {
    "values": ["*"],
    "types": {
      "pkg": {
        "read": true,
        "write": true
      },
      {
        "user": {
          "read": true,
          "write": true
        }
      }
    }
  }
]

API

We have rich, interactive API docs out-of-the-box with the help of our friends Scalar. The docs can be found at the registry root when running vsr locally (ex. run npx -y vltpkg/vsr &/or check out this repo & run npm run dev)

Notable API features include:

• Complete npm-compatible registry API
• Semver range resolution for package version requests
• Support for URL-encoded complex semver ranges (e.g., %3E%3D1.0.0%20%3C2.0.0 for >=1.0.0 <2.0.0)
• Background refresh of stale package data
• Minimal JSON responses for faster installs

npm Client Compatibility

The following commands should work out-of-the-box with npm & any other npm "compatible" clients although their specific commands & arguments may vary (ex. vlt, yarn, pnpm & bun)

Configuration

To use vsr as your registry you must either pass a registry config through a client-specific flag (ex. --registry=... for npm) or define client-specific configuration which stores the reference to your registry (ex. .npmrc for npm). Access to the registry & packages is private by default although an "admin" user is created during setup locally (for development purposes) with a default auth token of "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx".

; .npmrc
registry=http://localhost:1337
//localhost:1337/:_authToken=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Supported `npm` Client Commands

Support	Commannd
✅	access
✅	access list packages
✅	access get status
✅	access set status
🕤	access set mfa
✅	access grant
✅	access revoke
🕤	adduser - PUT /-/org/@<org>/<user>: Adds/updates a user (requires admin privileges)
⏳	audit
✅	bugs
✅	dist-tag add
✅	dist-tag rm
✅	dist-tag ls
✅	deprecate
✅	docs
✅	exec
✅	install
⏳	login
⏳	logout
🕤	org
✅	outdated
🕤	owner add
🕤	owner rm
🕤	owner ls
✅	ping
🕤	profile enable-2fa
🕤	profile disable-2fa
✅	profile get
🕤	profile set - PUT /-/npm/v1/user: Updates a user (requires auth)
✅	publish
✅	repo
✅	search
🕤	team
✅	view
✅	whoami

Package Registry Comparisons

Feature	vsr	npm	GitHub	Verdaccio	JSR	jFrog	Sonatype	Cloudsmith	Buildkite	Bit
License	FSL-1.1-MIT	Closed Source	Closed Source	MIT	MIT	Closed Source	Closed Source	Closed Source	Closed Source	Closed Source
Authored Language	JavaScript	JavaScript	Ruby/Go	TypeScript	Rust	-	-	-	-	-
Publishing	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
Installation	✅	✅	✅	✅	✴️	✅	✅	✅	✅	✅
Search	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
Scoped Packages	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
Unscoped Packages	✅	✅	❌	✅	❌	✅	✅	✅	✅	❌
Proxying Upstream Sources	✅	❌	✴️	✅	❌	✅	✅	✅	✅	❌
Hosted Instance	✅	✅	✅	❌	❌	✅	✅	✅	✅	✅
Hosted Instance Cost	$	-	$$$$	-	-	$$$$	$$$$	$$$$	$$$	$$$
Self-Hosted Instance	✅	❌	✴️	✅	✅	✅	✅	❌	❌	❌
Self-Hosted Instance Cost	🆓	-	$$$$$	$	$	$$$$$	$$$$$	-	-	-
Hosted Public Packages	⏳	✅	❌	❌	✅	❌	❌	❌	❌	✅
Hosted Private Packages	🕤	✅	✅	❌	✅	❌	❌	❌	❌	✅
Hosted Private Package Cost	-	$$	🆓	❌	❌	❌	❌	❌	❌	🆓
Granular Access/Permissions	✅	✴️	❌	✅	❌	✴️	✴️	✴️	✴️	❌
Manifest Validation	✅	✴️	❌	❌	✴️	✴️	✴️	❌	❌	❌
Audit	🕤	✴️	❌	✴️	✴️	✴️	✴️	✴️	❌	❌
Events/Hooks	🕤	❌	✅	❌	❌	✅	✅	✅	❌	❌
Plugins	🕤	❌	❌	✅	❌	✅	✅	✅	❌	❌
Multi-Cloud	🕤	❌	❌	✅	✅	❌	❌	❌	❌	❌
Documentation Generation	🕤	❌	❌	❌	✅	❌	❌	❌	❌	✴️
Unpackaged Files/ESM Imports	🕤	❌	❌	❌	✴️	❌	❌	❌	❌	❌
Variant Support	🕤	❌	❌	❌	✴️	❌	❌	❌	❌	❌

Legend:

• ✅ implemented
• ✴️ supported with caveats
• ⏳ in-progress
• 🕤 planned
• ❌ unsupported
• $ expense estimation (0-5)

Roadmap

v1.0.0

Status	Feature
⏳	api: package insights (powered by socket)
⏳	api: audit (powered by socket)
⏳	api: rate-limiting
🕤	web: package pages
🕤	web: search
🕤	web: user login (ex. npm login / --auth-type=web)
🕤	web: user account management
🕤	web: user registration
🕤	web: admin user management

v1.x

Status	Feature
🕤	mfa access provisioning
🕤	orgs
🕤	teams
🕤	staging
🕤	events/hooks
🕤	plugins/middleware
🕤	variants/distributions

License

This project is licensed under the Functional Source License (FSL-1.1-MIT).

Features

• Package Management: Publish, install, and manage npm packages
• Scoped Packages: Full support for scoped packages (e.g., @scope/package)
• Semver Range Support: Install packages using semver ranges (e.g., >=1.0.0, *)
• URL-encoded Semver Ranges: Handle complex semver ranges with spaces and special characters (e.g., >=1.0.0 <2.0.0)
• Proxying: Optional proxying to upstream registry for missing packages
• Caching: Efficient caching of package metadata and tarballs
• Security: Built-in support for package integrity verification
• Minimal JSON: Support for minimal JSON responses to reduce bandwidth
• Background Updates: Automatic background updates of stale package data
• Dist-tag Management: Manage package distribution tags
• Protected Tags: Special protection for the "latest" dist-tag which cannot be deleted
• Proxy Restrictions: Dist-tag operations not allowed on proxied packages
• Access Management: Granular access control for packages with read-only or read-write permissions
• Package Access Controls: Get and set package access status (private/public)
• User-level Permission Management: Grant and revoke package access permissions for users
• Manifest confusion checks: Ensure consistency and integrity of published packages
• Semver range resolution for package manifests: Accurate resolution of package versions
• Support for URL-encoded complex semver ranges: Handle complex semver ranges with spaces and special characters
• Dist-tag management for package versions: Manage package distribution tags
• Background refresh of stale package data: Keep package data up-to-date
• Minimal JSON responses for faster installs: Reduce bandwidth usage

API Endpoints

Package Manifest

Get a package manifest for a specific version or semver range:

# Get latest version
GET /<pkg-name>

# Get specific version
GET /<pkg-name>/1.0.0

# Get using dist tag
GET /<pkg-name>/latest

# Get using semver range
GET /<pkg-name>/>=1.0.0
GET /<pkg-name>/*

# Get using complex semver range (URL-encoded)
GET /<pkg-name>/%3E%3D1.0.0%20%3C2.0.0  # Equivalent to ">=1.0.0 <2.0.0"
GET /<pkg-name>/%5E1.0.0%7C%7E2.0.0     # Equivalent to "^1.0.0||~2.0.0"

The manifest endpoint supports:

• Explicit versions (e.g., 1.0.0)
• Dist tags (e.g., latest)
• Semver ranges (e.g., >=1.0.0, *)
• URL-encoded complex semver ranges with spaces and special characters

For semver ranges, the registry will:

1. Try to resolve the range from local database
2. If proxying is enabled, try to resolve from upstream registry
3. Return the highest matching version if found
4. Return a proper npm-style response if no matching version is found

URL-encoded Semver Ranges

npm clients often use complex semver ranges with spaces and special characters, which need to be URL-encoded in HTTP requests. This registry properly handles decoding and processing these ranges. Examples:

Range	URL-encoded	Description
>=1.0.0 <2.0.0	%3E%3D1.0.0%20%3C2.0.0	Version 1.0.0 or higher, but less than 2.0.0
^1.0.0||~2.0.0	%5E1.0.0%7C%7C%7E2.0.0	Compatible with 1.0.0 OR approximately 2.0.0
>1.0.0-beta <1.0.0	%3E1.0.0-beta%20%3C1.0.0	Greater than 1.0.0-beta but less than 1.0.0

Dist Tags

Manage package distribution tags:

# List all dist-tags for a package
GET /-/package/<pkg-name>/dist-tags

# Get a specific dist-tag
GET /-/package/<pkg-name>/dist-tags/<tag>

# Create or update a dist-tag
PUT /-/package/<pkg-name>/dist-tags/<tag>

# Delete a dist-tag
DELETE /-/package/<pkg-name>/dist-tags/<tag>

The registry also supports the npm client commands:

# List all dist-tags for a package
npm dist-tag ls <pkg-name>

# Add a new dist-tag
npm dist-tag add <pkg-name>@<version> <tag>

# Remove a dist-tag
npm dist-tag rm <pkg-name> <tag>

Dist-tags allow you to mark specific versions with labels like "latest", "beta", "next", "stable", etc.

Important restrictions:

• The "latest" tag is special and cannot be deleted, as it's the default tag used when installing packages without an explicit version or tag.
• Dist-tag operations (add, remove, list) cannot be performed on proxied packages. These operations are only available for packages that are published directly to this registry.
• Dist-tag names cannot be valid semver ranges (e.g., ">=1.0.0", "^1.0.0", "1.x", "*") to avoid confusion between version requests and tag names. This matches npm's behavior.

Access Management

Control access to packages with fine-grained permissions:

# Get the access level for a package
GET /-/package/<pkg-name>/access

# Set the access level for a package
PUT /-/package/<pkg-name>/access
# Request body: { "status": "private" }

# List packages a user has access to
GET /-/package/list?user=<username>

# Grant access to a package for a user
PUT /-/package/<pkg-name>/collaborators/<username>
# Request body: { "permission": "read-only" } or { "permission": "read-write" }

# Revoke access to a package from a user
DELETE /-/package/<pkg-name>/collaborators/<username>

The registry supports the npm client commands:

# List packages a user has access to
npm access list packages [<user>]

# Get the access level for a package
npm access get status <pkg>

# Set the access level for a package
npm access set status <pkg> <private|public>

# Grant access to a package for a user
npm access grant <read-only|read-write> <user> <pkg>

# Revoke access to a package from a user
npm access revoke <user> <pkg>

Important notes:

• All packages in this registry are private by default.
• Public access setting is not currently supported by this registry.
• Access management operations require appropriate permissions:
  • To grant or revoke access, you need user write permissions.
  • To set package access status, you need package write permissions.

Testing

VSR uses Vitest for testing. The test suite is organized into different categories for easier maintenance and focused testing.

Running Tests

The main test command runs all tests:

npm test

You can run specific test suites using the following commands:

# Run all tests
npm run test:run

# Run cache and waituntil tests
npm run test:cache

# Run improved waituntil tests
npm run test:improved

# Run manifest slimming tests
npm run test:slim

# Run integrity validation tests
npm run test:integrity

# Run packument tests
npm run test:packument

# Run JSON format tests
npm run test:json-format

Test Configuration

All tests use a single consolidated configuration file (vitest.config.js) that determines which tests to run based on the VITEST_MODE environment variable. This approach simplifies configuration management and makes it easier to add new test suites.

To add a new test suite:

1. Add a new entry to the testConfigs object in vitest.config.js
2. Create a new npm script in package.json that sets the appropriate VITEST_MODE