ci: add JFrog OIDC proxy for npm registry access#223
Closed
pffigueiredo wants to merge 24 commits intomainfrom
Closed
ci: add JFrog OIDC proxy for npm registry access#223pffigueiredo wants to merge 24 commits intomainfrom
pffigueiredo wants to merge 24 commits intomainfrom
Conversation
neondatabase-protected-runner-group cannot reach npm registries directly. Route through databricks.jfrog.io using OIDC authentication. Bun reads .npmrc for registry config. Same pattern as neondatabase/neon-js. Co-authored-by: Isaac
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
The default working-directory is landing/, and landing/.npmrc already exists. Bun stops walking up when it finds an .npmrc, so the root-level file was never read. Append JFrog registry config to landing/.npmrc. Co-authored-by: Isaac
.npmrc was correctly written but bun may not read it for tarball downloads. Add BUN_CONFIG_REGISTRY env var as belt-and-suspenders, plus a debug step to verify .npmrc contents. Co-authored-by: Isaac
landing/.npmrc has no trailing newline, so >> concatenated the registry line onto the save-exact line, producing invalid config. Co-authored-by: Isaac
The runner image (ubuntu-22-04-crowdstrike-image) has a TLS-intercepting proxy. Bun uses BoringSSL with bundled root CAs and doesn't read the system CA store. NODE_EXTRA_CA_CERTS makes bun trust the system bundle where CrowdStrike's CA cert is installed. Co-authored-by: Isaac
Need to understand: does the CA file exist? Is there a proxy? Can curl reach npm/JFrog? What cert does the TLS handshake present? Co-authored-by: Isaac
Co-authored-by: Isaac
Diagnostics show registry.npmjs.org is network-blocked on the runner (SSL_ERROR_SYSCALL), not a cert issue. JFrog is reachable. But bun with --frozen-lockfile may use cached resolution URLs ignoring the registry override. Test without --frozen-lockfile first. Co-authored-by: Isaac
Need to verify if JFrog rewrites tarball URLs or leaves them pointing to registry.npmjs.org (which is network-blocked on the runner). Co-authored-by: Isaac
bun.lock stores empty resolved URLs which default to registry.npmjs.org for tarball downloads. Deleting the lockfile forces bun to resolve everything fresh from the configured JFrog registry, including tarballs. Confirmed locally: without lockfile, bun correctly uses JFrog URLs. Co-authored-by: Isaac
- Remove debug/diagnostic steps from pr.yml - Remove NODE_EXTRA_CA_CERTS (not needed — fix was lockfile deletion) - Add landing/bunfig.toml pointing to npm-proxy.dev.databricks.com for local development behind the corporate proxy Co-authored-by: Isaac
- bunfig.toml: set default registry to npm-proxy.dev.databricks.com - .npmrc: override @neondatabase scope (global ~/.npmrc points it to the blocked registry.npmjs.org) - Regenerate bun.lock via the Databricks proxy registry.npmjs.org is network-blocked on Databricks machines. npm-proxy.dev.databricks.com proxies all packages including @neondatabase scoped ones. Co-authored-by: Isaac
bunfig.toml points to npm-proxy.dev.databricks.com (local dev proxy) which CI runners can't reach. Override it with JFrog URL in CI. Revert .npmrc @neondatabase scope (only needed locally, and bunfig.toml handles it). Co-authored-by: Isaac
.npmrc has @neondatabase scope pointing to npm-proxy.dev.databricks.com (local dev only). CI overwrites .npmrc entirely with JFrog config to avoid reaching unreachable dev proxy. Co-authored-by: Isaac
The scope override was working around a stale global ~/.npmrc that pointed @neondatabase packages at the blocked registry.npmjs.org. The Databricks proxy already has @neondatabase packages, so the override is unnecessary once the global config is fixed. Co-authored-by: Isaac
Default registry already handles all packages including @neondatabase. Co-authored-by: Isaac
Drop JFrog CLI, OIDC auth, .npmrc overwrite. The committed bunfig.toml already points to npm-proxy.dev.databricks.com which needs no auth. If CI runners can reach it, this is all we need. Co-authored-by: Isaac
npm-proxy.dev.databricks.com is only reachable from dev machines, not from neondatabase-protected-runner-group CI runners. JFrog OIDC is required for CI. Consolidated the .npmrc and bunfig.toml overwrite into a single Configure step. Co-authored-by: Isaac
Don't commit internal proxy URL to public repo. Instead: - bin/setup-registry.sh generates bunfig.toml locally (run once) - bunfig.toml is gitignored - CI generates its own bunfig.toml with JFrog in the workflow Co-authored-by: Isaac
No manual setup needed — bun install triggers preinstall which generates bunfig.toml if missing. CI already writes its own bunfig.toml before install, so the hook is a no-op there. Co-authored-by: Isaac
Avoids committing internal proxy URL to public repo. The script reads whatever registry the developer has in their global ~/.npmrc. Databricks devs get the proxy, external contributors get npmjs.org. Co-authored-by: Isaac
Instead of letting bun hang with cryptic ConnectionRefused errors, detect registry.npmjs.org and tell the developer to configure a proxy in their global ~/.npmrc. Co-authored-by: Isaac
Check BUN_CONFIG_REGISTRY env var first, then npm config, then default. Error message gives three options: global ~/.npmrc, env var for this session, or manual bunfig.toml creation. Co-authored-by: Isaac
Co-authored-by: Isaac
4 tasks
Collaborator
Author
|
Superseded by #224 which includes JFrog OIDC setup + pnpm migration in a single PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
bun installthrough Databricks npm proxies sinceregistry.npmjs.orgis network-blocked on Databricks machinesdatabricks.jfrog.io) with auth, overwrites.npmrc+bunfig.toml, deletes lockfile before installbunfig.tomlpointing tonpm-proxy.dev.databricks.com(no auth needed)798530b)Why delete the lockfile?
Bun ignores registry config for tarball downloads when a lockfile exists — it always uses
registry.npmjs.org. Deleting the lockfile forces fresh resolution through the configured proxy. Version drift is mitigated bysave-exact=true.Test plan
rm -rf node_modules bun.lock && bun install)This pull request was AI-assisted by Isaac.