Add package.json extractor for lockfile-less JS projects by jbcibois-ddhq · Pull Request #126 · DataDog/datadog-sbom-generator

jbcibois-ddhq · 2026-04-14T14:43:35Z

🚀 Motivation

JavaScript projects that only ship a package.json (no lockfile) currently produce no SBOM output. This adds a dedicated extractor so these projects get dependency coverage.

📚 Documentation

Document	Link or Detail
RFC	N/A
Incident	N/A
Jira Ticket	K9VULN-13485

📝 Summary

New PackageJSONExtractor that parses dependencies, devDependencies, and optionalDependencies from package.json files.

Version handling:

Exact pins ("lodash": "4.17.21") → version = "4.17.21"
Range specifiers ("lodash": "^4.17.21") → version = "" + new datadog:version-range SBOM property holding the raw specifier (e.g. "^4.17.21")
Non-version values (file:, latest, URLs) → both fields empty

npm alias resolution: "react18": "npm:react@18.3.1" emits a package named react at version 18.3.1, consistent with the npm-lock parser.

Deduplication: by resolvedName@version+versionRange, so aliased variants pointing to the same package at different versions are both preserved.

Lockfile suppression (hasLockfileInAncestors): extraction is skipped when a lockfile (package-lock.json, yarn.lock, pnpm-lock.yaml) exists in the same directory or any ancestor up to context.RootDir. This handles both standalone projects and monorepo workspaces. The check is bypassed when lockfile parsers are disabled via --enable-parsers.

node_modules/ filtering: ShouldExtract rejects paths containing node_modules/ to avoid extracting per-package manifests inside installed dependencies.

🧪 Testing

New tests were added for new logic.
Existing tests were updated for new logic, and not only so that they pass!
Benchmark results prove that performance is the same or better.

🚧 Staging validation

Deployed and monitored using Datadog dashboards.
Proof that it works as expected, including profiling or UX screenshots.

🆘 Recovery

Notes for on-call - select only one:

The change can be rolled back.
Do not roll back. Why?:

datadog-prod-us1-5 · 2026-04-14T14:44:55Z

🎯 Code Coverage (details)
• Patch Coverage: 93.24%
• Overall Coverage: 85.19% (+0.40%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 549617d | Docs | Datadog PR Page | Give us feedback!}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a53af776fb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 01d126b26d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8d01ffd4c0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b68e24bb79

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 61b94650d1

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-04-15T08:31:32Z

+			if _, exists := packages[dedupKey]; exists {
+				continue
+			}


Merge dep groups when deduplicating same package version

The dedup path drops later occurrences of the same resolvedName@version with continue, so dependencies declared in multiple sections (for example both dependencies and devDependencies) retain only one DepGroups value. This loses section metadata and can misclassify direct dependency context, whereas other JS matching paths in this repo append/merge groups instead of discarding them.

Useful? React with 👍 / 👎.

First occurrence wins. The extractor iterates sections in order: dependencies (prod), devDependencies (dev), optionalDependencies (optional) and deduplicates by name@version. So if lodash@4.17.21 appears in both dependencies and devDependencies, it's reported as prod only. npm itself treats a package listed in both dependencies and devDependencies as a prod dependency

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4c22f9bdaf

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-04-15T09:02:43Z

+	if len(lockfiles) > 0 && hasLockfileInAncestors(f, context.RootDir, lockfiles) {
+		return []lockfile.PackageDetails{}, nil


Respect scan exclusions when suppressing package.json

Extract short-circuits to an empty package list whenever hasLockfileInAncestors finds a lockfile on disk, but that presence check is independent of scanner filtering. In scanDir, lockfiles can be skipped by .gitignore/--exclude before parsing, so this early return can suppress package.json fallback even when no lockfile parser will actually run, yielding missing JavaScript dependencies in those filtered scans.

Useful? React with 👍 / 👎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d4dfe717f4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 667ad6b478

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-05-04T15:35:51Z

💡 Codex Review

datadog-sbom-generator/pkg/lockfile/javascript/parse-package-json.go

Line 81 in a8e6a1d

    
           var exactVersionRegex = cachedregexp.MustCompile(`^v?\d+(\.\d+)*(-[\w.]+)?(\+[\w.]+)?$`)

Treat '=' semver pins as exact versions

Dependencies pinned as "=1.2.3" are currently classified as ranged because exactVersionRegex does not accept an optional leading =. In npm this syntax is an exact pin, so these entries end up with Version="" and only VersionRange populated, which produces versionless PURLs and reduces vulnerability matching accuracy for otherwise fully pinned dependencies.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e6a5bce75e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-05-04T15:52:13Z

+
+func (e PackageJSONExtractor) ShouldExtract(path string) bool {
+	normalized := filepath.ToSlash(path)
+	if strings.Contains(normalized, "/"+nodeModulesPath) || strings.HasPrefix(normalized, nodeModulesPath) {


Match node_modules as a full path segment

ShouldExtract still filters by substring/prefix ("/"+nodeModulesPath and HasPrefix) rather than segment boundaries, so directories like node_modules2 or node_modules_backup are incorrectly treated as node_modules and their package.json files are skipped. This causes false negatives in repos that legitimately use similarly named directories. Fresh evidence after the earlier thread: the updated guard now uses strings.Contains(normalized, "/"+nodeModulesPath), which still matches paths such as path/to/node_modules2/foo/package.json.

Useful? React with 👍 / 👎.

rjcoulter22

I think we should gate this under the --manifest-parsers flag like we do for pyproject.toml parsing as well? We would probably want to be consistent IMO 🙂

rjcoulter22 · 2026-05-05T14:14:40Z

+
+const (
+	packageJSONPackageManager      = models.NPM
+	packageJSONOfficiallySupported = true


If this is opt in shgould this be false? https://github.com/DataDog/datadog-sbom-generator/blob/main/pkg/lockfile/python/types.go#L176

rjcoulter22 · 2026-05-05T14:16:50Z

+	return active
+}
+
+func hasLockfileInAncestors(f lockfile.DepFile, rootDir string, lockfiles []string) bool {


Hm idt we do this for pyproject - should we make it consistent?

yes, it is important that lockfile parsers take precedence if we find one

rjcoulter22 · 2026-05-05T14:20:20Z

+}
+
+func (e PackageJSONExtractor) Extract(f lockfile.DepFile, context lockfile.ScanContext) ([]lockfile.PackageDetails, error) {
+	lockfiles := activeLockfiles(context.EnabledParsers)


what is activeLockfiles checking for?

It filters siblingLockfiles (package-lock.json, yarn.lock, pnpm-lock.yaml) down to only the ones whose parsers are enabled. If all lockfile parsers are disabled (e.g. --enable-parsers package.json only), it returns an empty slice, which tells Extract to skip the lockfile guard entirely and always extract from package.json.

rjcoulter22 · 2026-05-05T14:20:53Z

+				versionRange = effectiveSpec
+			}
+
+			dedupKey := resolvedName + "@" + version + versionRange


would we ever have a version and versionRange?

jbcibois-ddhq · 2026-05-11T09:37:15Z

I think we should gate this under the --manifest-parsers flag like we do for pyproject.toml parsing as well? We would probably want to be consistent IMO 🙂

@rjcoulter22 done

anderruiz

pkg/lockfile/javascript/parse-package-json.go — missing LocationRole

The PackageDetails literal built in Extract does not set LocationRole. Since this extractor reads directly from package.json (a manifest, not a lockfile), every package it emits should carry LocationRole: models.LocationRoleManifest — same as parse-pyproject-toml.go (the other manifest-only extractor introduced in PR #138).

Without it, p.LocationRole is the zero value in vulnerability_result.go, so the role field in the emitted PackageLocation is blank even though the location points to a manifest file.

packages[dedupKey] = lockfile.PackageDetails{
    // ...
    BlockLocation:   blockLocation,
    NameLocation:    nameLocation,
    VersionLocation: versionLocation,
+   LocationRole:    models.LocationRoleManifest,
}

anderruiz · 2026-04-15T09:53:10Z

+  "name": "alias-collision",
+  "dependencies": {
+    "react": "17.0.2",
+    "react18": "npm:react@18.3.1"


We need to complete it at least with the cases included here

jbcibois-ddhq requested a review from a team as a code owner April 14, 2026 14:43

jbcibois-ddhq requested a review from anderruiz April 14, 2026 14:43

chatgpt-codex-connector Bot reviewed Apr 14, 2026

View reviewed changes

Comment thread pkg/lockfile/javascript/parse-package-json.go Outdated

Comment thread pkg/lockfile/javascript/parse-package-json.go Outdated

chatgpt-codex-connector Bot reviewed Apr 14, 2026

View reviewed changes

Comment thread pkg/lockfile/javascript/parse-package-json.go Outdated

chatgpt-codex-connector Bot reviewed Apr 15, 2026

View reviewed changes

Comment thread pkg/lockfile/javascript/parse-package-json.go Outdated

chatgpt-codex-connector Bot reviewed Apr 15, 2026

View reviewed changes

Comment thread pkg/lockfile/javascript/parse-package-json.go

jbcibois-ddhq force-pushed the jb.cibois/add-package-json-support branch 3 times, most recently from f5807e2 to 61b9465 Compare April 15, 2026 08:25

chatgpt-codex-connector Bot reviewed Apr 15, 2026

View reviewed changes

jbcibois-ddhq marked this pull request as draft April 15, 2026 13:03

jbcibois-ddhq force-pushed the jb.cibois/add-package-json-support branch 2 times, most recently from d13b1bd to 68c111a Compare May 4, 2026 13:45

jbcibois-ddhq marked this pull request as ready for review May 4, 2026 14:09

chatgpt-codex-connector Bot reviewed May 4, 2026

View reviewed changes

Comment thread pkg/lockfile/javascript/parse-package-json.go Outdated

jbcibois-ddhq added 5 commits May 4, 2026 16:53

Add PackageJSONFilePath constant and extractor types

55b03ab

Add VersionRange to PackageDetails and export as SBOM property

047b379

Add package.json extractor

4fe85d0

Add package.json extractor tests and fixtures

4ce9f7b

Update README and integration snapshots

5849bec

jbcibois-ddhq force-pushed the jb.cibois/add-package-json-support branch from 634d9c9 to 5849bec Compare May 4, 2026 14:53

chatgpt-codex-connector Bot reviewed May 4, 2026

View reviewed changes

Comment thread pkg/lockfile/javascript/parse-package-json.go Outdated

Drop duplicate PURL range entries, first alias key wins

540c53f

jbcibois-ddhq force-pushed the jb.cibois/add-package-json-support branch from 667ad6b to 540c53f Compare May 4, 2026 15:24

Fix node_modules path check to use segment boundaries

a8e6a1d

Handle =1.2.3 exact pin syntax

e6a5bce

chatgpt-codex-connector Bot reviewed May 4, 2026

View reviewed changes

rjcoulter22 reviewed May 5, 2026

View reviewed changes

jbcibois-ddhq added 3 commits May 11, 2026 11:24

Gate package.json extractor under --manifest-parsers flag

7f72b05

Make resolveNpmAlias private

44a8d4f

Add comments explaining magic numbers in package.json extractor

549617d

anderruiz reviewed May 13, 2026

View reviewed changes

anderruiz mentioned this pull request May 13, 2026

test(lockfile): add invariant — LocationRole must be set when BlockLocation is valid #149

Merged

2 tasks

		if len(lockfiles) > 0 && hasLockfileInAncestors(f, context.RootDir, lockfiles) {
		return []lockfile.PackageDetails{}, nil

Conversation

jbcibois-ddhq commented Apr 14, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Motivation

📚 Documentation

📝 Summary

🧪 Testing

🚧 Staging validation

🆘 Recovery

Uh oh!

datadog-prod-us1-5 Bot commented Apr 14, 2026 • edited by datadog-official Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented May 4, 2026

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot May 4, 2026

Choose a reason for hiding this comment

Uh oh!

rjcoulter22 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jbcibois-ddhq commented Apr 14, 2026 •

edited by atlassian Bot

Loading

datadog-prod-us1-5 Bot commented Apr 14, 2026 •

edited by datadog-official Bot

Loading