Skip to content

formatjson.ps1 - Sort manifest root keys by schema.json and child keys alphabetically #6494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 25 commits into
base: develop
Choose a base branch
from
+202 −73
Open

formatjson.ps1 - Sort manifest root keys by schema.json and child keys alphabetically #6494

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
21fd5d8
Reorder properties to wanted order
o-l-a-v Sep 19, 2025
16ea655
Add sort function to json.ps1
o-l-a-v Sep 19, 2025
98ce740
Take PSCustomObject instead, let parse_json do the parsing
o-l-a-v Sep 19, 2025
5983d28
Added sorting manifest as a step
o-l-a-v Sep 19, 2025
cd06740
Make variable name more telling
o-l-a-v Sep 19, 2025
a579f69
Added changes to changelog
o-l-a-v Sep 19, 2025
c0b41a2
Don't use aliases
o-l-a-v Sep 19, 2025
25a0b59
Use existing function parse_json instead
o-l-a-v Sep 19, 2025
2c21e14
Make parse_json function more readable
o-l-a-v Sep 19, 2025
d9f07a3
Fix goof by me
o-l-a-v Sep 19, 2025
37aa459
Read and parse schema.json once
o-l-a-v Sep 19, 2025
e440802
Add some failproof to make sure only expected keys are present
o-l-a-v Sep 20, 2025
802b26f
Remove debug
o-l-a-v Sep 20, 2025
37d4073
Sort childs alphabetically
o-l-a-v Sep 20, 2025
13bd45f
Only overwrite JSON file if new content is not empty
o-l-a-v Sep 20, 2025
8f696f8
Rename function and add synopsis
o-l-a-v Sep 20, 2025
d25b11e
Use Scoop function abort instead of PowerShell native Throw
o-l-a-v Sep 20, 2025
f3dd97e
Updated changelog to correct name of new function
o-l-a-v Sep 22, 2025
1cdae35
Use System.IO.File WriteAllText, because ConvertToPrettyJson returns …
o-l-a-v Sep 22, 2025
18a5f00
WriteAllText does not write the final newline, so reverting back to W…
o-l-a-v Sep 22, 2025
c02d6d8
Use uin16 instead of byte, just in case Scoop manifest ever gets more…
o-l-a-v Sep 22, 2025
80ab7ac
Validate that input is not null or empty
o-l-a-v Sep 22, 2025
9f429a1
Moved function to manifest.ps1 + made sorting of child keys recursive
o-l-a-v Sep 22, 2025
203e5a4
Apply nitpicks from CodeRabbit, except the case of handling arrays as…
o-l-a-v Sep 22, 2025
981c302
Apply one more nitpick
o-l-a-v Sep 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
## [Unreleased](https://github.com/ScoopInstaller/Scoop/compare/v0.5.3...develop)

### Features

- **schema**: Reorder root properties to match the wanted order.
- **json**: Add function `Sort-ScoopManifestRootProperties` which orders a manifest's root properties to match `schema.json`.
- **formatjson**: Use `Sort-ScoopManifestRootProperties` to sort manifests root properties.

### Bug Fixes

- **scoop-download:** Fix function `nightly_version` not defined error ([#6386](https://github.com/ScoopInstaller/Scoop/issues/6386))
Expand Down
22 changes: 16 additions & 6 deletions bin/formatjson.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ param(
[String] $App = '*',
[Parameter(Mandatory = $true)]
[ValidateScript( {
if (!(Test-Path $_ -Type Container)) {
if (-not (Test-Path $_ -Type Container)) {
throw "$_ is not a directory!"
} else {
$true
Expand All @@ -34,11 +34,21 @@ param(
$Dir = Convert-Path $Dir

Get-ChildItem $Dir -Filter "$App.json" -Recurse | ForEach-Object {
$file = $_.FullName
# beautify
$json = parse_json $file | ConvertToPrettyJson
# Path of file
$file = [string] $_.'FullName'

# convert to 4 spaces
$json = $json -replace "`t", ' '
# Parse JSON
$json = [PSCustomObject](parse_json -path $file)

# Sort JSON root properties
$json = [PSCustomObject](Sort-ScoopManifestRootProperties -JsonAsObject $json)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to sort the keys recursively with global/local order?

Copy link
Contributor Author

@o-l-a-v o-l-a-v Sep 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't fully understand what you're asking for here, about the local/global order.

Recursice ordering keys by the schema.json should be possible. Or just sort child keys alphabetically.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

local/global order

My description might be redundant. Jsut ignore it. Focus on recursive sorting.

I thought that when sorting keys like architecture.64bit/32bit , local order are used because the configuration file contains architecture.64bit/32bit. However, when sorting keys like architecture.64bit.url/hash, the configuration file lacks this entry, causing it to fall back to the global order.

As I'm not sure how to describe this, I called it local/global order.

Scoop/schema.json

Lines 217 to 231 in 37aa459

"architecture": {
"type": "object",
"additionalProperties": false,
"properties": {
"32bit": {
"$ref": "#/definitions/autoupdateArch"
},
"64bit": {
"$ref": "#/definitions/autoupdateArch"
},
"arm64": {
"$ref": "#/definitions/autoupdateArch"
}
}
},

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added sorting of first level child keys if parent is of type PSCustomObject. Going even deeper will be more complex. It's doable, but I think what we got now is a good start.

Copy link
Contributor Author

@o-l-a-v o-l-a-v Sep 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe there are existing PowerShell or C# projects that can sort a JSON file by schema.json and handle all the complexity of going deeper we can use?

In Python there is: https://github.com/ikonst/jschon-sort.

But as I already said, I think this serves as a very good start for getting more uniform manifests.


# Beautify
$json = [string](ConvertToPrettyJson -data $json)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am concerned whether the sort is a stable sort algorithm for keys that are not present in the index list.

Could multiple executions yield different results?

Copy link
Contributor Author

@o-l-a-v o-l-a-v Sep 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO there could and should be a check for keys we don't want to be present. Doesn't this exist already?

To make formatjson.ps1 safer we could do: If key is not present in the schema.json

  • Optian a) fail execution
  • Option b) remove key

Copy link

@z-Fng z-Fng Sep 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this exist already?

Does it? It will fail execution currently.

Sort-Object: D:\Scoop\apps\scoop\current\lib\json.ps1:241:9
Line |
 241 |          Sort-Object -Property @{
     |          ~~~~~~~~~~~~~~~~~~~~~~~~
     | Cannot convert value "-1" to type "System.Byte". Error: "Value was either too large or too small for an unsigned byte."

Copy link
Contributor Author

@o-l-a-v o-l-a-v Sep 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking more about that other parts of the validation should catch manifest errors before it gets to formatjson.ps1. Thus validating that the manifest doesn't contain invalid keys is a task that formatjson.ps1 shouldn't try to solve.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems that's what the validator is for?

https://github.com/ScoopInstaller/Scoop/tree/master/supporting/validator

Thus I don't think the new function Sort-ScoopManifestRootProperties should have a lot of validation logic. It's a feature that it throws if manifest does not comply with schema.json IMO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a check for parent keys not present in schema.json inside Sort-ScoopManifestRootProperties. Then users will get a better error message than what you posted in #6494 (comment).

Copy link
Contributor Author

@o-l-a-v o-l-a-v Sep 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we could write a warning, and just return the object as is / not try to sort the object at all. Instead of throwing.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems that's what the validator is for?

Yes, thanks for reminding.

I added a check for parent keys not present in schema.json inside Sort-ScoopManifestRootProperties. Then users will get a better error message than what you posted.

Totally agree. Great.


# Convert to 4 spaces
$json = [string]($json -replace "`t", ' ')

# Overwrite file content
[System.IO.File]::WriteAllLines($file, $json)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When JSON parsing fails, it will clear the manifest content. Should this behavior be adjusted to prevent contributors from losing their work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, maybe we should add some error handling.

}
37 changes: 37 additions & 0 deletions lib/json.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,3 +212,40 @@ function normalize_values([psobject] $json) {

return $json
}

function Sort-ScoopManifestRootProperties {
[OutputType([PSCustomObject])]

Param(
[Parameter(Mandatory)]
[PSCustomObject] $JsonAsObject
)

# Get wanted order from Scoop manifest schema
if ([string]::IsNullOrWhiteSpace($Script:WantedOrder)) {
Write-Debug -Message ('{0}\..\schema.json' -f $PSScriptRoot)
$Script:WantedOrder = [string[]](
(
parse_json -path (
'{0}\..\schema.json' -f $PSScriptRoot
)
).'properties'.'PSObject'.'Properties'.'Name'
)
}

# Create empty new object where properties will be added to
$SortedObject = [PSCustomObject]::new()

# Add properties from $Current to $Sorted ordered by $WantedOrder
$JsonAsObject.'PSObject'.'Properties'.'Name' |
Sort-Object -Property @{
'Expression' = {
[byte]($WantedOrder.IndexOf($_))
}
} | ForEach-Object -Process {
$null = Add-Member -InputObject $SortedObject -NotePropertyName $_ -NotePropertyValue $JsonAsObject.$_
}

# Return the sorted object
$SortedObject
}
12 changes: 9 additions & 3 deletions lib/manifest.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,16 @@ function manifest_path($app, $bucket) {
(Get-ChildItem (Find-BucketDirectory $bucket) -Filter "$(sanitary_path $app).json" -Recurse).FullName
}

function parse_json($path) {
if ($null -eq $path -or !(Test-Path $path)) { return $null }
function parse_json {
Param(
[Parameter(Mandatory)]
[string] $path
)
if ([string]::IsNullOrWhiteSpace($path) -or -not [System.IO.File]::Exists($path)) {
return $null
}
try {
Get-Content $path -Raw -Encoding UTF8 | ConvertFrom-Json -ErrorAction Stop
Get-Content -Path $path -Raw -Encoding UTF8 | ConvertFrom-Json -ErrorAction Stop
} catch {
warn "Error parsing JSON at '$path'."
}
Expand Down
126 changes: 63 additions & 63 deletions schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -533,6 +533,41 @@
"description": "A comment.",
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"version": {
"pattern": "^[\\w\\.\\-+_]+$",
"type": "string"
},
"description": {
"type": "string"
},
"homepage": {
"format": "uri",
"type": "string"
},
"license": {
"$ref": "#/definitions/license"
},
"notes": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"depends": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"suggest": {
"additionalProperties": false,
"patternProperties": {
"^(.*)$": {
"$ref": "#/definitions/stringOrArrayOfStrings"
}
},
"type": "object"
},
"url": {
"$ref": "#/definitions/uriOrArrayOfUris"
},
"hash": {
"$ref": "#/definitions/hash"
},
"architecture": {
"additionalProperties": false,
"properties": {
Expand All @@ -548,72 +583,49 @@
},
"type": "object"
},
"autoupdate": {
"$ref": "#/definitions/autoupdate"
"innosetup": {
"description": "True if the installer InnoSetup based. Found in https://github.com/ScoopInstaller/Main/search?l=JSON&q=innosetup",
"type": "boolean"
},
"bin": {
"$ref": "#/definitions/stringOrArrayOfStringsOrAnArrayOfArrayOfStrings"
"extract_dir": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"persist": {
"$ref": "#/definitions/stringOrArrayOfStringsOrAnArrayOfArrayOfStrings"
"extract_to": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"checkver": {
"$ref": "#/definitions/checkver"
"pre_install": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"cookie": {
"description": "Undocumented: Found at https://github.com/se35710/scoop-java/search?l=JSON&q=cookie",
"type": "object"
"installer": {
"$ref": "#/definitions/installer"
},
"depends": {
"post_install": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"description": {
"type": "string"
},
"env_add_path": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"env_set": {
"type": "object"
},
"extract_dir": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"extract_to": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"hash": {
"$ref": "#/definitions/hash"
},
"homepage": {
"format": "uri",
"type": "string"
},
"innosetup": {
"description": "True if the installer InnoSetup based. Found in https://github.com/ScoopInstaller/Main/search?l=JSON&q=innosetup",
"type": "boolean"
"bin": {
"$ref": "#/definitions/stringOrArrayOfStringsOrAnArrayOfArrayOfStrings"
},
"installer": {
"$ref": "#/definitions/installer"
"shortcuts": {
"$ref": "#/definitions/shortcutsArray"
},
"license": {
"$ref": "#/definitions/license"
"persist": {
"$ref": "#/definitions/stringOrArrayOfStringsOrAnArrayOfArrayOfStrings"
},
"notes": {
"pre_uninstall": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"post_install": {
"$ref": "#/definitions/stringOrArrayOfStrings"
"uninstaller": {
"$ref": "#/definitions/uninstaller"
},
"post_uninstall": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"pre_install": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"pre_uninstall": {
"$ref": "#/definitions/stringOrArrayOfStrings"
},
"psmodule": {
"additionalProperties": false,
"properties": {
Expand All @@ -623,27 +635,15 @@
},
"type": "object"
},
"shortcuts": {
"$ref": "#/definitions/shortcutsArray"
},
"suggest": {
"additionalProperties": false,
"patternProperties": {
"^(.*)$": {
"$ref": "#/definitions/stringOrArrayOfStrings"
}
},
"type": "object"
},
"uninstaller": {
"$ref": "#/definitions/uninstaller"
"checkver": {
"$ref": "#/definitions/checkver"
},
"url": {
"$ref": "#/definitions/uriOrArrayOfUris"
"autoupdate": {
"$ref": "#/definitions/autoupdate"
},
"version": {
"pattern": "^[\\w\\.\\-+_]+$",
"type": "string"
"cookie": {
"description": "Undocumented: Found at https://github.com/se35710/scoop-java/search?l=JSON&q=cookie",
"type": "object"
}
},
"if": {
Expand Down
Loading