Re-organize and upgrade NPM dependencies (#5021)

### Motivation

There are several problems with the existing dependency management via
npm.json and npm.py.

**No control over importmap key**

Example: Newer Three.js versions come with three.module.js which points
to three.core.js. We can declare both files as a dependency, but they
use the same key "three" in the importmap. A similar problem came up in
#4163.

**"Manual" code modifications needed**

If a file like three.module.js references `./three.core.js`, we have to
replace the relative link because the file three.core.js isn't found at
this relative URL. Similarly there is an import of
`../utils/BufferGeometryUtils.js` in Three.js' GLTFLoader which we need
to replace with `BufferGeometryUtils`. These corrections happen in
npm.py.

**Complex dependencies hard or impossible to integrate**

There is an extra script at scripts/codemirror/bundle.bash to integrate
CodeMirror. And we wanted to integrate ajv-formats for `ui.json_editor`
(https://cdn.jsdelivr.net/npm/[email protected]/+esm), but it includes
references to e.g. "/npm/[email protected]/+esm" which don't work in a local
NiceGUI app (see #4749).

### Implementation

For this PR I decided to move modules with dependencies into separate
directories (e.g. nicegui/elements/scene/) instead of holding all
dependencies in one giant lib/ directory. This includes codemirror,
scene, aggrid, joystick, json_editor, plotly, echarts, leaflet and
mermaid.

Each of these subdirectories contains an independent package.json,
bundler configuration (rollup or vite), src/ directory with an index.js,
a Git-ignored node-modules/ directory, and a generated dist/ directory
with the resulting (sometimes chunked) bundle. This way we can configure
the build individually for each UI element and don't have to find a
one-fits-all solution. Furthermore, we can rely on standard NPM tools
without the need for our own custom JSON format and Python scripts for
downloading and extracting packages. Now the build process is 1. `npm
install` and 2. `npm run build`.

For registering the module in NiceGUI, I introduced a new `esm`
parameter when subclassing an `Element`, e.g.:
```py
class JsonEditor(Element, component='json_editor.js', esm={'nicegui-json-editor': 'dist'}):
    ...
```
This defines an entry to the importmap named "nicegui-json-editor"
pointing to the relative directory dist/. This replaces the definition
of individual `dependencies=[...]`, which we will probably deprecate.

### Progress

The PR is still work in progress. The change has quite some implications
that still need to be implemented.

- [x] I chose a meaningful title that completes the sentence: "If
applied, this PR will..."
- [x] The implementation is complete.
    - [x] remove deprecated parameters from `Element.__init_subclass__`
    - [x] create custom package.jsons per UI element
- [x] create package.json for NiceGUI's core dependencies (Vue, Quasar,
Tailwind)
    - [x] remove npm.json/npm.py
- [x] update examples for custom components (number_checker,
signature_pad)
    - [x] update gitattributes
    - [x] generate DEPENDENCIES.md?
    - [x] what about minification and maps?
- [x] Pytests are not necessary.
- [x] Documentation has been updated.

Author: falkoschindler

.gitattributes

@@ -1,8 +1,6 @@
 nicegui/elements/tailwind.py       linguist-generated
 nicegui/elements/tailwind_types/** linguist-generated
-nicegui/elements/lib/**            linguist-vendored
-nicegui/scripts/codemirror/*.json  linguist-vendored
-nicegui/scripts/codemirror/node_modules linguist-vendored
+nicegui/elements/*/dist/**         linguist-vendored
 nicegui/static/es-module-shims.js  linguist-vendored
 nicegui/static/fonts/**            linguist-vendored
 nicegui/static/fonts.css           linguist-vendored
@@ -16,3 +14,4 @@ website/**                         linguist-documentation
 website/static/fuse.js@*           linguist-vendored
 examples/**                        linguist-documentation
 examples/fullcalendar/lib/**       linguist-vendored
+**/package-lock.json               linguist-vendored

.gitignore

@@ -1,7 +1,6 @@
 __pycache__/
 *.egg-info/
 .*.swp
-dist
 /test.py
 *.pickle
 screenshots/
@@ -12,4 +11,3 @@ venv
 .DS_Store
 .env
 node_modules/
-package-lock.json

.pre-commit-config.yaml

@@ -1,7 +1,7 @@
 default_language_version:
   python: python3.9
 default_install_hook_types: [pre-commit, pre-push]
-default_stages: [commit]
+default_stages: [pre-commit]
 
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
@@ -15,7 +15,6 @@ repos:
       - id: trailing-whitespace
         exclude: &exclusions |
           (?x)^(
-            nicegui/elements/lib/.*|
             nicegui/static/es-module-shims\.js|
             nicegui/static/fonts/.*|
             nicegui/static/fonts\.css|
@@ -25,7 +24,8 @@ repos:
             nicegui/static/tailwindcss\..*|
             nicegui/static/vue\..*|
             website/static/fuse\.js\@.*|
-            examples/fullcalendar/lib/.*
+            examples/fullcalendar/lib/.*|
+            .*\.js\.map
           )$
       - id: end-of-file-fixer
         exclude: *exclusions

CONTRIBUTING.md

@@ -228,28 +228,27 @@ The title should match the example folder name when [snake case converted](https
 
 ## Node dependencies
 
-We use [npm.json](https://github.com/zauberzeug/nicegui/blob/main/npm.json) to pin the versions of the node dependencies.
+We use package.json files to pin the versions of node dependencies.
+There is a `package.json` file in the root directory for core dependencies
+and additional `package.json` files in `nicegui/elements/.../` directories for individual UI elements.
 They are usually updated by the maintainers during major releases.
 
 To update or add new dependencies, we follow these steps:
 
 1. Use `npm` or other derivative tools, modify the `package.json` file with new versions or add dependencies.
-   - We **never** modify the version in `package.json` or `npm.json` directly, since there may be conflicts between versions.
-   - `npx npm-check-updates -u --target semver` is a good starting point for updating dependencies.
 2. Run `npm install` to install the new dependencies.
    Any conflicts in installation will be caught at this moment.
-3. Run `pin_versions.py` to update the `npm.json` file.
-4. Run `npm.py` to download dependencies into the `nicegui/static/` and `nicegui/elements/lib/` directories.
-5. Remember to commit `package.json` in addition to the new dependencies and `DEPENDENCIES.md`, such that Dependabot can catch outdated dependencies with security issues.
+3. Run `npm run build` to copy the dependencies into the `nicegui/static/` directory or
+   to bundle the dependencies in the `nicegui/elements/.../` directories.
 
-Apart from updating Node libraries, the following tools are used to update other resources:
+The following tools are used to update other resources:
 
-- scripts/codemirror/bundle.bash for managing the CodeMirror dependency
 - fetch_google_fonts.py for fetching the Google Fonts
 - fetch_languages.py to update the list of supported languages in language.py
 - fetch_milestone.py to prepare the release notes for a given milestone
 - fetch_sponsors.py to update the list of sponsors on the website and in the README.md file
 - fetch_tailwind.py to update NiceGUI's Tailwind API
+- summarize_dependencies.py to update the dependencies in the DEPENDENCIES.md file
 
 ## Pull requests
 

DEPENDENCIES.md

@@ -1,19 +1,23 @@
 # Included Web Dependencies
 
-- vue: 3.4.38 ([MIT](https://opensource.org/licenses/MIT))
-- quasar: 2.16.9 ([MIT](https://opensource.org/licenses/MIT))
-- tailwindcss: 3.4.10 ([MIT](https://opensource.org/licenses/MIT))
-- socket.io: 4.7.5 ([MIT](https://opensource.org/licenses/MIT))
-- es-module-shims: 1.10.0 ([MIT](https://opensource.org/licenses/MIT))
-- aggrid: 32.1.0 ([MIT](https://opensource.org/licenses/MIT))
-- codemirror: 6.0.1 ([MIT](https://opensource.org/licenses/MIT))
-- echarts: 5.5.1 ([Apache-2.0](https://opensource.org/licenses/Apache-2.0))
-- echarts-gl: 2.0.9 ([BSD-3-Clause](https://opensource.org/licenses/BSD-3-Clause))
-- leaflet: 1.9.4 ([BSD-2-Clause](https://opensource.org/licenses/BSD-2-Clause))
-- leaflet-draw: 1.0.4 ([MIT](https://opensource.org/licenses/MIT))
-- mermaid: 11.0.2 ([MIT](https://opensource.org/licenses/MIT))
-- nipplejs: 0.10.2 ([MIT](https://opensource.org/licenses/MIT))
-- plotly: 2.35.0 ([MIT](https://opensource.org/licenses/MIT))
-- three: 0.168.0 ([MIT](https://opensource.org/licenses/MIT))
-- tween: 25.0.0 ([MIT](https://opensource.org/licenses/MIT))
-- vanilla-jsoneditor: 0.23.8 ([ISC](https://opensource.org/licenses/ISC))
+- es-module-shims: 2.6.2 ([MIT](https://opensource.org/licenses/MIT))
+- quasar: 2.18.2 ([MIT](https://opensource.org/licenses/MIT))
+- socket.io: 4.8.1 ([MIT](https://opensource.org/licenses/MIT))
+- tailwindcss: 3.4.17 ([MIT](https://opensource.org/licenses/MIT))
+- vue: 3.5.18 ([MIT](https://opensource.org/licenses/MIT))
+- ag-grid-community: ^34.1.1 ([MIT](https://opensource.org/licenses/MIT))
+- codemirror: ^6.0.2 ([MIT](https://opensource.org/licenses/MIT))
+- @codemirror/language-data: ^6.5.1 ([MIT](https://opensource.org/licenses/MIT))
+- @codemirror/theme-one-dark: ^6.1.3 ([MIT](https://opensource.org/licenses/MIT))
+- @uiw/codemirror-themes-all: ^4.24.2 ([MIT](https://opensource.org/licenses/MIT))
+- @babel/runtime: ^7.28.2 ([MIT](https://opensource.org/licenses/MIT))
+- echarts: ^6.0.0 ([Apache-2.0](https://opensource.org/licenses/Apache-2.0))
+- echarts-gl: ^2.0.9 ([BSD 3-Clause](https://opensource.org/licenses/BSD-3-Clause))
+- nipplejs: ^0.10.2 ([MIT](https://opensource.org/licenses/MIT))
+- vanilla-jsoneditor: ^3.8.0 ([ISC](https://opensource.org/licenses/ISC))
+- leaflet: ^1.9.4 ([BSD-2-Clause](https://opensource.org/licenses/BSD-2-Clause))
+- leaflet-draw: ^1.0.4 ([MIT](https://opensource.org/licenses/MIT))
+- mermaid: ^11.9.0 ([MIT](https://opensource.org/licenses/MIT))
+- plotly.js: ^3.1.0 ([MIT](https://opensource.org/licenses/MIT))
+- three: ^0.179.1 ([MIT](https://opensource.org/licenses/MIT))
+- @tweenjs/tween.js: ^25.0.0 ([MIT](https://opensource.org/licenses/MIT))

examples/editable_ag_grid/main.py

@@ -35,7 +35,7 @@ async def delete_selected():
     aggrid = ui.aggrid({
         'columnDefs': columns,
         'rowData': rows,
-        'rowSelection': 'multiple',
+        'rowSelection': {'mode': 'multiRow'},
         'stopEditingWhenCellsLoseFocus': True,
     }).on('cellValueChanged', handle_cell_value_change)
 

examples/local_file_picker/local_file_picker.py

@@ -31,7 +31,7 @@ def __init__(self, directory: str, *,
             self.add_drives_toggle()
             self.grid = ui.aggrid({
                 'columnDefs': [{'field': 'name', 'headerName': 'File'}],
-                'rowSelection': 'multiple' if multiple else 'single',
+                'rowSelection': {'mode': 'multiRow' if multiple else 'singleRow'},
             }, html_columns=[0]).classes('w-96').on('cellDoubleClicked', self.handle_double_click)
             with ui.row().classes('w-full justify-end'):
                 ui.button('Cancel', on_click=self.close).props('outline')

examples/node_module_integration/README.md

@@ -6,7 +6,8 @@ We chose this package to demonstrate a very simple node module which has a depen
 namely the [is-number](https://www.npmjs.com/package/is-number) package.
 Using NPM, we can easily install both packages and bundle them into a single file which can be used in the app.
 The package.json file defines the is-odd dependency and some dev dependencies for bundling the node module,
-the webpack.config.js file specifies the entry point for the node module,
+the rollup.config.mjs file configures the bundling process,
+the src/index.mjs file is the entry point for the node module,
 and number_checker.js as well as number_checker.py define a new UI element to be used in the NiceGUI app main.py.
 
 1. First, install all third-party node modules (assuming you have NPM installed):
@@ -23,6 +24,8 @@ and number_checker.js as well as number_checker.py define a new UI element to be
    npm run build
    ```
 
+   This will create a dist directory containing the is-odd module.
+
 3. Finally, you can run the app as usual:
 
    ```bash

examples/node_module_integration/dist/index.js

@@ -0,0 +1,13 @@
+function e(e){return e&&e.__esModule&&Object.prototype.hasOwnProperty.call(e,"default")?e.default:e}
+/*!
+ * is-number <https://github.com/jonschlinkert/is-number>
+ *
+ * Copyright (c) 2014-2018, Jon Schlinkert.
+ * Released under the MIT License.
+ */var r,t,n,u;function o(){return t?r:(t=1,r=function(e){var r=+e;return r-r===0&&(r===e||"string"==typeof e&&(0!==r||""!==e.trim()))})}
+/*!
+ * is-odd <https://github.com/jonschlinkert/is-odd>
+ *
+ * Copyright (c) 2015-2017, Jon Schlinkert.
+ * Released under the MIT License.
+ */var a=function(){if(u)return n;u=1;const e=o();return n=function(r){const t=Math.abs(r);if(!e(t))throw new TypeError("expected a number");if(!Number.isInteger(t))throw new Error("expected an integer");if(!Number.isSafeInteger(t))throw new Error("value exceeds maximum safe integer");return t%2==1}}(),i=e(a);export{i as default};

examples/node_module_integration/number_checker.js

@@ -1,7 +1,6 @@
+import isOdd from "is-odd";
+
 export default {
-  async mounted() {
-    await import("is-odd");
-  },
   methods: {
     isOdd(number) {
       return isOdd(number);