V4.0.0

.github/workflows/main.yaml

@@ -69,6 +69,7 @@ jobs:
         pytest tests/test_manifests.py --tests=./specifications/json-ld-api/tests --loader=${{ matrix.loader }}
         pytest tests/test_manifests.py --tests=./specifications/json-ld-framing/tests --loader=${{ matrix.loader }}
         pytest tests/test_manifests.py --tests=./specifications/normalization/tests --loader=${{ matrix.loader }}
+        pytest tests/test_manifests.py --tests=./specifications/rdf-canon/tests --loader=${{ matrix.loader }}
         pytest --ignore ./tests/test_manifests.py
       env:
         LOADER: ${{ matrix.loader }}

.gitmodules

@@ -7,3 +7,6 @@
 [submodule "specifications/normalization"]
 	path = specifications/normalization
 	url = https://github.com/json-ld/normalization.git
+[submodule "specifications/rdf-canon"]
+	path = specifications/rdf-canon
+	url = https://github.com/w3c/rdf-canon.git

CHANGELOG.md

@@ -1,6 +1,60 @@
 # pyld ChangeLog
 
-## 3.3.0 - unreleased
+## 4.0.0 - unreleased
+
+### Added
+
+- Migrate `canon.py` to `rdflib`
+  - Added `rdflib` dependency in all relevant config and code files.
+  - Added `util.py`:
+    - Added the functions `from_legacy_dataset()` and `to_legacy_dataset()` to convert an `rdflib.Dataset` to an RDFJS-like `dict` and back wherever needed. 
+    - Added unittests in `tests/test_util.py` for these functions.
+- Implement RDFC1.0
+  - Added new class `RDFC10` (subclass of `URDNA2015`) in `canon.py`
+  - Added the RDFC1.0 test-suite in `tests/runtests.py`
+    - Added support for testing blank-node identifier maps. 
+    - Added support for testing with different hashing algorithms
+
+### Removed
+
+- **BREAKING**: Removed the internal `pyld.nquads` parser/serializer module.
+
+### Fixed
+
+- Preserved RDF literal lexical forms when converting through RDFLib, including canonical double output, large numeric values, and compound literal handling.
+- Fixed `iri_resolver.unresolve()` query/fragment reconstruction while cleaning up the resolver docstring.
+- Invalid IRI base values within lists are now skipped when serializing to RDF. Fixes [toRdf#tli12](https://w3c.github.io/json-ld-api/tests/toRdf-manifest#tli12) and [toRdf#tli14](https://w3c.github.io/json-ld-api/tests/toRdf-manifest#tli14).
+- Literals with invalid `@language` no longer output triples. Fixes [toRdf#twf05](https://w3c.github.io/json-ld-api/tests/toRdf-manifest#twf05).
+- IRI validation now detects multiple # fragments so these IRI's are skipped during toRDF. Fixes [toRdf#te111](https://w3c.github.io/json-ld-api/tests/toRdf-manifest#te111) and [toRdf#te112](https://w3c.github.io/json-ld-api/tests/toRdf-manifest#te112).
+- In `_create_node_map()`, ignored `@id: None` subjects are now dropped before they enter the node map to prevent `_graph_to_rdf()` from crashing when sorting mixed `None` and string keys. Fixes [toRdf#te122](https://w3c.github.io/json-ld-api/tests/toRdf-manifest#te122).
+- Use a helper for generalized N-Quads normalization in the test runner, since rdflib cannot parse blank-node predicates. Fixes [toRdf#te075](https://w3c.github.io/json-ld-api/tests/toRdf-manifest#te075).
+
+### Changed
+- **BREAKING**: Migrated `jsonld.to_rdf()`, `jsonld.from_rdf()`, and N-Quads parsing/serialization to use `rdflib.Dataset` and RDFLib terms directly.
+  - `jsonld.to_rdf()` now returns an `rdflib.Dataset` by default when no output `format` is requested.
+  - Added `legacyMode` to `jsonld.to_rdf()` to return the previous RDF.js-like dataset `dict`.
+  - `jsonld.from_rdf()` now accepts both `rdflib.Dataset` inputs and the previous legacy dataset `dict` shape.
+  - Updated RDF-related tests for RDFLib dataset and N-Quads serialization behavior.
+- **BREAKING**: nquads input/output now delegates to RDFLib instead of PyLD’s custom parser and serializer.
+- Migrate `canon.py` to `rdflib`:
+  - Now use `rdflib` for RDF term type checking (e.g., checking is something is a bnode), looping over triples/quads, nquads serialization and constructing RDF terms  (custom deepcopy is no longer needed) 
+  - Move nquads parsing from`JsonLdProcessor.normalize()` to `URDNA2015.main()` so all parsing and serialization is handled by the same class. 
+  - Move the main logic to `URDNA2015._canonicalize(self, dataset: Dataset)` while keeping input and output in `URDNA2015.main()`. The method `URDNA2015._canonicalize(self, dataset: Dataset)` accepts an `rdflib.Dataset` and returns a tuple with 
+      - the canonicalized result as a nquads `str` and 
+      - the blank node identifier map as `dict`.
+  - The method `URDNA2015 .main(self, dataset: str | dict | Dataset, options)` now 
+    - accepts a `rdflib.Dataset` object in addition to an nquads `str` or the original RDFJS-like`dict`. 
+    - returns 
+      - a `str`: the serialized nquads result, or 
+      - a `dict`: the result as RDFJS-like dataset or the blank node identifier map when the new parameter `outputMap` is `True`.
+  - The hashing algorithm is now an class attribute `URDNA2015.hash_algorithm` so it  configurable (required for RDFC1.0)
+  - The `permutations()` function now uses `itertools.permutations` instead of a custom implementation.
+  - Replacements for rdflib's `_nq_row` and `_quoteLiteral` (these should eventually move to a fix for rdflib's nquads serializer).
+  - Re-enabled all skipped URDNA2015, URDNA2012 tests in `tests/runtests.py`
+- If the result of a test is a dict and the expected value is a string, the expected value is now parsed as JSON (needed for testing blank-node identifier maps).
+- **BREAKING**: Removed small Python 2-era compatibility code and simplified a few internal collection checks.
+
+## 3.3.0 - 2026-09-02
 
 ### Added
 - `pyld.FileDocumentLoader`: a document loader for local `file:` URLs with optional root confinement.

README.md

@@ -2,7 +2,8 @@
 
 [Documentation](https://digitalbazaar.github.io/pyld/) |
 [Installation](#installation) | [Usage](#usage) | [Advanced
-Topics](#advanced-topics) | [Contributing](./CONTRIBUTING.md) |
+Topics](#advanced-topics) | [What's new in version 4](#whats-new-in-version-4) |
+[Contributing](./CONTRIBUTING.md) |
 [Changelog](./CHANGELOG.md)
 
 ## Introduction
@@ -41,9 +42,31 @@ memory footprint in order to operate.
 ## Requirements
 
 * Python (3.10 or later)
+* rdflib (7.3 or later)
 * [Requests](http://docs.python-requests.org/) (optional)
 * [aiohttp](https://aiohttp.readthedocs.io/) (optional)
 
+## What's new in version 4
+
+PyLD 4 moves RDF handling to [RDFLib](https://rdflib.readthedocs.io/) and adds
+RDF Dataset Canonicalization 1.0 support.
+
+The main developer-facing changes are:
+
+* `jsonld.to_rdf()` returns an `rdflib.Dataset` by default when no output
+  `format` is requested.
+* `jsonld.to_rdf(doc, {'legacyMode': True})` returns the RDF.js-like dataset
+  `dict` used by PyLD versions lower than 4.
+* `jsonld.from_rdf()` accepts `rdflib.Dataset`, N-Quads strings, and the legacy
+  dataset `dict`.
+* `jsonld.normalize()` defaults to `RDFC10`; pass `{'algorithm': 'URDNA2015'}`
+  if older canonicalization output must remain stable.
+* The internal `pyld.nquads` parser/serializer module has been removed. Use the
+  JSON-LD APIs with `format: 'application/n-quads'` or RDFLib directly.
+
+See the full [version 4 upgrade notes](https://digitalbazaar.github.io/pyld/whats-new-version-4/)
+and the [`4.0.0` changelog entry](./CHANGELOG.md#400---unreleased).
+
 ## Installation
 
 PyLD can be installed with a [pip](http://www.pip-installer.org/)
@@ -138,10 +161,10 @@ flattened = jsonld.flatten(doc)
 framed = jsonld.frame(doc, frame)
 # document transformed into a particular tree structure per the given frame
 
-# normalize a document using the RDF Dataset Normalization Algorithm
-# (URDNA2015), see: https://www.w3.org/TR/rdf-canon/
+# normalize a document using RDF Dataset Canonicalization 1.0
+# (RDFC10), see: https://www.w3.org/TR/rdf-canon/
 normalized = jsonld.normalize(
-    doc, {'algorithm': 'URDNA2015', 'format': 'application/n-quads'})
+    doc, {'algorithm': 'RDFC10', 'format': 'application/n-quads'})
 # normalized is a string that is a canonical representation of the document
 # that can be used for hashing, comparison, etc.
 
@@ -153,6 +176,15 @@ rdf = jsonld.to_rdf(
 # flatten() and frame() accept the same identifierIssuer option
 ```
 
+`jsonld.to_rdf()` returns an `rdflib.Dataset` when no output `format` is
+requested. To keep the RDF.js-like dataset `dict` used by PyLD versions lower
+than 4, pass
+`{'legacyMode': True}`:
+
+```python
+dataset = jsonld.to_rdf(doc, {'legacyMode': True})
+```
+
 ## Features & conformance
 
 This library aims to conform with the following W3C Recommendations:

docs/.pages

@@ -1,11 +1,14 @@
 nav:
   - Quickstart:
       - index.md
+      - whats-new-version-4.md
       - installation.md
       - conformance.md
-  - API:
-      - reference
   - CLI:
       - cli
+  - Tutorials:
+      - tutorials
+  - API:
+      - reference
   - Project:
       - project

docs/examples/context_resolver_sqlite_cache.py

@@ -0,0 +1,21 @@
+import json
+from pathlib import Path
+
+from cachetools import LRUCache
+
+from pyld import ContextResolver, SqliteCacheRequestsDocumentLoader, jsonld
+
+loader = SqliteCacheRequestsDocumentLoader(
+    sqlite_file_path=Path("/tmp/pyld_example_context_cache.sqlite"),
+)
+resolver = ContextResolver(LRUCache(maxsize=1000), loader)
+
+doc = {
+    "@context": "https://schema.org/",
+    "name": "Example Person",
+}
+options = {"documentLoader": loader, "contextResolver": resolver}
+
+expanded = jsonld.expand(doc, options=options)
+
+print(json.dumps(expanded, indent=2))

docs/examples/document_loaders/caching_modes.py

@@ -0,0 +1,31 @@
+import json
+import tempfile
+from pathlib import Path
+from uuid import uuid4
+
+from requests_cache import CachedSession
+
+from pyld import (
+    RequestsDocumentLoader,
+    SqliteCacheRequestsDocumentLoader,
+    jsonld,
+)
+
+doc = {
+    "@context": {"name": "https://schema.org/name"},
+    "name": "Earth",
+}
+
+no_cache_loader = RequestsDocumentLoader()
+
+memory_cache_loader = RequestsDocumentLoader(
+    session=CachedSession(backend="memory", cache_control=True),
+)
+
+persistent_cache_loader = SqliteCacheRequestsDocumentLoader(
+    sqlite_file_path=Path(tempfile.gettempdir()) / f"pyld-{uuid4()}.sqlite",
+)
+
+result = jsonld.expand(doc, options={"documentLoader": persistent_cache_loader})
+
+print(json.dumps(result, indent=2))

docs/examples/from_rdf_dataset.py

@@ -0,0 +1,15 @@
+import json
+
+from rdflib import Dataset
+
+from pyld import jsonld
+
+nquads = (
+    '<http://dbpedia.org/resource/Earth> '
+    '<http://schema.org/name> "Earth" .\n'
+)
+dataset = Dataset().parse(data=nquads, format="nquads")
+
+doc = jsonld.from_rdf(dataset)
+
+print(json.dumps(doc, indent=2))

docs/examples/inject_context.py

@@ -0,0 +1,16 @@
+import json
+
+from pyld import jsonld
+
+doc = {
+    "name": "Ada Lovelace",
+    "homepage": "https://example.com/ada",
+}
+context = {
+    "name": "http://schema.org/name",
+    "homepage": {"@id": "http://schema.org/url", "@type": "@id"},
+}
+
+expanded = jsonld.expand(doc, {"expandContext": context})
+
+print(json.dumps(expanded, indent=2))

docs/examples/inject_context_compact.py

@@ -0,0 +1,24 @@
+import json
+
+from pyld import jsonld
+
+doc = {
+    "name": "Ada Lovelace",
+    "homepage": "https://example.com/ada",
+}
+source_context = {
+    "name": "http://schema.org/name",
+    "homepage": {"@id": "http://schema.org/url", "@type": "@id"},
+}
+output_context = {
+    "name": "http://schema.org/name",
+    "url": {"@id": "http://schema.org/url", "@type": "@id"},
+}
+
+compacted = jsonld.compact(
+    doc,
+    output_context,
+    {"expandContext": source_context},
+)
+
+print(json.dumps(compacted, indent=2))

docs/examples/legacy_helpers.py

@@ -0,0 +1,20 @@
+import json
+
+from pyld.util import from_legacy_dataset, to_legacy_dataset
+
+legacy_dataset = {
+        'http://example.org': [
+            {
+                'subject': {'type': 'blank node', 'value': '_:s1'},
+                'predicate': {'type': 'IRI', 'value': 'p1'},
+                'object': {'type': 'literal', 'value': 'o1'},
+            }
+        ]
+}
+
+
+rdflib_dataset = from_legacy_dataset(legacy_dataset)
+print(rdflib_dataset.serialize(format="nquads"))
+
+legacy_dataset = to_legacy_dataset(rdflib_dataset)
+print(json.dumps(legacy_dataset, indent=2))

docs/examples/normalize.py

@@ -9,6 +9,15 @@
     },
 }
 
+# Normalize with RDFC10
+normalized = jsonld.normalize(
+    doc,
+    {"format": "application/n-quads"},
+)
+
+print(normalized)
+
+# Normalize with URDNA2015
 normalized = jsonld.normalize(
     doc,
     {"algorithm": "URDNA2015", "format": "application/n-quads"},

docs/examples/normalize_output_map.py

@@ -0,0 +1,19 @@
+import json
+
+from pyld import jsonld
+
+doc = {
+    "@type": "http://schema.org/Person",
+    "http://schema.org/name": "Manu Sporny",
+    "http://schema.org/url": {"@id": "http://manu.sporny.org/"},
+    "http://schema.org/image": {
+        "@id": "http://manu.sporny.org/images/manu.png"
+    },
+}
+
+identifier_map = jsonld.normalize(
+    doc,
+    {"algorithm": "RDFC10", "outputMap": True},
+)
+
+print(json.dumps(identifier_map, indent=2))

docs/examples/open_jsonld_file.py

@@ -0,0 +1,11 @@
+import json
+from pathlib import Path
+
+from pyld import jsonld
+
+path = Path(__file__).resolve().parent / "data" / "person.jsonld"
+doc = json.loads(path.read_text())
+
+expanded = jsonld.expand(doc)
+
+print(json.dumps(expanded, indent=2))

docs/examples/open_jsonld_file_to_rdf.py

@@ -0,0 +1,11 @@
+import json
+from pathlib import Path
+
+from pyld import jsonld
+
+path = Path(__file__).resolve().parent / "data" / "person.jsonld"
+doc = json.loads(path.read_text())
+
+nquads = jsonld.to_rdf(doc, {"format": "application/n-quads"})
+
+print(nquads)

docs/examples/persistent_document_loader_cache.py

@@ -0,0 +1,18 @@
+import json
+import tempfile
+from pathlib import Path
+from uuid import uuid4
+
+from pyld import SqliteCacheRequestsDocumentLoader, jsonld
+
+cache_path = Path(tempfile.gettempdir()) / f"pyld-{uuid4()}.sqlite"
+loader = SqliteCacheRequestsDocumentLoader(sqlite_file_path=cache_path)
+
+doc = {
+    "@context": {"name": "https://schema.org/name"},
+    "name": "Earth",
+}
+
+expanded = jsonld.expand(doc, options={"documentLoader": loader})
+
+print(json.dumps(expanded, indent=2))

docs/examples/rdflib_parse.py

@@ -0,0 +1,16 @@
+import json
+
+from rdflib import Dataset
+
+from pyld import jsonld
+
+nquads = (
+    '<http://dbpedia.org/resource/Earth> '
+    '<http://schema.org/name> "Earth" .\n'
+)
+dataset = Dataset().parse(data=nquads, format="nquads")
+
+doc = jsonld.from_rdf(dataset)
+compacted = jsonld.compact(doc, {"name": "http://schema.org/name"})
+
+print(json.dumps(compacted, indent=2))