Idea for safely serializing JSON data

[crowecawcaw]

Problem

type: TEXT embedded files are the only main way Deadline Cloud integrations pass structured config (init-data, run-data) to adaptors. Template authors write YAML as raw text with {{Param.*}} placeholders, and the runtime does raw text substitution after serialization. If the substituted value contains characters that break the YAML quoting chosen at serialization time, the worker gets a parse error.

Example

embeddedFiles:
  - name: initData
    filename: init-data.yaml
    type: TEXT
    data: |
      scene_file: '{{Param.Cinema4DFile}}'

{{ requires YAML quoting, so the serializer single-quotes the value. When the path /Users/artist's work/scene.c4d is substituted, the result is:

scene_file: '/Users/artist's work/scene.c4d'

Broken YAML — the apostrophe terminates the string early.

Affected templates

Many Deadline Cloud integrations create YAML files as strings like this. Taking C4D as the sample, the fix required a custom _yaml_utils.py module that splits values into "safe for yaml_dump" vs "must be unquoted plain scalars". Every plugin would need to replicate this.

Why repr_json() from RFC 6 isn't sufficient

repr_json() can wrap individual values safely — JSON double-quoted strings are valid YAML:

data: |
  scene_file: {{repr_json(Param.Cinema4DFile)}}
  output_path: {{repr_json(Param.OutputPath)}}

This handles apostrophes, double quotes, backslashes, colon-space, #, empty strings, etc. correctly. It's a viable near-term mitigation.

But it has two drawbacks:

1. Opt-in per value — every submitter author must remember to wrap every path value. Miss one and you have the same silent bug. It doesn't widen the pit of success.
2. Can only serialize values, not structure — the EXPR type system has no dict/map type, so you can't construct the entire init-data as a single object and serialize it. You're still hand-writing the YAML structure as text and wrapping individual leaf values. The keys, indentation, and colons are all unprotected raw text.

Proposed solution: type: JSON embedded files

Add a new embedded file variants where data is a structured object rather than raw text. The runtime resolves {{Param.*}} at the value level, then serializes the result to JSON when writing the file.

Schema

<EmbeddedFileStructured>:
  name: <Identifier>
  type: "JSON"
  filename: <Filename>           # @optional
  data: <object>                 # @fmtstring[host] — format strings resolved per-value

(JSON is valid YAML, so it will solve the YAML use case as well.)

Example

embeddedFiles:
  - name: initData
    filename: init-data.yaml
    type: JSON
    data:
      scene_file: "{{Param.Cinema4DFile}}"
      take: Main
      output_path: "{{Param.OutputPath}}"
      activate_error_checking: "{{Param.ActivateErrorChecking}}"

The serializer handles quoting correctly because substitution happens on the parsed data structure, not inside a serialized text blob.

Why this over repr_json()

• Correct by default — no per-value wrapping to forget
• Handles the full document — structure and values are both safe, not just leaf scalars
• Fits existing schema — <EmbeddedFile> already has a type discriminator

References

• Cinema 4D bug: Bug: Path names with single quotes with takes do not work aws-deadline/deadline-cloud-for-cinema-4d#397
• Cinema 4D fix: fix: fix yaml delimeter in paths aws-deadline/deadline-cloud-for-cinema-4d#408
• <EmbeddedFileText> spec: §6.1
• RFC 6 (repr_json()): RFC 0006