Skip to content

ref variables cause multiplied output #2453

@pabigot

Description

@pabigot

Describe the bug
I'm trying to copy a field present in a list of objects to each object in a list that is a member of the object. A method that uses ref variable operators produces multiple copies, possibly the product of the two list lengths. An alternative technique that replaces the inner ref variable operator with a relative form assignment only replicates as many documents as there are items in the outer list. This suggests a problem with ref variables introducing a loop, which does not appear to be documented behavior. (A loop could be expected for a relative form assignment, though the number of replications doesn't seem to match expectations for that case.)

Version of yq: 4.47.1
Operating system: linux
Installed via: binary release

Input Yaml
Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less)
data.yml:

%YAML 1.2
---
# Comment
top:
  - n: d1
    f: f1
    h:
      # Set for 1
      - hn: h1a
      - hn: h1b
      - hn: h1c
  - n: d2
    f: f2
    h:
      # Set for 2
      - hn: h2a
      - hn: h2b
      - hn: h2c

Command
The command you ran:

yq '.top[] ref $p | $p.h[] ref $h | $h.hf = $p.f' data.yml

Actual behavior

Six copies of the document:

%YAML 1.2
---
# Comment
top:
  - n: d1
    f: f1
    h:
      # Set for 1
      - hn: h1a
        hf: f1
      - hn: h1b
        hf: f1
      - hn: h1c
        hf: f1
  - n: d2
    f: f2
    h:
      # Set for 2
      - hn: h2a
        hf: f2
      - hn: h2b
        hf: f2
      - hn: h2c
        hf: f2
... Four copies elided ...
%YAML 1.2
---
# Comment
top:
  - n: d1
    f: f1
    h:
      # Set for 1
      - hn: h1a
        hf: f1
      - hn: h1b
        hf: f1
      - hn: h1c
        hf: f1
  - n: d2
    f: f2
    h:
      # Set for 2
      - hn: h2a
        hf: f2
      - hn: h2b
        hf: f2
      - hn: h2c
        hf: f2

Expected behavior

One copy:

%YAML 1.2
---
# Comment
top:
  - n: d1
    f: f1
    h:
      # Set for 1
      - hn: h1a
        hf: f1
      - hn: h1b
        hf: f1
      - hn: h1c
        hf: f1
  - n: d2
    f: f2
    h:
      # Set for 2
      - hn: h2a
        hf: f2
      - hn: h2b
        hf: f2
      - hn: h2c
        hf: f2

Additional context
Add any other context about the problem here.

If the command run is:

yq '.top[] ref $t | $t |= .h[] |= .f=$t.f' data.yaml

then two copies are produced. Note that the gratuitous | $t stage is required to avoid this diagnostic:

mira[223]$ yq '.top[] ref $t |= .h[] |= .f=$t.f' /tmp/data.yml 
Error: must use variable with a pipe, e.g. `exp as $x | ...`

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions