Using jsonpath filter on object variable fails

[luke-pp]

In my company's GraphQL APIs, we can get back some deeply nested json responses with a lot of data, and often we want to perform many different assertions against various parts of the response - ideally we'd like to organize these assertions by capturing one part of the response, and then performing assertions on each captured subsection separately. Here's a simplified example using the star wars API to show what I mean:

POST https://swapi-graphql.netlify.app/.netlify/functions/index
```graphql
query ExampleQuery($first: Int, $after: String) {
  allPlanets(first: $first, after: $after) {
    edges {
      cursor
      node {
        name 
        diameter
        population
        filmConnection {
          films {
            title
          }
        }
      }
    }
  }
}
variables {
	"first": 10
}
```

HTTP 200
[Captures]
bespin: jsonpath "$.data.allPlanets.edges[?(@.node.name == 'Bespin')]" nth 0
tatooine: jsonpath "$.data.allPlanets.edges[0]"

[Asserts]
variable "bespin" jsonpath "$.node.diameter" == 118000
variable "bespin" jsonpath "$.node.population" == 6000000

variable "tatooine" jsonpath "$.node.diameter" == 10465
variable "tatooine" jsonpath "$.node.population" == 200000

In this example, we want to do a couple of assertions about each planet, so we first capture the json objects for Bespin and Tatooine, and then perform assertions against these objects using the jsonpath filter. We could get the same behaviour by just using jsonpath assertions along the lines of

jsonpath "$.data.allPlanets.edges[?(@.node.name == 'Bespin')].node.diamter" nth 0 == 118000

but you can imagine it gets a bit more frustrating in a real workflow outside of the star wars API example.

What is the current bug behavior?

I get errors in the console:

error: Filter error
  --> .\hurl-issue.hurl:31:19
   |
   | POST https://swapi-graphql.netlify.app/.netlify/functions/index
   | ...
31 | variable "bespin" jsonpath "$.node.diameter" == 118000
   |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^ invalid filter input: object
   |

error: Filter error
  --> .\hurl-issue.hurl:32:19
   |
   | POST https://swapi-graphql.netlify.app/.netlify/functions/index
   | ...
32 | variable "bespin" jsonpath "$.node.population" == 6000000
   |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ invalid filter input: object
   |

error: Filter error
  --> .\hurl-issue.hurl:34:21
   |
   | POST https://swapi-graphql.netlify.app/.netlify/functions/index
   | ...
34 | variable "tatooine" jsonpath "$.node.diameter" == 10465
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^ invalid filter input: object
   |

error: Filter error
  --> .\hurl-issue.hurl:35:21
   |
   | POST https://swapi-graphql.netlify.app/.netlify/functions/index
   | ...
35 | variable "tatooine" jsonpath "$.node.population" == 200000
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ invalid filter input: object
   |

Steps to reproduce

Run the hurl file above

What is the expected correct behavior?

Ideally, the jsonpath should work on the variable object in the same way it works on a full response body/string such that the assertions in the example file pass.

Execution context

I don't think it's particularly relevant to this issue, but

• Hurl Version (hurl --version):
  hurl 4.3.0 (Windows) libcurl/8.4.0-DEV Schannel zlib/1.3 nghttp2/1.58.0
  Features (libcurl): alt-svc AsynchDNS HSTS HTTP2 IPv6 Largefile libz NTLM SPNEGO SSL SSPI Unicode UnixSockets
  Features (built-in): brotli

Possible fixes

I believe the issue is here:

hurl/packages/hurl/src/runner/filter/jsonpath.rs

Line 33 in 9cfc5b9

match value {

The jsonpath filter runner currently only accepts string values, and assumes it needs to parse the string into a json value before evaluating the json path. In our example, the first jsonpath evaluates to fetch the full planet object and capture it to a variable as an object which works fine. But then when we try to use the jsonpath filter on the object to perform assertions against specific items in it, it fails because it is an object rather than a string. I think if we had a second match clause that let us pass through objects (possibly also arrays?) it would help eg:

match value {
    Value::String(text) => {
        let json = match serde_json::from_str(text) {
            Err(_) => {
                return Err(RunnerError::new(
                    source_info,
                    RunnerErrorKind::QueryInvalidJson,
                    false,
                ));
            }
            Ok(v) => v,
        };
        eval_jsonpath_json(&json, expr, variables)
    }
    Value::Object => {
        eval_jsonpath_json(&value, expr, variables)
    }
    v => {
        let kind = RunnerErrorKind::FilterInvalidInput(v._type());
        Err(RunnerError::new(source_info, kind, assert))
    }
}

[jcamiel]

Hi @luke-pp

It sounds reasonable, don't know if it's easy to implement @fabricereix but the features seems "natural" and user can expect it to work.

[fabricereix]

Hello,
Yes, we could treat the Hurl Object as a JSON Object implicitly and apply jsonpath on it.
We will have to convert each of its value to a JSON entity. We could also make it fail if the conversion does not make sense.

[jcamiel]

Just changed it from "bug" to "enhancement"