Add hurlfmt --out=curl option

[hunter-gatherer8]

Problem to solve

While hurl is very handy to play with unfamiliar APIs and even to store API request collections, it isn't very interoperable with anything else, and in the end you'd often want to "copy" the request to share with someone else or transform it into a code in some programming language to use in an actual project, which is usually very trivial, but tedious, since it's often just re-typing all the QueryStringParams in a slightly different format.

While there are dozens of possible desired output formats, curl CLI can be considered a common denominator, and there are multiple tools to convert curl command into different programming languages, as well as import them into similar tools like Postman/Bruno. So while in a perfect world hurlfmt could serve as a converter between all sorts of formats, really just being able to output a chain of curl commands covers a lot of ground.

Proposal

Add hurlfmt --out=curl option.

hurlfmt already can convert curl to hurl (#316), but, surprisingly, there currently is no option to do it other way around.

If there is a chain of requests, produce several curl commands. If templates are used in request.hurl, I propose to leave variable names verbatim. Everything else (assertions, captures, etc) can be ignored.

It would be really nice, if the converter could be very explicit and include the default hurl headers and such, even if not specified in the source request.hurl file, since this can make a difference in the real world. But would be also useful without it, if it's complicated for some reason.

[fabricereix]

Hi @hunter-gatherer8,
You can take a look at the following discussion #2615.
You can get curl commands when running hurl in verbose mode.
Exporting directly to a Hurl script to curl is not possible due to dynamic values captured between requests.

[jcamiel]

@fabricereix as it's a rather recurring question/issue, maybe we can introduce a --curl option that takes a file and produces a list of curl commands if the Hurl execution file is successful?

$ hurl --curl commands.txt --test *.hurl

curl has a --libcurl command that produces a main.c from the curl command line.

We could use --curl in our integration test (instead of grepping --verbose). Grepping --verbose would still be useful if the test fails for instance.

[jcamiel]

@hunter-gatherer8 if you're OK, we're changing the title of the issue to "Add --curl option"

[hunter-gatherer8]

@jcamiel surely you know better how to approach this, but I formulated it this way consciously, for several reasons:

1. While curl is the most common option, there are infinitely many output formats you could possibly want. What if sometime in the future you decide that it'd be really useful to convert *.hurl to Bruno? Or if there is some other new tool? Surely you wouldn't want to add --bruno flag to hurl. So I think it kinda makes sense to design the API in a way for it to be extendible in the future. Like hurl --convert=curl|json|... instead of --json, --curl, etc. Also, without --convert it's kinda ambiguous what these flags mean. Does --curl specify the transport? Is --json used to force application/json MIME? It's easy to find out, but all of this is a possibility and isn't obvious for a new user.
2. Capturing the curl commands as they are executed is super useful indeed, but I think it's a slightly different use-case, and in fact not always desirable. Sometimes you want to send another person "a request instruction" (so, basically, a curl command) without actually executing it. It may be a paid API, or a destructive instruction that you don't want to execute yourself, just communicate to another person how to do it.
3. You've said it yourself, that this will work only if hurl execution was successful, but this doesn't always have to be the case. Maybe you just want to send another person a chain of requests to show them that their service fails and to ask if you are doing something wrong?

So, in all honesty, all of this is tangential to hurl primary purpose as a testing tool, and more has to do with the fact, that *.hurl files are way more pleasant to write than actual curl commands, while your peers are not necessarily familiar with hurl and being able to convert hurl files to something else allows you to worry less about what they actually use, and keep all your writings in hurl anyway.

While I somehow didn't manage to find #2615 on my own, I already acknowledged in the description, that handling templating/dynamic variables in curl is obviously impossible, and I think it's totally OK to translate them verbatim (maybe replacing {{ myVar }} with $myVar, when converting to a curl command, since it would be a more direct translation in bash). It's supposed to be just an utility to save you from having to manually do all these small edits to a format, so no big deal if it doesn't actually produce working commands. On the other hand, if one day you decide that you need one more output format in hurlfmt --out=..., this new format may have a translation for some hurl options that curl doesn't.

All of this being said, surely just being able to capture curl requests without grepping --verbose also would be a big usability improvement and may be useful for other use-cases.

[jcamiel]

Thanks @hunter-gatherer8 thanks for the feedback!

Regarding conversion, we've introduced, since the first versions of Hurl, the possibility to convert a Hurl file to a JSON representation (hurlfmt --out json my-file.hurl). This way, one can export a Hurl file to any tool of choice (whether StepCI Bruno etc...). We've purposely done this because we do not want to support every possible format, the burden is not on the maintainers to support a particular format. curl is a particular case because we're explicitely using curl as the HTTP engine, and curl is the lingua franca of HTTP exchanges actually.

I still like to have an option --curl to have an explicit command instead of grepping so I will make another issue!