v0.34.0

k6 v0.34.0 is here! 🎉 It introduces the already announced k6/execution API and includes some more enhancements and a bunch of minor bug fixes.

New k6 JavaScript API - k6/execution

The k6/execution module allows the fetching of information about the current VU, instance (mostly relevant in distributed/cloud) or scenario execution state through a series of exported properties.

How to use

Some of the previously suggested solutions with the globally available __VU and __ITER values, such as for getting a unique object per iteration from an array or SharedArray, can be done by using the scenario.iterationInTest property, which is guaranteed to be unique across VUs, even for distributed or cloud tests. For example:

import exec from "k6/execution";
import { SharedArray } from "k6/data";

const data = new SharedArray("my dataset", function(){
  return JSON.parse(open('my-large-dataset.json'));
})

export const options = {
  scenarios :{
    "use-all-the-data": {
      executor: "shared-iterations",
      vus: 100,
      iterations: data.length,
      maxDuration: "1h"
    }
  }
}

export default function() {
  // this is unique even in the cloud
  var item = data[exec.scenario.iterationInTest];
  http.post("https://httpbin.test.k6.io/anything?endpoint=amazing", item)
}

You can read the full documentation here.

Enhancements and UX improvements

• Warn Windows users on importing dependencies or opening files as absolute paths (#2078).
• Pass setup data object into handleSummary callback (#2103). Thanks, @SamuelJohnson01997!

Breaking changes

• The deprecated outputs Datadog and Kafka have been removed (#2081).

Bugs fixed!

• Use the POST HTTP request method instead of GET for pushing logs to Loki (#2100).
• Encode the blacklistIPs option using the CIDR notation in JSON (#2083).
• ext.loadimpact option has the same precedence as the script configuration during the consolidation process (#2099).
• The WebSocket connection used for tailing logs from the k6 Cloud is reestablished in the case of an unexpected error (#2090).

Internals

• A simpler and clearer API has been added as an alternative to common.Bind, which also gives JS modules and extensions easy access to some useful internal objects and runtime information (#2108). This API is not yet stable, it's very likely to change more in future k6 versions.
• Speeding ups TC39 tests using a pool of Babel compilers (#1839).
• Goja and some internal dependencies have been updated adding the native support for Arrow functions, Destructuring, Default arguments and Computed properties features. For the same reason, the relative Babel's plugins supporting those features are not required anymore so they have been disabled (#2109, #2092).

v0.33.0

k6 v0.33.0 is here! 🎉 It's a small release that includes a bunch of minor bugfixes and enhancements, but is also laying the groundwork for some major new features like the upcoming k6/execution API in k6 v0.34.0.

Acquired by Grafana Labs

Load Impact, the company behind k6, was acquired by Grafana Labs! 🎉 Nothing changes regarding the k6 development for now, and any changes in the future will only be in the direction of accelerating our existing roadmap and plans, as well as better integration between k6 and the awesome Grafana tools. For more details, see the official Grafana press release.

Enhancements and UX improvements

• The --verbose help message and the statsd warning message were improved (#2005). Thanks, @vishalkuo!
• The noColor k6 option and the current UI state are now propagated to the handleSummary() function. The state object has the isStdErrTTY, isStdOutTTY and testRunDurationMs keys (#1975).
• The error message when an HTTP request times out was improved (previously it was context deadline exceeded) and it now has an error_code value of 1050 (#2008). Thanks, @vishalkuo!
• Script errors will no longer have the confusing GoError prefix in their error messages (#1775).
• All custom metric objects now have a name property (#2058 and #2076). Thanks, @olimpias and @david-gourde!
• Top-level JS arrays will now be properly encoded when sent in the body of a application/x-www-form-urlencoded request (#2060). Thanks, @noelzubin!

Bugs fixed!

• The minIterationDuration option was uninterruptible and could delay the stopping of a scenario even after gracefulStop had expired. (#2035).
• The error_code detection for HTTP/2, x509 and TLS (and potentially others) was unreliable (#2025).
• k6 used to panic when responseType was binary, but there was no response body actually returned, e.g. when there was an HTTP error (#2041).
• The throw option was not respected when there was an invalid URL (#2045). Thanks, @gchaincl!
• k6 would return an exit code of 103 instead of 107 for script errors when initializing non-service VUs (#2046).
• Deleted library versions from cdnjs could previously cause a panic (#2047).
• The correct error message for missing files was not shown when the filename contained spaces (#1973).
• The regular expressions for the github and cdnjs "magic" loaders were slightly wrong (#2066).
• A potential (harmless) data race could have been caused by an unintentional copying of a data struct (#2067).
• The segmentation of small ramping-arrival-rate scenarios was not optimal (#1863).

Internals

• The default end-of-test summary is now completely generated by the same k6-summary JS code that is hosted on jslib.k6.io (#1975). That PR also improved the k6 TTY detection and removed a few Go dependencies and code hacks, though it also caused us to bump the minimum required Go version for compiling k6 to Go 1.16 (because of its usage of go:embed).
• Arrival-rate executors will no longer create a new goroutine for every new iteration (#1957 and #2038).
• We have enabled GitHub's CodeQL checks for the Go parts of the repo (#1961). Thanks, @jfcg!
• We have added the necessary k6 core changes for providing execution information to scripts #1863! This was the groundwork for the extended replacement of the __VU and __ITER execution context variables we plan to introduce. The new API will be able to return other information as well, for example which scenario the current iteration is in, what's the number of the current VU/iteration globally across all k6 instances, or in the current scenario, etc. These APIs are still not available to JS scripts, but we plan to expose them via the k6/x/execution xk6 extension for now and iterate on them in the following weeks, releasing a stable version in k6 v0.34.0.

Breaking changes

• The k6 cloud exit code for a failed cloud test was changed from 99 to 97 (#2046).
• The default value of K6_STATSD_TAG_BLOCKLIST and K6_DATADOG_TAG_BLACKLIST is now vu,iter,url (#2063).
• The __ITER execution context variable is no longer set in setup() and teardown() due to #1863. This might be better classified as removing a previously undefined behavior instead of a breaking change, but it's still worth mentioning.

v0.32.0

k6 v0.32.0 is here! 🎉 It's a smaller release, featuring mostly chores that we never got to or that needed to be done in this release cycle, but also includes a significant performance improvement for arrival-rate executors and some breaking changes.

Move out of Bintray

Bintray has stopped servicing users on 1st of May, which meant that we needed to move the deb and rpm repositories out of there before then. Please follow the new installation instructions and the "Note about Bintray" to find out how to remove the old repository if you have used them.

Notable changes

Move all outputs to new Output interface introduced in v0.31.0 and other related changes

We started on this in the previous v0.31.0 release and now all internal outputs implement the new Output interface we provided for xk6 output extensions. Additionally one of the old built-in outputs is being deprecated and one renamed:

1. The kafka output has been somewhat neglected since it was added 3 years ago. Additionally it brings a lot of complexity and isn't well understood by anyone on the team. Given that output extensions are possible since the last version, the Kafka output has been moved to one. The built-in output will continue to work for a few more k6 versions, emitting a deprecation warning when used, so that everyone has time to transition to the extension. All future improvements will happen only in the extension, the built-in output is frozen until it's dropped.
2. We are also deprecating/renaming the datadog output. It should've probably always been just a configuration of the statsd output and now in k6 v0.32.0, it is going to be just that. We've added a new K6_STATSD_ENABLE_TAGS option to the statsd output, which, when enabled (it's false by default), will send metric tags the same way the datadog output did before. That is, instead of using the datadog output, you should use the statsd one with K6_STATSD_ENABLE_TAGS=true. Additionally, the new K6_STATSD_TAG_BLOCKLIST option can be used to not send tags that the user doesn't want to, similar to the old K6_DATADOG_TAG_BLACKLIST option.
   This makes it cleaner to also emit metrics to other services that accept the same data and tag formats, such as New Relic, Amazon Cloudwatch, and statsd >v0.9.0. The old datadog output will still work for a few k6 versions, emitting a warning to switch to statsd when used.

Apart from a message about them being deprecated, nothing should be changed from the actual refactoring yet, but we advise users to use the proposed alternatives starting with this release.

json output emits thresholds defined on the metrics (#1886)

Previous to this change thresholds were not included in the json output. Now the Metric JSON object will get its thresholds field properly populated.

Thanks to @codebien for this contribution!

cloud output has an option to abort the test if aborted from the cloud (#1965)

In v0.26.0 we made the cloud output stop emitting metrics if it gets a particular error from the backend, as that meant that the test was aborted in the cloud. Now we added K6_CLOUD_ABORT_ON_ERROR to be able to say that it should not only stop emitting metrics, but also stop the execution of the local k6 test run. The default configuration is false, so it is backwards compatible. This also works when the test is aborted by the user or if cloud execution limits are reached, which would also lead to the test being aborted.

Full stack traces for init context and setup/teardown exceptions (#1971)

For a long time, if there was an exception in either the init context or in the setup() or teardown() invocations, the stack trace would be just the last line, making it really hard to debug issues there. Now there is a full stack trace and, as such errors will result in aborted k6 execution, k6 run will also exit with exit code 107, signalling a script error.

Considerable performance improvements for arrival rate executors (#1955)

Due to a wrong re-usage for a particular internal data structure, the arrival rate executors were having a much worse performance than expected. With this release they should be a lot more performant, especially with large numbers of VUs.

Updating the majority of our dependencies and dropping some

The list is too long and we have been postponing updating for quite some time now, but we have finally updated all our dependencies that we don't want to drop. This could lead to some stability problems, which is why it was done early on in the cycle. While the team has not found any regressions, given all the updates we could have missed something so please open a issue if you find anything.

Some notable updates:

• goja, the JS engine we use got support for let/const which allowed us to disable a Babel plugin. Previous to this if you had a particularly long script sometimes Babel took upwards of 30 minutes to transpile. Now even our worst contender that previously took 51 minutes is transpiled in less than a minute 🎉. Also globalThis is now available.
• updating the gRPC libraries fixed bug (#1928) and probably others. (#1937)

ArrayBuffer is now supported in all JS APIs dealing with binary data, including in WebSocket messages (#1841)

Besides the minor breaking changes (see the "Breaking changes" section below), it's now possible to send binary WS messages with the socket.sendBinary() function and to receive binary messages with the binaryMessage event handler:

const binFile = open('./file.pdf', 'b');

export default function () {
  ws.connect('http://wshost/', function(socket) {
    socket.on('open', function() {
      socket.sendBinary(binFile);
    });

    socket.on('binaryMessage', function(msg) {
      // msg is an ArrayBuffer, so we can wrap it in a typed array directly.
      new Uint8Array(msg);
    });
  });
}

Official arm64 releases for macOS and Linux (#2000)

We will now publish binary arm64 releases for macOS and Linux for new k6 releases. Support for these new architectures should be stable, given Go's cross-platform compatibility, but please report any issues you experience.

Other enhancements and UX improvements

• Options: k6 will now warn you on unrecognised configuration JS options in most cases instead of just silently ignoring them. (#1919)
• error_code: the tag error_code should now be set more accurately in some cases. (#1952)
• TLS: dropped support for SSLv3 encryption. This was dropped by Go, but now we no longer consider ssl3.0 a valid value for the tlsVersion k6 option. Thanks @codebien! (#1897)

Bugs fixed!

• Arrival-rate executors could in some cases report twice as many used VUs than what was actually true. (#1954 fixed by #1955)
• In cases of an error while reading the response body, the newly added responseCallback in v0.31.0 would be evaluated with the returned status code, while the reported one would be 0, as the response errored out and k6 does not return incomplete responses. Now responseCallback will also receive a 0 status. (#1962)
• Fix Kafka output not being usable with the InfluxDB format after v0.31.0 changes. (#1914)
• Error out with a user friendly message if ramping-vus executor would've not run a single iteration instead of just doing nothing. (#1942)

Internals

• JS: Added a way for JS modules to have per VU initialization and object. This can also be used by xk6 modules by implementing go.k6.io/k6/js/modules#HasModuleInstancePerVU. (Part of #1911)

Breaking changes

Support for ArrayBuffer in all k6 JS APIs (#1841)

Continuing from k6 v0.31.0, we're finalizing the transition to ArrayBuffer values for working with binary data. This release introduces some changes that might break scripts that relied on the previous array of integers or string result types, returned by some of our JS APIs. Specifically these cases now return ArrayBuffer instead: open(..., 'b'), HTTP response bodies for requests that specified responseType: 'binary' (including when http.batch() is used), crypto.randomBytes(), hasher.digest('binary') and encoding.b64decode().
The previous default behavior of returning string from encoding.b64decode() can be replicated with a new optional format argument and a value of "s": encoding.b64decode("5bCP6aO85by-Li4=", "url", "s").
Most of these shouldn't cause issues if the script is simply passing the values to another k6 API (e.g. opening a file as binary and passing it to http.post()), b...

v0.31.1

k6 v0.31.1 is a patch release with a single bugfix.

The bugfix is about the cloud output and the new http_req_failed metric in k6 v0.31.0. Due to additional state being used for its transport to the k6 cloud, and a misunderstanding of what a functional call from a library dependency does, the http_req_failed values were always set to 1. This did not affect any other output or the end of test summary. (#1908)

v0.31.0

k6 v0.31.0 is here! 🎉 It's a smaller release with some significant performance improvements, a new http_req_failed metric and changes to the output subsystem that enable output extensions with xk6!

New features

Output cleanup and extensions (#1874)

The state of k6's output packages has been a development pain point for a long time, which made it difficult to add new outputs in a consistent way. In the refactor done in v0.31.0, this has been mostly addressed and outputs now implement a simpler and cleaner Output interface.

In addition to this, it is now possible to implement custom k6 output extensions in Go with xk6! This is very useful if you use a system that's currently not supported by the built-in outputs, or need some custom handling of the metrics k6 produces.

Writing output extensions is done very similarly to how JS module extensions are currently written, though instead of calling js/modules.Register(), you should implement the new Output interface and call output.RegisterExtension() with your constructor.

We are working on the proper documentation for this, as well as the overdue xk6 documentation about JS extensions, so keep a lookout for those on k6.io/docs.

Marking requests as failed (#1856)

It's now possible to declare expected HTTP response statuses for either the entire test or for individual HTTP requests, and k6 will emit a new http_req_failed metric as well as tag HTTP metrics with expected_response: <bool>. By default, k6 will now fail requests that return HTTP 4xx/5xx response codes.

For example:

import http from 'k6/http';

// Set expected statuses globally for all requests.
http.setResponseCallback(http.expectedStatuses({min: 200, max: 399}, 418));

export default function () {
  // This request will be marked as failed.
  http.get('https://httpbin.test.k6.io/status/400');
  // This request will be considered as "passed" because of the responseCallback override.
  http.get('https://httpbin.test.k6.io/status/400', { responseCallback: http.expectedStatuses(400) });
}

Running this script will produce a summary like:

http_req_duration..............: avg=204.57ms min=203.31ms med=204.57ms max=205.82ms p(90)=205.57ms p(95)=205.7ms
  { expected_response:true }...: avg=203.31ms min=203.31ms med=203.31ms max=203.31ms p(90)=203.31ms p(95)=203.31ms
http_req_failed................: 50.00% ✓ 1   ✗ 1

Note the new http_req_duration sub-metric for expected responses only, and the new http_req_failed Rate metric. This new metric and metric tag have many potential use cases, and one of the most important ones is the ability to set better thresholds. For example:

• 'http_req_failed': ['rate<0.1'], i.e. fail the test if more than 10% of requests fail.
• 'http_req_duration{expected_response:true}': ['p(95)<300', 'p(99.9)<500'] - fail the test if the the 95th percentile HTTP request duration is above 300ms or the 99.9th percentile is above 500ms; specifying expected_response:true here may be important, because a lot of times failed requests may return more quickly than normal ones, thus skewing the results and wrongly satisfying the threshold.

If the response callback is not specified, the default expected statuses will be {min: 200, max: 399}. The previous behavior of not emitting anything can be achieved by setting the callback to null, i.e. http.setResponseCallback(null). Additionally, the expected_response tag can be disabled by removing it from the default list of system tags, e.g. k6 run --system-tags 'proto,subproto,status,method,url,name,group,check,error,error_code,tls_version,scenario,service'.

The http.setResponseCallback() is planned to allow arbitrary JS functions to process responses in the future, but for now only the http.expectedStatuses() callback is supported.

Other enhancements and UX improvements

• JS: Because of the awesome improvements to goja, the JS runtime k6 uses, it's no longer necessary for k6 to load core.js to polyfill missing JS features when using the default --compatibility-mode=extended. So in v0.31.0 core.js has been dropped entirely, yielding some significant CPU and memory usage improvements. The actual numbers will depend on the use case, but for simple tests users can expect a memory drop of about 2MB per VU (from ~2.7MB to ~600KB), and a slight CPU decrease of about 5-10%. For more complex tests with a lot of JS code this benefit won't be as pronounced. Another benefit of this change is that initializing VUs and starting a test is substantially faster than before! (#1824)
• JS: Also because of goja improvements, some unused Babel plugins were disabled which should have minor performance benefits as well. (#1822)
• JS: Expanded ArrayBuffer support in most internal modules, so now you can pass ArrayBuffer to http.file(), in k6/encoding and k6/crypto functions. This makes working with binary files more efficient as it doesn't require string translations. In upcoming versions we plan to expand this to the WebSocket module, as well as make some potentially breaking changes for APIs that currently return an array of integers or string (see the details in the Breaking Changes announcement below). (#1800)
• The Docker image base was updated to Alpine 3.13. Thanks @andriisoldatenko! (#1821)
• The Debian package now includes ca-certificates as a dependency. Thanks @Bablzz! (#1854)

Bugs fixed!

• Execution: Aborting a test during VU initialization (e.g. with ^C) will now properly propagate to any used outputs. (#1869)
• Execution: A race condition between the Engine and the outputs' finalization code was fixed, ensuring that all metrics are properly emitted before exiting. (#1869)
• Execution: Another race condition in the Engine was fixed, which may have resulted in the end-of-test summary missing some of the last test metric data. (#1888)
• Cloud: the test name is now properly validated and will raise an error if not set via the ext.loadimpact.name JS option or config, or the K6_CLOUD_NAME environment variable. (#1870)
• JS: Babel is now also run on compilation errors, which improves support of some obscure language features. (#1861)
• JS: SharedArray introduced in v0.30.0 can now be iterated with forEach. (#1848)

Internals

• JS: SharedArray was rewritten using goja.DynamicArray making it more performant and easier to reason about. (#1848)
• JS: Some TC39 tests for unsupported features were disabled, improving the runtime of the test suite. (#1816)
• CI: Some more tests were enabled on Windows. (#1855)

Breaking changes

• JS: While we don't expect the core.js removal and Babel changes to impact the vast majority of users, those were substantial changes in how k6 interprets JS and a minority of users might experience issues with their tests. Please report any unexpected JavaScript errors by creating a GitHub issue. In particular Promise is now undefined, and some unused Babel plugins like transform-es2015-for-of and transform-regenerator were also removed. This means that some workarounds like the ones mentioned here and here also won't work as is and will need additional polyfills and plugins to work properly.

Planned future breaking changes

The following are not breaking changes in this release, but we'd like to announce them so users can prepare for them in upcoming releases (likely k6 v0.32.0).

• JS: The ArrayBuffer changes in this release are backwards compatible and shouldn't cause any issues, but in v0.32.0 some JS APIs that currently return an array of integers or string for binary data will return ArrayBuffer instead. This is the case for open() when used with the 'b' argument, response bodies for requests that specify responseType: 'binary', crypto.randomBytes(), hasher.digest('binary'), and encoding.b64decode(). Response.json() and Response.html() will also probably stop working when used with requests that specify responseType: 'binary'. These changes shouldn't be a problem for most users that were simply using these values to pass them to other internal modules (e.g. opening a binary file and passing it to http.post()), but if the scripts modified the binary data or depended on the current array of integers or string values they will need to be adapted to use [typed arrays](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_...

v0.30.0

k6 v0.30.0 is here! 🎉 It was a bit of a slow after-holiday release, but it still packs a few major new features and improvements that users have been requesting for a long time!

New features

Share memory between VUs using read-only arrays (#1739)

k6 has long had an issue with the handling of big data files with test fixtures. For example, if you have a huge users.json file with test users for your application:

[
  {"username": "user1", "password": "password1", "other": "some-long-data-....-1"},
  {"username": "user2", "password": "password2", "other": "some-long-data-....-2"},
  // ... ~1 million more users or more... :D
  {"username": "user999999", "password": "password999999", "other": "some-long-data-....-999999"}
]

If you just use JSON.parse(open('users.json')) in your script, then every VU will have a copy of the whole huge data set. Every VU in k6 is a separate JavaScript runtime, so there wasn't a thread-safe way to share data between them. Until now, that is!

We've added a new built-in SharedArray object in the new k6/data module that allows VUs to share read-only data:

import { SharedArray } from 'k6/data';
import { sleep } from 'k6';
import http from 'k6/http';

let users = new SharedArray('someName', function () {
    // This function will be called only once, in the first init context
    // execution. Every other VU will just get a memory-safe read-only reference
    // to the already loaded data.
    console.log('Loading users.json, this happens only once...');
    // You are not restricted to JSON, you can do anything - parse a CSV or XML
    // file, generate random data, etc. - as long as you return an array.
    return JSON.parse(open('users.json'));
});

export let options = { vus: 10, duration: '30s' };
export default function () {
    let randomUserID = Math.floor(Math.random() * users.length);
    let user = users[randomUserID]; // alternatively, we can also use __VU and/or __ITER
    console.log(`VU ${__VU} is running iteration ${__ITER} with user ${user.username}...`);
    http.post('https://httpbin.test.k6.io/post', JSON.stringify(user));
    sleep(Math.random() * 2); // or, better yet, use arrival-rate
}

Notice how Loading users.json is logged only once, but each VU uses the users variable like a normal JS array. The data is read only once and we have just a single copy of the huge array in memory! Behind the scenes, k6 uses a JS Proxy to transparently copy only the row each VU requests in users[randomUserID] to it. This on-demand copying is a bit inefficient, but it's still leagues better than having a copy of the huge array in every VU! And you can avoid the copying in every iteration by pinning the data used by every VU and having let user = users[randomUserID] in the init context!

And yes, you can have multiple SharedArray objects in the same script, just make sure to give them unique names - this is what someName in the script above was for. Because VUs are independent JS runtimes, we need some way to differentiate between the different shared memory objects, so we require them to have unique names. These names are also the IDs that any xk6 extensions would need to use to access them.

This works both locally and in the cloud. We advise everyone who deals with large data files to wrap them in a SharedArray and give this new feature a try. The required script changes should be minimal, while the memory usage should be significantly lower. Hopefully, we can finally consider one of the biggest blockers k6 users have had for a long time solved! 🎉

Support a handleSummary() callback at the end of the test (#1768)

You can now export a function called handleSummary() and k6 will call it at the end of the test run, after even teardown(). handleSummary() will be called with a JS object containing the same information that is used to generate the end-of-test summary and --summary-export, and allows users to completely customize how the end-of-test summary looks like.

Besides customizing the end-of-test CLI summary (if handleSummary() is exported, k6 will not print the default), you can also transform the summary data to various machine or human-readable formats and save it to files. This allows the creation of JS helper functions that generate JSON, CSV, XML (JUnit/xUnit/etc.), HTML, etc. files from the summary data. Even binary formats like PDF are not out of reach, potentially, with an appropriate JS library that works in k6! You can also send the generated reports to a remote server by making an HTTP request with them (or using any of the other protocols k6 already supports)! Here's a simple example:

import http from 'k6/http';
import k6example from 'https://raw.githubusercontent.com/loadimpact/k6/master/samples/thresholds_readme_example.js';
export default k6example; // use some predefined example to generate some data
export const options = { vus: 5, iterations: 10 };

// These are still very much WIP and untested, but you can use them as is or write your own!
import { jUnit, textSummary } from 'https://jslib.k6.io/k6-summary/0.0.1/index.js';

export function handleSummary(data) {
    console.log('Preparing the end-of-test summary...');

    // Send the results to some remote server or trigger a hook
    let resp = http.post('https://httpbin.test.k6.io/anything', JSON.stringify(data));
    if (resp.status != 200) {
        console.error('Could not send summary, got status ' + resp.status);
    }

    return {
        'stdout': textSummary(data, { indent: ' ', enableColors: true}), // Show the text summary to stdout...
        'junit.xml': jUnit(data), // but also transform it and save it as a JUnit XML...
        'summary.json': JSON.stringify(data), // and a JSON with all the details...
        // And any other JS transformation of the data you can think of,
        // you can write your own JS helpers to transform the summary data however you like!
    }
}

k6 expects handleSummary() to return a {key1: value1, key2: value2, ...} map. The values can be string or ArrayBuffer and represent the generated summary report contents. The keys should be strings and determine where the contents will be displayed or saved: stdout for standard output, stderr for standard error, or a path to a file on the system (which will be overwritten).

The format of the data parameter is similar but not identical to the data format of --summary-export. The format of --summary-export remains unchanged, for backwards compatibility, but the data format for this new k6 feature was made more extensible and had some of the ambiguities and issues from the previous format fixed. We can't cover the new format in the release notes, though you can easily see what it contains by using return { 'stdout': JSON.stringify(data)}; in handleSummary()! 😄

This feature is only available for local k6 run tests for now, though we plan to support k6 cloud tests eventually. And, as mentioned in the snippet above, the JS helper functions that transform the summary in various formats are far from final, so keep an eye on jslib.k6.io for updates. Or, better yet, submit PRs with improvements and more transformations at https://github.com/loadimpact/jslib.k6.io 😄

Other enhancements and UX improvements

• CI: k6 releases for Windows will now be digitally signed, which should reduce the number and severity of warnings Windows users see; the warnings would hopefully disappear altogether once Microsoft sees enough usage of the signed k6 releases to trust us (#1746). The installer and binary were also enhanced with more metadata and had their look updated with the new k6 logo and styling (#1727).
• JS: goja, the JS runtime k6 uses, was updated to its latest master version. This includes a few bugfixes and support for several new features, so --compatibility-mode=base is even more feature-rich at no additional runtime cost. We are contributing patches to goja in an effort to completely drop core.js and have the benefit of lower CPU and memory usage per VU even with --compatibility-mode=extended in the next k6 version! 🎉
• Config: integer values for duration and similar time values in the exported script options and environment variables are now treated as milliseconds. Similarly, the timeout option in http.Params can now be "stringy", e.g. "30s", "1m10s", etc. (#1738).
• HTTP: k6 now accepts ArrayBuffer values for the HTTP request body (#1776). This is a prelude/MVP for us gradually adopting ArrayBuffer for all binary data in k6 (#1020).
• Docker: We've added WORKDIR /home/k6 to our official Dockerfile (#1794).

Bugs fixed!

• HTTP: updated the golang.org/x/crypto and golang.org/x/net dependencies, which should have resolved some corner case issues with HTTP/2 connections, since k6 depends on golang.org/x/net/http2 (#1734).
• HTTP: fixed a couple of issues with blockHostnames that prevented zero-length matches for wildcards, as well as the ex...

v0.29.0

k6 v0.29.0 is here! 🎉 It's a feature-packed release with tons of much-requested changes and additions, a lot of them implemented by awesome external contributors! ❤️

As promised in the previous release notes, we're trying to stick to a roughly 8-week release cycle, so you can expect the next k6 version at the start of January 2021, barring any bugfix releases before that.

New features

Initial support for gRPC (#1623)

k6 now supports unary gRPC calls via the new k6/net/grpc built-in module. Streaming RPCs are not yet supported and the JS API is in beta, so there might be slight breaking changes to the API in future k6 versions, but it's a good start on the road to fully supporting this much-requested protocol!

This is a simple example of how the new module can be used with grpcb.in:

import grpc from "k6/net/grpc";

let client = new grpc.Client();
// Download addsvc.proto for https://grpcb.in/, located at:
// https://raw.githubusercontent.com/moul/pb/master/addsvc/addsvc.proto
// and put it in the same folder as this script.
client.load(null, "addsvc.proto");

export default () => {
    client.connect("grpcb.in:9001", { timeout: "5s" });

    let response = client.invoke("addsvc.Add/Sum", {
        a: 1,
        b: 2
    });
    console.log(response.message.v); // should print 3

    client.close();
}

You can find more information and examples how to use k6's new gRPC testing capabilities in our documentation.

Huge thanks to @rogchap for adding this feature!

New options for configuring DNS resolution (#1612)

You can now control some aspects of how k6 performs DNS resolution! Previously, k6 would have cached DNS responses indefinitely (#726) and always picked the first resolved IP (#738) for all connections. This caused issues, especially when load testing services that relied on DNS for load-balancing or auto-scaling.

For technical reasons explored in (#726), k6 v0.29.0 still doesn't respect the actual TTL value of resolved IPs, that will be fixed in a future k6 version. For now, it simply allows users to specify a global static DNS TTL value and resolution strategy manually. It also has better defaults! Now, by default, the global DNS TTL value is 5 minutes and, if the DNS resolution returned multiple IPs, k6 will pick a random (preferably IPv4) one for each connection.

You can also configure this behavior with the new --dns CLI flag, the K6_DNS environment variable, or the dns script/JSON option. Three DNS resolution options are exposed in this k6 version: ttl, select, and policy.

Possible ttl values are :

• 0: no caching at all - each request will trigger a new DNS lookup.
• inf: cache any resolved IPs for the duration of the test run (the old k6 behavior).
• any time duration like 60s, 5m30s, 10m, 2h, etc.; if no unit is specified (e.g. ttl=3000), k6 assumes milliseconds. The new default value is 5m.

Possible select values are:

• first - always pick the first resolved IP (the old k6 behavior).
• random - pick a random IP for every new connection (the new default value).
• roundRobin - iterate sequentially over the resolved IPs.

Possible policy values are:

• preferIPv4: use IPv4 addresses, if available, otherwise fall back to IPv6 (the new default value).
• preferIPv6: use IPv6 addresses, if available, otherwise fall back to IPv4.
• onlyIPv4: only use IPv4 addresses, ignore any IPv6 ones.
• onlyIPv6: only use IPv6 addresses, ignore any IPv4 ones.
• any: no preference, use all addresses (the old k6 behavior).

Here are some configuration examples:

k6 run --dns "ttl=inf,select=first,policy=any" script.js # this is the old k6 behavior
K6_DNS="select=random,ttl=5m,policy=preferIPv4" k6 cloud script.js # new default behavior
# syntax for the JSON config file or for the exported script `options`:
echo '{"dns": {"select": "roundRobin", "ttl": "1h33m7s", "policy": "onlyIPv6"}}' > config.json
k6 run --config "config.json" script.js

Support for Go extensions (#1688)

After some discussions (#1353) and exploration of different approaches for Go-based k6 extensions, we've settled on adopting something very similar to caddy's extensions. In short, xk6 (modeled after xcaddy) is a small stand-alone tool that will be able to build custom k6 binaries with 3rd party extensions bundled in. The extensions can be simple Git repositories (no central infrastructure needed!) with Go modules. They will be fully compiled, not interpreted, a part of the final custom k6 binary users will be able to build with k6.

xk6 is not yet stable or documented, so any extension authors will struggle until we stabilize and document everything in the coming weeks. The important part is that the k6 changes that would allow xk6 to work were implemented in #1688, so k6 v0.29.0 is the first version compatible with xk6!

Expect more information soon, but for a brief example, xk6 will work somewhat like this:

xk6 build v0.29.0 --with github.com/k6io/xk6-k8s --with github.com/k6io/[email protected]

./k6 run some-script-with-sql-and-k8s.js

Thanks, @andremedeiros, for pushing us to add plugins in k6 and for making a valiant attempt to harness Go's poor plugin API! Thank you, @mardukbp, for pointing us towards the xcaddy approach and explaining its benefits!

Support for setting local IPs, potentially from multiple NICs (#1682)

You can now specify a list of source IPs, IP ranges and CIDRs for k6 run, from which VUs will make requests via the new --local-ips CLI flag or K6_LOCAL_IPS environment variable. The IPs will be sequentially given out to VUs, allowing you to distribute load between different local addresses. This option doesn't change anything on the OS level, so the IPs need to already be configured on the OS level in order for k6 to be able to use them.

The biggest use case for this feature is splitting the network traffic from k6 between multiple network adapters, thus potentially greatly increasing the available network throughput. For example, if you have 2 NICs, you can run k6 with --local-ips="<IP-from-first-NIC>,<IP-from-second-NIC>" to balance the traffic equally between them - half of the VUs will use the first IP and the other half will use the second. This can scale to any number of NICs, and you can repeat some local IPs to give them more traffic. For example, --local-ips="<IP1>,<IP2>,<IP3>,<IP3>" will split VUs between 3 different source IPs in a 25%:25%:50% ratio.

Thanks to @ofauchon, @srguglielmo, and @divfor for working on previous iterations of this!

New option for blocking hostnames (#1666)

You can now block network traffic by hostnames with the new --block-hostnames CLI flag / K6_BLOCK_HOSTNAMES environment variable / blockHostnames JS/JSON option. Wildcards are also supported at the beginning, allowing you to easily block a domain and all of its subdomains. For example, this will make sure k6 never attempts to connect to any k6.io subdomain (test.k6.io, test-api.k6.io, etc.) and www.example.com:

export let options = {
  blockHostnames: ["*.k6.io" , "www.example.com"],
};

Thanks to @krashanoff for implementing this feature!

UX and enhancements

• HTTP: The gjson library k6 uses for handling the HTTP Response.json(selector) behavior was updated, so we now support more modifiers like @flatten and multipaths (#1626). Thanks, @sondnm!
• HTTP: The status text returned by the server can now be accessed from the new Response.status_text field (#1649). Thanks, @lcd1232!
• HTTP: --http-debug now emits extra UUID values that can be used to match HTTP requests and their responses (#1644). Thanks, @repl-david-winiarski!
• Logging: A new allowedLabels sub-option is added to the Loki configuration (#1639).
• Cloud: when aborting a k6 cloud test with Ctrl+C, k6 will now wait for the cloud service to fully abort the test run before returning. A second Ctrl+C will cause it to immediately exit (#1647), (#1705). Thanks, @theerapatcha!
• JS: k6 will now attempt to recover from Go panics that occur in VU code, so they will be treated similarly to JS exceptions (#1697). This is just a precaution that should never be needed. panics should not happen and if one occurs, please report it in our issue tracker, since it's most likely a bug in k6.

Bugs fixed!

• JS: goja, the JS runtime k6 uses, was updated to its latest version, to fix some issues with regular expressions after its previous update (#1707).
• JS: Prevent loops with --compatibility-mode=extended when Babel can transpile the code but goja can't parse it (#1651).
• JS: Fixed a bug that rarely caused a context canceled error message to be shown ([#1677](https://github.com/loadimpact/k6/pull...

v0.28.0

k6 v0.28.0 is here! 🎉 It's a small release that adds some much requested features and a few important bugfixes!

Starting with this release, we'll be trying to stick to a new 8-week fixed release schedule for new k6 versions. This release comes ~8 weeks after v0.27.0 was released, and k6 v0.29.0 should be released in mid-November.

New features and enhancements!

Cloud execution logs (#1599)

Logs from distributed k6 cloud test runs will now be shown in the terminal that executed the k6 cloud command, as well as in the k6 cloud web app on app.k6.io! 🎉 This means that, if your script contains console.log() / console.warn() / etc. calls, or some of your requests or iterations fail, you'd be able to see that and debug them much more easily! Even --http-debug data should be proxied, up to 10000 bytes per message. To prevent abuse and not to overwhelm any user terminals, cloud logs are rate-limited at 10 messages per second per instance, for now, but that should be more than enough to debug most issues!

This feature is enabled by default, though you can disable it with k6 cloud --show-logs=false script.js.

Pushing k6 logs to loki (#1576)

k6 can now push its execution logs to a loki server! This can be done via the new --log-output CLI flag or the K6_LOG_OUTPUT environment variable option. For example, k6 run --log-output "loki=https://my-loki-server/loki/api/v1/push,limit=100,level=info,msgMaxSize=10000" will push up to 100 k6 log messages per second, of severity INFO and up, truncated to 10000 bytes, to https://my-loki-server.

Optional port to host mappings (#1489)

@calavera added an extension for the host mapping feature. Now you can specify different port numbers via the hosts option, like this:

import http from 'k6/http';

export let options = {
    hosts: {
        'test.k6.io': '127.0.0.1:8080',
    },
};

Support for specifying data types to InfluxDB fields (#1395)

@TamiTakamiya added support for specifying the data type (int/float/bool/string) of fields that are emitted to InfluxDB outputs.

In order to specify the data type, you should:

• Use the environment variable K6_INFLUXDB_TAGS_AS_FIELDS, which is used to specify which k6 metric tag values should be sent as nonindexable fields (instead of tags) to an InfluxDB output. This is specified as a comma-separated string, and is now extended to optionally allow specifying a data type to each name.
• Each pair of field name and its data type is represented in the format (name):(data_type), for example, event_processing_time:int.
• One of four data types (int, float, bool and string) can be specified to one field name.
• When the colon and a data_type are omitted, for example transaction_id, it is interpreted as a string field.

A complete example can look like this: export K6_INFLUXDB_TAGS_AS_FIELDS="vu:int,iter:int,url:string,boolField:bool,floatField:float"

Note: If you have existing InfluxDB databases that contain fields whose data types are different from the ones that you want to save in future k6 test executions, you may want to create a new database or change field names as the current InfluxDB offers limited support for changing fields' data type. See the InfluxDB documentation for more details.

Support for automatic gzip-ing of the CSV output result (#1566)

@thejasbabu added support to gzip archiving the file emitted by the CSV output on the fly. To use it, simply append .gz at the end of the file name, like this: k6 run --out csv=test.csv.gz test.js

UX

• Various spacing and progress bar rendering issues were improved (#1580).
• The k6 ASCII logo was made a bit more proportional (#1615). Thanks, @rawtaz!
• The docker-compose example setup from the k6 repo now contains a built-in simple dashboard (#1610). Thanks, @jeevananthank!
• Some logs now have a source field specifying if a log comes from console, http-debug or stacktrace (when an exception has bubbled to the top of the iteration).

Bugs fixed!

• Network: IPv6 support was fixed as a part of the new hosts port mapping (#1489). Thanks, @calavera!
• Metrics: Fixed the wrong name metric tag for redirected requests (#1474).
• UI: Fixed a divide by zero panic caused by some unusual execution environments that present a TTY, but return 0 for the terminal size (#1581).
• Config: Fixed the parsing of K6_DATADOG_TAG_BLACKLIST (#1602).
• Config: Fixed marshaling of tlsCipherSuites and tlsVersion (#1603). Thanks, @berndhartzer!
• WebSockets: Fixed a ws.SetTimeout() and ws.SetInterval() panic when float values were passed (#1608).

Internals

• goja, the JavaScript runtime k6 uses, was updated to the latest version. This means that k6 with --compatibility-mode=base now supports some standard library features from ES6 (goja's PR), though no new syntax yet. In future versions we plan to drop some current core.js modules that are no longer needed, which should greatly reduce memory usage per VU (#1588).
• Go modules are now used to manage the k6 dependencies instead of dep (#1584).

Breaking changes

• k6 cloud will now proxy execution logs back to the client machine. To disable this behavior, use k6 cloud --show-logs=false.

• --http-debug request and response dumps are now emitted through the logging sub-system, to facilitate the cloud log proxying (#1577).

v0.27.1

k6 v0.27.1 is a minor release with a few bugfixes and almost no functional changes compared to v0.27.0.

The biggest fix was resolving a panic (and some k6 login errors) when k6 was ran through git bash / Mintty on Windows (#1559).

k6 will now work in those terminals, however, if you're using git bash or Mintty as your terminal on Windows, you might not get the best user experience out of k6. Consider using a different terminal like Windows Terminal, PowerShell or Cmder. Alternatively, to work around the issues with the incompatible terminals, you can try running k6 through winpty, which should already be preinstalled in your git bash environment: winpty k6 run script.js.

If you're using the Windows Subsystem for Linux (WSL), you are probably going to get better experience by using the official Linux k6 binary or .deb package. For all other cases of running k6 on Windows, the normal k6 Windows binary / .msi package should work well.

Other minor fixes and changes:

• The Go version that k6 is compiled with was updated to 1.14.6, to incorporate the latest Go fixes (#1563).
• If the throw option is enabled, warnings for failed HTTP requests will no longer be logged to the console (#1199).
• Metric sample packets sent to the cloud with k6 run --out cloud can now be sent in parallel via the new K6_CLOUD_METRIC_PUSH_CONCURRENCY option, with a default value of 1 (#1569).
• The gracefulRampDown VU requirement calculations for the ramping-vus executor were greatly optimized for large test runs (#1567).
• Fixed a rare bug where dropped_iterations wouldn't be emitted by the per-vu-iterations executor on time due to a race (#1357).
• Metrics, including checks, from setup() and teardown(), were not correctly shown in local k6 runs (#949).

v0.27.0

k6 v0.27.0 is here! 🎉

This is a milestone release containing a major overhaul to the execution subsystem of k6, along with many improvements and bug fixes.

New features and enhancements!

New execution engine (#1007)

After 1.5 years in the making, the k6 team is proud to release the first public version of the new execution engine, offering users new ways of modeling advanced load testing scenarios that can more closely represent real-world traffic patterns.

These new scenarios are entirely optional, and the vast majority of existing k6 scripts and options should continue to work the same as before. There are several minor breaking changes and fixes of previously undefined behavior, but please create a new issue if you find some issue we haven't explicitly noted as a breaking change.

See the documentation for details and examples, or keep reading for the summary.

New executors

Some of the currently possible script execution patterns were formalized into standalone executors:

• shared-iterations: a fixed number of iterations are "shared" by all VUs, and the test ends once all iterations are executed. This executor is equivalent to the global vus and iterations (plus optional duration) options.
• constant-vus: a fixed number of VUs execute as many iterations as possible for a specified amount of time. This executor is equivalent to the global vus and duration options.
• ramping-vus: a variable number of VUs execute as many iterations as possible for a specified amount of time. This executor is equivalent to the global stages option.
• externally-controlled: control and scale execution at runtime via k6's REST API or the CLI.

You'd still be able to use the global vus, iterations, duration, and stages options, they are not deprecated! They are just transparently converted to one of the above executors underneath the hood. And if your test run needs just a single, simple scenario, you may never need to use more than these shortcut options. For more complicated use cases however, you can now fine-tune any of these executors with additional options, and use multiple different executors in the same test run, via the new scenarios option, described below.

Additionally, besides the 4 "old" executor types, there are 3 new executors, added to support some of the most frequently requested load testing scenarios that were previously difficult or impossible to model in k6:

• per-vu-iterations: each VU executes a fixed number of iterations (#381).
• constant-arrival-rate: iterations are started at a specified fixed rate, for a specified duration. This allows k6 to dynamically change the amount of active VUs during a test run, to achieve the specified amount of iterations per period. This can be very useful for a more accurate representation of RPS (requests per second), for example. See #550 for details.
• ramping-arrival-rate: a variable number of iterations are executed in a specified period of time. This is similar to the ramping VUs executor, but instead of specifying how many VUs should loop through the script at any given point in time, the iterations per second k6 should execute at that point in time can be specified.

It's important to also note that all of these executors, except the externally-controlled one, can be used both in local k6 execution with k6 run, and in the distributed cloud execution with k6 cloud. This even includes "old" executors that were previously unavailable in the cloud, like the shared-iterations one. Now, you can execute something like k6 cloud --iterations 10000 --vus 100 script.js without any issues.

Execution scenarios and executor options

Multiple execution scenarios can now be configured in a single test run via the new scenarios option. These scenarios can run both sequentially and in parallel, and can independently execute different script functions, have different executor types and execution options, and have custom environment variables and metrics tags.

An example using 3 scenarios:

import http from 'k6/http';
import { sleep } from 'k6';

export let options = {
    scenarios: {
        my_web_test: { // some arbitrary scenario name
            executor: 'constant-vus',
            vus: 50,
            duration: '5m',
            gracefulStop: '0s', // do not wait for iterations to finish in the end
            tags: { test_type: 'website' }, // extra tags for the metrics generated by this scenario
            exec: 'webtest', // the function this scenario will execute
        },
        my_api_test_1: {
            executor: 'constant-arrival-rate',
            rate: 90, timeUnit: '1m', // 90 iterations per minute, i.e. 1.5 RPS
            duration: '5m',
            preAllocatedVUs: 10, // the size of the VU (i.e. worker) pool for this scenario
            maxVUs: 10, // we don't want to allocate more VUs mid-test in this scenario

            tags: { test_type: 'api' }, // different extra metric tags for this scenario
            env: { MY_CROC_ID: '1' }, // and we can specify extra environment variables as well!
            exec: 'apitest', // this scenario is executing different code than the one above!
        },
        my_api_test_2: {
            executor: 'ramping-arrival-rate',
            startTime: '30s', // the ramping API test starts a little later
            startRate: 50, timeUnit: '1s', // we start at 50 iterations per second
            stages: [
                { target: 200, duration: '30s' }, // go from 50 to 200 iters/s in the first 30 seconds
                { target: 200, duration: '3m30s' }, // hold at 200 iters/s for 3.5 minutes
                { target: 0, duration: '30s' }, // ramp down back to 0 iters/s over the last 30 second
            ],
            preAllocatedVUs: 50, // how large the initial pool of VUs would be
            maxVUs: 100, // if the preAllocatedVUs are not enough, we can initialize more

            tags: { test_type: 'api' }, // different extra metric tags for this scenario
            env: { MY_CROC_ID: '2' }, // same function, different environment variables
            exec: 'apitest', // same function as the scenario above, but with different env vars
        },
    },
    discardResponseBodies: true,
    thresholds: {
        // we can set different thresholds for the different scenarios because
        // of the extra metric tags we set!
        'http_req_duration{test_type:api}': ['p(95)<250', 'p(99)<350'],
        'http_req_duration{test_type:website}': ['p(99)<500'],
        // we can reference the scenario names as well
        'http_req_duration{scenario:my_api_test_2}': ['p(99)<300'],
    }
};

export function webtest() {
    http.get('https://test.k6.io/contacts.php');
    sleep(Math.random() * 2);
}

export function apitest() {
    http.get(`https://test-api.k6.io/public/crocodiles/${__ENV.MY_CROC_ID}/`);
    // no need for sleep() here, the iteration pacing will be controlled by the
    // arrival-rate executors above!
}

As shown in the example above and the documentation, all executors have some additional options that improve their flexibility and facilitate code reuse, especially in multi-scenario test runs:

• Each executor has a startTime property, which defines at what time, relative to the beginning of the whole test run, the scenario will start being executed.
• Executors have a new gracefulStop property that allows for iterations to complete gracefully for some amount of time after the normal executor duration is over (#879, #1033). The ramping-vus executor additionally also has gracefulRampDown, to give iterations time to finish when VUs are ramped down. The default value for both options is 30s, so it's a slight breaking change, but the old behavior of immediately interrupting iterations can easily be restored by setting these options to 0s.
• Different executors can execute different functions other than the default exported one. This can be specified by the exec option in each scenarios config, and allows for more flexibility in organizing your tests, easier code reuse, building test suites, etc.
• To allow for even greater script flexibility and code reuse, you can specify different environment variables and tags in each scenario, via the new env and tags executor options respectively.
• k6 may now emit a new dropped_iterations metric in the shared-iterations, per-vu-iterations, constant-arrival-rate and ramping-arrival-rate executors; this is done if it can't run an iteration on time, depending on the configured rates (for the arrival-rate executors) or scenario maxDuration (for the iteration-based executors), so it's generally a sign of a poor config or an overloaded system under test (#1529).

We've also introduced new --execution-segment and --execution-segment-sequence options, which allow for relatively easy partitioning of test runs across multiple k6 instances. Initially this applies to the test execution (all new executor types are supported!), but opens the door to test data partitioning, an often requested feature. See #997 for more details.

UX

• CLI: There are separate descriptions and real-time thread-safe progress bars for each individual executor.
• CLI: Improve module import error message ([#1439](https://github.com/loadimpact/...