v0.40.0

k6 v0.40.0 is here! This release includes:

• Breaking changes to some undocumented and unintentional edge behaviors.
• New experimental modules and first-class support for JavaScript classes.
• Bugs and refactorings to pave the way for future features.

Finally, the Roadmap goes over the plans for the next cycles.

Breaking changes

• #2632 During the refactoring to set tags to metric.add in the order they are provided, we discovered that you could provide tags as a key-value pair map multiple times in the same call. This was never the intended use and was never documented. As it was undocumented, and as such functionality makes no sense alongside every other API k6 has, we decided to remove this ability.
• #2582 [For extensions using the event loop] Previously, if RegisterCallback result was called twice, the second call would silently break the event loop. This has never been expected behavior, and calling it twice is always a bug in the code using it. Now, calling the RegisterCallback result twice leads to a panic.
• #2596 The tainted property of the Metric type is no longer outputted by the JSON output. That property was likely always going to have a false value as it was outputted at the beginning of the test.

Main module/script no longer pollutes the global scope #2571

During the ESM changes, we found that anything defined in the main module scope was also accessible globally. This was because it was directly evaluated in the global scope.
This has now been remedied and is no longer the case. This is a breaking change, but given that the whole point of modules (CommonJS or ESM) is to separate them, this is obviously rather a bug than a feature.

On that note, we've seen reports by people who have this global accessibility of the main module (intentionally or not). Still, it seems relatively rare, with only a few usages in a script. So if you need to access something globally, our suggested workaround is to set it explicitly on the global object globalThis.

k6/ws now respects the throw option #2247

k6/http has used the throw option to figure out whether it should throw an exception on errors or return a response object with an error set on it (and log it).

This functionality is finally also available for k6/ws, which previously would've always thrown an exception and thus involved more scripting in handling it (#2616).

This is a minor breaking change. By default, throw is false, so it now no longer throws an exception but instead returns a Response with error property.

Thank you, @fatelei, for making this change!

New Features

Experimental modules #2630 and #2656

As mentioned in the v0.39.0 release notes, we're happy to announce that this release brings experimental modules. The main idea behind this initiative is to get community feedback earlier, which will help us improve them faster. We encourage you to try experimental modules out and provide feedback through the community forums or GitHub issues.

This release contains three experimental modules:

• k6/experimental/redis - support for interaction with Redis
• k6/experimental/websockets - a new Websockets API that copies the "web" WebSocket API
• k6/experimental/timers - adding setTimeout/clearTimeout and setInterval/clearInterval implementations.

Important to highlight that the k6 team does not guarantee backward compatibility for these modules and may change or remove them altogether. Also, their import paths, starting with k6/experimental, will break when the modules stop being experimental. Of course, we are going to try to limit those breaking changes to a minimum and, when possible, do them in a backward compatible way for at least a version.

Redis example

Here is a fairly big example using xk6-redis as an experimental module to keep track of data in Redis:

import { check } from "k6";
import http from "k6/http";
import redis from "k6/experimental/redis"; // this will be `k6/x/redis` if you are using it as extension
import { textSummary } from "https://jslib.k6.io/k6-summary/0.0.1/index.js";

export const options = {
  scenarios: {
    usingRedisData: {
      executor: "shared-iterations",
      vus: 10,
      iterations: 200,
      exec: "measureUsingRedisData",
    },
  },
};

// Instantiate a new redis client
const redisClient = new redis.Client({
  addrs: __ENV.REDIS_ADDRS.split(",") || new Array("localhost:6379"), // in the form of "host:port", separated by commas
  password: __ENV.REDIS_PASSWORD || "",
});

// Prepare an array of crocodile ids for later use
// in the context of the measureUsingRedisData function.
const crocodileIDs = new Array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9);

export function setup() {
  redisClient.sadd("crocodile_ids", ...crocodileIDs);
}

export function measureUsingRedisData() {
  // Pick a random crocodile id from the dedicated redis set,
  // we have filled in setup().
  redisClient
    .srandmember("crocodile_ids")
    .then((randomID) => {
      const url = `https://test-api.k6.io/public/crocodiles/${randomID}`;
      const res = http.get(url);

      check(res, {
        "status is 200": (r) => r.status === 200,
        "content-type is application/json": (r) =>
          r.headers["Content-Type"] === "application/json",
      });

      return url;
    })
    .then((url) => redisClient.hincrby("k6_crocodile_fetched", url, 1));
}

export function teardown() {
  redisClient.del("crocodile_ids");
}

export function handleSummary(data) {
  redisClient
    .hgetall("k6_crocodile_fetched")
    .then((fetched) => Object.assign(data, { k6_crocodile_fetched: fetched }))
    .then((data) =>
      redisClient.set(`k6_report_${Date.now()}`, JSON.stringify(data))
    )
    .then(() => redisClient.del("k6_crocodile_fetched"));

  return {
    stdout: textSummary(data, { indent: "  ", enableColors: true }),
  };
}

This example also showcases how to write some data and clean up after yourself.

The extension does not run a Redis server. You need to separately handle running, scaling, and connecting infrastructure to Redis.

The xk6-redis repository has more examples, and the module is documented in the official k6 documentation.

WebSockets example

This is a rewrite of the current WebSocket example at https://test-api.k6.io/.

This showcases how a single VU can run multiple WebSockets connections asynchronously and how to stop them after a period using the timeout and interval functions.

import { randomString, randomIntBetween } from "https://jslib.k6.io/k6-utils/1.1.0/index.js";
import { WebSocket } from "k6/experimental/websockets"
import { setTimeout, clearTimeout, setInterval, clearInterval } from "k6/experimental/timers"

let chatRoomName = 'publicRoom'; // choose your chat room name
let sessionDuration = randomIntBetween(5000, 60000); // user session between 5s and 1m


export default function() {
  for (let i = 0; i < 4; i++) {
    startWSWorker(i)
  }
}

function startWSWorker(id) {
  let url = `wss://test-api.k6.io/ws/crocochat/${chatRoomName}/`;
  let ws = new WebSocket(url);
  ws.addEventListener("open", () => {
    ws.send(JSON.stringify({ 'event': 'SET_NAME', 'new_name': `Croc ${__VU}:${id}` }));

    ws.addEventListener("message", (e) => {
      let msg = JSON.parse(e.data);
      if (msg.event === 'CHAT_MSG') {
        console.log(`VU ${__VU}:${id} received: ${msg.user} says: ${msg.message}`)
      }
      else if (msg.event === 'ERROR') {
        console.error(`VU ${__VU}:${id} received:: ${msg.message}`)
      }
      else {
        console.log(`VU ${__VU}:${id} received unhandled message: ${msg.message}`)
      }
    })


    let intervalId = setInterval(() => {
      ws.send(JSON.stringify({ 'event': 'SAY', 'message': `I'm saying ${randomString(5)}` }));
    }, randomIntBetween(2000, 8000)); // say something every 2-8seconds


    let timeout1id = setTimeout(function() {
      clearInterval(intervalId)
      console.log(`VU ${__VU}:${id}: ${sessionDuration}ms passed, leaving the chat`);
      ws.send(JSON.stringify({ 'event': 'LEAVE' }));
    }, sessionDuration);

    let timeout2id = setTimeout(function() {
      console.log(`Closing the socket forcefully 3s after graceful LEAVE`);
      ws.close();
    }, sessionDuration + 3000);

    ws.addEventListener("close", () => {
      clearTimeout(timeout1id);
      clearTimeout(timeout2id);
      console.log(`VU ${__VU}:${id}: disconnected`);
    })
  });
}

Note that no k6 iterations finish if any WebSocket is still open or if a timeout or an interval is not cleared or triggered. This means that your script must take care of clearing all intervals and closing the WebSocket at some point. However, k6 still kills the whole process if it takes too long to stop after the maximum test duration is reached.

Current issues and ...

v0.39.0

k6 v0.39.0 is here! 🎉 It's a small release that includes a bunch of bugfixes and minor enhancements. Much of our focus was on some upcoming big changes. You can read about what's coming up next in the Roadmap and future plans section.

Enhancements and UX improvements

• #2274 and #2560 improved the csv output with support for a new timeFormat option. The possible values are unix (default) and rfc3399. You can also configure it through the K6_CSV_TIME_FORMAT environment variable. Thanks, @rpocklin!
• #2509 added the clear() and delete() methods to the CookieJar object from the k6/http module. Thanks, @Maksimall89!
• #2282 increased the precision of the iteration-progress bar in the UI. Thanks, @m3hm3t and @darkaether!
• #2568 added more descriptive error messages when there were problems with parsing a config file.

Bug fixes

• #2523 fixed a gRPC marshaling error when any.proto was used for a type. Thanks, @Flowersea!
• #2534 fixed the return type of Selection.map() from the k6/html module to the correct object types instead of a forced array of strings.
• #2502 made it so k6 waits for scenario executors to fully finish before updating their final progress in the UI, preventing misleading red crosses (#2500).
• #2524 fixed a bug where GoError string contained missing URL values (#2537).
• #2530 fixed a wrong error message on remote resolution.
• #2542 fixed a bug where Rate metric and sub-metric values were shown as NaN in the end-of-test summary if there were no measured values for them during the test run.
• #2558 fixed a panic when trying to set the value of a vu.tags element from k6/execution to null or undefined.
• #2567 fixed a panic when trying to access some k6/execution properties outside of a VU context, e.g. trying to access execution.scenario in setup().

Maintenance and internal improvements

• #2550 updated the used Go version to 1.18.
• #2524, #2551, #2552, #2553, #2554, #2555 updated various Go dependencies in k6.
• #2583 added a deprecation warning for thresholds that use the url, error, vu or iter tags, which will become un-indexable in the future.

Roadmap and future plans

As the lack of big changes in this release suggests, we've focused the last few months' efforts on a few areas that haven't yet been merged into the core of k6.

In this section, we'd like to inform the community about important features that we're currently working on - our short-term roadmap in a sense. We'll also use it to give notice of breaking changes we plan to make in the near future.

k6/experimental/* JS modules

Over the last several k6 releases, among a lot of other refactorings, we've added support for JavaScript event loops (#2228) in k6 VUs and added a new Go API for exposing built-in and xk6 extension modules to user scripts (announcement, docs). This has given us (and any xk6-extension authors!) the ability to better support various asynchronous streaming/messaging/etc. protocols (#882).

We've started building some of these newly possible APIs as xk6 extensions first, to be able to iterate on them more quickly and get some user feedback while we are building them. xk6-websockets, xk6-timers and xk6-redis are some of the first such APIs, but we plan to also work on support for gRPC streaming (#2020), messaging protocols (#1269), a new and better HTTP API (#2461) and many others in the future!

We want to eventually include a lot of these APIs in the k6 core as built-in modules that users can directly use, without needing to mess with xk6 or Go compilation. However, because we try to keep the built-in k6 APIs stable and backwards-compatible, we want to get more user feedback before we do that, while we are still free to iterate and make (hopefully minor) breaking changes.

So, we decided to create a new middle ground between the unstable and purely external xk6 extensions and the stable built-in k6 APIs―built-in k6/experimental/* modules! Our goal is that, starting with the next k6 v0.40.0 release, we'll start releasing some or all of these core-bound extensions as built-in k6 modules under these k6/experimental/ import paths. This will let k6 users, both OSS and Cloud, to give us feedback and help us improve the APIs before we stabilize them.

As is hopefully clear from the name, our usual guarantees of API stability won't apply to these modules while they are still experimental. We reserve the right to make breaking changes in experimental modules, up to and including completely dropping them. We don't expect big breaking changes will need to happen often, but we want to be clear they aren't impossible. Finally, when an API has been stabilized and made available under a regular import path, we'll deprecate its experimental import path. To make the transition easier, both import paths will be available simultaneously for at least one k6 version.

Native support for ECMAScript modules

At the moment, k6 has support for ECMAScript modules (ESM, i.e. import, export, etc.) via automatic transpilation of scripts by the built-in Babel.js. That mostly works, but it has caused some performance and compatibility problems (#824 and #2168 among others), so we want to support ESM modules and all other ES6 features directly in k6, without the need for Babel.js (#2296). goja, the JavaScript runtime we use to evaluate k6 scripts, doesn't yet have native ESM support, so we are currently working on adding it there, to then be able to support ECMAScript modules natively in k6!

That work has been ongoing for a while and we're making progress, but it will likely not be ready in time for the next k6 v0.40.0 release. We are mentioning it here because we will probably need to make a few minor breaking changes and fixes of currently undefined behavior in k6 to support ESM modules natively.

For example, at the moment, some values like the consolidated script options were unintentionally made available globally, in all JS modules of a test, instead of just in the exported options value from the main JS module. That is not the intended or documented behavior, it's somewhere between a bug and undefined behavior, and we'll need to fix it (#2571) and other similar issues like it, starting in k6 v0.40.0. We don't expect many scripts to break because of these fixes, but we'll nevertheless announce them in the release notes of the k6 version that they happen in.

Refactoring metrics

Over the last several k6 releases, we've also slowly been refactoring and improving the metrics internals in k6 (see #2071, #2330, #2426, #2463, #2433, #2442, among others). This has already brought many side benefits and minor bugfixes, and we still have a lot of work left (e.g. #1889, #572, #2430, #1831), but we've finally reached the point where we can almost start implementing some major new features effectively!

One of the upcoming next big steps is the introduction of a "time series" concept internally in k6 (#2580). We'll start to efficiently group samples (i.e. metric measurements) with the same metric and tags into a single TimeSeries, which ...

v0.38.3

k6 v0.38.3 is a patch release containing a single fix

Threshold over already defined sub-metrics will result in an error (#2538)

There was a bug where we were checking if a submetric had already been added. Unfortunately, we didn't check that this will work with the one submetric we have by default http_req_duration{expected_response:true}. After v0.38.0 defining a threshold on it would result in an error.

As this definitely shouldn't happen in that case and we don't see a particular case where that will be problematic - adding a submetric again just reuses the already added one instead.

This issue has been addressed in #2539, and k6 v0.38.3 will now lead you add a threshold on http_req_duration{expected_response:true}.

v0.38.2

k6 v0.38.2 is a patch release containing a couple of bugfixes!

Threshold over sub-metrics without samples would result in NaN (#2520)

There was a bug in thresholds applied to sub-metrics set to abortOnFail: leading k6 to evaluate thresholds that would have likely aborted before they had a chance of passing (because no samples for the given metric were recorded yet). This bug would have led to such thresholds' results value to be NaN rather than a numerical value. The following script, for instance:

import { check, sleep } from 'k6';

import http from 'k6/http';

export const options = {
  scenarios: {
    iWillFail: {
      exec: 'iWillFail',
      executor: 'constant-vus',
      startTime: '2s',
      vus: 1,
      duration: '30s',
    },
  },

  thresholds: {
    'checks{type:read}': [{ threshold: 'rate>0.9', abortOnFail: true }],
  },
};

export function iWillFail() {
  let res = http.get(`https://test-api.k6.io/`);

  check(res, {
    'read status is 200': (r) => r.status === 200,
  }, { type: 'read' });

  sleep(1);
}

Would result in the following:

✗ { type:read }...: NaN% ✓ 0 ✗ 0  
vus...............: 0 min=0 max=0
vus_max...........: 1 min=1 max=1

This issue was introduced by recent changes to how we handle thresholds in the k6 engine and is now addressed in v0.38.2.

Sub-metrics without values rendered below an incorrect parent metric (#2518)

There was in how thresholds over sub-metrics that didn't receive any samples would be displayed under an incorrect parent metric. For instance, the following script:

import { Counter } from 'k6/metrics';

const counter1 = new Counter("one");
const counter2 = new Counter("two");

export const options = {
    thresholds: {
        'one{tag:xyz}': [],
    },
};

export default function() {
    console.log('not submitting metric1');
    counter2.add(42);
}

Would have led to the following output, where the {tag:xyz} sub-metric is displayed under iterations instead of one:

data_received........: 0 B 0 B/s
data_sent............: 0 B 0 B/s
iteration_duration...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
iterations...........: 1 499.950005/s
  { tag:xyz }........: 0 0/s
two..................: 42 20997.90021/s

When we would have expected it to produce:

one..................: 0 0/s
  { tag:xyz }........: 0 0/s
two..................: 42

This issue has been addressed in #2519, and k6 v0.38.2 now displays sub-metrics under their actual parents, even when they have received no samples.

Special thanks to @efdknittlfrank, who reported and helped us track down the issue.

v0.38.1

k6 v0.38.1 is a patch release containing a bugfix!

Threshold sub-metric selectors containing reserved symbols would fail (#2512)

There was a bug in threshold sub-metric selector parsing, which led to errors when users would use specific symbols such as {, } or : as part of their definitions. For instance, thresholds used for sub-metrics with URL Grouping like http_req_duration{name:"http://example.com/${}"} would have led to failures in v0.38.0.

The error messages for invalid metric, sub-metric and threshold definitions were also improved.

Special thanks to @efdknittlfrank, who reported and helped us track down the issue.

v0.38.0

k6 v0.38.0 is here! 🎉

New Features!

AWS JSLib

There's a new addition to the officially supported k6 JavaScript libraries: k6-jslib-aws.
This library lets users interact with a selection of AWS services directly from their scripts. The library currently implements support for the S3 and the Secrets Manager services.

The AWS JS lib documentation has examples and details on how to use the library in your scripts.

Accessing the consolidated and derived options from the default function (#2493)

The k6/execution core module now lets you access the consolidated and derived options that k6 computed before it started executing the test. You can access consolidated options through the exec.test.options property. Note that consolidated options are frozen and cannot be modified. The k6 execution module's documentation has examples and details on how to use the functionality in your scripts.

import exec from "k6/execution";

export const options = {
    vus: 10,
    duration: "30s",
};

export default function () {
    console.log(exec.test.options.scenarios.default.vus); // 10
}

Tagging metric values with the current scenario stage

With the new consolidated script options, we've added a few helper functions to the k6-jslib-utils library. You can use them to automatically tag all the emitted metric samples by k6 with the currently running stage.

The k6 documentation has examples and details on how to use it.

Dumping SSL keys to an NSS formatted key log file (#2487)

This release adds the ability to dump SSL keys while making TLS connections.
You then can use these keys to decrypt all traffic that k6 generates.

To accomplish this, set the SSLKEYLOGFILE environment variable to some file path and run k6.
This will populate the file with the keys.
Then you can use Wireshark to capture the traffic, decrypt it, and use that for debugging.

Here's an example that uses curl to inspect TLS traffic.

Breaking Changes

console methods now pretty print objects and arrays (2375)

For convenience, all console methods such as console.log() and console.info() will now automatically JSON.stringify() objects and arrays passed to them. Thus, instead of console.log({'foo': 'bar'}) printing [object Object], it will now print {'foo': 'bar'}, which will make the experience of debugging k6 scripts easier and more intuitive.

To achieve the previous behavior, cast the Object to a String, as in console.log(String({'foo': 'bar'})).

export default function () {
    console.log([1, 2, "test", ["foo", "bar"], { user: "Bob" }]);
    // before: 1,2,test,foo,bar,[object Object]
    // after: [1,2,"test",["foo","bar"],{"user":"Bob"}]
}

The Go types in the stats package were moved to the metrics package #2433

For convenience and to facilitate further developments, the types and functionalities that used to live in k6's stats package have been moved to the metrics package. The stats package is, as of v0.38.0, removed in favor of the metrics package. Besides, #2442 removed the stats.New function in favor of initializing new metric via a register.NewMetric call instead.

Deprecation

• #2499 removed support for the deprecated maxVUs option.
  It had been removed in k6 v0.27.0, however using the CLI flag resulted only in a deprecation warning.
  Now, using this flag will generate an error.
• This release drops some leftovers from the previous version of our JS module Go APIs. As of v0.38.0, these are now unsupported:
  • The deprecated common.Bind (#2488) and common.BindToGlobal (#2451) functions.
  • The context-based (common/context.go #2488) utils have also been removed.

Enhancements and UX improvements

Stricter thresholds' evaluation before the execution starts (#2330)

k6 v0.37.0 already improved threshold parsing by switching its underlying implementation from JavaScript to Go. k6 v0.38.0 introduces two additional improvements:

• k6 will now parse and evaluate thresholds before the execution starts. If a threshold is invalid, as described below, k6 will immediately exit without starting the load test.
• k6 will now detect invalid thresholds:

  export const options = {
      // ...
      thresholds: {
          // Incorrect thresholds expressions:
          http_req_failed: ["rave<0.01"], // e.g. "rave" is not a valid aggregation method
          // Thresholds applying to a non-existing metrics:
          iDoNotExist: ["p(95)<200"], // e.g. the metric 'iDoNotExist' does not exist
          // Thresholds applying an aggregation method that's unsupported by the metric they apply to:
          my_counter: ["p(95)<200"], // Counter metrics do not support the p() aggregation method
      },
  };

Disabling colors (#2410)

In addition to the --no-color CLI flag, the ANSI color escape codes emitted by k6 can now also be disabled by setting the NO_COLOR or K6_NO_COLOR environment variables, following the NO_COLOR standard.

# No color output
K6_NO_COLOR=true k6 run script.js

# No color output
NO_COLOR= k6 run script.js

Support for encrypted TLS private keys (#2488)

You can now use passphrase-protected private keys when authenticating with TLS. Using the password property of an options' tlsAuth object, you can now indicate the passphrase to decrypt a private key. Note that this support is limited to the scope of RFC1423 and does not support PKCS8 keys, as they're not yet supported by the Golang standard library.

export const options = {
    tlsAuth: [
        {
            domains: ["example.com"],
            cert: open("mycert.pem"),
            key: open("mycert-key.pem"),
            password: "mycert-passphrase",
        },
    ],
};

Thanks, @Gabrielopesantos, for the contribution.

Improve JSON output's performance (#2436)

The JSON output was optimized and now should be around 2x more performant at outputting metrics. This means that it either can export twice as many metrics, or use half the resources to do the same amount of metrics.

As a side effect, there is a slight breaking change: the tags field is no longer sorted.

Treat panics as interrupt errors (#2453)

We changed the behavior of how k6 treats Go panics, which may happen because of bugs in k6 or in a JavaScript k6 extension. Previously, the behavior was to catch the panic and log it as an error.

Starting with v0.38.0, whenever k6 observes a Go panic, it logs an error like before, but more importantly, it will abort the script execution and k6 will exit with a non-0 exit code. This will help extension authors to identify issues in their extensions more easily.

Miscellaneous

• #2411 The k6 command-line UI (logo, test description, and progress bars) can now effectively be disabled using the --quiet flag.
• #2429 lib/types now exposes the source of the NullHostnameTrie to simplify access to an original list of the hostnames.

Extensions

PoC for a new Web Sockets JS API

We built a new xk6 extension, https://github.com/grafana/xk6-websockets, with a proof of concept implementation for a new JavaScript Web Sockets API. This API uses the global event loops introduced in k6 v0.37.0 to allow a single VU to have multiple concurrent web socket connections open simultaneously, greatly reducing the resources needed for large tests. It also is a step towards supporting the official Web Sockets JS standard, potentially allowing the usage of more third-party JS libraries in the future.

Please share any feedback you have about the new extension since it's likely that we'll adopt a future version of it into the k6 core in one of the next several k6 releases.

gRPC module refactored to enable gRPC extensions to use it

#2484 moved out in a new dedicated Go lib/netext/grpcext package all the parts not strictly required from the js/k6/net/grpc module for binding the gRPC operations and the JavaScript runtime. It facilitates the development of extensions based on gRPC without the direct dependency on the goja runtime. Furthermore, the new [Dial](https://github.com/grafana/k6/blob/bef458906f6884a99843573223028981c0a8b8db/l...

v0.37.0

k6 v0.37.0 is here! 🎉 Mainly it contains fixes and ongoing efforts with refactoring.

New Features!

Added experimental basic event loop (#882)

We added basic event loop support in k6 (#2228 and #2373) 🎉 This was just the first step and isn't used by any of the existing k6 JS APIs yet. For now, it is only available to xk6 extensions like this one that adds support for setTimeout(), setInterval(), clearTimeout() and clearInterval().

Expect to see more changes related to event loops in the next k6 releases, where event loops will start being used by some core k6 modules! For example, by improving some existing JavaScript APIs to have support for callbacks or return Promise values, so they can be used asynchronously. We expect this change will unlock a lot of previously difficult use cases (see #882), though we'll likely iterate on these new APIs as experimental extensions for a while, to stabilize them before we merge them into the core.

ℹ️ If you are an extension developer, please use it and give your feedback. But take into consideration that it's likely that the current Go API may change.

Added an option to output k6 logs to a file through --log-output (#2285)

This is on top of the already supported options for sending logs to stdout/stderr and to Grafana Loki. This new feature speaks for itself with simple usage examples:

k6 run --log-output file=./k6.log --logformat json ./examples/stages.js

And one more with a defined minimal log level:

k6 run --log-output file=./k6.log,level=info --logformat json ./examples/stages.js

Thanks, @alyakimenko for the contribution!

Docs: Using file output

Breaking changes

Introduced stricter thresholds parsing (#2400)

In the past, thresholds were evaluated using a JavaScript runtime. For a multitude of reasons, this wasn't satisfying. As of v0.37.0, thresholds are now parsed directly in Go. As a result, k6 will now return an error message on thresholds that do not strictly match the documented specification, instead of just silently ignoring them. Another change is that when a non syntactically correct threshold expression is detected, k6 will immediately interrupt its execution before even starting the load test run.

Below you can find examples of the thresholds expressions that won't work anymore:

export const options = {
    thresholds: {
        "http_req_duration": [
            // although the aggregation method and values are correct, 
            // the equal sign is invalid; use == or ===
            "rate=200",
            // thresholds do not support javascript expressions anymore
            "throw new Error('wat')",
            // it fails, as foo is not a valid threshold expression's aggregation method keyword
            "foo>500",
        ],
    },
};

Extensions

v0.37.0 finalizes (#2376) the switching of our internal modules (gRPC module refactoring) to a new Go/JavaScript module API.

⚠️ It's important to highlight that the old API (e.g. methods like context.WithRuntime, common.Bind and others #2384) is deprecated and will be removed in the next k6 release (v0.38.0). For this release, every extension that isn't using the new API will get a warning message like this:

WARN[0000] Module 'k6/x/sql' is using deprecated APIs that will be removed in k6 v0.38.0, for more details on how to update it see https://k6.io/docs/extensions/guides/create-an-extension/#advanced-javascript-extension

We did migrations for some xk6 extensions (see connected issues to the task #2344). The pull requests can serve as examples on how to transition your extension to the new API.

Docker Repository

We migrated our Docker Hub repository from loadimpact/k6 to grafana/k6 (#2377).

docker run -i grafana/k6 run - <script.js

We will continue publishing our docker image releases as both loadimpact/k6 and grafana/k6 for several more releases, but if you use the old one in your local or CI environments, please plan the migration.

Enhancements and UX improvements

• We continued work on the source map feature: add samples (#2339) and improve loading and parsing (#2355).
• Updated installation from source instructions (#2359). Thanks, @karitham, for making the change.
• Enabled more TC39 tests: optional-chaining (#2338) and async (#2396).
• We updated our Code of Conduct and a few GitHub URLs (#2416).

Bugs fixed!

• http.head started taking a body as a second argument (#2402) by mistake in v0.36.0. It is now back to its old signature http.head(url, [params]).
• Fixed options.scenarios' JSON marshaling (#2392).
• Metrics' names display again in k6's Rest API (#2421).
• We improved argument validation for check(#2387), http.batch(#2415) and metrics' constructors (#2427).

Internals

• We updated our CI to improve developer experience. Dependency and linter checks now run only for pull requests (#2403).
• This release also contains a few refactoring PRs that fix linters errors (#2334, #2331 and #2341), remove global variable usage (#2336, #2358, #2353 and #2357) and remove an unnecessary dependency (#2313) which makes our codebase more consistent and maintainable.
• The headers parameter in k6's GRPC module is marked as deprecated (#2370).
• Switched envconfig to our own fork (#2337) in order to abstract the os package and improve testability.

v0.36.0

k6 v0.36.0 is here! 🎉 It introduces a couple of new features which enhance its usability, includes a number of fixes, and the result of ongoing refactoring efforts.

New Features!

Source Maps support (#2082)

Following #2082, k6 now has support for Source Maps. k6 will try to load source maps either from the file system(s) or from inside the script, based on the standard(-ish) //#sourceMappingURL= comments. Furthermore, as k6 internally uses Babel to transform ES6+ scripts to ES5.1+, it will now make use of its ability to generate source maps, including combining with previously generated ones, to report correct line numbers. This should fix #1804; however, we anticipate some more issues will arise, and further tweaking will be necessary.

Thus, given an imported.js module such as:

export function f1() {
  throw "line 2";
}

  throw "line 6";
}

export function f3() {
  throw "line 10";
}

and a k6 test script importing it as such:

import { f2 } from "./imported.js"

export default function() {
  f2();
}

Previous versions of k6 would report an error stack trace indicating an invalid line number in imported.js (10):

ERRO[0000] line 6
        at f2 (file:///some/path/imported.js:10:61(2))
        at file:///some/path/sourcemap.js:4:20(4) executor=per-vu-iterations scenario=default source=stacktrace

Starting with v0.36.0 and source maps support, k6 would now report the exception at the correct line in imported.js:

ERRO[0000] line 6
        at f2 (file:///some/path/imported.js:6:2(2))
        at file:///some/path/loadtest.js:4:2(4)
        at native executor=per-vu-iterations scenario=default source=stacktrace

Temporary warning

Note that if a file size is greater than 250kb and the internal Babel is needed, Babel will not generate source map. This is because during internal testing it was found this takes 3x to 4x more memory, potentially leading to OOM (standing for "Out Of Memory", a state in which the OS kills a process for using too much memory) on bigger inputs. If required, you can control the accepted file size limit via the temporary K6_DEBUG_SOURCEMAP_FILESIZE_LIMIT=524288 environment variable; which will be removed after we no longer rely on Babel (#2296). A pre-generated source map will always be loaded. For more details, check #2345.

Ability to abort tests (#2093)

Thanks to the contribution of @gernest (#2093), k6 now has the ability to abort a test run from within the test script. The newly added test.abort() function in the k6/execution module allows k6 scripts to immediately abort the test execution - the VU that called it will abort immediately and any other VUs in the same or other instances (in the case of k6 cloud) will also be interrupted and abort soon after. Local k6 run tests will exit with a code of 108, so this event can also be easily detected in a CI script.

Aborting is possible during initialization:

import exec from "k6/execution";
exec.test.abort();

As well as inside the default function:

import exec from "k6/execution";

export default function() {
  // Note that you can abort with a specific message too
  exec.test.abort("this is the reason");
}

export function teardown() {
  console.log("teardown will still be called after test.abort()");
}

k6 inspect extended output (#2279)

Following #2279, the k6 inspect command now supports an --execution-requirements flag. When used, the command's output will include fields related to the execution requirements, by deriving k6's configuration from the execution context, and including the maxVUs and totalDuration fields in the output.

Forcing HTTP/1 protocol (#2222)

Thanks to the work of @sjordhani22, #2222 made it possible to force k6 to use version 1.1 of the protocol when firing HTTP requests.

It can be done by setting the http2client=0 value in the GODEBUG environment variable:

GODEBUG=http2client=0 k6 run testscript.js

N.B: the usage of the GODEBUG variable is considered temporary, and expected to change in the future. If you start using this feature, keep an eye out for potential future changes.

Extensions

v0.36.0 marks the switch of some of our internal modules to a new Go/JavaScript module API. We expect this change to make the process of developing internal JavaScript modules and advanced JavaScript extensions easier and more streamlined in the future. Although this switch to a new API does not introduce breaking changes for existing extensions yet, we anticipate deprecating the old extension API (e.g. common.Bind(), lib.WithState(), etc.) at an undecided point in the future.

For more details, see: #2243, #2241, #2239, #2242, #2226, and #2232.

Breaking changes

Restricting file opening to init context

VUs are now restricted to only open() files that were also opened in the init context of the first VU - the one that was initialized to get the exported options from the JS script (__VU==0). While it was somewhat possible to open files only in other VUs (e.g __VU==2) in the past, it was unreliable. #2314 ensures that k6 would now throw an error in a similar scenario. This means that you can still open files only for some VUs, but you need to have opened all of those files in the initial VU (__VU==0).

let file;

if (__VU == 0) {
  open("./file1.bin")
  open("./file2.bin")
} else if (__VU % 2 == 0) {
  file = open("./file1.bin")
} else {
  file = open("./file2.bin")
}

export default () => {
  // use file for something
}

Bugs Fixed!

• We addressed an issue uncovered by our community, which kept our users from using GRPC with multiple services definition in a single proto file. This issue was solved in #2265.
• Thanks to the contribution of @Resousse, we've now updated k6's go-ntlmssp dependency. The updating PR #2290 indeed fixes issues with NTLM Authentication backends returning two authorization headers.

Maintenance

• We have refactored our implementation of the RampingVU executor, for better clarity and maintainability. See #2155.
• #2316 relaxed quite a few of the code linting rules we applied to k6's code. It also revamped our Makefile, so the new make ci-like-lint target will run the exact same golangci-lint version that will be used in our GitHub Actions CI pipeline.
• #2304 prepared the removal of external dependencies from k6's JSONAPI compliant REST API, and deprecated the api.v1's client.Call method in favor of its newer client.CallAPI counterpart. It allows us to both reduce our reliance on external dependencies and improve its maintainability.
• We have updated our Goja dependency, our JS interpreter, to its latest available version. Unfortunately, some of the new features are not always usable, yet. Namely, Goja now supports the optional chaining syntax, but the Babel version we use presently does not. Which means that if Babel needs to be used, optional chaining can't be. See #2317 and #2238.
• Thanks to @knittl, #2312 upgraded loadimpact/k6 docker image base to Alpine 3.15.

Known Bugs

• #2226 introduced an unintended breaking change to http.head(). The signature in k6 v0.35.0 was http.head(url, [params]) and was inadvertently changed to http.head(url, [body], [params]) in v0.36.0. That change will be reverted in k6 v0.37.0, but until then, we suggest users use the stable http.request('HEAD', url, null, params) API for HTTP HEAD requests that need to specify custom parameters. Thanks, @grantyoung, for reporting the problem (#2401)!

v0.35.0

k6 v0.35.0 is here! 🎉 It introduces several new features that nicely enhance its usability and also contains a whole lot of fixes and ongoing efforts with refactoring.

In total, we have closed 14 issues. We have also branched out three new xk6 extensions during this release ⭐

New features

Ability to set VU-wide custom metric tags (#2172)

k6 now supports setting tags for VUs as part of the Execution API with an easy key-value interface. These tags are attached to the metrics emitted by the VU. Example usage:

import http from 'k6/http';
import exec from 'k6/execution';

export const options = {
    duration: '10s',
    vus: 3,
};

export default function () {
    exec.vu.tags['mytag'] = 'value';
    exec.vu.tags['vuId'] = exec.vu.idInTest;

    console.log(`mytag is ${exec.vu.tags['mytag']} and my VU's ID in tags ${exec.vu.tags['vuId']}`);

    // the metrics these HTTP requests emit will get tagged with `mytag` and `vuId`:
    http.batch(['https://test.k6.io', 'https://test-api.k6.io']);
}

One of the most requested use cases for this feature is that now we can tag all metrics with the current stage number. With a bit of JS code it is possible to calculate which stage of a ramping-vus or ramping-arrival-rate scenario the VU is currently in. This in turn allows the setting of thresholds only on the metrics that were emitted in specific stages of the test run! 🎉

There are some caveats, however: values can be only of String, Number or Boolean type, while values of other types will result either in a warning or an exception if throw option is enabled. Additionally, given that k6 has a whole bunch of system tags, one should be careful with using them as keys. You can read complete information about VU tags in k6/execution docs.

Initial basic support for JS promises

With the goja update in #2197, you can now make a Promise and chain it in your k6 scripts:

export default function () {
    var p = new Promise((resolve, reject) => {
        console.log('do something promising!');
        reject('here');
    });

    p.then(
        (s) => { console.log('fulfilled with', s) },
        (s) => { console.log('rejected with', s) },
    );
}

It must be noted that Promises are not used by k6 itself yet but this addition is a stepping stone for implementing async functionality in future releases. Thanks, @dop251, for your awesome work in developing goja! ❤️

Support for gRPC server reflection (#2160)

k6's gRPC capabilities were extended with a support for server reflection which allows one to use gRPC even without a proto file at hand. In other words, the following script is now possible:

import grpc from 'k6/net/grpc';
import { check } from "k6";

let client = new grpc.Client();

export default () => {
	client.connect("127.0.0.1:10000", {plaintext: true, reflect: true})
	const response = client.invoke("main.RouteGuide/GetFeature", {
		latitude: 410248224,
		longitude: -747127767
	})

	check(response, {"status is OK": (r) => r && r.status === grpc.StatusOK});
	console.log(JSON.stringify(response.message))

	client.close()
}

You can read more about the protocol here. Thanks, @joshcarp, for putting a lot of effort into this feature!

Other changes and UX improvements

• Support for cookie jars in k6/ws (#2193).
• Forbid metric.Add calls to let NaN values through. Instead, k6 will log nice warnings or throw an exception if --throw is enabled (#1876, #2219).
• Support for compression in websockets (#2162). Thanks, @cooliscool!
• Switch to camel case for CLI options to the outputs (#2150). Thanks, @josephwoodward!
• Much neater error message on nil response body (#2195). Thanks, @daniel-shuy!

New xk6 extensions

xk6-browser

xk6-browser is a browser automation extension which relies on Chrome Devtools Protocol. With xk6-browser, you can interact with the browser to test your web applications end-to-end while accessing all of the k6 core features, including protocol-level APIs and other k6 extensions. It’s a single tool for both protocol and browser-level testing.

The browser extension comes with an API that aims for rough compatibility with the Playwright API for NodeJS, meaning k6 users do not have to learn an entirely new API.

xk6-output-remote-write

Prometheus is now officially supported in k6 OSS with a xk6-output-remote-write extension. This is an output extension with implementation for Prometheus Remote-Write protocol which means that beyond Prometheus, any compatible remote-write solution can be used with it. You can read the full guide to using the extension in the relevant tutorial.

xk6-output-influxdb

After hard work at working out how to integrate InfluxDB v2 API, it was decided to pull that integration into a new xk6-output-influxdb extension for now. The built-in influxdb output in k6 still supports only InfluxDB v1, as before, with some minor optimizations (#2190).

Please try out the new extensions and tell us what you think!

Breaking changes

• The addition of common metrics registry (#2071) no longer allows defining custom metrics with the same name as one of the builtin metrics, e.g. new Counter("http_req_duration") will now abort. Similarly, an attempt to redefine a metric with the same name but with different type will error out. Builtin metrics may no longer be referenced as global objects in xk6 extensions either.
• Fix inconsistency in environment variables' names: use K6_NO_SETUP and K6_NO_TEARDOWN options instead of NO_SETUP and NO_TEARDOWN (#2140).
• Module interfaces were changed as part of refactoring efforts. Any JS module that needs access to the VU must now implement the new interfaces. This change can impact some xk6 extensions (#2234).

Bugs fixed!

• Fix of a misleading sorting of custom submetrics in the default end-of-test summary (#2198). Thanks, @knittl!
• Fix for extensions depending on afero.FS: implement a newer version of the afero.FS interface for internal filesystems so that extension depending on that or newer version can be built (#2216).
• Fix for websockets: websockets now use the global User-Agent setting (#2151). Thanks, @cooliscool!
• Fixes for tests, Github actions, and Loki integration (#2205, #2153, #2220).

Maintenance

• Update a whole bunch of dependencies (#2159, #2170, #2165).
• Switch to Go 1.17 (#2156). Thanks, @b5710546232!
• Get rid of mapstructure (#2223).
• Minor but necessary cleanups (#2164, #2192).

hacktoberfest

k6 participated in this year's hacktoberfest and we would like to thank all contributors! Here're some additional improvements made by the community members:

• Add multi-message WebSockets tests (#2184).
• Try out the new and shiny Github forms which are already improving the formatting of k6's new issues (#2174,#2179).
• An improved writing style and correctness in our README (#2189, #2152, #2169, #2181) and in some other places (#2182).

Thank you, @knittl, @cooliscool, @josephwoodward, @b5710546232, @Nontw, @divshacker, @daniel-shuy, @Sayanta66, @marooncoder09, @idivyanshbansal, @saintmalik, @EricSmekens, for helping make k6 better 😄

v0.34.1

k6 v0.34.1 is a patch release with a few minor bugfixes:

• There was a minor bug in the new k6/execution API added to k6 v0.34.0 - some of its properties weren't usable with the externally-controlled executor (#2132).
• goja, the JavaScript runtime that k6 uses, was updated to its latest version (#2135), which fixed a couple of bugs:
  • A newly introduced JS bug from k6 v0.34.0, where rest parameters were undefined when the functions also had an internal lambda (#2131). Thanks for reporting, @efdknittlfrank!
  • An old JS bug, first introduced in k6 v0.28.0, which caused Response.json() to not have a length property when the response was a JSON array, i.e.response.json().hasOwnProperty('length') was returning false (#2133). Thanks for reporting, @julien-sugg!