Bot releases are visible (Hide)

k6 - https://github.com/grafana/k6/releases/tag/v0.50.0 Latest Release

Published by github-actions[bot] 7 months ago

k6 v0.50.0 is here 🎉!

This release:

Adds support for uploading files from the browser module.
Introduces the options.cloud option.
Stabilizes the previously experimental timers module as the k6/timers module.
Brings JSON Web Key support to the k6/experimental/webcrypto module.

Breaking changes

websockets#60 allows manually setting the name tag, which also overwrites the url tag with the name value. This change makes it consistent with the logic that was implemented in k6 v0.41. Thanks, @mkadirtan for contributing!

Browser APIs to Async

In future releases, we are going to be moving most of the synchronous browser APIs to asynchronous ones (promisifying them). We expect this will affect most of our users, so we are posting this upfront before making the change. Here are the reasons for making this large breaking change:

Most browser APIs use some form of long-running IO operation (networking) to perform the requested action on the web browser against the website under test. We need to avoid blocking JavaScript's runtime event loop for such operations.
We're going to add more asynchronous event-based APIs (such as page.on) that our current synchronous APIs would block.
To align with how developers expect to work with JavaScript APIs.
To have better compatibility with Playwright.

You can find a list of all the APIs that we expect to convert to async in a comment in issue browser#428.

Awaiting on something that’s not a thenable just returns that value, which means you can add the await keyword against APIs that will become async to future proof your test scripts.

New features

Add support for uploading files from the browser module browser#1097, browser#1244

You can now upload files using the available input forms on the website under test. The new API is setInputFiles which can be called from a page, frame or elementHandle types. It can upload one or more files encoded in the test script. To upload files from the local file system, work with the experimental fs module.

For the following examples, we will use the HTML file:

<html>

<body>
    <form method="POST" action="/upload" enctype="multipart/form-data">
        <input type="file" name="upl" id="upload" multiple />
        <input type="submit" value="Send" />
    </form>
</body>

</html>

Uploading a file can be achieved with the following script:

// Import the k6 encoder module.
import encoding from 'k6/encoding';
...
export default async function () {
  const page = browser.newPage();

  await page.goto(url)

  // Encode and upload some data into a plain text file called test.txt.
  page.setInputFiles('input[id="upload"]', { name: 'test.txt', mimetype: 'text/plain', buffer: encoding.b64encode('Hello World') })
  
  // Click on the submit button on the form to upload the file.
  const submitButton = page.locator('input[type="submit"]')
  await Promise.all([page.waitForNavigation(), submitButton.click()])

  page.close();
}

Uploading multiple files can be done with the use of an array:

page.setInputFiles('input[id="upload"]',
    [{ name: 'test.txt', mimetype: 'text/plain', buffer: encoding.b64encode('Hello World') },
    { name: 'test.json', mimetype: 'text/json', buffer: encoding.b64encode('{"message": "Hello World"}') }])

Thanks to @bandorko! 🙇 🎉

Introducing options.cloud #3348, #3407

In this release, we introduce a new way of defining cloud options. From now on, you can use options.cloud instead of options.ext.loadimpact.

To migrate, you can move the loadimpact object to the root of the options object and rename it to cloud. For example:

export let options = {
    ext: {
        loadimpact: {
            name: "Legacy way of defining cloud options",
            projectID: 12345,
        }
    }
};

export let options = {
    cloud: {
        name: "Current way of defining cloud options",
        projectID: 12345,
    }
};

All scripts with legacy options.ext.loadimpact will continue to function as before. There's no planned sunset date for the legacy option, but we highly encourage using options.cloud going forward. For more details about cloud options, refer to Cloud options.

Timers API becomes part of the k6 core #3587

With this release, the timers API is no longer experimental and can be imported as k6/timers instead of as k6/experimental/timers. The later will be supported until v0.52.0.

You can also contribute to the discussion on making the current timer exports globally available in #3589, or just give it a 👍.

JSON Web Key support in `k6/experimental/webcrypto` module webcrypto#61

The experimental webcrypto module now supports the JSON Web Key (JWK) format, using the importKey and exportKey methods.

This allows you to import and export keys in the JWK format for the supported algorithms.

const generatedKey = await crypto.subtle.generateKey({name: "AES-CBC", length: "256"}, true, [ "encrypt", "decrypt"]);

const exportedKey = await crypto.subtle.exportKey("jwk", generatedKey);

UX improvements and enhancements

browser#1197, browser#1202, browser#1203, browser#1221 adds the ability to upload screenshots to a remote location.
browser#1209 adds a shadow DOM usage example.
browser#1233 returns actionable errors for evaluate APIs.
browser#1228, browser#1232, browser#1235 injects the testRunId into the window.k6 object for external applications to query (for example, Grafana Faro).

Browser Context Isolation browser#1112

With this release, we have overhauled and (tremendously) improved the performance and stability of the browser module. It's now possible to run tests with a larger number of VUs concurrently without any performance issues. Previously, when running tests with multiple VUs concurrently, each VU's browser context would attach to the pages from the other VUs' browser contexts. This led to unexpected behavior and performance issues and, to an extent, reduced the module's capability to run multi-VU tests.

Bug fixes

#3653 fixes a connectivity issue with non-lowercase options.hosts.
browser#1215 fixes a data race during logging that panics.
browser#1238 fixes fill functionality for textarea. Thanks @bandorko for the fix! 🙇 🎉
browser#1242 fixes XPath evaluation on DocumentFragment.

Maintenance and internal improvements

browser#1164, browser#1166, browser#1171,
browser#1173, browser#1175, browser#1179,
browser#1183, browser#1186, browser#1188,
browser#1189, browser#1190, browser#1191,
browser#1193, browser#1163, browser#1205,
browser#1217 refactors internals to improve stability.
browser#850, browser#1211, browser#1212,
browser#1214, browser#1216 refactors to work with errors.Join and sets the minimum Go version to 1.20.
browser#1220 adds more logging.
browser#1112 fixes deadlock issues when running multiple VUs, iterations, and Chrome instances.
browser#1246 removes logging of in-flight requests when a request fails.
#3586 fixes file traversal for the test.
#3588 updates codeql GitHub action to v3.

webcrypto#62 fixes display error message in the console and does minor maintenance.
webcrypto#60 leverages some of the k6 APIs to handle JavaScript operations.
webcrypto#59 makes newTestSetup rely on k6's modulestest.
webcrypto#58 addresses linter issues related to repeated static strings.
3633 updates k6 dependencies.

k6 -

Published by github-actions[bot] 9 months ago

k6 v0.49.0 is here 🎉! This release:

Adds a built-in web dashboard that displays test results in real time.
Introduces clear functionality to the browser module's locator classes.
Merges the gRPC experimental module back into the gRPC core module.
Enables the ability to get the selection from an element in k6/html.
Collects internal modules and outputs used by a script.
Prepares k6/experimental/timers for stabilization.

Breaking changes

#3494 stops updating loadimpact/k6 docker image. If you still use it, please migrate to the grafana/k6 image.
browser#1111 removes timeout option for isVisible and isHidden since the API no longer waits for the element to appear on the page.

New features

Web Dashboard

The new web dashboard brings real-time visualization to load testing. This feature allows users to monitor test progress and analyze
results dynamically, enhancing the overall testing experience.

Real-time test results

Activate this feature using the environment variable K6_WEB_DASHBOARD=true. For this initial release, the dashboard is not enabled by default to allow users to opt into this new experience as it evolves.

K6_WEB_DASHBOARD=true k6 run script.js

Once enabled and the test script is running, navigate to http://localhost:5665 in your web browser to access the dashboard.

k6 Web Dashboard Overview

Test report

The web dashboard also offers an HTML test report (see an example) for detailed analysis, enabling easy sharing and downloading capabilities for
collaboration.

To access and download the report, click on the Report button in the dashboard's top right corner or use the K6_WEB_DASHBOARD_EXPORT environment variable.

K6_WEB_DASHBOARD=true K6_WEB_DASHBOARD_EXPORT=test-report.html k6 run script.js

Add `clear` to the `locator` class browser#1149

The new clear method on the locator class clears the text boxes and input fields. This is useful when navigating to a website where the text boxes and input fields already contain a value that needs to be cleared before filling it with a specific value.

import { check } from 'k6';
import { browser } from 'k6/experimental/browser';

export const options = {
  scenarios: {
    ui: {
      executor: 'shared-iterations',
      options: {
        browser: {
            type: 'chromium',
        },
      },
    },
  },
}

export default async function() {
  const context = browser.newContext();
  const page = context.newPage();

  await page.goto('https://test.k6.io/my_messages.php', { waitUntil: 'networkidle' });
  
  // To mimic an input field with existing text.
  page.locator('input[name="login"]').type('admin');

  check(page, {
    'not_empty': p => p.locator('input[name="login"]').inputValue() != '',
  });

  // Clear the text.
  page.locator('input[name="login"]').clear();

  check(page, {
    'empty': p => p.locator('input[name="login"]').inputValue() == '',
  });

  page.close();
}

Add tracing to the browser module browser#1100

The browser module now generates traces that provide a representation of its inner workings, such as API methods executed (for example browser.newPage and page.goto), page navigations, and Web Vitals measurements.

Currently, the instrumented methods are a subset of all the methods exposed by the browser module API, but this will be extended in the future.

The traces generation for the browser module depends on the overall k6 traces option introduced in v0.48.0. Check out the documentation to learn more about it.

gRPC streaming API becomes part of the k6 core #3490

With this release, gRPC's streaming API becomes part of the core's k6/net/grpc module. The experimental k6/experimental/grpc has been back-merged into the core.

You can still use import k6/experimental/grpc for a couple of releases, but it's deprecated and will be removed in the future (planned in k6 version v0.51.0).

To migrate your scripts, replace k6/experimental/grpc with k6/net/grpc in your script imports, and the code should work as before.

k6/html: Extract selection from element #3519

k6/html has been around for a while and allows you to search within an HTML document with a jQuery-like API called Selection, and also has support for the more standard Element that represents DOM element.

For a long time, you could get an Element from a Selection using the .get(index), but you couldn't get back to a Selection from an Element.

This is not a common case, but one that requires quite a bit of code. For example, see the following jQuery snippet:

let li = http.get("https://test.k6.io").html().find("li");
li.each(function(_, element) {
    // here element is an element not a selection
    // but what if for each li we want to select something more?
    // in jquery that will be:
   let container = $(element).closest('ul.header-icons');
    // but what should `$` do?
    // in a browser there is only 1 html document that you have access to
    // in k6 though you can be working with multiple ones, so `$` can't know which one it should
    // work against
});

In order to support the above example, you can use selection, without going to the element:

let li = http.get("https://test.k6.io").html().find("li");
for (; li.size() > 0; li = li.next()) {
    let ul = li.closest('ul.header-icons'); // li here is still a selection and we iterate over it.
}

This is not always possible though, and arguably isn't what most users will naturally do.

Because of this, we have now added a new .selection() which returns a selection for its element.

   let li = http.get("https://test.k6.io").html().find("li");
   li.each(function(_, element) {
      let container = element.selection().closest('ul.header-icons');
        // .. more code
   });

Thanks to @Azhovan! 🙇 🎉

Collect usage data on imported internal modules and outputs #3525

k6 now collects usage data of the modules and outputs that are being used when the usage report is enabled. The data collection is only related to the built-in k6 modules and outputs. Private, custom modules and extensions are never collected. The usage report is enabled by default in k6, but it is possible to opt-out using the no-usage-report option.

We always want to improve the product, but at the same time, we need to pay attention to where we allocate our resources. Having data of what are the most used modules and outputs gives us better confidence to make decisions because we are supported by data.
The data can let us know what percentage of our users will benefit from the introduction of a new feature and also, how many of them would be impacted in case of a breaking change.

UX improvements and enhancements

#3529 enables the k6 cloud traces output by default.
#3440 adds a fallback for using built-in certificates if the OS provides none. Thanks to @mem for working on it!
browser#1104 adds support for browser module traces metadata. Users can define key-value metadata that will be included as attributes in every generated span.
browser#1135 improves the array output from console in the k6 logs.
browser#1137, browser#1145 improves the error messages displayed when Chrome or Chromium isn't found.
#3543 replaces documentation URLs to grafana.com/docs/k6/latest/.

Bug fixes

#3485 fixes the REST API always logging a 200 status code response, which was found as part of fixing lint issues in the code.
browser#1129 mitigates the risk of panics when the website under test uses the console.
browser#1133 fixes BigInt parsing.
browser#1108, browser#1110 fixes isVisible and isHidden so that it doesn't wait for an element to match with the given selector, allowing it to continue on with the test script when elements are not on the page.
browser#1121 fixes dblClick so that it works with onDblClick and performs two clicks on the specified element.
browser#1152 fixes a nil pointer dereference when navigating around on SPA websites.

Maintenance and internal improvements

#3204 internal refactor to make future distributed execution work easier. With a small fix to tests in #3531. Thanks to @na-- 🎉.
Lint fixes throughout the k6 code base #3460, #3462, #3463, #3478, #3479, #3480, #3481, #3482, #3483, #3484, #3485, #3495.
#3473 refinements to the release process.
Dependency updates across k6 #3500, #3501, #3502, #3503, #3509, #3513, #3537, #3539, #3540.
#3489 migrates pull-requests assignment to CODEOWNERS from GitHub Action.
#3496 checks for security issues with a scheduled trivy scan.
#3517 adds unit tests to the cloadapi package. This is the first contribution by external contributor @nilskch. Thanks for this @nilskch 🙇.
#3520 stops using deprecated by golang net.Dialer.DualStack option.
#3526 refactors to JavaScript package test around open and require and their handling of paths.
#3527 generates test certificates for more tests during the test. This, among other things, fixes macOS tests.
#3528 enables macOS tests in GitHub Actions.
browser#1134 adds a new error type when parsing objects.
browser#1107, browser#1109 refactor internals.

Roadmap

As mentioned earlier, there's work in progress to make xk6-timers stable as part of the next release. You can find more information on issue #3297.

k6 - https://github.com/grafana/k6/releases/tag/v0.48.0

Published by github-actions[bot] 11 months ago

k6 v0.48.0 is here 🎉! This release includes:

Numerous long-awaited breaking changes.
A new k6 new subcommand to generate a new test script.
A new k6/experimental/fs module for file interactions.
CPU and network throttling support for the k6 browser module.

Breaking changes

This release includes several breaking changes, mainly cleaning up deprecations from previous versions. They should have a straightforward migration process, and not heavily impact existing users.

#3448 limits metric names, aligning to both OpenTelemetry (OTEL) and Prometheus name requirements, while still being limited to 128 ASCII characters. Warnings about the limit started in v0.45.
#3439 changes the Client signature in k6/experimental/redis module. Refer to the module-related section below.
#3350 removes the grpc.invoke()'s parameter headers, deprecated in k6 v0.37. Use the metadata parameter instead.
#3389 removes the --logformat flag, deprecated in v0.38. Use the --log-format flag instead.
#3390 removes all CSV output's CLI arguments, deprecated in v0.35. This change makes the CSV output consistent with other output formats.
#3365 removes the k6 convert CLI command, deprecated in v0.41. Use the har-to-k6 package instead.
#3451 removes logic that would attempt to prepend a https:// scheme to module specifiers that were not recognized. Deprecated in k6 v0.25. Use full URLs if you want to load remote modules instead.

New features

Add `k6 new` subcommand #3394

k6 now has a new subcommand that generates a new test script. This is useful for new users who want to get started quickly, or for experienced users who want to save time when creating new test scripts. To use the subcommand, open your terminal and type:

k6 new [filename]

If no filename is provided, k6 uses script.js as the default filename. The subcommand will create a new file with the provided name in the current directory, and populate it with a basic test script that can be run with k6 run.

Add a `k6/experimental/fs` module #3165

k6 now has a new k6/experimenal/fs module providing a memory-efficient way to handle file interactions within your test scripts. It currently offers support for opening files, reading their content, seeking through it, and retrieving metadata about them.

Unlike the traditional open function, which loads a file multiple times into memory, the filesystem module reduces memory usage by loading the file as little as possible, and sharing the same memory space between all VUs. This approach significantly reduces the memory footprint of your test script and lets you load and process large files without running out of memory.

For more information, refer to the module documentation.

This example shows the new module usage:

import fs from 'k6/experimental/fs';

// k6 doesn't support async in the init context. We use a top-level async function for `await`.
//
// Each Virtual User gets its own `file` copy.
// So, operations like `seek` or `read` won't impact other VUs.
let file;
(async function () {
  file = await open('bonjour.txt');
})();

export default async function () {
  // About information about the file
  const fileinfo = await file.stat();
  if (fileinfo.name != 'bonjour.txt') {
    throw new Error('Unexpected file name');
  }

  const buffer = new Uint8Array(128);

  let totalBytesRead = 0;
  while (true) {
    // Read into the buffer
    const bytesRead = await file.read(buffer);
    if (bytesRead == null) {
      // EOF
      break;
    }

    // Do something useful with the content of the buffer
    totalBytesRead += bytesRead;

    // If bytesRead is less than the buffer size, we've read the whole file
    if (bytesRead < buffer.byteLength) {
      break;
    }
  }

  // Check that we read the expected number of bytes
  if (totalBytesRead != fileinfo.size) {
    throw new Error('Unexpected number of bytes read');
  }

  // Seek back to the beginning of the file
  await file.seek(0, SeekMode.Start);
}

Redis (m)TLS support and new Client constructor options #3439, xk6-redis/#17

In this release, the k6/experimental/redis module receives several important updates, including breaking changes.

Connection URLs

The Client constructor now supports connection URLs to configure connections to Redis servers or clusters. These URLs can be in the format redis://[[username][:password]@][host][:port][/db-number] for standard connections, or rediss://[[username][]:password@]][host][:port][/db-number] for TLS-secured connections. For more details, refer to the documentation.

Example usage

import redis from 'k6/experimental/redis';

const redisClient = new redis.Client('redis://someusername:somepassword@localhost:6379/0');

Revamped Options object

The Client constructor has been updated with a new Options object format. This change aligns the module with familiar patterns from Node.js and Deno libraries, offering enhanced flexibility and control over Redis connections. For more details, refer to the documentation.

This example shows the usage of the new Options object:

import redis from 'k6/experimental/redis';

const redisClient = new redis.Client({
  socket: {
    host: 'localhost',
    port: 6379,
  },
  username: 'someusername',
  password: 'somepassword',
});

(m)TLS support

The Redis module now includes (m)TLS support, enhancing security for connections. This update also improves support for Redis clusters and sentinel modes (failover). For connections using self-signed certificates, enable k6's insecureSkipTLSVerify option (set to true).

This example shows the configuration of a TLS connection:

import redis from 'k6/experimental/redis';

const redisClient = new redis.Client({
  socket: {
    host: 'localhost',
    port: 6379,
    tls: {
      ca: [open('ca.crt')],
      cert: open('client.crt'), // client certificate
      key: open('client.key'), // client private key
    },
  },
});

Add tracing instrumentation #3445

k6 now supports a new traces output option that allows you to configure the output for traces generated during its execution. This option can be set through the --traces-output argument in the k6 run command or by setting the K6_TRACES_OUTPUT environment variable.

Currently, no traces are generated by k6 itself, but this feature represents the first step towards richer tracing functionalities in k6 and its extensions.

By default traces output is set to none, and currently the only supported output is otel which uses the opentelemetry-go's Open Telemetry API and SDK implementations. The format for the otel traces output configuration is the following:

--traces-output=<endpoint>[,opt1=val1,opt2=val2]

Where opts can be one of the following options:

proto: Specifies the protocol to use in the connection to the traces backend. Supports grpc (default) and http.
header.<header_name>: Specifies an additional header to include in the connection to the traces backend.

Example:

K6_TRACES_OUTPUT=https://traces.k6.io/v1/traces,proto=http,header.Authorization=Bearer token

Add support for browser module's `page.throttleCPU` browser#1095

The browser module now supports throttling the CPU from chrome/chromium's perspective by using the throttleCPU API, which helps emulate slower devices when testing the website's frontend. It requires an argument of type CPUProfile, which includes a rate field that is a slow-down factor, where 1 means no throttling, 2 means 2x slowdown, and so on. For more details, refer to the documentation.

...
  const context = browser.newContext();
  const page = context.newPage();

  try {
    page.throttleCPU({ rate: 4 });
...

Add support for browser module's `page.throttleNetwork` browser#1094

The browser module now supports throttling the characteristics of the network from chrome/chromium's perspective by using the throttleNetwork API, which helps emulate slow network connections when testing the website's frontend. It requires an argument of type NetworkProfile, with a definition of:

export interface NetworkProfile {
    /*
     * Minimum latency from request sent to response headers received (ms).
     */
    latency: number;
    
    /*
     * Maximal aggregated download throughput (bytes/sec). -1 disables download
     * throttling.
     */
    download: number;
    
    /*
     * Maximal aggregated upload throughput (bytes/sec). -1 disables upload
     * throttling.
     */
    upload: number;
}

You can either define your own network profiles or use the ones we have defined by importing networkProfiles from the browser module. For more details, refer to the documentation.

import { browser, networkProfiles } from 'k6/experimental/browser';
...
  const context = browser.newContext();
  const page = context.newPage();

  try {
    page.throttleNetwork(networkProfiles['Slow 3G']);
...

k6's documentation is moving under grafana.com/docs/k6

It's not directly part of the k6 v0.48 release, but we believe it is worth mentioning that we're moving the documentation from k6.io/docs to grafana.com/docs/k6.

The legacy documentation space k6.io/docs will be available for a while, but we encourage you to update your bookmarks and links to the new domain.

UX improvements and enhancements

browser#1074 adds a new browser.closeContext() method to facilitate closing the current active browser context.
#3370 adds a new flag --profiling-enabled which enables exposing pprof profiling endpoints. The profiling endpoints are exposed on the same port as the HTTP REST API under the /debug/pprof/ path. This can be useful for extension developers.
#3442 adds a new --version flag, which has the same output as k6 version command. Thanks, @ffapitalle!
#3423 adds an environment variable K6_INFLUXDB_PROXY to the InfluxDB output which allows specifying proxy. Thanks, @IvanovOleg!
#3398 enables k6 cloud traces by default.
#3400 sets a binary-based cloud output (a.k.a. cloud output v2) as the default version for streaming metrics from a local test run via -o cloud.
#3452 adds fsext.Abs helper function.

Bug fixes

#3380 corrects console.debug(), aligning -v output to --console-output and stdout.
#3416 prints the stack trace when there's an exception in handleSummary().
#3438 prevents an error on HTTP requests with content-encoding header and HTTP statuses known for having no body.
browser#1077 fixes browserContext.clearPermissions to clear permissions without panic.
browser#1042 fixes browserContext.waitForEvent which involved promisifying the waitForEvent API.
browser#1078 fixes request interception deadlock to improve stability.
browser#1101 fixes page.$ so that it returns null when no matches with given selector are found.
#3397, #3427, #3417 update goja dependency. Fixes a possible panic and proper handling circular types at JSON.stringify. Fixes an issue about dumping the correct stack trace when an error is re-thrown.
browser#1106 fixes an NPE on NavigateFrame when navigate occurs in the same document.
browser#1096 fixes a panic when trying to interact within nested iframes. Thanks, @bandorko!

Maintenance and internal improvements

#3378 fixes usage of gh in GitHub actions creating the OSS release.
#3386, #3387, #3388, browser#1047 update dependencies.
#3393, #3399 fix lint issues in the js package.
#3381 disables temporarily ARM tests on GitHub Actions.
#3401, #3469 refactors a Makefile, removes make ci-like-lint in favor of make lint. Updates a golangci-lint version to v1.55.2.
#3410 fixes the tests reference in the all rule of the Makefile. Thanks, @flyck!
#3402 adds a test-case for the k6 cloud.
#3421 updates dependencies for xk6 integration tests.
browser#1075, browser#1076 refactors clearPermissions and grantPermissions.
browser#1043 refines tests.
browser#1069, browser#1090 refactor internal.
browser#1102 uses force and noWaitAfter in frame.newAction.
#3443 mentions that k6-core team aims to support the last two major golang versions for building a k6 binary.
#3437 switches k6 cloud traces to a new hostname.
#3429 increases timeout expectations for the TestSetupTimeout test.
#3446 moves log tokenizer to lib/strvals package.

Roadmap

Graduating from experimental

It has been a while since we've introduced the k6/experimental namespace. This namespace was specifically created to test new features before we fully committed to them. Thanks to it, we have been able to iterate on features and receive valuable feedback from the community before adding them to the core of k6.

In the following releases, we're going to graduate k6/experimental/grpc and k6/experimental/timers.

These modules' "experimental" versions will remain available for a couple of releases, but the goal is to remove the "experimental" imports for them in favor of the core-only imports.

New dashboard features

We're happy to announce our work on a new, upcoming dashboard feature. Based on the xk6-dashboard extension, this upcoming feature will enable you to visualize your test runs and their results in your web browser, in real time. The k6 maintainers team is starting to work towards its integration into the core of k6, and we're aiming to release it in the next couple of releases.

While the final user-experience might differ, you can already try it out by following the instructions in the xk6-dashboard repository. We update the extension on a regular basis as we're converging towards the first release of the feature in k6. Go ahead and give it a try! Let us know what you think about it!

k6 - https://github.com/grafana/k6/releases/tag/v0.47.0

Published by mstoykov about 1 year ago

k6 v0.47.0 is here 🎉! This release includes:

Deprecations

#3347 The built-in statsd output option has been deprecated, and users should use the xk6-output-statsd extension instead. See #2982 for future plans.
#3288 Loading remote modules now requires users to prepend them with https://. Before, k6 would try to resolve importing remote modules by prepending https:// if it was missing. This behavior has been deprecated and will be fully removed in the next release (v0.48.0).

New features

Add gRPC's binary metadata support #3234, xk6-grpc#46

The k6 gRPC modules (k6/net/grpc and k6/experimental/grpc) now support handling binary metadata that uses the -bin postfix, according to the gRPC specification.

let resp = client.invoke("grpc.testing.TestService/EmptyCall", {}, { metadata: { "X-Load-Tester-bin": new Uint8Array([2, 200]) } })

Thanks to @sapphire-janrain for the contribution!

Add gRPC's reflection metadata support #3343, xk6-grpc#46

The k6 gRPC modules (k6/net/grpc and k6/experimental/grpc) now support adding metadata to reflection requests by using a new connection parameter reflectMetadata.

Higher precision for Trend metrics in Grafana Cloud k6 #3302

Grafana Cloud k6 is now able to store and visualize Trend metrics up to 3 digits of precision for decimal numbers.

Docker support for browser-based tests #3199

k6 is now publishig Docker images that include Chromium web browser. This allows k6 users to run tests that use Browser API without having to install Chrome first. Check the "A note on running browser tests" section of the Overview page on DockerHub for details.

Docker images for ARM64 architecture #3320

The k6's release process now builds and pushes dedicated Docker images for ARM64. Check k6's tags page on DockerHub for details.

New authentication methods and HTTP headers API for Prometheus remote write output xk6-output-prometheus-remote#143, xk6-output-prometheus-remote#145, xk6-output-prometheus-remote#147

The experimental Prometheus remote write output now supports two new authentication methods: Bearer token and TLS certificates. Check out the documentation to learn more about how to define them using the new environment variables.
We've also added the K6_PROMETHEUS_RW_HTTP_HEADERS that defines a new and more convenient way to set custom HTTP headers to pass through each flush metrics' request.

Improved the browser module's cookie API

The browser module now provides a more complete and robust API for handling cookies. The cookie API was stabilized by defining a new Cookie class (browser#1008, browser#1030) that can be used while creating and retrieving cookies. This enabled us to add a new browserContext.cookies([urls]) method (browser#1005) that returns all cookies from the current browser context. The new API also supports filtering cookies by URL (browser#1016).

That led to fixing a bug where the expires field was not being set correctly while adding cookies using the context.addCookie() method (browser#1031). Lastly, the existing context.clearCookies() method was fixed to clear all cookies from the current browser context (browser#1040).

const context = browser.newContext();
context.addCookies([
    {name: 'foo', value: 'bar', url: 'https://test.k6.io'},
    {name: 'baz', value: 'qux', url: 'https://grafana.com'},
]);

const cookies = context.cookies('https://test.k6.io');
console.log(cookies.length);         // 1
console.log(cookies[0].name);        // foo
console.log(cookies[0].value);       // bar

context.clearCookies();
console.log(context.cookies.length); // 0

Add support for browser module's `page.on('console')` browser#1006

Allows users to register a handler to be executed every time the console API methods are called from within the page's JavaScript context. The arguments passed into the handler are defined by the ConsoleMessage class.

page.on('console', msg => {
    check(msg, {
        'assertConsoleMessageType': msg => msg.type() == 'log',
        'assertConsoleMessageText': msg => msg.text() == 'this is a console.log message 42',
        'assertConsoleMessageArgs0': msg => msg.args()[0].jsonValue() == 'this is a console.log message',
        'assertConsoleMessageArgs1': msg => msg.args()[1].jsonValue() == 42,
    });
});

page.evaluate(() => console.log('this is a console.log message', 42));

UX improvements and enhancements

#3338, xk6-grpc#48 Adds support for the gRPC reflection protocol v1.
#3290 Adds error logging when executing setup and teardown via REST API. Thanks to @kmtym1998 for the contribution!
#3327 Adds commit identifier for the k6 build when running k6 version.
#3340 Updates k6 *-with-browser Docker images to automatically set the no-sandbox environment variable.
#3335 The character limit for metric names increased from 63 to 128 after the OpenTelemetry update. k6 will return an error starting on the next release (v0.48.0) if users hit the limit.
browser#1007 Adds a k6 object (window.k6 = {};) to help identify k6 browser module tests.
browser#1022 Refactors the check in examples/fillform.js so that it matches the type definitions and documentation for check.

Bug fixes

xk6-grpc#47 Fixes the premature closing of a gRPC stream when a stream's client has finished sending. Thanks to @thiagodpf for reporting!
#3344, xk6-grpc#49 Adds support for Google's protobuf wrappers. Thanks to @zibul444 for reporting!
#3308 Updates goja version, and fixes a compiler bug when a class is declared in a function with an argument.
browser#1039 Fixes goja conversions while adding and retrieving cookies.
browser#1038 Fixes read/write data race for edge case with remote browsers.
browser#1034 Fixes page.reload & page.setContent to use the default navigation timeout over the default timeout.
browser#1033 Fixes the page timeouts so it is actually used after being set.

Maintenance and internal improvements

#3342 Updates xk6-grpc to the latest version. This change brings all the latest fixes and improvements to the experimental gRPC module.
#3271,#3272 Updates the golangci version and adds the interfacebloat linter.
#3279 Fixes the CI not publishing the SBOM file on a new release.
#3283 Updates the Go version in k6's CI used to build the binaries.
#3341, #3339 Updates goja, includes runtime initialization speed-up and a fix for source indexes.
#3311 Updates the alpine image version that is used as the base of the k6 Docker image.
browser#1043, browser#1021, browser#1019, browser#1014 Fixes xk6-browser tests.
browser#1000, browser#1024 Refines xk6-browser issue and PR templates.
browser#1003, browser#1009, browser#1010 Internal changes to xk6-browser.
browser#997 Updates xk6-browser readme.
browser#962 CI fixes.
browser#1035 Refactors int64 timeout to time.Duration, to help avoid confusion as to whether a timeout is in milliseconds or seconds.

Roadmap

Native ECMAScript Modules support

Work on this epic issue has been picked up and there is some progress in the underlying implementation.

One of the main internal changes will be dropping Babel, which is currently used to transpile ESM code to CommonJS.

For users, it will mean better JavaScript support as this change will automatically get object spread working and likely faster startup for big scripts. In the future, this also means JavaScript compatibility will be easier to add, since it'll only need to be supported in the JavaScript VM we use - goja.

There's a risk that some k6 tests using both CommonJS and ECMAScript modules syntax will change in behavior. In practice, using both should never really be done as they're not compatible.

Because there are risks involved, we are going to do this very carefully and only once we have done a lot of tests. That means this is currently planned for v0.49.0, but with the majority of the remaining work done in the v0.48.0 cycle.

This way, we will also be able to have most of our users test these changes from our master branch, using Docker images, for example.

Future breaking changes

There are several changes in the next release that are entering the final stage of their deprecation period. That means the next release will include the following breaking changes:

#3065 Metric names will be limited to 128 characters and a set of allowed symbols. This was going to be changed to a 63 character limit to ensure compatibility with OpenTelemetry standards, but since then, the OTel limit has been updated to 255 characters. You can find more details about the changes in the linked issue.
#3365 The k6 converter command will be removed.
#3350 The headers param for the gRPC module will be removed. Users should use the metadata property instead.
#3287 Loading remote modules will require users to include the https:// protocol in their URLs, otherwise it will return an error.
CSV output no longer accepts arguments in snake case (for example, fileName should be used instead of file_name).
The --logformat flag for defining the Log format option will be removed. --log-format should be used instead.

We recommend checking this list to see if you are impacted, and updating your scripts or CI/CD processes to avoid any issues. If you have any feedback, please open an issue.

k6 - v0.46.0

Published by github-actions[bot] about 1 year ago

k6 v0.46 is here 🎉! This release includes:

xk6-browser extension version bump to v1.0.2, which includes breaking changes.
Send custom headers to Loki log output
Cloud Traces, the new integration between k6 and Tempo
Ability to configure TLS per gRPC connection
Cloud output v2
And many UX improvements, bug fixes, and internal improvements

Breaking Changes

Browser

In this release, the xk6-browser extension version is bumped up to v1.0.2, as it includes multiple breaking changes concerning options, browser lifecycle, and metrics.

Options devtools, env and proxy are deprecated (browser#868, browser#870, browser#872). Additionally, browser launch/connect options are no longer defined in the corresponding JS API methods, instead the test execution related options are now defined inside the browser scenario options (see #3036), and the other more "environmental options", such as headless, debug, executablePath, are set as ENV vars. Also, the slowMo option is no longer supported, although it might be supported again in the future through a different API (browser#876).

Metrics also went through a few changes. The Web Vitals metrics are renamed to use the browser_ prefix and short namings (e.g.: webvital_first_input_delay -> browser_web_vital_fid) (browser#885, browser#903), and the rating metric is removed, as it is now set as a label in the corresponding Web Vitals metrics (browser#915).
The browser HTTP metrics have also been modified, as these are now split from the HTTP module ones, so there are new browser_ prefixed HTTP metrics, specifically for request duration and failed requests (browser#916).

Another big change introduced in this version affects the way the browser lifecycle is handled. Users no longer have to explicitly initialize/close the browser instance through the JS API. Instead, a browser instance will be automatically initialized/closed at the beginning/end of each iteration if the browser type is set in scenario options (see #3036). This also means that the chromium entity from k6/experimental/browser import path is no longer valid. Instead, the browser entity provides access to the browser methods such as browser.newPage() (browser#910, browser#944). This change means that the browser.on() method is no longer applicable, and therefore it has been removed (browser#919).

Additionally, related to import changes, the browser module version is no longer an exported field (browser#923).

Last but not least, this release also includes constraints on browser contexts usage as now only one browserContext is allowed per iteration, which means that the user can create a new browserContext, close it, and then create it again; but can not have more than one "live" browserContext. Instead, scenarios should be used to separate independent actions in a test (browser#929, browser#945).

With all these changes, a simple browser test now looks like:

import { browser } from 'k6/experimental/browser';

export const options = {
  scenarios: {
    ui: {
      executor: 'shared-iterations',
      options: {
        browser: {
          type: 'chromium', // chromium is the only supported browser type so as long as
                            // the option is set, Chromium/Google Chrome will be used
        },
      },
    },
  },
};

export default async function () {
  const page = browser.newPage();
  try {
    await page.goto('https://grafana.com')
  } finally {
    page.close();
  }
}

Deprecations

#3121 deprecates the loadimpact/k6 docker image. We recommend switching to the grafana/k6 image as soon as possible.

New features

Loki log output sending additional headers #3227

k6 has been able to send logs to Loki for nearly 3 years since v0.28.0 but didn't support any way to authenticate.

Now, it can be configured to send additional headers on every request.

This can be done by using the new header config option, similar to label:

k6 --log-output=loki=http://example.org,header.X-My-Header=123,header.Authorization=mytoken ...

The example above will now send the header X-My-Header with the value 123 and the Authorization header with the value mytoken.

Thanks to @federicotdn for adding this feature.

Cloud Traces #3100, #3202

This release supports the new integration between k6 and Tempo in the cloud. Grafana Cloud k6 and Grafana Cloud Tempo customers will be able to start their traces in k6 using the existing k6/experimental/tracing module to enrich their test run analysis page with metrics and aggregations from tracing data.

The new Cloud traces will work "out of the box" for k6 cloud runs. In case of k6 run execution, the K6_CLOUD_TRACES_ENABLED environment variable has to be set to true.

Ability to configure TLS per gRPC connection #3159, xk6-grpc#25

The k6/net/grpc and k6/experimental/grpc modules now support configuring TLS per-connection via ConnectParams. This is useful when connecting to a single gRPC server, using different TLS settings for each VUs, as well as when connecting to multiple gRPC servers from the same VU.

import grpc from "k6/experimental/grpc";
import { check } from "k6";
import { SharedArray } from "k6/data";
import exec from "k6/execution";

// note: the services in this example don't exist. If you would like
// to run this example, make sure to replace the URLs, and
// the cacerts, cert, key, and password variables.
const grpcArgs = new SharedArray("grpc", () => {
    // Using SharedArray here so that not every VU gets a copy of every certificate a key
    return [
        {
            host: "foo1.grpcbin.test.k6.io:9001",
            plaintext: false,
            params: {
                tls: {
                    cacerts: [open("cacerts0.pem")],
                    cert: open("cert0.pem"),
                    key: open("key0.pem"),
                },
            },
        },
        {
            host: "foo2.grpcbin.test.k6.io:9002",
            params: {
                plaintext: false,
                tls: {
                    cacerts: open("cacerts1.pem"),
                    cert: open("cert1.pem"),
                    key: open("key1.pem"),
                    password: "cert1-passphrase",
                },
            },
        },
    ];
});

const client = new grpc.Client();

export default () => {
    if (__ITER === 0) {
        // Take one config and use it for this one VU
        let grpcArg = grpcArgs[exec.vu.idInTest % grpcArgs.length];
        client.connect(grpcArg.host, grpcArg.params);
    }

    const response = client.invoke("hello.HelloService/SayHello", {
        greeting: "Bert",
    });

    check(response, {
        "status is OK": (r) => r && r.status === grpc.StatusOK,
    });

    console.log(JSON.stringify(response.message));
};

Thanks @chrismoran-mica for the contribution 🙇‍♂️.

Cloud Output v2 #3117

After years of great service, we decided to refresh the k6 Cloud output introducing a more efficient end-to-end solution for ingesting the generated tests' metrics. The main change regards the protocol used for flushing metrics that is now a binary-based payload over HTTP.

The new output reduces the resources a load generator uses for tests that produce many metrics. There is no significant difference in the user experience; it's expected to be the same.

The one thing worth highlighting is that the new output is strict about tags, and it'll drops tags if they are reserved. For example:

A custom tag named test_run_id, since that is reserved for internal k6 Cloud operations
Any tag with a key that starts with two underscores (__), that is marked by Prometheus convention as reserved

This is not yet the default Cloud output for the test runs executed from local machines (k6 run -o cloud), but it is expected to be transparently enabled in the upcoming weeks.

The full list of related PRs: #3104, #3108, #3120, #3125, #3162, #3169, #3182, #3186, #3187, #3193, #3195, #3206, #3226, #3157, #3172.

UX improvements and enhancements

#3176 Adds a js/promises package, which enables extension developers to easily create promises that will be dispatched to the eventloop using the New function.
#3181 Adds a RunOnEventLoop method to the modulestest.Runtime type, which allows extensions developers to run code on the event loop from their tests.
xk6-grpc#19 Fixes EOF (end of file) error that was produced when writing to a closed gRPC stream. Additionally, if no error handlers are set up, the error is logged to the console.
browser#881 Adds a new browser_http_req_failed metric.
browser#901 Adds support for shadow DOM piercing in locator. Thanks to @tmc for its implementation.
browser#953 Improves Web Vitals reporting for better accuracy and consistency.

Bug fixes

#3178, xk6-grpc#23 Fixes the registration of nested protobuf messages in the gRPC module. Thanks @thiagodpf for reporting and fixing the bug!
#3194 Fixes loki log output exiting before push is finished.
#3231 Fixes the tracing module sampling option to default to 1.0 when not set by the user.
browser#866 Disables the timeout on browser process execution.
browser#942 Fixes deadlock in frameNavigated.
browser#943 Fixes a race condition in navigation and lifecycle events.
browser#979 Fixes a deadlock on pagehide event evaluation when page.close() is called.
browser#980 Fixes Loki log output for console log serializer.
browser#991 Fixes data directory not being cleared during browser.close() panic.

Maintenance and internal improvements

#3112 Adds an event system for core JS modules.
#3170 Upgraded the version of gRPC k6 demo service dependency to address a security vulnerability ([CVE-2023-32731]).
#3211 Updates the experimental gRPC module version used by k6.
#3210 Updates the xk6-output-prometheus-remote version used by k6.
#3136 Updates k6 Go dependencies.
#3131 Updates our Pull Request template to be more structured and include a checklist for contributors.
#3177 Updates the version of golangci-lint we use to the latest version.
#3163 Fixes our gRPC example.
#3196 Logs packages lint fixes, and test for loki flushing at end.
browser#849, browser#972, browser#977, browser#978, browser#983 Log fixes and improvements.
browser#889, browser#889, browser#902, browser#935, browser#936 Internal refactors.
browser#904, browser#959 Test fixes.
browser#906 Parses the scenarios JSON definition from the K6_INSTANCE_SCENARIOS environment variable.
browser#918 Replaces the usage of os.LookupEnv for k6 LookupEnv and abstracts environment variables lookup.
browser#984 Ensures CDP requests message ID are unique at the connection scope.

k6 - v0.45.1

Published by github-actions[bot] about 1 year ago

k6 v0.45.1 is a patch release that fixes the build process for extensions:

#3252, #3253 Due to the deletion of the go.buf.build registry that is used by the Prometheus remote write output, the building process for extensions has been broken.

There are no functional changes in k6 compared to v0.45.0.

k6 - v0.45.0

Published by github-actions[bot] over 1 year ago

k6 v0.45.0 is here 🎉! This release includes:

Experimental gRPC streaming support.
Update scripts in the cloud without running tests.
JS Metadata API.
A lot of internal changes and bugfixes.

Breaking changes

#3066 k6 will warn users about metric names incompatible with OpenTelemetry or Prometheus. This is planned to become an error with v0.48.0.
browser#851 Remove existing browser namespaced metrics. These have been deprecated in favor of Web Vitals metrics.

New features

Experimental gRPC module with streaming support #3107

There is a new experimental module k6/experimental/grpc. It is a copy of the k6/net/grpc module with added stream support #2020.

This example shows server streaming:

import { Client, Stream } from 'k6/experimental/grpc';
import { sleep } from 'k6';

const COORD_FACTOR = 1e7;
// to run this sample, you need to start the grpc server first.
// to start the grpc server, run the following command in k6 repository's root:
// go run -mod=mod examples/grpc_server/*.go
// (golang should be installed)
const GRPC_ADDR = __ENV.GRPC_ADDR || '127.0.0.1:10000';
const GRPC_PROTO_PATH = __ENV.GRPC_PROTO_PATH || '../../grpc_server/route_guide.proto';

let client = new Client();

client.load([], GRPC_PROTO_PATH);

export default () => {
  client.connect(GRPC_ADDR, { plaintext: true });

  const stream = new Stream(client, 'main.FeatureExplorer/ListFeatures', null);

  stream.on('data', function (feature) {
    console.log(
      'Found feature called "' +
        feature.name +
        '" at ' +
        feature.location.latitude / COORD_FACTOR +
        ', ' +
        feature.location.longitude / COORD_FACTOR
    );
  });

  stream.on('end', function () {
    // The server has finished sending
    client.close();
    console.log('All done');
  });

  stream.on('error', function (e) {
    // An error has occurred and the stream has been closed.
    console.log('Error: ' + JSON.stringify(e));
  });

  // send a message to the server
  stream.write({
    lo: {
      latitude: 400000000,
      longitude: -750000000,
    },
    hi: {
      latitude: 420000000,
      longitude: -730000000,
    },
  });

  sleep(0.5);
};

You can just replace k6/net/grpc import with k6/experimental/grpc to use the new functionality. Documentation for the module is available here.

In the future, this functionality will be moved to the k6/net/grpc module.

You can now only upload a test to the cloud without running it #3030

For years users have wanted to be able to update the test that is saved in the cloud but not run it at this exact point.

This is now possible by adding --upload-only when invoking k6 cloud as in k6 cloud --upload-only script.js.

This is likely going to be most useful in a CI on the actual test script project. Now that CI can just run k6 cloud --upload-only new-version-of-script.js on "release".

And later on that newer version will be used. For example by a scheduled run.

Setting sample metadata API #3037

Support for high-cardinality metrics metadata was added in v0.41.0, but it wasn't accessible from test scripts. It's now possible to set or delete metadata for the whole VU with a similar API as used for tags:

import exec from "k6/execution";

export default () => {
  exec.vu.metrics.metadata["my_cool_id"] = "a very unique value";
  // all metrics from here on will have this metadata set
  delete exec.vu.metrics.metadata["my_cool_id"];
  // all metrics from here on will *not* have the metadata set
}

This also introduces the sub-object metrics on the vu object.
Apart from metadata it has another property tags. This is meant to be the new way to set tags instead of using exec.vu.tags.

There are no current plans to replace exec.vu.tags with exec.vu.metrics.tags.

UX improvements and enhancements

#3099 replace "breached" with "crossed" in logs around thresholds. Thanks to @MattDodsonEnglish 🙇.
#3102 Better error message when SharedArray constructor is provided with an async function. This is not supported, but the original message wasn't very clear.
#3089 Add Software Bill of Materials (SBOM) reports to k6 releases. Thanks to @SadFaceSmith 🙇.
goja#510 JSON.parse will now fail with a friendlier error message.

Bug fixes

browser#852 Fix Locator.WaitFor for detached and hidden states.
browser#859 Fix remote object parsing when subtype is null.

Maintenance and internal improvements

#2991 Refactor JS modules system so that is usable in tests. Which allowed enabling the tc39 tests for modules #3040.
#3025 Internally stop referring to afero and use an internal package to do all file system interaction. That package still uses afero.
#3036 and #3053 Add options to scenarios for usage by browser module.

#3058 fix repetitive the. Thank you, @cuishuang 🙇.

#3064, #3070, #3075 and #3106 Go dependencies updates.
#3067 Add method to retrieve all registered metrics.
#3068 Add metric Sink constructor.
#3078 Pin base Docker builder image to Alpine 3.17. Thank you, @arukiidou 🙇.
#3086 Fix downloading .golangci.yml for PRs from forks.
#3088 Make TestDNSResolver less flaky.
#3094 Fix example from the run command. Thanks to @rocktwotj 🙇.
#3095 Maintenance update of .golangci.yml.
#3103 Fix lint and logical issues in k6/data module tests.
#3045, #3049, #3073 and #3044 New issues are now automatically assigned to maintainers, to improve response time on issues. Both new issue and new PR assignments are now not using external actions.
#3109 Add a way to get the cloudapi Client's base URL. Thanks to @yorugac 🙇.

Roadmap

We're excited to share our public roadmap, outlining the upcoming features and improvements we have planned.

We hope this updated roadmap provides a clear overview of our plans for k6's future development. As always, we welcome feedback, corrections, and suggestions to make this roadmap more comprehensive, accessible, and valuable for the k6 community.

Cloud output v2

Work on a new version of the cloud output has been ongoing over this cycle.

While functionally it is now mostly complete, we feel like more testing is still needed and some smaller issues need to be ironed out.

Over the next cycle we will be testing it internally, and in v0.46.0 it will be generally available as the default Cloud output. It will still be possible to use the current version via an option, but we plan to gradually deprecate it.

The new output has some benefits over the previous one:

Binary (protobuf) format instead of JSON #2963
Samples aggregation for every metric instead of only for HTTP ones #3071
HDR Histogram generation for trend-type metrics #3027

This in general makes the payload sent for tests with a lot of samples much smaller, which also in most cases has turned out to lower the CPU and memory usage.

Other related PRs: #3041, #3061, #3063, #3072, #3082, #3083, #3085, #3098, #3105

k6 - v0.44.1

Published by github-actions[bot] over 1 year ago

k6 v0.44.1 is a patch release that fixes a couple of packaging issues:

#3055 due to an oversight, the k6 package signing key in our RPM repository wasn't updated when its expiration date was extended in March.
#3060 fixed building of Docker image due to a missing pinned ca-certificates version.

There are no functional changes in k6 compared to v0.44.0.

k6 - v0.44.0

Published by github-actions[bot] over 1 year ago

k6 v0.44.0 is here! 🎉 This release includes:

A new k6/experimental/webcrypto module implementing (partially) the Web Crypto API specification.
A sampling option for the experimental tracing module.
Memory usage improvements.
Bug fixes and UX improvements.

Some highlights from the k6/experimental/browser module are:

locator.click is now asynchronous, which is a breaking change.
browserContext.addCookies has now been implemented.
browserType.Connect has been implemented so k6 can now connect to an already running Chrome/Chromium browser instance.
Web vitals are natively supported when working with the browser module.

Breaking changes

The browser module is still in an experimental stage, and therefore breaking changes are expected as we are improving the APIs to make them more user-friendly.

browser#790 Converted locator.click to async to have feature parity with page.click and elementHandle.click. Users must remember to work with promise.All and page.waitForNavigation() when a click action results in navigation.

A locator.click action that doesn't result in navigation can be used like so:
```
const tails = page.locator("input[value='Bet on tails!']");
await tails.click(),
```
A locator.click action that does result in a navigation can be used like so:
```
const tails = page.locator("input[value='Bet on tails!']");
await Promise.all([
  page.waitForNavigation(),
  tails.click(),
]);
```
browser#817 We've removed --no-sandbox from the default Chrome launch arguments. Now Chrome will launch with a sandbox, which is a more secure way of running the browser. If you are running tests under a root user, the browser will no longer launch unless the --no-sandbox argument is supplied. You can still pass this flag when launching a new Chrome instance using the args parameter on chromium.launch:
```
const browser = chromium.launch({
  args: ['no-sandbox'],
});
```
browser#844 Removed the exported version param from the root module. Users should from now on reference the k6 version instead of the browser module version.
browser#838 Removed the first meaningful paint metric. This metric is being deprecated across all the browsers, because the metric's definition relies on browser-specific implementation details, and we've now introduced web vitals in the browser module which is a reliable industry standard way to measure frontend performance. You can find more details here.
browser#843 Removed the build step from Github Actions. From this release onwards, no new standalone browser binaries will be built and available from the releases section. The latest version of the browser module will be available in the k6 binary which can be found in the k6 releases page.

New features

A new `k6/experimental/webcrypto` module implementing the Web Crypto API specification #3007

This release includes a new k6/experimental/webcrypto module partially implementing the Web Crypto API specification in k6.

This example shows encrypting and decrypting of a "Hello, World!" string using AES-CBC algorithm.

import { crypto } from 'k6/experimental/webcrypto';

export default async function () {
  const key = await crypto.subtle.generateKey(
    {
      name: 'AES-CBC',
      length: 256,
    },
    true,
    ['encrypt', 'decrypt']
  );

  const encoded = stringToArrayBuffer('Hello, World!');
  const iv = crypto.getRandomValues(new Uint8Array(16));

  const ciphertext = await crypto.subtle.encrypt(
    {
      name: 'AES-CBC',
      iv: iv,
    },
    key,
    encoded
  );

  const plaintext = await crypto.subtle.decrypt(
    {
      name: 'AES-CBC',
      iv: iv,
    },
    key,
    ciphertext
  );

  console.log(
    'deciphered text == original text: ',
    arrayBufferToHex(plaintext) === arrayBufferToHex(encoded)
  );
}

function arrayBufferToHex(buffer) {
  return [...new Uint8Array(buffer)].map((x) => x.toString(16).padStart(2, '0')).join('');
}

function stringToArrayBuffer(str) {
  var buf = new ArrayBuffer(str.length * 2); // 2 bytes for each char
  var bufView = new Uint16Array(buf);
  for (var i = 0, strLen = str.length; i < strLen; i++) {
    bufView[i] = str.charCodeAt(i);
  }
  return buf;
}

You can see the list of currently supported APIs and algorithms in the project's README. Documentation for the module is available here.

Add sampling capabilities to the experimental tracing module #2886

This release adds sampling capabilities to the tracing module. You can now specify a sampling rate with the sampling option when initializing a Client, or in the tracing.InstrumentHTTP function.

`browserContext.addCookies` browser#760

Cookies can now be added to a BrowserContext and all new Pages created from this context will have the cookie assigned to them. Thanks @zucchinho for implementing this feature!

const context = browser.newContext()
context.addCookies([{name: 'myCookie', value: 'hello world', url: 'https://test.k6.io'}])
const page = context.newPage()

`browserType.Connect` browser#800

There are cases where the user may want to connect to a remote browser instance where they have more control over the browser lifecycle, such as when working in a resource bound environment. This feature enables users to connect to a manually started Chrome/Chromium browser instance. It's a simple case of replacing browser.launch with browser.connect and supplying the CDP url as the first argument. Not all launch options will work with connect since the browser instance should already have started prior to working with connect. Since we assume that the user had decided to take ownership of starting the browser, we have made browser.close a NOOP when working with browser.connect, so the user will need to close the browser themselves.

const browser = chromium.connect('ws://127.0.0.1:1234/devtools/browser/e3bb7e53-ad0f-46f3-ae89-a8416868f4ce')
const page = browser.newPage();

Web Vitals are now natively supported by the browser module browser#836 browser#847

Web vitals are the defacto way for developers to measure their frontend performance using the core metrics:

Largest contentful paint (LCP)
First input delay (FID)
Cumulative layout shift (CLS)

These measurements are now calculated for all tests without any additional work from your side. Simply run your test as you have been doing and you will be presented with the new metrics in the output. This is the output after running the examples/fillform.js script:

webvital_cumulative_layout_shift..........: avg=0        min=0        med=0        max=0        p(90)=0        p(95)=0
webvital_cumulative_layout_shift_good.....: 1       0.323332/s
webvital_first_contentful_paint...........: avg=278.86ms min=141.1ms  med=229.39ms max=466.1ms  p(90)=418.76ms p(95)=442.43ms
webvital_first_contentful_paint_good......: 3       0.969995/s
webvital_first_input_delay................: avg=300µs    min=200µs    med=300µs    max=399.99µs p(90)=379.99µs p(95)=389.99µs
webvital_first_input_delay_good...........: 2       0.646663/s
webvital_interaction_to_next_paint........: avg=16ms     min=16ms     med=16ms     max=16ms     p(90)=16ms     p(95)=16ms
webvital_interaction_to_next_paint_good...: 1       0.323332/s
webvital_largest_content_paint............: avg=303.6ms  min=141.1ms  med=303.6ms  max=466.1ms  p(90)=433.6ms  p(95)=449.85ms
webvital_largest_content_paint_good.......: 2       0.646663/s
webvital_time_to_first_byte...............: avg=205.23ms min=104.79ms med=188.39ms max=322.5ms  p(90)=295.67ms p(95)=309.08ms
webvital_time_to_first_byte_good..........: 3       0.969995/s

You may have noticed other metrics in there too. We rely on the web-vitals JS library which exposes a few more metrics, so we've left them in for you to experiment with. You can find more details on all the browser module metrics in our documentation.

You will no longer see browser_first_contentful_paint in the summary, and instead you can work with webvital_first_contentful_paint.

UX improvements and enhancements

#2906 Added multiple date-time formats for CSV output. Thanks, @Azanul!
#2916 k6 started to show the actual binary's name in the usage help. Thanks, @ariasmn!
#2942 Reference 'k6 cloud' instead of 'Load Impact' in docs and errors.
#2985 Added support of async functions for setup and handleSummary.
#2901 Added a warning when the number of time series exceeds 100 000, which could lead to potential out-of-memory issues.
#2997 Added a new exit code (109), used on a go panic.
browser#788 Updated the browser readme to highlight that it is now a module in k6.
browser#803 Users are now warned if the browser.close method is called more than once.
browser#820 Added error handling to wildcard selectors, which cleans up the error output in the terminal.
browser#848 Multiple k6 instances can now connect to one browser to run concurrent tests. This update empowers high-concurrency browser testing with multiple VUs and instances. Using the new browserType.Connect API, users can now connect to an existing browser instance and execute concurrent tests, which was not possible previously.

Bug fixes

#2984 Fixed wrongly assigned HTTP POST and PUT methods for the tracing.instrumentHTTP. Thanks, @marcin-maciej-seweryn!
#2928 Handled a new behavior of filepath.Join on windows with go1.20, which could cause issues for the k6 archive and k6 cloud commands.
#2915 Fixed check that could return incorrect values for some cases with many preallocated VUs.
#2953 Fixed active VU reporting by arrival-rate executors.
#3006 xk6-websockets updated to v0.2.0 which fixes a lock up of the whole k6.
#3023 Fixed Trend.Max's support of negative values.
browser#781 Fixed mapping of response object's function from jSON to json.
browser#779 Cleared Zombie processes on panic.
browser#834 Fixed page.close so that it closes the current page and not the whole browser context.

Maintenance and internal improvements

Improved the per-VU buffer pool #2879

Improved the per-VU buffer pool which should greatly reduce memory usage, at a minor expense of higher CPU usage and lower request throughput. In some cases, this change can reduce memory usage up to 50%.

Thanks to @davidpst for the contribution!

Other minor changes in this release:

#3004 Changed eventloop.WaitOnRegistered to execute all scheduled callbacks.
#2881 Refactored how modules are loaded. This is a preparation for the upcoming ESM support.
#2920 Updated Go version that we use for k6 compilation to 1.20.x. The docker image is also updated to use alpine:3.17.
#2986 Refactored goja's isNullish to be a part of js/common.
#2960 Refactored sirupsen/logrus usage.
#2999 Directly embed lib.TestPreInitState in js/common.InitEnvironment.
#2892 Added z/OS build flags for IBM z/OS compatibility. Thanks, @msradam!
#2833 Fixed detected memory leaks.
#2931, #2940, #2895, #3002 Updated k6's dependencies. Added goja's generator support.
#2947, #2943, #2946, #3009, #3012, #2894 Tests' fixes, refactoring, and improvements.
#2891, #2921, #2923, #2990, #2995, #3016, #2989 Linters and formatting fixes.
#3005 The samples catalog was renamed to examples.
browser#776 Fixed a test for preset flags for Chrome on macOS.
browser#782, browser#783, browser#826 Fixed and refactored the Go-JS mapping.
browser#797, browser#832 Fixed multi browser close.
browser#796, browser#810 Refactored browserContext.SetExtraHTTPHeaders to work with errors and ErrFatal.
browser#798, browser#799 Added more tests for the Go-JS mapping layer.
browser#802 Added a helper to assert on the logs and dump the logs for easier debugging.
browser#807 Fixed incorrect keyboard key code on up/down key presses.
browser#819 Browser.Launch now transitions to Browser.Connect when a CDP URL is provided in an environment variable.
browser#821, browser#824, browser#830 Upgraded dependencies and fixed breaking changes.

k6 - v0.43.1

Published by github-actions[bot] over 1 year ago

k6 v0.43.1 is a patch release containing a few bugfixes:

#2926 fixed a panic in setup() code when vu.iterationInScenario from k6/execution was used.
#2934 fixed a wrongly printed internal output ID to the stdout UI.
#2938 fixed a synchronization bug that caused k6 to get stuck after the end-of-test summary when sending the usage report took more than 3s. Thanks for reporting this, @ichasepucks!

k6 - v0.43.0

Published by github-actions[bot] over 1 year ago

k6 v0.43.0 is here! 🎉

Notable changes in this release include:

xk6-browser is now bundled in k6 as an experimental module, and usable without a separate binary or compilation step!
Native support for JavaScript's async/await.
A new experimental module for distributed tracing.
Large refactoring of core components to simplify the code base, improve maintainability, and increase test coverage.
Bug fixes, UX improvements, and maintenance.

Keep reading for the details.

Breaking changes

#2807 Use non-zero exit codes for tests aborted by Ctrl+C or the REST API.

Aborting a test run with Ctrl+C will now exit with code 105, and stopping via the REST API will exit with code 103.

New Features

xk6-browser is now a built-in module #2884

This release includes xk6-browser as an experimental module. This means you can now also use the main k6 binary for browser automation and end-to-end testing, without needing to build a custom binary with xk6.

All xk6-browser scripts that work with v0.8.0 will continue to work with the built-in module in k6 v0.43.0. To use them, change the import path from k6/x/browser to k6/experimental/browser, and set the environment variable K6_BROWSER_ENABLED to true. The requirement to specify the environment variable is temporary and may be removed in a future k6 release. It was added to minimize the risks with k6 unexpectedly launching a browser (or another process) from k6 scripts. It's also a mechanism we use in the k6 Cloud, where browser tests are currently disabled.

For details, review the script example, or the updated browser module documentation.

The module is currently under the experimental namespace, which means we reserve the decision to introduce breaking changes in the future. However, our mid-term goal is to drop the experimental label and make browser support a stable part of the k6 feature set, eventually enabling it in k6 Cloud as well.

Native support for JavaScript's `async`/`await` #2830

In v0.35.0 we added support for asynchronous functionality in k6 scripts with the addition of Promise.

While useful, the experience wasn't very friendly. Scripts had to use the .then() API to chain Promises, instead of the await syntax available in most other JavaScript runtimes, and the async keyword wasn't supported. Some workarounds were possible, but it required a separate build pipeline to transpile the syntax into the older ES5.1+ standard supported by k6.

That is, until now! 🎉 With invaluable help from @dop251, who maintains goja, the JS VM k6 uses, v0.43.0 brings native async/await to your k6 scripts. This functionality works just as you'd expect in other JS runtimes, and makes working with async APIs much more convenient. For details, review the following http.asyncRequest() example.

One caveat to note: async functions can't be passed to group() or check(). These functions are incompatible with asynchronous behavior, so you will get an error if trying to do so.

Experimental JavaScript module for distributed tracing #2853 #2854 #2855

This release brings a new experimental JavaScript module that adds distributed tracing support to k6. With one call in init context, you can instrument your test script's HTTP requests. If the system you're testing is instrumented in the same way, this module brings visibility to SUT behavior for the lifetime of each request.

An example:

import tracing from 'k6/experimental/tracing';
import http from 'k6/http';

tracing.instrumentHTTP({
  propagator: 'w3c',
});

export default () => {
  http.get('https://httpbin.test.k6.io/get', {
    headers: {
      'X-Example-Header': 'instrumented/get',
    },
  });
};

For details and examples, refer to the tracing module documentation.

`http.asyncRequest` #2877

The k6/http module has a new asyncRequest function that takes the same arguments as http.request(), but returns a Promise that, when used with await, will be resolved with a Response object. This gives you more control over script execution, as potentially the most time-consuming calls—making HTTP requests—will no longer block the thread of execution.

An example issuing a POST request:

import http from 'k6/http';

export default async function () {
  const resPromise = http.asyncRequest(
    'POST', 'https://httpbin.test.k6.io/post', { name: 'Bert' });
  // Do something else here, make other requests, etc.
  // Then when you're ready to use the response:
  const resp = await resPromise;
  console.log(resp.json().form.name); // Bert
}

This is one of the first steps towards migrating our APIs to be asynchronous, and similar changes can be expected in the future.

You can read more about asyncRequest in the documentation.

Enhancements and UX improvements

#2754, #2805 The output of the k6 version command has been enhanced to also show the version of all extensions built into the k6 binary produced by xk6. Thanks, @HarrisChu!
#2800 Improved handling of the Ctrl+C signal to gracefully abort the test during VU initialization.
#2803 Ensure the REST API server is shut down after the test ends.
#2867 Added the ability to display test run details and logs from k6 Cloud.
#2890 Added a method for JavaScript modules to lookup environment variables without directly accessing the os package.
#2910 Added a bit more context when parsing the script options, so that it is more obvious what fails.

Bug fixes

#2829 The csv output now correctly shows vu and iter system tags. This fixes a regression introduced in v0.40.0. Thanks, @leonyork!
#2851 Calling k6/execution.test.abort() within a group() now correctly exits the k6 process with code 108. Thanks for reporting this, @pomeh!
#2896 Fixed a panic in k6/ws when using Socket.setInterval() with values between 0 and 1.
#2903 Fixed a regression introduced in v0.42.0 where k6 will load the wrong module if the same import specifier has already been loaded, but they are pointing to different absolute paths, based on the files they are imported from.
#2907 Fixed the exit code and run_status value when the cloud output aborts a test.

Maintenance and internal improvements

#2809, #2810, #2812, #2813, #2815, #2885, #2893 A core component of test execution, the Engine, was removed, and the behavior heavily refactored. This change simplifies the code base and unblocks further improvements.
#2821, #2864 Our high-level integration test suite was refactored and expanded, increasing the coverage of behavior that closely replicates real-world usage.
#2803 Enabled checking for goroutine leaks.
#2883 The goja runtime has been updated.
#2845 Lint fixes in the k6/http module.
#2831 Compatibility with TC39 error messages was improved.
#2846 The initialization of js.Bundle was simplified.
#2861, #2870 Some deprecated features and dependencies in our CI pipeline were removed.
#2882 Our Dockerfile was improved with some linter suggestions. Thanks, @kempsterc!

Full Changelog: https://github.com/grafana/k6/compare/v0.42.0...v0.43.0

k6 - v0.42.0

Published by github-actions[bot] almost 2 years ago

k6 v0.42.0 is here! 🎉

This release includes:

A tiny breaking change to improve WebSockets response handling.
A new experimental output.
More features in our experimental WebSocket module.
Wildcard support for hosts.
Some bug fixes, UX improvements, and maintenance.

Breaking changes

#2712 k6/ws returns an HTTP response for failed connections instead of an undefined behavior. Thanks, @brietaylor.

New Features

Experimental Prometheus Remote Write Output #2784

This release brings a new builtin Output to any Prometheus Remote Write implementations (e.g. Prometheus, Mimir). This is an experimental feature, and future releases could introduce breaking changes.

The following example uses k6 run with the new output. It uses the defaults options, such as the Remote Write server URL (http://localhost:9090/api/v1/write):

k6 run -o experimental-prometheus-rw script.js

It supports the new and convenient experimental Native Histogram feature, added in Prometheus v2.40.0. It's not enabled by default, but we expect to make it the default way to map k6 Trend metrics once the Prometheus project signals that its mature enough and when more Remote Write implementations support it. For now, if you want to use it, you need to set the environment variable K6_PROMETHEUS_RW_TREND_AS_NATIVE_HISTOGRAM to true.

You can find complete documentation with more examples, use cases, and available configurations.

More features for the experimental websockets module #2786

The k6/experimental/websockets module that we announced in the v0.40.0 release got an update that extends its functionality.

It brings some useful features that the k6/ws module already has, like cookies, custom headers, compression and tags customization support, the syntax to define event handlers (onopen, onmessage, etc.) and ping/pong functionality.

This is still an experimental module, but with the recent changes we think it's usable for most users. So whether you're writing a new WebSocket test, or currently using the k6/ws module, we invite you to give it a try, and report any issues in the project's issue tracker.

This example customizes tags for a WebSocket connection, sets up handlers using the new on* syntax, and demonstrates the ping/pong feature.

import { WebSocket } from "k6/experimental/websockets";
import {
  setTimeout,
  clearTimeout,
  setInterval,
  clearInterval
} from "k6/experimental/timers";

const CLOSED_STATE = 3

export default function () {
  const params = {
    "tags": {
      "my_tag": "hello"
    }
  };

  const ws = new WebSocket('ws://localhost:10000', null, params);

  ws.onopen = () => {
    console.log('connected');
    ws.send(Date.now().toString());
  };

  let intervalId = setInterval(() => {
    ws.ping();
    console.log("Pinging every 1 sec (setInterval test)");
  }, 1000);

  let timeout1id = setTimeout(function () {
    console.log('2 seconds passed, closing the socket');
    clearInterval(intervalId);
    ws.close();
  }, 2000);

  ws.onclose = () => {
    clearTimeout(timeout1id);
    console.log('disconnected');
  };

  ws.onping = () => {
    console.log("PING!");
  };

  ws.onpong = () => {
    console.log("PONG!");
  };

  // Multiple event handlers on the same event
  ws.addEventListener("pong", () => {
    console.log("OTHER PONG!");
  });

  ws.onmessage = (m) => {
    let parsed = parseInt(m.data, 10)
    if (Number.isNaN(parsed)) {
      console.log('Not a number received: ', m.data);
      return
    }

    console.log(`Roundtrip time: ${Date.now() - parsed} ms`);

    let timeoutId = setTimeout(function () {
      if (ws.readyState == CLOSED_STATE) {
        console.log("Socket closed, not sending anything");

        clearTimeout(timeoutId);
        return;
      }

      ws.send(Date.now().toString());
    }, 500);
  };

  ws.onerror = (e) => {
    if (e.error != "websocket: close sent") {
      console.log('An unexpected error occurred: ', e.error);
    }
  };
};

The module docs has a complete reference, and some examples.

Wildcard support for `hosts` option #2747

Thanks to the great effort from @eugercek, the hosts option now accepts domains that contain a wildcard at the beginning.
It can be helpful for setting multiple subdomains of the same domain, so instead of setting subdomain1.k6.io': '1.2.3.4', 'subdomain2.k6.io': '1.2.3.4' it is possible to use the wildcard for setting directly *.k6.io: '1.2.3.4'.

export const options = {
  hosts: {
    '*.k6.io': '1.2.3.4',
  },
}

Enhancements and UX improvements

#2660 Pre-loads the operating system TLS certificates. Thanks, @tbourrely.
#2791 Initializes VUs for setup() and teardown() only if they are defined in the script.

Bug fixes

#2759 Ensures the evaluation of thresholds over trend metrics' median.
#2759 Fixes a few potential Output data races for interrupted test runs.
#2767 Fixes the emission of ws_session_duration when setup throws an error.
#2773 Ensures that JavaScript runtime makes only one copy of the exports for each module including built-in ones.

Maintenance and internal improvements

We had a few minor changes in this release:

#2757 goja runtime has been updated.
#2768 WS.Connect() has been refactored.
#2770 Refactored parts of the js module.
#2782 Covered more relative path test cases for require and open.
#2789, #2792, #2795, #2796 Improved stability of the integration tests.
#2791 Optimized the performance of the internal Trie implementation. Thanks, @eugercek.

Full Changelog: https://github.com/grafana/k6/compare/v0.41.0...v0.42.0

k6 - v0.41.0

Published by github-actions[bot] almost 2 years ago

k6 v0.41.0 is here! 🎉 It has relatively few user-facing changes, but includes massive internal improvements that pave the way for some awesome features for the near future. Unfortunately, this work also required us to make a few minor breaking changes.

Breaking changes

Changes in the `url`, `iter` and `vu` system metric tags

As we warned in the release notes for k6 v0.39.0 and v0.40.0, we've been fundamentally refactoring the metrics sub-systems of k6. We now have efficient support for time series, which required a few minor user-facing breaking changes:

If URL grouping is used for HTTP requests (that is, if the http.url helper is used or the name metric tag is specified), then the url tag in the resulting http_req_* metric samples will also have the same value as the name tag. Previously, k6 did this only for the cloud output, but now it does this universally (#2703).
The vu and iter system tags, which are disabled by default, have been transformed into high-cardinality metrics metadata instead. It means that they will no longer be usable in thresholds, and various outputs may emit them differently or ignore them completely (#2726).

Changes in the Go metrics APIs

While the user-facing changes from our metrics refactoring are few and relatively minor, and there are no changes to JavaScript APIs yet, we have extensively refactored our internal Go APIs (#2594, #2726, #2727). The metrics.Sample and metrics.TagSet types are now entirely different. We also have high-cardinality metadata attributes in each Sample and at the VU level (see the combined TagsAndMeta code and how it is used in the per-VU State object).

`k6 convert` is officially deprecated (#2714)

k6 convert has been a sub-command to convert a HAR file recording of HTTP traffic into a preliminary k6 script that makes roughly the same requests. It has been long neglected and softly deprecated in favor of the newer and more feature-rich har-to-k6 standalone converter.

We have now officially deprecated k6 convert. The command still works and will continue to do so for a few more k6 versions. However, it's not visible from k6 --help and will emit a warning when used. Please see the documentation for the standalone har-to-k6 converter and open an issue (or comment on an existing one) if you have any problems with it.

New Features, enhancements, and UX improvements

#2679 added support for maxReceiveSize and maxSendSize parameters in the gRPC's Client.connect() method. Thanks, @ariasmn!
#2605 introduced a new --exclude-env-vars CLI flag to k6 archive that causes it to not include the provided environment variables in the resulting archive bundle's metadata.json file.
#2700 added support for loading gRPC protoset files. Thanks, @jklipp!

Bug fixes

#2678 fixed the Docker image labels. Thanks, @knittl, for reporting the problem (#2677)!
#2689 fixed the REST API's Content-Type response header. Thanks, @wingyplus!
#2691 fixed the detailed k6 version information embedded in the k6 releases.
#2693 fixed a bug that made the k6 event loop unusable when a Promise rejection was left unhandled.
#2696 fixed a problem with HTTP redirects with empty Location headers (#2474) by updating the Go version we use to compile k6 to 1.19.x. Thanks, @agilob!
#2705 fixed a panic in the k6/net/grpc module (#2661). Thanks, @c47gao and @robpickerill!
#2738 fixed a panic when a Promise was rejected with an undefined reason.
#2739 fixed hidden stack traces in certain types of errors originating from k6's Go code.

Maintenance and internal improvements

We had a few minor changes in this release:

#2687 improved our logging tests. Thanks, @nsmith5!
#2696 updated the used Go version to 1.19.x and the Alpine version in our Docker image to 3.16. Thanks, @agilob!
#2707, #2708, #2709, #2710 updated most of the Go dependencies k6 has.
#2716 refactored how custom JS tags are applied to metrics and cleaned up validation for invalid tag values.

We also have a couple of significant improvements that will help us develop exciting new features soon:

Metric time series (#2594)

Previous to #2594, k6 didn't have an efficient way to group metric samples with the same tags. It meant that a whole class of applications for processing and aggregating metrics were nearly impossible to do or, at best, very inefficient.

At the cost of some minor breaking changes, we now have a performant internal representation to group metric samples with the same tags at the time of the action that generated them, i.e. the time of metric measurement. With this, k6 can efficiently group samples for the same action (e.g. an HTTP request to a specific URL) over time and construct time series with them.

Internal support for high-cardinality metrics metadata (#2726, #2727)

As described in the previous section, the efficient grouping of metric samples into time series works well for relatively low-cardinality data. However, k6 needed some way to attach high-cardinality metadata as well. This is necessary for data that's unique or random, such as Trace and Span IDs in distributed tracing or user IDs in tests with huge data sets.

k6 v0.41.0 has added support for attaching high-cardinality metadata to metric samples, and the vu and iter system tags have been transformed into such metadata (see the breaking changes section above), but it is not yet accessible from user scripts. There is no JavaScript API to modify this metadata, only built-in k6 Go modules and xk6 Go extensions can make use of it, for now.

k6 - v0.40.0

Published by github-actions[bot] about 2 years ago

k6 v0.40.0 is here! This release includes:

Breaking changes to some undocumented and unintentional edge behaviors.
New experimental modules and first-class support for JavaScript classes.
Bugs and refactorings to pave the way for future features.

Finally, the Roadmap goes over the plans for the next cycles.

Breaking changes

#2632 During the refactoring to set tags to metric.add in the order they are provided, we discovered that you could provide tags as a key-value pair map multiple times in the same call. This was never the intended use and was never documented. As it was undocumented, and as such functionality makes no sense alongside every other API k6 has, we decided to remove this ability.
#2582 [For extensions using the event loop] Previously, if RegisterCallback result was called twice, the second call would silently break the event loop. This has never been expected behavior, and calling it twice is always a bug in the code using it. Now, calling the RegisterCallback result twice leads to a panic.
#2596 The tainted property of the Metric type is no longer outputted by the JSON output. That property was likely always going to have a false value as it was outputted at the beginning of the test.

Main module/script no longer pollutes the global scope #2571

During the ESM changes, we found that anything defined in the main module scope was also accessible globally. This was because it was directly evaluated in the global scope.
This has now been remedied and is no longer the case. This is a breaking change, but given that the whole point of modules (CommonJS or ESM) is to separate them, this is obviously rather a bug than a feature.

On that note, we've seen reports by people who have this global accessibility of the main module (intentionally or not). Still, it seems relatively rare, with only a few usages in a script. So if you need to access something globally, our suggested workaround is to set it explicitly on the global object globalThis.

`k6/ws` now respects the `throw` option #2247

k6/http has used the throw option to figure out whether it should throw an exception on errors or return a response object with an error set on it (and log it).

This functionality is finally also available for k6/ws, which previously would've always thrown an exception and thus involved more scripting in handling it (#2616).

This is a minor breaking change. By default, throw is false, so it now no longer throws an exception but instead returns a Response with error property.

Thank you, @fatelei, for making this change!

New Features

Experimental modules #2630 and #2656

As mentioned in the v0.39.0 release notes, we're happy to announce that this release brings experimental modules. The main idea behind this initiative is to get community feedback earlier, which will help us improve them faster. We encourage you to try experimental modules out and provide feedback through the community forums or GitHub issues.

This release contains three experimental modules:

k6/experimental/redis - support for interaction with Redis
k6/experimental/websockets - a new Websockets API that copies the "web" WebSocket API
k6/experimental/timers - adding setTimeout/clearTimeout and setInterval/clearInterval implementations.

Important to highlight that the k6 team does not guarantee backward compatibility for these modules and may change or remove them altogether. Also, their import paths, starting with k6/experimental, will break when the modules stop being experimental. Of course, we are going to try to limit those breaking changes to a minimum and, when possible, do them in a backward compatible way for at least a version.

Here is a fairly big example using xk6-redis as an experimental module to keep track of data in Redis:

import { check } from "k6";
import http from "k6/http";
import redis from "k6/experimental/redis"; // this will be `k6/x/redis` if you are using it as extension
import { textSummary } from "https://jslib.k6.io/k6-summary/0.0.1/index.js";

export const options = {
  scenarios: {
    usingRedisData: {
      executor: "shared-iterations",
      vus: 10,
      iterations: 200,
      exec: "measureUsingRedisData",
    },
  },
};

// Instantiate a new redis client
const redisClient = new redis.Client({
  addrs: __ENV.REDIS_ADDRS.split(",") || new Array("localhost:6379"), // in the form of "host:port", separated by commas
  password: __ENV.REDIS_PASSWORD || "",
});

// Prepare an array of crocodile ids for later use
// in the context of the measureUsingRedisData function.
const crocodileIDs = new Array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9);

export function setup() {
  redisClient.sadd("crocodile_ids", ...crocodileIDs);
}

export function measureUsingRedisData() {
  // Pick a random crocodile id from the dedicated redis set,
  // we have filled in setup().
  redisClient
    .srandmember("crocodile_ids")
    .then((randomID) => {
      const url = `https://test-api.k6.io/public/crocodiles/${randomID}`;
      const res = http.get(url);

      check(res, {
        "status is 200": (r) => r.status === 200,
        "content-type is application/json": (r) =>
          r.headers["Content-Type"] === "application/json",
      });

      return url;
    })
    .then((url) => redisClient.hincrby("k6_crocodile_fetched", url, 1));
}

export function teardown() {
  redisClient.del("crocodile_ids");
}

export function handleSummary(data) {
  redisClient
    .hgetall("k6_crocodile_fetched")
    .then((fetched) => Object.assign(data, { k6_crocodile_fetched: fetched }))
    .then((data) =>
      redisClient.set(`k6_report_${Date.now()}`, JSON.stringify(data))
    )
    .then(() => redisClient.del("k6_crocodile_fetched"));

  return {
    stdout: textSummary(data, { indent: "  ", enableColors: true }),
  };
}

This example also showcases how to write some data and clean up after yourself.

The extension does not run a Redis server. You need to separately handle running, scaling, and connecting infrastructure to Redis.

The xk6-redis repository has more examples, and the module is documented in the official k6 documentation.

This is a rewrite of the current WebSocket example at https://test-api.k6.io/.

This showcases how a single VU can run multiple WebSockets connections asynchronously and how to stop them after a period using the timeout and interval functions.

import { randomString, randomIntBetween } from "https://jslib.k6.io/k6-utils/1.1.0/index.js";
import { WebSocket } from "k6/experimental/websockets"
import { setTimeout, clearTimeout, setInterval, clearInterval } from "k6/experimental/timers"

let chatRoomName = 'publicRoom'; // choose your chat room name
let sessionDuration = randomIntBetween(5000, 60000); // user session between 5s and 1m


export default function() {
  for (let i = 0; i < 4; i++) {
    startWSWorker(i)
  }
}

function startWSWorker(id) {
  let url = `wss://test-api.k6.io/ws/crocochat/${chatRoomName}/`;
  let ws = new WebSocket(url);
  ws.addEventListener("open", () => {
    ws.send(JSON.stringify({ 'event': 'SET_NAME', 'new_name': `Croc ${__VU}:${id}` }));

    ws.addEventListener("message", (e) => {
      let msg = JSON.parse(e.data);
      if (msg.event === 'CHAT_MSG') {
        console.log(`VU ${__VU}:${id} received: ${msg.user} says: ${msg.message}`)
      }
      else if (msg.event === 'ERROR') {
        console.error(`VU ${__VU}:${id} received:: ${msg.message}`)
      }
      else {
        console.log(`VU ${__VU}:${id} received unhandled message: ${msg.message}`)
      }
    })


    let intervalId = setInterval(() => {
      ws.send(JSON.stringify({ 'event': 'SAY', 'message': `I'm saying ${randomString(5)}` }));
    }, randomIntBetween(2000, 8000)); // say something every 2-8seconds


    let timeout1id = setTimeout(function() {
      clearInterval(intervalId)
      console.log(`VU ${__VU}:${id}: ${sessionDuration}ms passed, leaving the chat`);
      ws.send(JSON.stringify({ 'event': 'LEAVE' }));
    }, sessionDuration);

    let timeout2id = setTimeout(function() {
      console.log(`Closing the socket forcefully 3s after graceful LEAVE`);
      ws.close();
    }, sessionDuration + 3000);

    ws.addEventListener("close", () => {
      clearTimeout(timeout1id);
      clearTimeout(timeout2id);
      console.log(`VU ${__VU}:${id}: disconnected`);
    })
  });
}

Note that no k6 iterations finish if any WebSocket is still open or if a timeout or an interval is not cleared or triggered. This means that your script must take care of clearing all intervals and closing the WebSocket at some point. However, k6 still kills the whole process if it takes too long to stop after the maximum test duration is reached.

Current issues and future improvements for the WebSockets API can be found in its issue tracker. Currently, documentation is available through MDN, though some features are not yet supported:

no Blob binary type - ArrayBuffer is the default
no onMessage and co. - only addEventListener

First-class support for JavaScript Classes

As part of updating goja, k6 got native support for classes. Again, that's native, as in not by transpiling by the internal Babel.

Because this actually implements classes as described in the latest ECMAScript specification, this also means we get a ton of additional class features that were never previously supported (for example, private fields). Additionally, at least one bug #1763 was fixed as a result of this, but probably many more as well.

Due to this fairly significant change, some code could behave differently. Please report any issues, though consider that it's possible that the new behavior is just the correct one.

Other updates from goja are:

optimizations around using strings and some access patterns
support for \u{01234} Unicode point syntax in regexp
Fixed a case where interrupting the VM did not work, especially around try/catch usage (#2600). This was particularly problematic for k6, as it could lead to k6 hanging.

Many thanks to @dop251 for continuing to improve goja!

New Test runtime for module extension developers #2598

While we develop extensions internally, we often need to repeatedly create the same structure. With the addition of the event loops, it is now required to set it up as well. Even k6 team members get parts of this wrong every once in a while, so we added a small type to be used by (extension) module developers to write tests easier (#2602).

This API will likely change and evolve as we add more functionality or as we change the k6 internal API.

Bug fixes

#2585 http.batch() now displays an error if it is not given exactly 1 argument(#1289). Thanks, @vanshaj!
#2596 Fixes a potential data race in the JSON output. Includes a breaking change where tainted property is no longer outputted. That property was (likely) always going to have the value false as it was outputted at the beginning of the test.
#2604 Fixes SSL keylogger not working with absolute paths.
#2637 Fixes setting the options rps to 0 or below leading to exceptions. Now setting it to 0 or below disables the limit. Thanks, @tbourrely. #2613
#2278 Reading options.tags directly was not possible. This was fixed by accident by #2631. k6/execution is still the recommended way to access the final options of the test.

Maintenance and internal improvements

#2590 Updates direct dependencies without any interesting changes apart goja.
#2591 Changes to the CI process to always build rpm/deb and windows packages and use nfpm to do it.
#2593 Internal cleanup after finally removing common.Bind.
#2597 Fixes go benchmarks we have broken over time.
#2599 Reformats //nolint comments as part of getting ready for go 1.19.
A bunch of fixes for tests #2589, #2620, #2625, #2643, #2647, #2648,
#2607 Fixes the build badge in the README. Thanks @AetherUnbound!
#2614 Fixes advice for RPM install on Amazon Linux.
#2615 Improves documentation of the RegisterCallback, following feedback on how hard it was to understand.
#2627 Create distinct test state objects for the pre-init and run phases.
#2635 Drop License header in each file.
#2636 Add SECURITY.md with instructions how to report security issues responsibly.
#2641 Fix spelling of lose. Thanks @spazm!
Update to golangci-lint v1.47.2 and enable a bunch more linters. #2609, #2611. Also, drop obsolete configurations #2619.

Roadmap and future plans

This section discusses our plans for future versions. Notice that two big ticket items are here again―ESM modules and metric refactoring. They remain on the roadmap mostly for the sheer size of the work required on both, but also for some unforeseen needed changes, which actually should make them better in the long run. It also so happens that it is vacation season so the k6 team rightfully is taking some personal time away.

Native support for ECMAScript modules

Native ESM support is coming. A PR to k6 is under development, and there's a branch that will become a PR to goja to add the support there. The k6 team is hopeful that this will land in the next version, v0.41.0!

It turned out that there were a lot more things to be added as functionality - dynamic import, and the tc39 group released the latest ECMAScript specification adding support of top-level await in modules. While neither k6 nor goja has support for the async/await syntax, yet, this changes significantly the internal of the change, which did require a not insignificant refactor.

Additionally, as previously mentioned, there were a bunch of changes to goja, including adding class support which also needed to be integrated.

A future breaking change is that using CommonJS style exports along import/export syntax in the same file will no longer be possible.

import http from "k6/http"

exports.default = function() {} // this will start to error out

It will still be possible to import a CommonJS module and require an ES module or use require in an ES module. Having a dependency cycle mixing and matching CommonJS and ESM is also unlikely to work properly, but might do so in particular cases.

This is really expected to have never been done by anyone as there isn't really much of a reason for this, but it is currently supported due to Babel transpiling everything to CommonJS behind the scenes.

Refactoring metrics

The refactoring of metrics is underway, with a PR for more performant internal representation. Unfortunately, this took longer to get to a stable state, and given the size and the type of change, the k6 team decided to hold merging it until very near the end of the cycle. It also doesn't have any noticeable change for most users. Instead, it will be merged early in the v0.41.0 release cycle, and then more changes to use it will be made through the release cycle.

Some of those changes include supporting non-indexable tags, which also have a read PR. This change is also essential for historical reasons connected with how the name tag works. As such, it also needed to be merged to release the internal metric refactor.

Future breaking change: as part of the many changes and experiments, we found out that we can keep url as an indexable tag. Previously the plan was that it along vu and iter will become non-indexable. However, the url tag is heavily used and enabled by default. Because of that (and other internal reasons), the new change will be that url will stay indexable, but if used with name will be overwritten to have the same value as name. Arguably this is what most users would want in the case when they are using name. We plan to add a non indexable raw_url tag for those that do not. As such, we no longer will be printing a warning when url is used in thresholds.

Even so, we did make a bunch of changes to the internals of k6 that will pave the way forward (#2629, #2631).

We are also still working on incorporating the newly developed time series data model for the Prometheus remote-write output extension. We are fixing bugs and improving the extension with the goal of eventually integrating it as a core built-in k6 output module in a future k6 release.

Distributed tracing support itself needs non-indexable tag support. Once that is merged, more work in that direction will be started.

k6 - v0.39.0

Published by github-actions[bot] over 2 years ago

k6 v0.39.0 is here! 🎉 It's a small release that includes a bunch of bugfixes and minor enhancements. Much of our focus was on some upcoming big changes. You can read about what's coming up next in the Roadmap and future plans section.

Enhancements and UX improvements

#2274 and #2560 improved the csv output with support for a new timeFormat option. The possible values are unix (default) and rfc3399. You can also configure it through the K6_CSV_TIME_FORMAT environment variable. Thanks, @rpocklin!
#2509 added the clear() and delete() methods to the CookieJar object from the k6/http module. Thanks, @Maksimall89!
#2282 increased the precision of the iteration-progress bar in the UI. Thanks, @m3hm3t and @DarkAEther!
#2568 added more descriptive error messages when there were problems with parsing a config file.

Bug fixes

#2523 fixed a gRPC marshaling error when any.proto was used for a type. Thanks, @Flowersea!
#2534 fixed the return type of Selection.map() from the k6/html module to the correct object types instead of a forced array of strings.
#2502 made it so k6 waits for scenario executors to fully finish before updating their final progress in the UI, preventing misleading red crosses (#2500).
#2524 fixed a bug where GoError string contained missing URL values (#2537).
#2530 fixed a wrong error message on remote resolution.
#2542 fixed a bug where Rate metric and sub-metric values were shown as NaN in the end-of-test summary if there were no measured values for them during the test run.
#2558 fixed a panic when trying to set the value of a vu.tags element from k6/execution to null or undefined.
#2567 fixed a panic when trying to access some k6/execution properties outside of a VU context, e.g. trying to access execution.scenario in setup().

Maintenance and internal improvements

#2550 updated the used Go version to 1.18.
#2524, #2551, #2552, #2553, #2554, #2555 updated various Go dependencies in k6.
#2583 added a deprecation warning for thresholds that use the url, error, vu or iter tags, which will become un-indexable in the future.

Roadmap and future plans

As the lack of big changes in this release suggests, we've focused the last few months' efforts on a few areas that haven't yet been merged into the core of k6.

In this section, we'd like to inform the community about important features that we're currently working on - our short-term roadmap in a sense. We'll also use it to give notice of breaking changes we plan to make in the near future.

`k6/experimental/*` JS modules

Over the last several k6 releases, among a lot of other refactorings, we've added support for JavaScript event loops (#2228) in k6 VUs and added a new Go API for exposing built-in and xk6 extension modules to user scripts (announcement, docs). This has given us (and any xk6-extension authors!) the ability to better support various asynchronous streaming/messaging/etc. protocols (#882).

We've started building some of these newly possible APIs as xk6 extensions first, to be able to iterate on them more quickly and get some user feedback while we are building them. xk6-websockets, xk6-timers and xk6-redis are some of the first such APIs, but we plan to also work on support for gRPC streaming (#2020), messaging protocols (#1269), a new and better HTTP API (#2461) and many others in the future!

We want to eventually include a lot of these APIs in the k6 core as built-in modules that users can directly use, without needing to mess with xk6 or Go compilation. However, because we try to keep the built-in k6 APIs stable and backwards-compatible, we want to get more user feedback before we do that, while we are still free to iterate and make (hopefully minor) breaking changes.

So, we decided to create a new middle ground between the unstable and purely external xk6 extensions and the stable built-in k6 APIs―built-in k6/experimental/* modules! Our goal is that, starting with the next k6 v0.40.0 release, we'll start releasing some or all of these core-bound extensions as built-in k6 modules under these k6/experimental/ import paths. This will let k6 users, both OSS and Cloud, to give us feedback and help us improve the APIs before we stabilize them.

As is hopefully clear from the name, our usual guarantees of API stability won't apply to these modules while they are still experimental. We reserve the right to make breaking changes in experimental modules, up to and including completely dropping them. We don't expect big breaking changes will need to happen often, but we want to be clear they aren't impossible. Finally, when an API has been stabilized and made available under a regular import path, we'll deprecate its experimental import path. To make the transition easier, both import paths will be available simultaneously for at least one k6 version.

Native support for ECMAScript modules

At the moment, k6 has support for ECMAScript modules (ESM, i.e. import, export, etc.) via automatic transpilation of scripts by the built-in Babel.js. That mostly works, but it has caused some performance and compatibility problems (#824 and #2168 among others), so we want to support ESM modules and all other ES6 features directly in k6, without the need for Babel.js (#2296). goja, the JavaScript runtime we use to evaluate k6 scripts, doesn't yet have native ESM support, so we are currently working on adding it there, to then be able to support ECMAScript modules natively in k6!

That work has been ongoing for a while and we're making progress, but it will likely not be ready in time for the next k6 v0.40.0 release. We are mentioning it here because we will probably need to make a few minor breaking changes and fixes of currently undefined behavior in k6 to support ESM modules natively.

For example, at the moment, some values like the consolidated script options were unintentionally made available globally, in all JS modules of a test, instead of just in the exported options value from the main JS module. That is not the intended or documented behavior, it's somewhere between a bug and undefined behavior, and we'll need to fix it (#2571) and other similar issues like it, starting in k6 v0.40.0. We don't expect many scripts to break because of these fixes, but we'll nevertheless announce them in the release notes of the k6 version that they happen in.

Refactoring metrics

Over the last several k6 releases, we've also slowly been refactoring and improving the metrics internals in k6 (see #2071, #2330, #2426, #2463, #2433, #2442, among others). This has already brought many side benefits and minor bugfixes, and we still have a lot of work left (e.g. #1889, #572, #2430, #1831), but we've finally reached the point where we can almost start implementing some major new features effectively!

One of the upcoming next big steps is the introduction of a "time series" concept internally in k6 (#2580). We'll start to efficiently group samples (i.e. metric measurements) with the same metric and tags into a single TimeSeries, which would unlock many of possibilities that were previously too inefficient to implement.

Another upcoming metrics-related change would be the refactoring of metric tags into two distinct types (#2584) - ones that can be used for indexing (so, in the TimeSeries mentioned above, in sub-metric thresholds, in certain outputs that do aggregation based on tags, etc. ) and ones that cannot (e.g. high-cardinality metadata that's often unique for every data point).

Unfortunately, we'll need to make a few minor breaking changes. The current url, error, vu and iter system tags will be made non-indexable by default, to reflect their usually very high cardinality, so they won't be usable in thresholds. Instead of error, the error_code tag should be used there. And instead of url, users should already use the name tag to avoid issues, see the documentation about URL grouping. k6 v0.39.0 still supports thresholds with these tags, it will just print a warning if they are used.

Prometheus remote-write output

Because of these changes, we'll finally be able to refactor and improve the Prometheus remote-write output extension. Our goal is to make it more efficient, polish its rough edges, and merge it into the k6 core as a built-in output module in the near future.

Tracing support

Another new feature these changes would unlock is support for distributed tracing (#2128).

Because of the upcoming support for high-cardinality unindexable metric tags, we'll be able to safely attach things like the unique trace and span IDs to every metric measurement without negatively affecting other parts of k6.

k6 - v0.38.3

Published by github-actions[bot] over 2 years ago

k6 v0.38.3 is a patch release containing a single fix

Threshold over already defined sub-metrics will result in an error (#2538)

There was a bug where we were checking if a submetric had already been added. Unfortunately, we didn't check that this will work with the one submetric we have by default http_req_duration{expected_response:true}. After v0.38.0 defining a threshold on it would result in an error.

As this definitely shouldn't happen in that case and we don't see a particular case where that will be problematic - adding a submetric again just reuses the already added one instead.

This issue has been addressed in #2539, and k6 v0.38.3 will now lead you add a threshold on http_req_duration{expected_response:true}.

k6 - v0.38.2

Published by github-actions[bot] over 2 years ago

k6 v0.38.2 is a patch release containing a couple of bugfixes!

Threshold over sub-metrics without samples would result in `NaN` (#2520)

There was a bug in thresholds applied to sub-metrics set to abortOnFail: leading k6 to evaluate thresholds that would have likely aborted before they had a chance of passing (because no samples for the given metric were recorded yet). This bug would have led to such thresholds' results value to be NaN rather than a numerical value. The following script, for instance:

import { check, sleep } from 'k6';

import http from 'k6/http';

export const options = {
  scenarios: {
    iWillFail: {
      exec: 'iWillFail',
      executor: 'constant-vus',
      startTime: '2s',
      vus: 1,
      duration: '30s',
    },
  },

  thresholds: {
    'checks{type:read}': [{ threshold: 'rate>0.9', abortOnFail: true }],
  },
};

export function iWillFail() {
  let res = http.get(`https://test-api.k6.io/`);

  check(res, {
    'read status is 200': (r) => r.status === 200,
  }, { type: 'read' });

  sleep(1);
}

Would result in the following:

✗ { type:read }...: NaN% ✓ 0 ✗ 0  
vus...............: 0 min=0 max=0
vus_max...........: 1 min=1 max=1

This issue was introduced by recent changes to how we handle thresholds in the k6 engine and is now addressed in v0.38.2.

Sub-metrics without values rendered below an incorrect parent metric (#2518)

There was in how thresholds over sub-metrics that didn't receive any samples would be displayed under an incorrect parent metric. For instance, the following script:

import { Counter } from 'k6/metrics';

const counter1 = new Counter("one");
const counter2 = new Counter("two");

export const options = {
    thresholds: {
        'one{tag:xyz}': [],
    },
};

export default function() {
    console.log('not submitting metric1');
    counter2.add(42);
}

Would have led to the following output, where the {tag:xyz} sub-metric is displayed under iterations instead of one:

data_received........: 0 B 0 B/s
data_sent............: 0 B 0 B/s
iteration_duration...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
iterations...........: 1 499.950005/s
  { tag:xyz }........: 0 0/s
two..................: 42 20997.90021/s

When we would have expected it to produce:

one..................: 0 0/s
  { tag:xyz }........: 0 0/s
two..................: 42

This issue has been addressed in #2519, and k6 v0.38.2 now displays sub-metrics under their actual parents, even when they have received no samples.

Special thanks to @efdknittlfrank, who reported and helped us track down the issue.

k6 - v0.38.1

Published by github-actions[bot] over 2 years ago

k6 v0.38.1 is a patch release containing a bugfix!

Threshold sub-metric selectors containing reserved symbols would fail (#2512)

There was a bug in threshold sub-metric selector parsing, which led to errors when users would use specific symbols such as {, } or : as part of their definitions. For instance, thresholds used for sub-metrics with URL Grouping like http_req_duration{name:"http://example.com/${}"} would have led to failures in v0.38.0.

The error messages for invalid metric, sub-metric and threshold definitions were also improved.

Special thanks to @efdknittlfrank, who reported and helped us track down the issue.

k6 - v0.38.0

Published by github-actions[bot] over 2 years ago

k6 v0.38.0 is here! 🎉

New Features!

AWS JSLib

There's a new addition to the officially supported k6 JavaScript libraries: k6-jslib-aws.
This library lets users interact with a selection of AWS services directly from their scripts. The library currently implements support for the S3 and the Secrets Manager services.

The AWS JS lib documentation has examples and details on how to use the library in your scripts.

Accessing the consolidated and derived options from the default function (#2493)

The k6/execution core module now lets you access the consolidated and derived options that k6 computed before it started executing the test. You can access consolidated options through the exec.test.options property. Note that consolidated options are frozen and cannot be modified. The k6 execution module's documentation has examples and details on how to use the functionality in your scripts.

import exec from "k6/execution";

export const options = {
    vus: 10,
    duration: "30s",
};

export default function () {
    console.log(exec.test.options.scenarios.default.vus); // 10
}

Tagging metric values with the current scenario stage

With the new consolidated script options, we've added a few helper functions to the k6-jslib-utils library. You can use them to automatically tag all the emitted metric samples by k6 with the currently running stage.

The k6 documentation has examples and details on how to use it.

Dumping SSL keys to an NSS formatted key log file (#2487)

This release adds the ability to dump SSL keys while making TLS connections.
You then can use these keys to decrypt all traffic that k6 generates.

To accomplish this, set the SSLKEYLOGFILE environment variable to some file path and run k6.
This will populate the file with the keys.
Then you can use Wireshark to capture the traffic, decrypt it, and use that for debugging.

Here's an example that uses curl to inspect TLS traffic.

Breaking Changes

`console` methods now pretty print objects and arrays (2375)

For convenience, all console methods such as console.log() and console.info() will now automatically JSON.stringify() objects and arrays passed to them. Thus, instead of console.log({'foo': 'bar'}) printing [object Object], it will now print {'foo': 'bar'}, which will make the experience of debugging k6 scripts easier and more intuitive.

To achieve the previous behavior, cast the Object to a String, as in console.log(String({'foo': 'bar'})).

export default function () {
    console.log([1, 2, "test", ["foo", "bar"], { user: "Bob" }]);
    // before: 1,2,test,foo,bar,[object Object]
    // after: [1,2,"test",["foo","bar"],{"user":"Bob"}]
}

The Go types in the `stats` package were moved to the `metrics` package #2433

For convenience and to facilitate further developments, the types and functionalities that used to live in k6's stats package have been moved to the metrics package. The stats package is, as of v0.38.0, removed in favor of the metrics package. Besides, #2442 removed the stats.New function in favor of initializing new metric via a register.NewMetric call instead.

Deprecation

#2499 removed support for the deprecated maxVUs option.
It had been removed in k6 v0.27.0, however using the CLI flag resulted only in a deprecation warning.
Now, using this flag will generate an error.
This release drops some leftovers from the previous version of our JS module Go APIs. As of v0.38.0, these are now unsupported:
- The deprecated common.Bind (#2488) and common.BindToGlobal (#2451) functions.
- The context-based (common/context.go #2488) utils have also been removed.

Enhancements and UX improvements

Stricter thresholds' evaluation before the execution starts (#2330)

k6 v0.37.0 already improved threshold parsing by switching its underlying implementation from JavaScript to Go. k6 v0.38.0 introduces two additional improvements:

k6 will now parse and evaluate thresholds before the execution starts. If a threshold is invalid, as described below, k6 will immediately exit without starting the load test.

k6 will now detect invalid thresholds:

export const options = {
    // ...
    thresholds: {
        // Incorrect thresholds expressions:
        http_req_failed: ["rave<0.01"], // e.g. "rave" is not a valid aggregation method
        // Thresholds applying to a non-existing metrics:
        iDoNotExist: ["p(95)<200"], // e.g. the metric 'iDoNotExist' does not exist
        // Thresholds applying an aggregation method that's unsupported by the metric they apply to:
        my_counter: ["p(95)<200"], // Counter metrics do not support the p() aggregation method
    },
};

Disabling colors (#2410)

In addition to the --no-color CLI flag, the ANSI color escape codes emitted by k6 can now also be disabled by setting the NO_COLOR or K6_NO_COLOR environment variables, following the NO_COLOR standard.

# No color output
K6_NO_COLOR=true k6 run script.js

# No color output
NO_COLOR= k6 run script.js

Support for encrypted TLS private keys (#2488)

You can now use passphrase-protected private keys when authenticating with TLS. Using the password property of an options' tlsAuth object, you can now indicate the passphrase to decrypt a private key. Note that this support is limited to the scope of RFC1423 and does not support PKCS8 keys, as they're not yet supported by the Golang standard library.

export const options = {
    tlsAuth: [
        {
            domains: ["example.com"],
            cert: open("mycert.pem"),
            key: open("mycert-key.pem"),
            password: "mycert-passphrase",
        },
    ],
};

Thanks, @Gabrielopesantos, for the contribution.

Improve JSON output's performance (#2436)

The JSON output was optimized and now should be around 2x more performant at outputting metrics. This means that it either can export twice as many metrics, or use half the resources to do the same amount of metrics.

As a side effect, there is a slight breaking change: the tags field is no longer sorted.

Treat panics as interrupt errors (#2453)

We changed the behavior of how k6 treats Go panics, which may happen because of bugs in k6 or in a JavaScript k6 extension. Previously, the behavior was to catch the panic and log it as an error.

Starting with v0.38.0, whenever k6 observes a Go panic, it logs an error like before, but more importantly, it will abort the script execution and k6 will exit with a non-0 exit code. This will help extension authors to identify issues in their extensions more easily.

Miscellaneous

#2411 The k6 command-line UI (logo, test description, and progress bars) can now effectively be disabled using the --quiet flag.
#2429 lib/types now exposes the source of the NullHostnameTrie to simplify access to an original list of the hostnames.

Extensions

PoC for a new Web Sockets JS API

We built a new xk6 extension, https://github.com/grafana/xk6-websockets, with a proof of concept implementation for a new JavaScript Web Sockets API. This API uses the global event loops introduced in k6 v0.37.0 to allow a single VU to have multiple concurrent web socket connections open simultaneously, greatly reducing the resources needed for large tests. It also is a step towards supporting the official Web Sockets JS standard, potentially allowing the usage of more third-party JS libraries in the future.

Please share any feedback you have about the new extension since it's likely that we'll adopt a future version of it into the k6 core in one of the next several k6 releases.

gRPC module refactored to enable gRPC extensions to use it

#2484 moved out in a new dedicated Go lib/netext/grpcext package all the parts not strictly required from the js/k6/net/grpc module for binding the gRPC operations and the JavaScript runtime. It facilitates the development of extensions based on gRPC without the direct dependency on the goja runtime. Furthermore, the new Dial function accepts a grpc.DialOption variadic for customizing the dialing operation.

Event loop testing

With this release, you can export the event loop added in v0.37.0. This lets extension developers test event-loop-dependent APIs.

There were also updates to modulestest.VU to support the new API. Head to GitHub to see it in action.

Bugs Fixed!

#2456: Fixed a very unlikely panic involving arrival rate executors and execution segments with very low VU counts.
#2349: Thanks to @rainingmaster it is now possible to leave the options' tlsAuth domains property empty.
#1346: Thresholds over custom metrics for which no data was collected will now be evaluated.
#2390: Thresholds over sub-metrics with empty tag values will now return the appropriate results.
#2480: Fixed an error occurring when passing the data property of an http.file() result as a request body.

Known issues

@hecnavsanz reported to us that in certain scenarios, k6's UI might inaccurately report a test as failed ❌ when it actually succeeded ✅. This is caused by some concurrency issue in our codebase that only affects the report's UI, and has no impact on the actual test result. A fix is in the works, but won't be ready for this version. We expect to ship it with the next k6 release instead.

Maintenance

Dependencies

#2479 updated k6's Goja version to 9037c2b61cbf.
#2449, #2446, #2444, and #2443 updated k6 Go dependencies to their latest compatible versions.

Other

#2504 updated our installation instructions to reflect the recent changes in the behavior of the go install command. Thanks, @JamieEdge, for your contribution!
#2437 refreshed our contribution guidelines.
#2431 refreshed out our dependencies update workflow, process, and guidelines.

k6 - v0.37.0

Published by github-actions[bot] over 2 years ago

k6 v0.37.0 is here! 🎉 Mainly it contains fixes and ongoing efforts with refactoring.

New Features!

Added experimental basic event loop (#882)

We added basic event loop support in k6 (#2228 and #2373) 🎉 This was just the first step and isn't used by any of the existing k6 JS APIs yet. For now, it is only available to xk6 extensions like this one that adds support for setTimeout(), setInterval(), clearTimeout() and clearInterval().

Expect to see more changes related to event loops in the next k6 releases, where event loops will start being used by some core k6 modules! For example, by improving some existing JavaScript APIs to have support for callbacks or return Promise values, so they can be used asynchronously. We expect this change will unlock a lot of previously difficult use cases (see #882), though we'll likely iterate on these new APIs as experimental extensions for a while, to stabilize them before we merge them into the core.

ℹ️ If you are an extension developer, please use it and give your feedback. But take into consideration that it's likely that the current Go API may change.

Added an option to output k6 logs to a file through --log-output (#2285)

This is on top of the already supported options for sending logs to stdout/stderr and to Grafana Loki. This new feature speaks for itself with simple usage examples:

k6 run --log-output file=./k6.log --logformat json ./examples/stages.js

And one more with a defined minimal log level:

k6 run --log-output file=./k6.log,level=info --logformat json ./examples/stages.js

Thanks, @alyakimenko for the contribution!

Docs: Using file output

Breaking changes

Introduced stricter thresholds parsing (#2400)

In the past, thresholds were evaluated using a JavaScript runtime. For a multitude of reasons, this wasn't satisfying. As of v0.37.0, thresholds are now parsed directly in Go. As a result, k6 will now return an error message on thresholds that do not strictly match the documented specification, instead of just silently ignoring them. Another change is that when a non syntactically correct threshold expression is detected, k6 will immediately interrupt its execution before even starting the load test run.

Below you can find examples of the thresholds expressions that won't work anymore:

export const options = {
    thresholds: {
        "http_req_duration": [
            // although the aggregation method and values are correct, 
            // the equal sign is invalid; use == or ===
            "rate=200",
            // thresholds do not support javascript expressions anymore
            "throw new Error('wat')",
            // it fails, as foo is not a valid threshold expression's aggregation method keyword
            "foo>500",
        ],
    },
};

Extensions

v0.37.0 finalizes (#2376) the switching of our internal modules (gRPC module refactoring) to a new Go/JavaScript module API.

⚠️ It's important to highlight that the old API (e.g. methods like context.WithRuntime, common.Bind and others #2384) is deprecated and will be removed in the next k6 release (v0.38.0). For this release, every extension that isn't using the new API will get a warning message like this:

WARN[0000] Module 'k6/x/sql' is using deprecated APIs that will be removed in k6 v0.38.0, for more details on how to update it see https://k6.io/docs/extensions/guides/create-an-extension/#advanced-javascript-extension

We did migrations for some xk6 extensions (see connected issues to the task #2344). The pull requests can serve as examples on how to transition your extension to the new API.

Docker Repository

We migrated our Docker Hub repository from loadimpact/k6 to grafana/k6 (#2377).

docker run -i grafana/k6 run - <script.js

We will continue publishing our docker image releases as both loadimpact/k6 and grafana/k6 for several more releases, but if you use the old one in your local or CI environments, please plan the migration.

Enhancements and UX improvements

We continued work on the source map feature: add samples (#2339) and improve loading and parsing (#2355).
Updated installation from source instructions (#2359). Thanks, @karitham, for making the change.
Enabled more TC39 tests: optional-chaining (#2338) and async (#2396).
We updated our Code of Conduct and a few GitHub URLs (#2416).

Bugs fixed!

http.head started taking a body as a second argument (#2402) by mistake in v0.36.0. It is now back to its old signature http.head(url, [params]).
Fixed options.scenarios' JSON marshaling (#2392).
Metrics' names display again in k6's Rest API (#2421).
We improved argument validation for check(#2387), http.batch(#2415) and metrics' constructors (#2427).

Internals

We updated our CI to improve developer experience. Dependency and linter checks now run only for pull requests (#2403).
This release also contains a few refactoring PRs that fix linters errors (#2334, #2331 and #2341), remove global variable usage (#2336, #2358, #2353 and #2357) and remove an unnecessary dependency (#2313) which makes our codebase more consistent and maintainable.
The headers parameter in k6's GRPC module is marked as deprecated (#2370).
Switched envconfig to our own fork (#2337) in order to abstract the os package and improve testability.

Package Rankings

Top 0.44% on Proxy.golang.org

Related Projects

k6-template-typescript

Template to use TypeScript with k6

09 Jun 2020 256

k6-template-es6

Template using Webpack and Babel to enable ES6 features in k6 tests

27 Mar 2020 78

grafana-presto

grafana with presto support

04 Sep 2014 24

k6

Breaking changes

Browser APIs to Async

New features

Add support for uploading files from the browser module browser#1097, browser#1244

Introducing options.cloud #3348, #3407

Timers API becomes part of the k6 core #3587

JSON Web Key support in k6/experimental/webcrypto module webcrypto#61

UX improvements and enhancements

Browser Context Isolation browser#1112

Bug fixes

Maintenance and internal improvements

Breaking changes

New features

Web Dashboard

Real-time test results

Test report

Add clear to the locator class browser#1149

Add tracing to the browser module browser#1100

gRPC streaming API becomes part of the k6 core #3490

k6/html: Extract selection from element #3519

Collect usage data on imported internal modules and outputs #3525

UX improvements and enhancements

Bug fixes

Maintenance and internal improvements

Roadmap

Breaking changes

New features

Add k6 new subcommand #3394

Add a k6/experimental/fs module #3165

Redis (m)TLS support and new Client constructor options #3439, xk6-redis/#17

Connection URLs

Example usage

Revamped Options object

(m)TLS support

Add tracing instrumentation #3445

Add support for browser module's page.throttleCPU browser#1095

Add support for browser module's page.throttleNetwork browser#1094

k6's documentation is moving under grafana.com/docs/k6

UX improvements and enhancements

Bug fixes

Maintenance and internal improvements

Roadmap

Graduating from experimental

New dashboard features

Deprecations

New features

Add gRPC's binary metadata support #3234, xk6-grpc#46

Add gRPC's reflection metadata support #3343, xk6-grpc#46

Higher precision for Trend metrics in Grafana Cloud k6 #3302

Docker support for browser-based tests #3199

Docker images for ARM64 architecture #3320

New authentication methods and HTTP headers API for Prometheus remote write output xk6-output-prometheus-remote#143, xk6-output-prometheus-remote#145, xk6-output-prometheus-remote#147

Improved the browser module's cookie API

Add support for browser module's page.on('console') browser#1006

UX improvements and enhancements

Bug fixes

Maintenance and internal improvements

Roadmap

Native ECMAScript Modules support

Future breaking changes

Breaking Changes

Browser

Deprecations

New features

Loki log output sending additional headers #3227

Cloud Traces #3100, #3202

Ability to configure TLS per gRPC connection #3159, xk6-grpc#25

Cloud Output v2 #3117

UX improvements and enhancements

Bug fixes

Maintenance and internal improvements

Breaking changes

New features

Experimental gRPC module with streaming support #3107

You can now only upload a test to the cloud without running it #3030

Setting sample metadata API #3037

UX improvements and enhancements

Bug fixes

Maintenance and internal improvements

JSON Web Key support in `k6/experimental/webcrypto` module webcrypto#61

Add `clear` to the `locator` class browser#1149

Add `k6 new` subcommand #3394

Add a `k6/experimental/fs` module #3165

Add support for browser module's `page.throttleCPU` browser#1095

Add support for browser module's `page.throttleNetwork` browser#1094

Add support for browser module's `page.on('console')` browser#1006

A new `k6/experimental/webcrypto` module implementing the Web Crypto API specification #3007

`browserContext.addCookies` browser#760

`browserType.Connect` browser#800

Native support for JavaScript's `async`/`await` #2830

`http.asyncRequest` #2877

Wildcard support for `hosts` option #2747

Changes in the `url`, `iter` and `vu` system metric tags

`k6 convert` is officially deprecated (#2714)

`k6/ws` now respects the `throw` option #2247

`k6/experimental/*` JS modules

Threshold over sub-metrics without samples would result in `NaN` (#2520)