artillery

The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.

MPL-2.0 License

Downloads
473.9K
Stars
7.9K
Committers
148
View Code on GitHub Visit Website View on X
Ecosystems: JavaScript, gRPC, Playwright, OpenTelemetry, TypeScript, Amazon Web Services

Bot releases are hidden (Show)

artillery - Artillery v2.0.9 Latest Release

Published by bernardobridge 7 months ago

Artillery v2 0 9

Publish Metrics

CLI & Core

This release brought to you by @InesNi @95th

artillery - Artillery v2.0.8

Published by bernardobridge 7 months ago

Artillery v2.0.8

Artillery Cloud

Metrics by Endpoint

CLI and Core

This release brought to you by @hassy @bernardobridge

artillery - Artillery v2.0.7

Published by hassy 8 months ago

Artillery v2.0.7

CLI and Core

  • Add support for loading custom code from ES modules (#2522) - docs
  • Add support for using async functions as hooks (#2522) - docs

Playwright

  • Update Playwright version to 1.42.1
  • Fix bug that prevented traces from Playwright tests from being sent to Datadog and Honeycomb (#2533)

AWS Fargate

  • Tests running in Fargate will now use AWS Distro for OpenTelemetry instead of Datadog Agent for sending traces to Datadog (#2481) - docs
  • Fix bug that prevented setting CPU and memory allocation with the --launch-config flag (#2526)

This release brought to you by @hassy @bernardobridge @InesNi

artillery - Artillery v2.0.6

Published by bernardobridge 8 months ago

2 0 6

Playwright

Artillery Cloud

CLI & Core

Publish Metrics

Fargate

This release brought to you by @bernardobridge, @InesNi, @hassy

artillery - Artillery v2.0.5

Published by bernardobridge 9 months ago

Artillery CLI v2.0.5

New release cadence

A new stable version of the Artillery CLI will now be released twice a month - on the first and third Monday of the month. Canary releases will continue being released from the main branch.

AWS Fargate

Playwright

publish-metrics plugin

AWS Lambda

CLI & Core

Artillery Cloud

This release brought to you by @InesNi, @hassy, @bernardobridge, @Georgegriff

artillery - Artillery v2.0.4

Published by bernardobridge 9 months ago

2 0 4

Fargate

CLI & Core

Playwright

OpenTelemetry

This release brought to you by @InesNi @hassy @bernardobridge

artillery - Artillery v2.0.3

Published by hassy 11 months ago

Fixes & Improvements

  • Fix a bug that prevented tests from running on AWS Lambda (#2355)
  • Fix a bug that prevented tests from running on AWS Fargate (#2354)
artillery - Artillery v2.0.2

Published by hassy 11 months ago

  • Tests running locally or on AWS ECS/Fargate can be stopped through the dashboard now
artillery - Artillery v2.0.1

Published by hassy 11 months ago

Artillery CLI v2.0.1

CLI & Core

  • Add time unit support for think steps in scenarios (#2272)
  • metrics-by-endpoint plugin is now enabled by default to capture per-URL metrics in HTTP tests. The metrics will be available only in the JSON report and not output to the console, unless the plugin is loaded explicitly by the user. (#2274)
  • Configure CPU and memory for Fargate tests with --cpu and --memory flags (#2301)
  • Fix issue with $env not resolving correctly in the config section (#2265)
  • Fix an issue with --scenario-name not picking up the correct scenario (#2314)
  • Add a new fake-data plugin to allow for realistic test data to be generated in your tests (#2318) (docs)
  • Fix handling of metrics with non-alphanumeric characters in ensure checks (#2284)
  • Improve error handling in the expect plugin (#2298)

OpenTelemetry

  • Add experimental tracing support for Playwright tests, with tracing of test.step() calls, page navigation events, and Core Web Vitals metrics (#2259)
  • Improve conformance to OTel semantic conventions:
    • Include error messages for spans with HTTP response codes of 400 or higher (#2285)
    • Include HTTP_REQUEST_CONTENT_LENGTH, HTTP_USER_AGENT and HTTP_FLAVOR attributes (#2285)
  • Fix issue with virtual user ID attribute (vu.uuid) not always being set correctly
  • Fix inconsistencies in error handling (#2298 & #2285)

Playwright

  • Add experimental tracing support via OpenTelemetry with publish-metrics plugin (#2259)
  • Fix FID Core Web Vital metric not being emitted as expected

ECS/Fargate

  • Configure CPU and memory for Fargate tests with --cpu and --memory flags (#2301)

Socket.IO

  • Allow validation of any number of arguments in response/acknowledge with new args/on options (#2171)
  • More flexible validations with ability to wait for multiple responses (#2139)
  • Fix issue with displaying match errors in the report

This release brought to you by @bernardobridge @InesNi @josh-coderpad @hassy

artillery - Artillery v2.0.0-38

Published by bernardobridge 12 months ago

release

New features & improvements

Engines

CLI/Core

Plugins

publish-metrics - OpenTelemetry

Fixes

Artillery Dashboard (join the waitlist)

Engines

CLI/Core

Plugins

This release brought to you by @InesNi, @bernardobridge, @kettanaito and @hassy.

artillery - Artillery v2.0.0-37

Published by hassy about 1 year ago

Artillery v2.0.0-37

New features & improvements

  • Add support for specifying time units in load phase configs (#2084) - (docs)
  • Add OpenTelemetry support (#2073, #2113) - (docs)
  • Add color output to ensure check summary (#2074)
  • Add support for using application keys Datadog appKey support (#2067)
  • Update Playwright engine to use Playwright v1.37.1 (#2105)

Other fixes

  • Fix issue with tests not running as expected when filenames shared a common prefix (#2092)
  • Improve handling of runtime errors in workers on Fargate
  • Fix issue with some console logs not showing when running tests on Fargate
  • Routine dependency updates

This release brought to you by @bernardobridge @InesNi @yvesDenis @hassy

artillery - Artillery v2.0.0-36

Published by hassy about 1 year ago

New features

Fixes & improvements

Artillery Dashboard (join the waitlist)

Core

Plugins

This release brought to you by @inesni @bernardobridge @hassy

artillery - Artillery v2.0.0-35

Published by hassy over 1 year ago

New features

  • Splunk and New Relic integrations now have support for sending test started/finish events via publish-metrics (#1963 & #1958)

Fixes & improvements

  • Fix issue with sending reports to Mixpanel in publish-metrics (#1986)
  • Fix issue with ensure plugin reading its settings from config.ensure
  • Fix issue with sending test metadata to Artillery Cloud dashboard (#1954)
  • Fix issue with config.payload used along with config.environments (#1964)
  • Dependency updates to address security warnings in dependencies

With contributions from: @bernardobridge @InesNi @hassy

artillery - Skytrace v0.0.7

Published by hassy over 1 year ago

Fixes & Improvements

  • Fix an issue with handling of JSON responses which would make JSON-based expectations and checks fail even when the response was correct JSON
artillery - Skytrace v0.0.6

Published by hassy over 1 year ago

New features

JMESPath expectations

JMESPath is a popular query language for JSON supported by many tools such as AWS CLI and Azure CLI.

Skytrace now supports for setting expectations via JMESPath expressions. An expression that evaluates to a truthy value will result in a successful check.

# Check that the first object in the list of results has a title field:
skytrace http get http://lab.artillery.io/movies -e 'jmespath: "[0].title != null"'

Connected to http://lab.artillery.io

HTTP/1.1 200 OK
x-powered-by: Express
content-type: application/json; charset=utf-8
content-encoding: gzip
etag: W/"558f-PLaaZcOcQVqOfQBMyp4Dw3RncCE"
date: Thu, 22 Jun 2023 14:07:01 GMT
connection: keep-alive
keep-alive: timeout=5
transfer-encoding: chunked
server: Fly/a0b91024 (2023-06-13)
via: 1.1 fly.io
fly-request-id: 01H3HPW44SMJEXCJ6KSVRPQXPM-ams


 DNS Lookup | TCP Connection | SSL Handshake | Time to First Byte | Content Transfer
  1ms       |  35ms          |     N/A       |     53ms           |   12ms          |
            |                |               |                    |                 |
            1ms              |               |                    |                 |
                          36ms               |                    |                 |
                                           36ms                   |                 |
                                                                89ms                |
                                                                               total:101ms


Body stored in: /var/folders/yd/nyh8vtm17w92h5hm4pdhs8j00000gn/T/2023522-24253-1hq5ccb.f6czg.json

Expectations:

*  ok jmespath [0].title != null

Fixes & improvements

  • Print base URL only in "Connecting" message
  • Fix POST requests not being available
artillery - Artillery v2.0.0-34

Published by hassy over 1 year ago

New run:lambda command

Tests can now be run on AWS Lambda with the new run:lambda command. (Running tests on AWS Lambda with run --platform aws:lambda will continue working as expected too).

The new command makes running tests on AWS Lambda less verbose and more clear, for example:

artillery run:lambda --region eu-west-1 my-script.yml

Rather than:

artillery run --platform aws:lambda --platform-opt region=eu-west-1 my-script.yml

JMESPath expectations

The expect plugin can now evaluate JMESPath expressions as checks. If the expression evaluates to a truthy value, the check will pass. For example:

- get:
    url: "/foo/1234"
    expect:
      jmespath: "title != null"

More flexibility in metrics-by-endpoint reporting

Two new configuration options are now available to help reduce the number of unique metrics produced by by larger & more complex test suites:

  • stripQueryString - ignore all query strings on a URL for purposes of reporting its metrics, i.e. /foo?bar=1 and /foo?baz=2 will be reported as the same /foo endpoint
  • and ignoreUnnamedRequests - ignore any requests that don't have a name attribute set explicitly

Other fixes & improvements

  • A custom role may be set for AWS Lambda workers with --lambda-role-arn flag for run:lambda command

Community shout outs

@cjsaurusrex

artillery - Artillery v2.0.0-33

Published by hassy over 1 year ago

AWS Fargate support

This release adds support for running tests on AWS Fargate. Scale out your load tests horizontally from any of the six supported regions.

artillery run:fargate --count 10 --region us-east-1 my-script.yml

Supported regions:

  • us-east-1
  • us-west-1
  • eu-west-1
  • eu-central-1
  • ap-south-1
  • ap-northeast-1

AWS Fargate is a serverless solution that requires no infrastructure to be managed, and unlike AWS Lambda does not impose a restriction on the duration of your load tests.

Artillery's Playwright integration is fully supported on Fargate, to let you run browser-based load tests at scale with ease.

Playwright support

Playwright support is now built into Artillery. The integration in artillery-engine-playwright is now bundled into the Artillery CLI.

  • The version of Playwright used by the integration has been updated to v1.3.4
  • Web Vitals tracking has been improved to provide more accurate measurements
  • Playwright scenario file may now be loaded via testFunction attribute rather than flowFunction
  • flow attribute may now be omitted in Playwright engine scenarios

Other fixes & improvements

  • Pass lambdaRoleArn to AWS Lambda engine correctly (https://github.com/artilleryio/artillery/pull/1920)
  • Fix issue with returning matched groups from regexp capture (https://github.com/artilleryio/artillery/pull/1922)
  • Track & report downloaded data in HTTP tests via http.downloaded_bytes metric
  • Environment variables may now be accessed through the special $env variable (the current way of accessing those via $processEnvironment is still supported but will be deprecated in future)
  • Dependency updates to address deprecation and security notices

Contributor shout outs

@tancredi @jpilla @BanksySan

artillery - Skytrace v0.0.5

Published by hassy over 1 year ago

New Features

  • Skytrace can run existing Artillery scripts now. Use skytrace run on an existing Artillery script to run it as an end-to-end test.
artillery - Artillery v2.0.0-32

Published by hassy over 1 year ago

Changelog

New features

  • Splunk integration in publish-metrics plugin. You can now export data from Artillery tests to the Splunk observability platform.
  • Apdex support via the new apdex plugin. Calculate an industry standard measure of users' satisfaction based on HTTP metrics in Artillery tests.
  • Add --record and --key flags for the run command for enabling Artillery Cloud reporting for individual test runs
  • The namespace for metrics-by-endpoint-produced metrics may be customized now

Fixes & improvements

Community shout-outs

  • @InesNi @jeffhall4
artillery - Artillery v2.0.0-31

Published by hassy over 1 year ago

What's Changed

New Features

Fixes & Improvements

  • AWS Lambda - fix issue with the size of zipped Lambda packages which would prevent tests from running (#1855)
  • AWS Lambda - fix issue with running AWS Lambda tests on Windows (#1846)
  • HTTP: Fix issue with cookies not working inside loop under some conditions (#1826)
  • Fix issue with CSV payload data not being split across multiple worker threads (#1829)
  • HTTP: Query strings are now shown in HTTP debug output (#1817)
  • Fix axis display on generated HTML reports (#1815)

Community shout-outs

  • @InesNi, @anoopknayak, @MagicKriss, @chinh-tran - new features & fixes
  • @mplaul - help with debugging & tracking down the AWS Lambda issue
Package Rankings
Top 0.8% on Npmjs.org
Top 5.28% on Proxy.golang.org
Related Projects
typescript-backend-toolkit

TypeScript Backend Toolkit - Express.js - Swagger (OpenAPI 3.0) Autogenered docs, Zod - BullMQ - ...

11 Aug 2024 195
playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and...

15 Nov 2019 66,091