The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.
MPL-2.0 License
Bot releases are visible (Hide)
replaceSpanNameRegex
config option (https://github.com/artilleryio/artillery/pull/2607) - Docs
sampleRate
tracing option set (https://github.com/artilleryio/artillery/pull/2601)plugins.publish-metrics.spans.exported
) when tracing is enabled. (https://github.com/artilleryio/artillery/pull/2597)This release brought to you by @InesNi @95th
Published by bernardobridge 7 months ago
ARTILLERY_CLOUD_API_KEY
environment variable from being picked up by tests on Fargate when using --dotenv
(https://github.com/artilleryio/artillery/pull/2585)ETIMEOUT
or ECONNRESET
(https://github.com/artilleryio/artillery/pull/2574) - docs
This release brought to you by @hassy @bernardobridge
Published by hassy 8 months ago
async
functions as hooks (#2522) - docs
--launch-config
flag (#2526)This release brought to you by @hassy @bernardobridge @InesNi
Published by bernardobridge 8 months ago
expect
from @playwright/test
work with Typescript support (https://github.com/artilleryio/artillery/pull/2488) - Documentation
config.bundling.external
to set packages that can’t be bundled as external (https://github.com/artilleryio/artillery/pull/2488) - Documentation
{{ $testId }}
in scenarios/config and vuContext.vars.$testId
in custom code (https://github.com/artilleryio/artillery/pull/2499)test_id
attribute to metrics and traces sent via the OpenTelemetry reporter (https://github.com/artilleryio/artillery/pull/2505) - Documentation
This release brought to you by @bernardobridge, @InesNi, @hassy
Published by bernardobridge 9 months ago
A new stable version of the Artillery CLI will now be released twice a month - on the first and third Monday of the month. Canary releases will continue being released from the main
branch.
testIdAttribute
setting (https://github.com/artilleryio/artillery/pull/2441) - Documentation
publish-metrics
pluginartillery-plugin-publish-metrics
on Lambda (https://github.com/artilleryio/artillery/pull/2457). This is a temporary workaround to get tests to run on Lambda reliably again$randomString(length)
utility to return correct number of characters and allow length larger than 10 (https://github.com/artilleryio/artillery/pull/2342)This release brought to you by @InesNi, @hassy, @bernardobridge, @Georgegriff
Published by bernardobridge 9 months ago
eu-west-2
, eu-west-3
, us-east-2
, us-west-2
, ca-central-1
, ap-east-1
, ap-northeast-2
, ap-southeast-1
, ap-southeast-2
, ap-northeast-1
, me-south-1
, sa-east-1
) (https://github.com/artilleryio/artillery/pull/2358, https://github.com/artilleryio/artillery/commit/29a4861d36ca4035e2d0269b0d4154c2fa7072b8)--variables
flag in run-fargate
command (https://github.com/artilleryio/artillery/pull/2391) - Documentation
--region
flag (https://github.com/artilleryio/artillery/commit/960b5ae505221bf0cbf7f840a5c5562923af65c6)browser.page.codes.<code>
metric (https://github.com/artilleryio/artillery/pull/2383) - Documentation
before
/ after
hooks that prevented the Playwright engine from working (https://github.com/artilleryio/artillery/pull/2396) - Documentation
test.step()
with OpenTelemetry Playwright tracing enabled (https://github.com/artilleryio/artillery/pull/2368)This release brought to you by @InesNi @hassy @bernardobridge
Published by hassy 11 months ago
Published by hassy 11 months ago
Published by hassy 11 months ago
think
steps in scenarios (#2272)metrics-by-endpoint
plugin is now enabled by default to capture per-URL metrics in HTTP tests. The metrics will be available only in the JSON report and not output to the console, unless the plugin is loaded explicitly by the user. (#2274)--cpu
and --memory
flags (#2301)$env
not resolving correctly in the config
section (#2265)--scenario-name
not picking up the correct scenario (#2314)fake-data
plugin to allow for realistic test data to be generated in your tests (#2318) (docs)ensure
checks (#2284)expect
plugin (#2298)test.step()
calls, page navigation events, and Core Web Vitals metrics (#2259)400
or higher (#2285)HTTP_REQUEST_CONTENT_LENGTH
, HTTP_USER_AGENT
and HTTP_FLAVOR
attributes (#2285)vu.uuid
) not always being set correctlypublish-metrics
plugin (#2259)FID
Core Web Vital metric not being emitted as expected--cpu
and --memory
flags (#2301)args
/on
options (#2171)match
errors in the reportThis release brought to you by @bernardobridge @InesNi @josh-coderpad @hassy
Published by bernardobridge 12 months ago
playwright
: Update Playwright version to latest (1.39.0
) (https://github.com/artilleryio/artillery/pull/2230).playwright
: Add a convenience test.step()
helper, allowing you to break your tests into steps that emit custom timing metrics (https://github.com/artilleryio/artillery/pull/2173) - Docs.--scenario-name
, allowing you to specify which scenario to run by its name in the test script (https://github.com/artilleryio/artillery/pull/2184) - Docs.mean
as a metric (https://github.com/artilleryio/artillery/pull/2163).context.scenario
(https://github.com/artilleryio/artillery/pull/2158) - Docs.socketio
: Allow nested options (e.g. namespace
) in emit without throwing validation error (https://github.com/artilleryio/artillery/commit/8b3f0351477709fad018df79efcadbfc7094558c, https://github.com/artilleryio/artillery/commit/2c714fc6a5e13aabb42f2e8f9ece85ac3a4d9f27) - Docs.socketio
: Allow emit
action to be an array of strings (https://github.com/artilleryio/artillery/pull/2167) - Docs.fargate
: Fix issue with --packages
flag.fargate
: Include apdex
plugin and playwright
engine by default in image, preventing additional installation in workers.capture
(https://github.com/artilleryio/artillery/pull/2127).config.engine
(https://github.com/artilleryio/artillery/pull/2101).apdex
- Allow frustrated
, satisfied
and tolerated
metrics to be checked with ensure
plugin (https://github.com/artilleryio/artillery/pull/2210, https://github.com/artilleryio/artillery/pull/2213).apdex
- Prevent plugin from crashing when using processor functions (https://github.com/artilleryio/artillery/pull/2195).publish-metrics
- Set headers as metadata for otel-grpc
exporter in OpenTelemetry reporter (https://github.com/artilleryio/artillery/commit/82cb40bc5ff39e744d09093420b7c79623997527) - Docs.publish-metrics
- Create endpoint path correctly so it works with managed instances of Dynatrace (https://github.com/artilleryio/artillery/commit/ccf622996b52c35ada681c6e26c5625f4cf6e1f3).publish-metrics
: Fix OpenTelemetry reporter debug logging (https://github.com/artilleryio/artillery/commit/f6ebc70d1a814158a8774a283f543caaa1c04086).This release brought to you by @InesNi, @bernardobridge, @kettanaito and @hassy.
Published by hassy about 1 year ago
ensure
check summary (#2074)appKey
support (#2067)This release brought to you by @bernardobridge @InesNi @yvesDenis @hassy
Published by hassy about 1 year ago
publish-metrics
plugin can now be configured to send Artillery metrics and events to Dynatrace. Read more in the docs. (https://github.com/artilleryio/artillery/commit/140ee923601f3fa5027a37ce67659b9ad7017719)config.http.defaults
- default headers and cookies for all requests in a test may now be set via config.http.defaults
(https://github.com/artilleryio/artillery/commit/938f9c472b415128062f0e38f0f4a8a14ee10db2)--variables
to be used anywhere in config
(https://github.com/artilleryio/artillery/commit/c08c6951095615711beea66d11e99dd9a0162cd8)publish-metrics
- Exit early if required config is missing for publish-metrics (https://github.com/artilleryio/artillery/commit/7871d398cfb462a5bc31d766df63762f82eb32fe)publish-metrics
- Reduce memory usage of the plugin (https://github.com/artilleryio/artillery/commit/338b070778d4aed99b53b3a4745e28c2f150e5e1)expect
- add useRequestNamesOnly
option (https://github.com/artilleryio/artillery/pull/1995)ensure
- fix bug where ensure config under config.plugins.ensure
would not get picked up (https://github.com/artilleryio/artillery/commit/d1136f37d492d61e91a41334549a08196980b26c)This release brought to you by @inesni @bernardobridge @hassy
Published by hassy over 1 year ago
publish-metrics
(#1963 & #1958)publish-metrics
(#1986)ensure
plugin reading its settings from config.ensure
config.payload
used along with config.environments
(#1964)With contributions from: @bernardobridge @InesNi @hassy
Published by hassy over 1 year ago
Published by hassy over 1 year ago
JMESPath is a popular query language for JSON supported by many tools such as AWS CLI and Azure CLI.
Skytrace now supports for setting expectations via JMESPath expressions. An expression that evaluates to a truthy value will result in a successful check.
# Check that the first object in the list of results has a title field:
skytrace http get http://lab.artillery.io/movies -e 'jmespath: "[0].title != null"'
Connected to http://lab.artillery.io
HTTP/1.1 200 OK
x-powered-by: Express
content-type: application/json; charset=utf-8
content-encoding: gzip
etag: W/"558f-PLaaZcOcQVqOfQBMyp4Dw3RncCE"
date: Thu, 22 Jun 2023 14:07:01 GMT
connection: keep-alive
keep-alive: timeout=5
transfer-encoding: chunked
server: Fly/a0b91024 (2023-06-13)
via: 1.1 fly.io
fly-request-id: 01H3HPW44SMJEXCJ6KSVRPQXPM-ams
DNS Lookup | TCP Connection | SSL Handshake | Time to First Byte | Content Transfer
1ms | 35ms | N/A | 53ms | 12ms |
| | | | |
1ms | | | |
36ms | | |
36ms | |
89ms |
total:101ms
Body stored in: /var/folders/yd/nyh8vtm17w92h5hm4pdhs8j00000gn/T/2023522-24253-1hq5ccb.f6czg.json
Expectations:
* ok jmespath [0].title != null
POST
requests not being availablePublished by hassy over 1 year ago
run:lambda
commandTests can now be run on AWS Lambda with the new run:lambda
command. (Running tests on AWS Lambda with run --platform aws:lambda
will continue working as expected too).
The new command makes running tests on AWS Lambda less verbose and more clear, for example:
artillery run:lambda --region eu-west-1 my-script.yml
Rather than:
artillery run --platform aws:lambda --platform-opt region=eu-west-1 my-script.yml
The expect
plugin can now evaluate JMESPath expressions as checks. If the expression evaluates to a truthy value, the check will pass. For example:
- get:
url: "/foo/1234"
expect:
jmespath: "title != null"
metrics-by-endpoint
reportingTwo new configuration options are now available to help reduce the number of unique metrics produced by by larger & more complex test suites:
stripQueryString
- ignore all query strings on a URL for purposes of reporting its metrics, i.e. /foo?bar=1
and /foo?baz=2
will be reported as the same /foo
endpointignoreUnnamedRequests
- ignore any requests that don't have a name
attribute set explicitly--lambda-role-arn
flag for run:lambda
command@cjsaurusrex
Published by hassy over 1 year ago
This release adds support for running tests on AWS Fargate. Scale out your load tests horizontally from any of the six supported regions.
artillery run:fargate --count 10 --region us-east-1 my-script.yml
Supported regions:
us-east-1
us-west-1
eu-west-1
eu-central-1
ap-south-1
ap-northeast-1
AWS Fargate is a serverless solution that requires no infrastructure to be managed, and unlike AWS Lambda does not impose a restriction on the duration of your load tests.
Artillery's Playwright integration is fully supported on Fargate, to let you run browser-based load tests at scale with ease.
Playwright support is now built into Artillery. The integration in artillery-engine-playwright
is now bundled into the Artillery CLI.
testFunction
attribute rather than flowFunction
flow
attribute may now be omitted in Playwright engine scenarioslambdaRoleArn
to AWS Lambda engine correctly (https://github.com/artilleryio/artillery/pull/1920)http.downloaded_bytes
metric$env
variable (the current way of accessing those via $processEnvironment
is still supported but will be deprecated in future)@tancredi @jpilla @BanksySan
Published by hassy over 1 year ago
skytrace run
on an existing Artillery script to run it as an end-to-end test.Published by hassy over 1 year ago
publish-metrics
plugin. You can now export data from Artillery tests to the Splunk observability platform.apdex
plugin. Calculate an industry standard measure of users' satisfaction based on HTTP metrics in Artillery tests.--record
and --key
flags for the run
command for enabling Artillery Cloud reporting for individual test runsmetrics-by-endpoint
-produced metrics may be customized nowPublished by hassy over 1 year ago
publish-metrics
plugin - configure Artillery to send test metrics to New Relic (docs)loop
under some conditions (#1826)