Bot releases are hidden (Show)

st2 - v3.8.1 Latest Release

Published by armab 10 months ago

3.8.1 - December 13, 2023

Fixed

Fix proxy auth mode in HA environments #5766 #6049
Contributed by @floatingstatic
Fix issue with linux pack actions failed to run remotely due to incorrect python shebang. #5983 #6042
Contributed by Ronnie Hoffmann (@ZoeLeah Schwarz IT KG)
Fix CI usses #6015
Contributed by Amanda McGuinness (@amanda11 intive)
Bumped paramiko to 2.10.5 to fix an issue with SSH Certs - paramiko/paramiko#2017 (security)
Contributed by @jk464
Avoid logging sensitive information in debug (fix #5977)
Fix codecov failures for stackstorm/st2 tests. #6035, #6046, #6048
Fix #4676, edge case where --inherit-env is skipped if the action has no parameters
Fix ST2 Client for Windows Clients. PWD is a Unix only Libary. #6071
Contributed by (@philipphomberger Schwarz IT KG)
Fix Snyk Security Finding Cross-site Scripting (XSS) in contrib/examples/sensors/echo_flask_app.py #6070
Contributed by (@philipphomberger Schwarz IT KG)
Update cryptography 3.4.7 -> 39.0.1, pyOpenSSL 21.0.0 -> 23.1.0, paramiko 2.10.5 -> 2.11.0 (security). #6055
Bumped eventlet to 0.33.3 and gunicorn to 21.2.0 to fix RecursionError bug in setting SSLContext minimum_version property. (security) #6061
Contributed by @jk464
Update orquesta to v1.6.0 to fix outdated dependencies (security). #6050
Fix KV value lookup in actions when RBAC is enabled #5934
Update version 3.1.15 of gitpython to 3.1.18 for py3.6 and to 3.1.37 for py3.8 (security). #6063
Update importlib-metadata from 3.10.1 to 4.8.3 for py3.6 and to 4.10.1 for py3.8 (security). #6072
Contributed by @jk464
For "local-shell-script" runner, on readonly filesystems, don't attempt to run chmod +x on script_action. Fixes #5591
Contributed by @jk464

Added

Move git clone to user_home/.st2packs #5845
Error on st2ctl status when running in Kubernetes. #5851
Contributed by @mamercad
Continue introducing pants <https://www.pantsbuild.org/docs>_ to improve DX (Developer Experience)
working on StackStorm, improve our security posture, and improve CI reliability thanks in part
to pants' use of PEX lockfiles. This is not a user-facing addition.
#5778 #5789 #5817 #5795 #5830 #5833 #5834 #5841 #5840 #5838 #5842 #5837 #5849 #5850
#5846 #5853 #5848 #5847 #5858 #5857 #5860 #5868 #5871 #5864 #5874 #5884 #5893 #5891
#5890 #5898 #5901 #5906 #5899 #5907 #5909 #5922 #5926 #5927 #5925 #5928 #5929 #5930
#5931 #5932 #5948 #5949 #5950
Contributed by @cognifloyd
Added a joint index to solve the problem of slow mongo queries for scheduled executions. #5805
Added publisher to ActionAlias to enable streaming ActionAlias create/update/delete events. #5763
Contributed by @ubaumann
Expose environment variable ST2_ACTION_DEBUG to all StackStorm actions.
Contributed by @maxfactor1
Python 3.9 support. #5730
Contributed by Amanda McGuinness (@amanda11 intive)
Run the st2 self-check in Github Actions and support the environment variable TESTS_TO_SKIP to skip tests when running st2-self-check. #5609
Contributed by @winem

Changed

Remove distutils dependencies across the project. #5992
Contributed by @AndroxxTraxxon

st2 - v3.8.0

Published by nzlosh almost 2 years ago

3.8.0 - November 18, 2022

Fixed

Fix redis SSL problems with sentinel #5660
Fix a bug in the pack config loader so that objects covered by an patternProperties schema
or arrays using additionalItems schema(s) can use encrypted datastore keys and have their
default values applied correctly. #5321

Contributed by @cognifloyd
Fixed st2client/st2client/base.py file to check for http_proxy and https_proxy environment variables for both lower and upper cases.

Contributed by @S-T-A-R-L-O-R-D
Fixed a bug where calling 'get_by_name' on client for getting key details was not returning any results despite key being stored. #5677

Contributed by @bharath-orchestral
Fixed st2client/st2client/base.py file to use https_proxy(not http_proxy) to check HTTPS_PROXY environment variables.

Contributed by @wfgydbu
Fixed schema utils to more reliably handle schemas that define nested arrays (object-array-object-array-string) as discovered in some
of the ansible installer RBAC tests (see #5684). This includes a test that reproduced the error so we don't hit this again. #5685
Fixed eventlet monkey patching so more of the unit tests work under pytest. #5689
Fix and reenable prance-based openapi spec validation, but make our custom x-api-model validation optional as the spec is out-of-date. #5709
Contributed by @cognifloyd
Fixed generation of st2.conf.sample to show correct syntax for [sensorcontainer].partition_provider (space separated key:value pairs). #5710
Contributed by @cognifloyd
Fix access to key-value pairs in workflow and action execution where RBAC rules did not get applied #5764

Contributed by @m4dcoder
Add backward compatibility to secret masking introduced in #5319 to prevent security-relative issues.
Migration to the new schema is required to take advantage of the full output schema validation. #5783

Contributed by @m4dcoder

Added

Added graceful shutdown for workflow engine. #5463
Contributed by @khushboobhatia01
Add ST2_USE_DEBUGGER env var as alternative to the --use-debugger cli flag. #5675
Contributed by @cognifloyd
Added purging of old tokens. #5679
Contributed by Amanda McGuinness (@amanda11 intive)
Begin introducing pants <https://www.pantsbuild.org/docs>_ to improve DX (Developer Experience)
working on StackStorm, improve our security posture, and improve CI reliability thanks in part
to pants' use of PEX lockfiles. This is not a user-facing addition. #5713 #5724 #5726 #5725 #5732 #5733 #5737 #5738 #5758 #5751 #5774 #5776 #5777 #5782
Contributed by @cognifloyd

Changed

BREAKING CHANGE for anyone that uses output_schema, which is disabled by default.
If you have [system].validate_output_schema = True in st2.conf AND you have added
output_schema to any of your packs, then you must update your action metadata.

output_schema must be a full jsonschema now. If a schema is not well-formed, we ignore it.
Now, output can be types other than object such as list, bool, int, etc.
This also means that all of an action's output can be masked as a secret.

To get the same behavior, you'll need to update your output schema.
For example, this schema:


    output_schema:
      property1:
        type: bool
      property2:
        type: str

should be updated like this:


    output_schema:
      type: object
      properties:
        property1:
          type: bool
        property2:
          type: str
      additionalProperties: false

#5319

Contributed by @cognifloyd

Changed the X-XSS-Protection HTTP header from 1; mode=block to 0 in the conf/nginx/st2.conf to align with the OWASP security standards. #5298

Contributed by @LiamRiddell
Use PEP 440 direct reference requirements instead of legacy PIP VCS requirements. Now, our *.requirements.txt files use
package-name@ git+https://url@version ; markers instead of git+https://url@version#egg=package-name ; markers. #5673
Contributed by @cognifloyd
Move from udatetime to ciso8601 for date functionality ahead of supporting python3.9 #5692
Contributed by Amanda McGuinness (@amanda11 intive)
Refactor tests to use python imports to identify test fixtures. #5699 #5702 #5703 #5704 #5705 #5706
Contributed by @cognifloyd
Refactor st2-generate-schemas so that logic is in an importable module. #5708
Contributed by @cognifloyd

Removed

Removed st2exporter service. It is unmaintained and does not get installed. It was
originally meant to help with analytics by exporting executions as json files that
could be imported into something like elasticsearch. Our code is now instrumented
to make a wider variety of stats available to metrics drivers. #5676
Contributed by @cognifloyd

st2 - v3.7.0

Published by cognifloyd over 2 years ago

https://stackstorm.com/2022/05/10/stackstorm-3-7-0-released/

3.7.0 - May 05, 2022

Added

Added st2 API get action parameters by ref. #5509
API endpoint /api/v1/actions/views/parameters/{action_id} accepts ref_or_id.
Contributed by @DavidMeu
Enable setting ttl for MockDatastoreService. #5468
Contributed by @ytjohn
Added st2 API and CLI command for actions clone operation.

API endpoint /api/v1/actions/{ref_or_id}/clone takes ref_or_id of source action.Request method body takes destination pack and action name. Request method body also takes optional parameter overwrite. overwrite = true in case of destination action already exists and to be overwritten.

CLI command st2 action clone <ref_or_id> <dest_pack> <dest_action> takes source ref_or_id, destination pack name and destination action name as mandatory arguments. In case destination already exists then command takes optional argument -f or --force to overwrite destination action. #5345

Contributed by @mahesh-orch.
Implemented RBAC functionality for existing KEY_VALUE_VIEW, KEY_VALUE_SET, KEY_VALUE_DELETE and new permission types KEY_VALUE_LIST, KEY_VALUE_ALL. RBAC is enabled in the st2.conf file. Access to a key value pair is checked in the KeyValuePair API controller. #5354
Contributed by @m4dcoder and @ashwini-orchestral
Added service deregistration on shutdown of a service. #5396
Contributed by @khushboobhatia01
Added pysocks python package for SOCKS proxy support. #5460
Contributed by @kingsleyadam
Added support for multiple LDAP hosts to st2-auth-ldap. #5535, https://github.com/StackStorm/st2-auth-ldap/pull/100
Contributed by @ktyogurt
Implemented graceful shutdown for action runner. Enabled graceful_shutdown in st2.conf file. #5428
Contributed by @khushboobhatia01
Enhanced 'search' operator to allow complex criteria matching on payload items. #5482
Contributed by @erceth
Added cancel/pause/resume requester information to execution context. #5554
Contributed by @khushboobhatia01
Added trigger.headers_lower to webhook trigger payload. This allows rules to match webhook triggers without dealing with the case-sensitive nature of trigger.headers, as triggers.headers_lower providers the same headers, but with the header name lower cased. #5038
Contributed by @Rand01ph
Added support to override enabled parameter of resources. #5506
Contributed by Amanda McGuinness (@amanda11 Intive)
Add new api.auth_cookie_secure and api.auth_cookie_same_site config options which specify values which are set for secure and SameSite attribute for the auth cookie we set when authenticating via token / api key in query parameter value (e.g. via st2web).

For security reasons, api.auth_cookie_secure defaults to True. This should only be changed to False if you have a valid reason to not run StackStorm behind HTTPs proxy.

Default value for api.auth_cookie_same_site is lax. If you want to disable this functionality so it behaves the same as in the previous releases, you can set that option to None.

#5248
Contributed by @Kami.
Add new st2 action-alias test <message string> CLI command which allows users to easily test action alias matching and result formatting.

This command will first try to find a matching alias (same as st2 action-alias match command) and if a match is found, trigger an execution (same as st2 action-alias execute command) and format the execution result.

This means it uses exactly the same flow as commands on chat, but the interaction avoids chat and hubot which should make testing and developing aliases easier and faster. #5143

#5143
Contributed by @Kami.
Add new credentials.basic_auth = username:password CLI configuration option.

This argument allows client to use additional set of basic auth credentials when talking to the StackStorm API endpoints (api, auth, stream) - that is, in addition to the token / api key native StackStorm auth.

This allows for simple basic auth based multi factor authentication implementation for installations which don't utilize SSO.

#5152
Contributed by @Kami.
Add new audit message when a user has decrypted a key whether manually in the container (st2 key get [] --decrypt)
or through a workflow with a defined config. #5594
Contributed by @dmork123
Added garbage collection for rule_enforcement and trace models #5596/5602
Contributed by Amanda McGuinness (@amanda11 intive)
Added garbage collection for workflow execution and task execution objects #4924
Contributed by @srimandaleeka01 and @amanda11

Changed

Minor updates for RockyLinux. #5552

Contributed by Amanda McGuinness (@amanda11 intive)
Bump black to v22.3.0 - This is used internally to reformat our python code. #5606
Updated paramiko version to 2.10.3 to add support for more key verification algorithms. #5600

Fixed

Fix deserialization bug in st2 API for url encoded payloads. #5536
Contributed by @sravs-dev
Fix issue of WinRM parameter passing fails for larger scripts.#5538
Contributed by @ashwini-orchestral
Fix Type error for time_diff critera comparison. convert the timediff value as float to match timedelta.total_seconds() return. #5462
Contributed by @blackstrip
Fix issue with pack option not working when running policy list cli #5534
Contributed by @momokuri-3
Fix exception thrown if action parameter contains {{ or {% and no closing jinja characters. #5556
contributed by @guzzijones12
Link shutdown routine and sigterm handler to main thread #5555
Contributed by @khushboobhatia01
Change compound index for ActionExecutionDB to improve query performance #5568
Contributed by @khushboobhatia01
Fix build issue due to MarkUpSafe 2.1.0 removing soft_unicode
Contributed by Amanda McGuinness (@amanda11 intive) #5581
Fixed regression caused by #5358. Use string lock name instead of object ID. #5484
Contributed by @khushboobhatia01
Fix st2-self-check script reporting falsey success when the nested workflows runs failed. #5487
Fix actions from the contrib/linux pack that fail on CentOS-8 but work on other operating systems and distributions. (bug fix) #4999 #5004
Reported by @blag and @dove-young contributed by @winem.
Use byte type lock name which is supported by all tooz drivers. #5529
Contributed by @khushboobhatia01
Fixed issue where pack index searches are ignoring no_proxy #5497
Contributed by @minsis
Fixed trigger references emitted by linux.file_watch.line. #5467

Prior to this patch multiple files could be watched but the rule reference of last registered file
would be used for all trigger emissions causing rule enforcement to fail. References are now tracked
on a per file basis and used in trigger emissions.

Contributed by @nzlosh
Downgrade tenacity as tooz dependency on tenacity has always been < 7.0.0 #5607
Contributed by @khushboobhatia01
Pin typing-extensions<4.2 (used indirectly by st2client) to maintain python 3.6 support. #5638

st2 -

Published by winem almost 3 years ago

https://stackstorm.com/2021/12/16/stackstorm-v3-6-0-released/

3.6.0 - October 29, 2021

Added

Added possibility to add new values to the KV store via CLI without leaking them to the shell history. #5164
st2.conf is now the only place to configure ports for st2api, st2auth, and st2stream.

We replaced the static .socket sytemd units in deb and rpm packages with a python-based generator for the
st2api, st2auth, and st2stream services. The generators will get <ip>:<port> from st2.conf
to create the .socket files dynamically. #5286 and st2-packages#706

Contributed by @nzlosh

Changed

Modified action delete API to delete action files from disk along with backward compatibility.

From CLI st2 action delete <pack>.<action> will delete only action database entry.
From CLI st2 action delete --remove-files <pack>.<action> or st2 action delete -r <pack>.<action>
will delete action database entry along with files from disk.

API action DELETE method with {"remove_files": true} argument in json body will remove database
entry of action along with files from disk.
API action DELETE method with {"remove_files": false} or no additional argument in json body will remove
only action database entry. #5304, #5351, #5360

Contributed by @mahesh-orch.
Removed --python3 deprecated flag from st2client. #5305

Contributed by Amanda McGuinness (@amanda11 Ammeon Solutions)

Contributed by @blag.
Fixed __init__.py files to use double quotes to better align with black linting #5299

Contributed by @blag.
Reduced minimum TTL on garbage collection for action executions and trigger instances from 7 days to 1 day. #5287

Contributed by @ericreeves.
update db connect mongo connection test - isMaster MongoDB command depreciated, switch to ping #5302, #5341

Contributed by @lukepatrick
Actionrunner worker shutdown should stop Kombu consumer thread. #5338

Contributed by @khushboobhatia01
Move to using Jinja sandboxed environment #5359

Contributed by Amanda McGuinness (@amanda11 Ammeon Solutions)
Pinned python module networkx to versions between 2.5.1(included) and 2.6(excluded) because Python v3.6 support was dropped in v2.6.
Also pinned decorator==4.4.2 (dependency of networkx<2.6) to work around missing python 3.8 classifiers on decorator's wheel. #5376

Contributed by @nzlosh
Add new --enable-profiler flag to all the servies. This flag enables cProfiler based profiler
for the service in question and dumps the profiling data to a file on process
exit.

This functionality should never be used in production, but only in development environments or
similar when profiling code. #5199

Contributed by @Kami.
Add new --enable-eventlet-blocking-detection flag to all the servies. This flag enables
eventlet long operation / blocked main loop logic which throws an exception if a particular
code blocks longer than a specific duration in seconds.

This functionality should never be used in production, but only in development environments or
similar when debugging code. #5199
Silence pylint about dev/debugging utility (tools/direct_queue_publisher.py) that uses pika because kombu
doesn't support what it does. If anyone uses that utility, they have to install pika manually. #5380
Fixed version of cffi as changes in 1.15.0 meant that it attempted to load libffi.so.8. #5390

Contributed by @amanda11, Ammeon Solutions
Updated Bash installer to install latest RabbitMQ version rather than out-dated version available
in OS distributions.

Contributed by @amanda11, Ammeon Solutions

Fixed

Correct error reported when encrypted key value is reported, and another key value parameter that requires conversion is present. #5328
Contributed by @amanda11, Ammeon Solutions
Make update_executions() atomic by protecting the update with a coordination lock. Actions, like workflows, may have multiple
concurrent updates to their execution state. This makes those updates safer, which should make the execution status more reliable. #5358

Contributed by @khushboobhatia01
Fix "not iterable" error for output_schema handling. If a schema is not well-formed, we ignore it.
Also, if action output is anything other than a JSON object, we do not try to process it any more.
output_schema will change in a future release to support non-object output. #5309

Contributed by @guzzijones
core.inject_trigger: resolve trigger payload shadowing by deprecating trigger param in favor of trigger_name.
trigger param is still available for backwards compatibility, but will be removed in a future release. #5335 and #5383

Contributed by @mjtice

st2 - v3.5.0

Published by amanda11 over 3 years ago

Added

Added web header settings for additional security hardening to nginx.conf: X-Frame-Options,
Strict-Transport-Security, X-XSS-Protection and server-tokens. #5183

Contributed by @shital.
Added support for limit and offset argument to the list_values data store
service method (#5097 and #5171).

Contributed by @anirudhbagri.
Various additional metrics have been added to the action runner service to provide for better
operational visibility. (improvement) #4846

Contributed by @Kami.
Added sensor model to list of JSON schemas auto-generated by make schemasgen that can be used
by development tools to validate pack contents. (improvement)
Added the command line utility st2-validate-pack that can be used by pack developers to
validate pack contents. (improvement)
Fix a bug in the API and CLI code which would prevent users from being able to retrieve resources
which contain non-ascii (utf-8) characters in the names / references. (bug fix) #5189

Contributed by @Kami.
Fix a bug in the API router code and make sure we return correct and user-friendly error to the
user in case we fail to parse the request URL / path because it contains invalid or incorrectly
URL encoded data.

Previously such errors weren't handled correctly which meant original exception with a stack
trace got propagated to the user. (bug fix) #5189

Contributed by @Kami.
Make redis the default coordinator backend.
Fix a bug in the pack config loader so that objects covered by an additionalProperties schema
can use encrypted datastore keys and have their default values applied correctly. #5225

Contributed by @cognifloyd.
Add new database.compressors and database.zlib_compression_level config option which
specifies compression algorithms client supports for network / transport level compression
when talking to MongoDB.

Actual compression algorithm used will be then decided by the server and depends on the
algorithms which are supported by the server + client.

Possible / valid values include: zstd, zlib. Keep in mind that zstandard (zstd) is only supported
by MongoDB >= 4.2.

Our official Debian and RPM packages bundle zstandard dependency by default which means
setting this value to zstd should work out of the box as long as the server runs
MongoDB >= 4.2. #5177

Contributed by @Kami.
Add support for compressing the payloads which are sent over the message bus. Compression is
disabled by default and user can enable it by setting messaging.compression config option
to one of the following values: zstd, lzma, bz2, gzip.

In most cases we recommend using zstd (zstandard) since it offers best trade off between
compression ratio and number of CPU cycles spent for compression and compression.

How this will affect the deployment and throughput is very much user specific (workflow and
resources available). It may make sense to enable it when generic action trigger is enabled
and when working with executions with large textual results. #5241

Contributed by @Kami.
Mask secrets in output of an action execution in the API if the action has an output schema
defined and one or more output parameters are marked as secret. #5250

Contributed by @mahesh-orch.

Changed

All the code has been refactored using black and black style is automatically enforced and
required for all the new code. (#5156)

Contributed by @Kami.
Default nginx config (conf/nginx/st2.conf) which is used by the installer and Docker
images has been updated to only support TLS v1.2 and TLS v1.3 (support for TLS v1.0 and v1.1
has been removed).

Keep in mind that TLS v1.3 will only be used when nginx is running on more recent distros
where nginx is compiled against OpenSSL v1.1.1 which supports TLS 1.3. #5183 #5216

Contributed by @Kami and @shital.
Add new -x argument to the st2 execution get command which allows
result field to be excluded from the output. (improvement) #4846
Update st2 execution get <id> command to also display execution log attribute which
includes execution state transition information.

By default end_timestamp attribute and duration attribute displayed in the command
output only include the time it took action runner to finish running actual action, but it
doesn't include the time it it takes action runner container to fully finish running the
execution - this includes persisting execution result in the database.

For actions which return large results, there could be a substantial discrepancy - e.g.
action itself could finish in 0.5 seconds, but writing data to the database could take
additional 5 seconds after the action code itself was executed.

For all purposes until the execution result is persisted to the database, execution is
not considered as finished.

While writing result to the database action runner is also consuming CPU cycles since
serialization of large results is a CPU intensive task.

This means that "elapsed" attribute and start_timestamp + end_timestamp will make it look
like actual action completed in 0.5 seconds, but in reality it took 5.5 seconds (0.5 + 5 seconds).

Log attribute can be used to determine actual duration of the execution (from start to
finish). (improvement) #4846

Contributed by @Kami.
Various internal improvements (reducing number of DB queries, speeding up YAML parsing, using
DB object cache, etc.) which should speed up pack action registration between 15-30%. This is
especially pronounced with packs which have a lot of actions (e.g. aws one).
(improvement) #4846

Contributed by @Kami.
Underlying database field type and storage format for the Execution, LiveAction,
WorkflowExecutionDB, TaskExecutionDB and TriggerInstanceDB database models has
changed.

This new format is much faster and efficient than the previous one. Users with larger executions
(executions with larger results) should see the biggest improvements, but the change also scales
down so there should also be improvements when reading and writing executions with small and
medium sized results.

Our micro and end to benchmarks have shown improvements up to 15-20x for write path (storing
model in the database) and up to 10x for the read path.

To put things into perspective - with previous version, running a Python runner action which
returns 8 MB result would take around ~18 seconds total, but with this new storage format, it
takes around 2 seconds (in this context, duration means the from the time the execution was
scheduled to the time the execution model and result was written and available in the database).

The difference is even larger when working with Orquesta workflows.

Overall performance improvement doesn't just mean large decrease in those operation timings, but
also large overall reduction of CPU usage - previously serializing large results was a CPU
intensive time since it included tons of conversions and transformations back and forth.

The new format is also around 10-20% more storage efficient which means that it should allows
for larger model values (MongoDB document size limit is 16 MB).

The actual change should be fully opaque and transparent to the end users - it's purely a
field storage implementation detail and the code takes care of automatically handling both
formats when working with those object.

Same field data storage optimizations have also been applied to workflow related database models
which should result in the same performance improvements for Orquesta workflows which pass larger
data sets / execution results around.

Trigger instance payload field has also been updated to use this new field type which should
result in lower CPU utilization and better throughput of rules engine service when working with
triggers with larger payloads.

This should address a long standing issue where StackStorm was reported to be slow and CPU
inefficient with handling large executions.

If you want to migrate existing database objects to utilize the new type, you can use
st2common/bin/migrations/v3.5/st2-migrate-db-dict-field-values migration
script. (improvement) #4846

Contributed by @Kami.
Add new result_size field to the ActionExecutionDB model. This field will only be
populated for executions which utilize new field storage format.

It holds the size of serialzed execution result field in bytes. This field will allow us to
implement more efficient execution result retrieval and provide better UX since we will be
able to avoid loading execution results in the WebUI for executions with very large results
(which cause browser to freeze). (improvement) #4846

Contributed by @Kami.
Add new /v1/executions/<id>/result[?download=1&compress=1&pretty_format=1] API endpoint
which can be used used to retrieve or download raw execution result as (compressed) JSON file.

This endpoint will primarily be used by st2web when executions produce very large results so
we can avoid loading, parsing and formatting those very large results as JSON in the browser
which freezes the browser window / tab. (improvement) #4846

Contributed by @Kami.
Update jinja2 dependency to the latest stable version (2.11.3). #5195
Update pyyaml dependency to the latest stable version (5.4). #5207
Update various dependencies to latest stable versions (bcrypt, appscheduler, pytz,
python-dateutil, psutil, passlib, gunicorn, flex, cryptography.
eventlet, greenlet, webob , mongoengine, pymongo, requests,
pyyaml, kombu, amqp, python-ldap).

#5215, https://github.com/StackStorm/st2-auth-ldap/pull/94

Contributed by @Kami.
Update code and dependencies so it supports Python 3.8 and Mongo DB 4.4 #5177

Contributed by @nzloshm @winem @Kami.
StackStorm Web UI (st2web) has been updated to not render and display execution results
larger than 200 KB directly in the history panel in the right side bar by default anymore.
Instead a link to view or download the raw result is displayed.

Execution result widget was never optimized to display very large results (especially for
executions which return large nested dictionaries) so it would freeze and hang the whole
browser tab / window when trying to render / display large results.

If for some reason you want to revert to the old behavior (this is almost never a good idea
since it will cause browser to freeze when trying to display large results), you can do that by
setting max_execution_result_size_for_render option in the config to a very large value (e.g.
max_execution_result_size_for_render: 16 * 1024 * 1024).

https://github.com/StackStorm/st2web/pull/868

Contributed by @Kami.
Some of the config option registration code has been refactored to ignore "option already
registered" errors. That was done as a work around for an occasional race in the tests and
also to make all of the config option registration code expose the same consistent API. #5234

Contributed by @Kami.
Update pyywinrm dependency to the latest stable version (0.4.1). #5212

Contributed by @chadpatt .
Monkey patch on st2stream earlier in flow #5240

Contributed by Amanda McGuinness (@amanda11 Ammeon Solutions)
Support % in CLI arguments by reading the ConfigParser() arguments with raw=True.

This removes support for '%' interpolations on the configuration arguments.

See https://docs.python.org/3.8/library/configparser.html#configparser.ConfigParser.get for
further details. #5253

Contributed by @winem.
Remove duplicate host header in the nginx config for the auth endpoint.
Update orquesta to v1.4.0.

Improvements

CLI has been updated to use or orjson when parsing API response and C version of the YAML
safe dumper when formatting execution result for display. This should result in speed up when
displaying execution result (st2 execution get, etc.) for executions with large results.

When testing it locally, the difference for execution with 8 MB result was 18 seconds vs ~6
seconds. (improvement) #4846

Contributed by @Kami.
Update various Jinja functiona to utilize C version of YAML safe_{load,dump} functions and
orjson for better performance. (improvement) #4846

Contributed by @Kami.
For performance reasons, use udatetime library for parsing ISO8601 / RFC3339 date strings
where possible. (improvement) #4846

Contributed by @Kami.
Speed up service start up time by speeding up runners registration on service start up by
re-using existing stevedore ExtensionManager instance instead of instantiating new
DriverManager instance per extension which is not necessary and it's slow since it requires
disk / pkg resources scan for each extension. (improvement) #5198

Contributed by @Kami.
Add new ?max_result_size query parameter filter to the GET /v1/executiond/<id> API
endpoint.

This query parameter allows clients to implement conditional execution result retrieval and
only retrieve the result field if it's smaller than the provided value.

This comes handy in the various client scenarios (such as st2web) where we don't display and
render very large results directly since it allows to speed things up and decrease amount of
data retrieved and parsed. (improvement) #5197

Contributed by @Kami.
Update default nginx config which is used for proxying API requests and serving static
content to only allow HTTP methods which are actually used by the services (get, post, put,
delete, options, head).

If a not-allowed method is used, nginx will abort the request early and return 405 status
code. #5193

Contributed by @ashwini-orchestral
Update default nginx config which is used for proxying API requests and serving static
content to not allow range requests. #5193

Contributed by @ashwini-orchestral
Drop unused python dependencies: prometheus_client, python-gnupg, more-itertools, zipp. #5228

Contributed by @cognifloyd.
Update majority of the "resource get" CLI commands (e.g. st2 execution get,
st2 action get, st2 rule get, st2 pack get, st2 apikey get, st2 trace get,
st2 key get, st2 webhook get, st2 timer get, etc.) so they allow for retrieval
and printing of information for multiple resources using the following notation:
st2 <resource> get <id 1> <id 2> <id n>, e.g. st2 action.get pack.show packs.get packs.delete

This change is fully backward compatible when retrieving only a single resource (aka single
id is passed to the command).

When retrieving a single source the command will throw and exit with non-zero if a resource is
not found, but when retrieving multiple resources, command will just print an error and
continue with printing the details of any other found resources. (new feature) #4912

Contributed by @Kami.

Fixed

Refactor spec_loader util to use yaml.load with SafeLoader. (security)
Contributed by @ashwini-orchestral
Import ABC from collections.abc for Python 3.10 compatibility. (#5007)
Contributed by @tirkarthi
Updated to use virtualenv 20.4.0/PIP20.3.3 and fixate-requirements to work with PIP 20.3.3 #512
Contributed by Amanda McGuinness (@amanda11 Ammeon Solutions)
Fix st2 execution get --with-schema flag. (bug fix) #4846

Contributed by @Kami.
Fix SensorTypeAPI schema to use class_name instead of name since documentation for pack
development uses class_name and registrar used to load sensor to database assign class_name
to name in the database model. (bug fix)
Updated paramiko version to 2.7.2, to go with updated cryptography to prevent problems
with ssh keys on remote actions. #5201

Contributed by Amanda McGuinness (@amanda11 Ammeon Solutions)
Update rpm package metadata and fix Provides section for RHEL / CentOS 8 packages.

In the previous versions, RPM metadata would incorrectly signal that the st2 package
provides various Python libraries which it doesn't (those Python libraries are only used
internally for the package local virtual environment).

https://github.com/StackStorm/st2-packages/pull/697

Contributed by @Kami.
Make sure st2common.util.green.shell.run_command() doesn't leave stray / zombie processes
laying around in some command timeout scenarios. #5220

Contributed by @r0m4n-z.
Fix support for skipping notifications for workflow actions. Previously if action metadata
specified an empty list for notify parameter value, that would be ignored / not handled
correctly for workflow (orquesta, action chain) actions. #5221 #5227

Contributed by @khushboobhatia01.
Clean up to remove unused methods in the action execution concurrency policies. #5268

st2 - v3.4.1

Published by blag over 3 years ago

https://stackstorm.com/2021/03/10/stackstorm-v3-4-1-security-fix/

Added

Update the service start up code to warn if the service locale encoding is not set to utf-8 #5184
Contributed by @Kami

Changed

Use sudo -E to fix GitHub Actions tests #5187
Contributed by @cognifloyd
Properly handle unicode strings in logs #5184
Contributed by @Kami

st2 - v3.4.0

Published by blag over 3 years ago

https://stackstorm.com/2021/03/04/v3-4-0-released/

Added

Added support for GitLab SSH URLs on pack install and download actions. (improvement) #5050
Contributed by @asthLucas
Added st2-rbac-backend pip requirements for RBAC integration. (new feature) #5086
Contributed by @hnanchahal
Added notification support for err-stackstorm. (new feature) #5051
Added st2-auth-ldap pip requirements for LDAP auth integartion. (new feature) #5082
Contributed by @hnanchahal
Added --register-recreate-virtualenvs flag to st2ctl reload to recreate virtualenvs from scratch. (part of upgrade instructions) [#5167]
Contributed by @winem and @blag

Changed

Updated deprecation warning for python 2 pack installs, following python 2 support removal. #5099
Contributed by @amanda11
Improve the st2-self-check script to echo to stderr and exit if it isn't run with a
ST2_AUTH_TOKEN or ST2_API_KEY environment variable. (improvement) #5068
Added timeout parameter for packs.install action to help with long running installs that exceed the
default timeout of 600 sec which is defined by the python_script action runner (improvement) #5084

Contributed by @hnanchahal
Upgraded cryptography version to 3.2 to avoid CVE-2020-25659 (security) #5095
Converted most CI jobs from Travis to GitHub Actions (all except Integration tests).

Contributed by @nmaludy, @winem, and @blag
Updated cryptography dependency to version 3.3.2 to avoid CVE-2020-36242 (security) #5151

Fixed

Pin chardet version as newest version was incompatible with pinned requests version #5101
Contributed by @amanda11
Fixed issue were st2tests was not getting installed using pip because no version was specified.
Contributed by @anirudhbagri
Added monkey patch fix to st2stream to enable it to work with mongodb via SSL. (bug fix) #5078 #5091
Fix nginx buffering long polling stream to client. Instead of waiting for closed connection
wait for final event to be sent to client. (bug fix) #4842 #5042

Contributed by @guzzijones
StackStorm now explicitly decodes pack files as utf-8 instead of implicitly as ascii (bug fix)
#5106, #5107
Fix incorrect array parameter value casting when executing action via chatops or using
POST /aliasexecution/match_and_execute API endpoint. The code would incorrectly assume the
value is always a string, but that may not be the cast - they value could already be a list and
in this case we don't want any casting to be performed. (bug fix) #5141

Contributed by @Kami.
Fix @parameter_name=/path/to/file/foo.json notation in the st2 run command which didn't
work correctly because it didn't convert read bytes to string / unicode type. (bug fix) #5140

Contributed by @Kami.
Fix broken st2 action-alias execute command and make sure it works
correctly. (bug fix) #5138

Contributed by @Kami.

Removed

Removed --python3 pack install option #5100
Contributed by @amanda11
Removed submit-debug-info tool and the st2debug component #5103
Removed check-licence script (cleanup) #5092

Contributed by @kroustou
Updated Makefile and CI to use Python 3 only, removing Python 2 (cleanup) #5090

Contributed by @blag
Remove st2resultstracker from st2ctl, the development environment and the st2actions setup.py (cleanup) #5108

Contributed by @winem

st2 - v3.3.0

Published by nmaludy almost 4 years ago

https://stackstorm.com/2020/10/22/stackstorm-v3-3-0-released/

Added

Add make command to autogen JSON schema from the models of action, rule, etc. Add check
to ensure update to the models require schema to be regenerated. (new feature)
Improved st2sensor service logging message when a sensor will not be loaded when assigned to a
different partition (@punkrokk) #4991
Add support for a configurable connect timeout for SSH connections as requested in #4715
by adding the new configuration parameter ssh_connect_timeout to the ssh_runner
group in st2.conf. (new feature) #4914

This option was requested by Harry Lee (@tclh123) and contributed by Marcel Weinberg (@winem).
Added a FAQ for the default user/pass for the tools/launch_dev.sh script and print out the
default pass to screen when the script completes. (improvement) #5013

Contributed by @punkrokk
Added deprecation warning if attempt to install or download a pack that only supports
Python 2. (new feature) #5037

Contributed by @amanda11
Added deprecation warning to each StackStorm service log, if service is running with
Python 2. (new feature) #5043

Contributed by @amanda11
Added deprecation warning to st2ctl, if st2 python version is Python 2. (new feature) #5044

Contributed by @amanda11

Changed

Switch to MongoDB 4.0 as the default version starting with all supported OS's in st2
v3.3.0 (improvement) #4972

Contributed by @punkrokk
Added an enhancement where ST2api.log no longer reports the entire traceback when trying to get a datastore value
that does not exist. It now reports a simplified log for cleaner reading. Addresses and Fixes #4979. (improvement) #4981

Contributed by Justin Sostre (@saucetray)
The built-in st2.action.file_writen trigger has been renamed to st2.action.file_written
to fix the typo (bug fix) #4992
Renamed reference to the RBAC backend/plugin from enterprise to default. Updated st2api
validation to use the new value when checking RBAC configuration. Removed other references to
enterprise for RBAC related contents. (improvement)
Remove authentication headers St2-Api-Key, X-Auth-Token and Cookie from webhook payloads to
prevent them from being stored in the database. (security bug fix) #4983

Contributed by @potato and @knagy
Updated orquesta to version v1.2.0.

Fixed

Fixed a bug where type attribute was missing for netstat action in linux pack. Fixes #4946

Reported by @scguoi and contributed by Sheshagiri (@sheshagiri)
Fixed a bug where persisting Orquesta to the MongoDB database returned an error
message: key 'myvar.with.period' must not contain '.'. This happened anytime an
input, output, publish or context var contained a key with a . within
the name (such as with hostnames and IP addresses). This was a regression introduced by
trying to improve performance. Fixing this bug means we are sacrificing performance of
serialization/deserialization in favor of correctness for persisting workflows and
their state to the MongoDB database. (bug fix) #4932

Contributed by Nick Maludy (@nmaludy Encore Technologies)
Fix a bug where passing an empty list to a with items task in a subworkflow causes
the parent workflow to be stuck in running status. (bug fix) #4954
Fixed a bug in the example nginx HA template declared headers twice (bug fix) #4966
Contributed by @punkrokk
Fixed a bug in the paramiko_ssh runner where SSH sockets were not getting cleaned
up correctly, specifically when specifying a bastion host / jump box. (bug fix) #4973

Contributed by Nick Maludy (@nmaludy Encore Technologies)
Fixed a bytes/string encoding bug in the linux.dig action so it should work on Python 3
(bug fix) #4993
Fixed a bug where a python3 sensor using ssl needs to be monkey patched earlier. See also #4832, #4975 and gevent/gevent#1016 (bug fix) #4976

Contributed by @punkrokk
Fixed bug where action information in RuleDB object was not being parsed properly
because mongoengine EmbeddedDocument objects were added to JSON_UNFRIENDLY_TYPES and skipped.
Removed this and added if to use to_json method so that mongoengine EmbeddedDocument
are parsed properly.

Contributed by Bradley Bishop (@bishopbm1 Encore Technologies)
Fix a regression when updated dnspython pip dependency resulted in
st2 services unable to connect to mongodb remote host (bug fix) #4997
Fixed a regression in the linux.dig action on Python 3. (bug fix) #4993

Contributed by @blag
Fixed a bug in pack installation logging code where unicode strings were not being
interpolated properly. (bug fix)

Contributed by @misterpah
Fixed a compatibility issue with the latest version of the logging library API
where the find_caller() function introduced some new variables. (bug fix) #4923

Contributed by @Dahfizz9897
Fixed another logging compatibility issue with the logging API in Python 3.
The return from the logging.findCaller() implementation now expects a 4-element
tuple. Also, in Python 3 there are new arguments that are passed in and needs to be
acted upon, specificall stack_info that determines the new 4th element in the returned
tuple. (bug fix) #5057

Contributed by Nick Maludy (@nmaludy Encore Technologies)

Removed

Removed Mistral workflow engine (deprecation) #5011

Contributed by Amanda McGuinness (@amanda11 Ammeon Solutions)
Removed CentOS 6/RHEL 6 support #4984

Contributed by Amanda McGuinness (@amanda11 Ammeon Solutions)
Removed our fork of codecov-python for CI and have switched back to the upstream version (improvement) #5002

st2 - v3.2.0

Published by armab over 4 years ago

https://stackstorm.com/2020/04/30/stackstorm-v3-2-0-released/

Added

Add support for blacklisting / whitelisting hosts to the HTTP runner by adding new
url_hosts_blacklist and url_hosts_whitelist runner attribute. (new feature)
stackstorm/st2#4757
Add user parameter to re_run method of st2client. stackstorm/st2#4785
Install pack dependencies automatically. stackstorm/st2#4769
Add support for immutable_parameters on Action Aliases. This feature allows default
parameters to be supplied to the action on every execution of the alias. stackstorm/st2#4786
Add get_entrypoint() method to ActionResourceManager attribute of st2client.
stackstorm/st2#4791
Add support for orquesta task retry. (new feature)
Add config option scheduler.execution_scheduling_timeout_threshold_min to better control the cleanup of scheduled actions that were orphaned. stackstorm/st2#4886

Changed

Install pack with the latest tag version if it exists when branch is not specialized.
(improvement) stackstorm/st2#4743
Implement "continue" engine command to orquesta workflow. (improvement) stackstorm/st2#4740
Update various internal dependencies to latest stable versions (apscheduler, eventlet,
kombu, amqp, pyyaml, mongoengine, python-gnupg, paramiko, tooz, webob, bcrypt).

Latest version of mongoengine should show some performance improvements (5-20%) when
writing very large executions (executions with large results) to the database. stackstorm/st2#4767
Improved development instructions in requirements.txt and dist_utils.py comment headers
(improvement) stackstorm/st2#4774
Add new actionrunner.stream_output_buffer_size config option and default it to -1
(previously default value was 0). This should result in a better performance and smaller
CPU utilization for Python runner actions which produce a lot of output.
(improvement)

Reported and contributed by Joshua Meyer (@jdmeyer3) stackstorm/st2#4803
Add new action_runner.pip_opts st2.conf config option which allows user to specify a list
of command line option which are passed to pip install command when installing pack
dependencies into a pack specific virtual environment. stackstorm/st2#4792
Refactor how orquesta handles individual item result for with items task. Before the fix,
when there are a lot of items and/or result size for each item is huge, there is a negative
performance impact on write to the database when recording the conductor state. (improvement)
Remove automatic rendering of workflow output when updating task state for orquesta workflows.
This caused workflow output to render incorrectly in certain use case. The render_workflow_output
function must be called separately. (improvement)
Update various internal dependencies to latest stable versions (cryptography, jinja2, requests,
apscheduler, eventlet, amqp, kombu, semver, six) stackstorm/st2#4819 (improvement)
Improve MongoDB connection timeout related code. Connection and server selection timeout is now
set to 3 seconds. Previously a default value of 30 seconds was used which means that for many
connection related errors, our code would first wait for this timeout to be reached (30 seconds)
before returning error to the end user. stackstorm/st2#4834
Upgrade pymongo to the latest stable version (3.10.0.). stackstorm/st2#4835 (improvement)
Updated Paramiko to v2.7.1 to support new PEM ECDSA key formats stackstorm/st2#4901 (improvement)
Remove .scrutinizer.yml config file. No longer used.
Convert escaped dict and dynamic fields in workflow db models to normal dict and dynamic fields.
(performnce improvement)
Add support for PEP 508 <https://www.python.org/dev/peps/pep-0508/stackstorm/st2#environment-markers>_
environment markers in generated requirements.txt files. (improvement) stackstorm/st2#4895
Use pip-compile from pip-tools instead of pip-conflict-checker (improvement) stackstorm/st2#4896
Refactor how inbound criteria for join task in orquesta workflow is evaluated to count by
task completion instead of task transition. (improvement)
The workflow engine orquesta is updated to v1.1.0 for the st2 v3.2 release. The version upgrade
contains various new features and bug fixes. Please review the release notes for the full list of
changes at https://github.com/StackStorm/orquesta/releases/tag/v1.1.0 and the st2 upgrade notes
for potential impact. (improvement)

Fixed

Fix the action query when filtering tags. The old implementation returned actions which have the
provided name as action name and not as tag name. (bug fix) stackstorm/st2#4828

Reported by @AngryDeveloper and contributed by Marcel Weinberg (@winem)
Fix the passing of arrays to shell scripts where the arrays where not detected as such by the
st2 action_db utility. This caused arrays to be passed as Python lists serialized into a string.

Reported by @kingsleyadam stackstorm/st2#4804 and contributed by Marcel Weinberg (@winem) stackstorm/st2#4861
Fix ssh zombies when using ProxyCommand from ssh config stackstorm/st2#4881 [Eric Edgar]
Fix rbac with execution view where the rbac is unable to verify the pack or uid of the execution
because it was not returned from the action execution db. This would result in an internal server
error when trying to view the results of a single execution.
Contributed by Joshua Meyer (@jdmeyer3) stackstorm/st2#4758
Fixed logging middleware to output a content_length of 0 instead of Infinity
when the type of data being returned is not supported. Previously, when the value was
set to Infinity this would result in invalid JSON being output into structured
logs. (bug fix) stackstorm/st2#4722

Contributed by Nick Maludy (@nmaludy Encore Technologies)
Fix the workflow execution cancelation to proceed even if the workflow execution is not found or
completed. (bug fix) stackstorm/st2#4735
Added better error handling to contrib/linux/actions/dig.py to inform if dig is not installed.
Contributed by JP Bourget (@punkrokk Syncurity) stackstorm/st2#4732
Update dist_utils module which is bundled with st2client and other Python packages so it
doesn't depend on internal pip API and so it works with latest pip version. (bug fix) stackstorm/st2#4750
Fix dependency conflicts in pack CI runs: downgrade requests dependency back to 0.21.0, update
internal dependencies and test expectations (amqp, pyyaml, prance, six) (bugfix) stackstorm/st2#4774
Fix secrets masking in action parameters section defined inside the rule when using
GET /v1/rules and GET /v1/rules/<ref> API endpoint. (bug fix) stackstorm/st2#4788 stackstorm/st2#4807

Contributed by @Nicodemos305 and @jeansfelix
Fix a bug with authentication API endpoint (POST /auth/v1/tokens) returning internal
server error when running under gunicorn and whenauth.api_url config option was not set.
(bug fix) stackstorm/st2#4809

Reported by @guzzijones
Fixed st2 execution get and st2 run not printing the action.ref for non-workflow
actions. (bug fix) stackstorm/st2#4739

Contributed by Nick Maludy (@nmaludy Encore Technologies)
Update st2 execution get command to always include context.user, start_timestamp and
end_timestamp attributes. (improvement) stackstorm/st2#4739
Fixed core.sendmail base64 encoding of longer subject lines (bug fix) stackstorm/st2#4795

Contributed by @stevemuskiewicz and @guzzijones
Update all the various rule criteria comparison operators which also work with strings (equals,
icontains, nequals, etc.) to work correctly on Python 3 deployments if one of the operators is
of a type bytes and the other is of a type unicode / string. (bug fix) stackstorm/st2#4831
Fix SSL connection support for MongoDB and RabbitMQ which wouldn't work under Python 3 and would
result in cryptic "maximum recursion depth exceeded while calling a Python object" error on
connection failure.

NOTE: This issue only affected installations using Python 3. (bug fix) stackstorm/st2#4832 stackstorm/st2#4834

Reported by @alexku7.
Fix the amqp connection setup for WorkflowExecutionHandler to pass SSL params. (bug fix) stackstorm/st2#4845

Contributed by Tatsuma Matsuki (@mtatsuma)
Fix dependency conflicts by updating requests (2.23.0) and gitpython (2.1.15). stackstorm/st2#4869
Fix orquesta syntax error for with items task where action is misindented or missing. (bug fix)
PR StackStorm/orquesta#195.
Fix orquesta yaql/jinja vars extraction to ignore methods of base ctx() dict. (bug fix)
PR StackStorm/orquesta#196. Fixes stackstorm/st2#4866.
Fix parsing of array of dicts in YAQL functions. Fix regression in YAQL/Jinja conversion
functions as a result of the change. (bug fix) PR StackStorm/orquesta#191.

Contributed by Hiroyasu Ohyama (@userlocalhost)

Removed

Removed Ubuntu 14.04 from test matrix stackstorm/st2#4897

st2 - v3.1.0

Published by m4dcoder over 5 years ago

Changed

Allow the orquesta st2kv function to return default for nonexistent key. (improvement) #4678
Update requests library to latest version (2.22.0) in requirements. (improvement) #4680
Disallow "decrypt_kv" filter to be specified in the config for values that are marked as
"secret: True" in the schema. (improvement) #4709
Upgrade tooz library to latest stable version (1.65.0) so it uses latest version of
grpcio library. (improvement) #4713
Update st2-pack-install and st2-pack-download CLI command so it supports installing
packs from local directories which are not git repositories. (improvement) #4713

Fixed

Fix orquesta st2kv to return empty string and null values. (bug fix) #4678
Allow tasks defined in the same task transition with fail to run for orquesta. (bug fix)
Fix workflow service to handle unexpected coordinator and database errors. (bug fix) #4704 #4705
Fix filter to_yaml_string to handle mongoengine base types for dict and list. (bug fix) #4700
Fix timeout handling in the Python runner. In some scenarios where action would time out before
producing any output (stdout, stder), timeout was not correctly propagated to the user. (bug fix)
#4713
Update st2common/setup.py file so it correctly declares all the dependencies and script
files it provides. This way st2-pack-* commands can be used in a standalone fashion just by
installing st2common Python package and nothing else. (bug fix) #4713
Fix st2-pack-download command so it works in the environments where sudo binary is not
available (e.g. Docker). (bug fix) #4713

st2 - v3.0.1

Published by m4dcoder over 5 years ago

Fixed

Fix a bug in the remote command and script runner so it correctly uses SSH port from a SSH config
file if ssh_runner.use_ssh_config parameter is set to True and if a custom (non-default)
value for SSH port is specified in the configured SSH config file
(ssh_runner.ssh_config_file_path). (bug fix) #4660 #4661
Update pack install action so it works correctly when python_versions pack.yaml metadata
attribute is used in combination with --python3 pack install flag. (bug fix) #4654 #4662
Add source_channel back to the context used by Mistral workflows for executions which are
triggered via ChatOps (using action alias).

In StackStorm v3.0.0, this variable was inadvertently removed from the context used by Mistral
workflows. (bug fix) #4650 #4656
Fix a bug with timestamp attribute in the execution.log attribute being incorrect when
server time where st2api is running was not set to UTC. (bug fix) #4668

Contributed by Igor Cherkaev. (@emptywee)
Fix a bug with some packs which use --python3 flag (running Python 3 actions on installation
where StackStorm components run under Python 2) which rely on modules from Python 3 standard
library which are also available in Python 2 site-packages (e.g. concurrent) not working
correctly.

In such scenario, package / module was incorrectly loaded from Python 2 site-packages instead of
Python 3 standard library which broke such packs. (bug fix) #4658 #4674
Remove policy-delayed status to avoid bouncing between delayed statuses. (bug fix) #4655
Fix a possible shell injection in the linux.service action. User who had access to run this
action could cause a shell command injection by passing a compromised value for either the
service or action parameter. (bug fix) #4675

Reported by James Robinson (Netskope and Veracode).
Replace sseclient library on which CLI depends on with sseclient-py. sseclient has
various issue which cause client to sometimes hang and keep the connection open which also causes
st2 execution tail command to sometimes hang for a long time. (improvement)
Truncate some database index names so they are less than 65 characters long in total. This way it
also works with AWS DocumentDB which doesn't support longer index name at the moment.

NOTE: AWS DocumentDB is not officially supported. Use at your own risk. (improvement) #4688 #4690

Reported by Guillaume Truchot (@GuiTeK)

st2 - v3.0.0

Published by Kami over 5 years ago

Added

Allow access to user-scoped datastore items using {{ st2kv.user.<key name> }} Jinja template
notation inside the action parameter default values. (improvement) #4463

Contributed by Hiroyasu OHYAMA (@userlocalhost).
Add support for new python_versions (list of string) attribute to pack metadata file
(pack.yaml). With this attribute pack declares which major Python versions it supports and
works with (e.g. 2 and 3).

For backward compatibility reasons, if pack metadata file doesn't contain that attribute, it's
assumed it only works with Python 2. (new feature) #4474
Update service bootstrap code and make sure all the services register in a service registry once
they come online and become available.

This functionality is only used internally and will only work if configuration backend is
correctly configured in st2.conf (new feature) #4548
Add new GET /v1/service_registry/groups and
GET /v1/service_registry/groups/<group_id>/members API endpoint for listing available service
registry groups and members.

Also add corresponding CLI commands - st2 service-registry group list, st2 service registry member list [--group-id=<group id>]

NOTE: This API endpoint is behind an RBAC wall and can only be viewed by the admins. (new feature)
#4548
Add support for ?include_attributes and ?exclude_attributes query param filter to the
GET /api/v1/executions/{id} API endpoint. Also update st2 execution get CLI command so it
only retrieves attributes which are displayed. (new feature) #4497

Contributed by Nick Maludy (@nmaludy Encore Technologies)
Add new --encrypted flag to st2 key set CLI command that allows users to pass in values
which are already encrypted.

This attribute signals the API that the value is already encrypted and should be used as-is.

st2 key load CLI command has also been updated so it knows how to work with values which are
already encrypted. This means that st2 key list -n 100 -j < data.json ; st2 key load data.json will now also work out of the box for encrypted datastore values (values which have
encrypted: True and secret: True attribute will be treated as already encrypted and used
as-is).

The most common use case for this feature is migrating / restoring datastore values from one
StackStorm instance to another which uses the same crypto key.

Contributed by Nick Maludy (Encore Technologies) #4547
Add source_channel to Orquesta st2() context for workflows called via ChatOps. #4600

Changed

Changed the inquiries API path from /exp to /api/v1. #4495
Refactored workflow state in orquesta workflow engine. Previously, state in the workflow engine
is not status to be consistent with st2. Other terminologies used in the engine are also revised
to make it easier for developers to understand. (improvement)
Update Python runner code so it prioritizes libraries from pack virtual environment over StackStorm
system dependencies.

For example, if pack depends on six==1.11.0 in pack requirements.txt, but StackStorm depends
on six==1.10.0, six==1.11.0 will be used when running Python actions from that pack.

Keep in mind that will not work correctly if pack depends on a library which brakes functionality used
by Python action wrapper code.

Contributed by Hiroyasu OHYAMA (@userlocalhost). #4571
Improved the way that the winrm-ps-script runner sends scripts to the target Windows
host. Previously the script was read from the local filesystem and serialized as one long
command executed on the command line. This failed when the script was longer than either
2047 or 8191 bytes (depending on Windows version) as the Windows command line uses this
as its maximum length. To overcome this, the winrm-ps-script runner now uploads the
script into a temporary directory on the target host, then executes the script.
(improvement) #4514

Contributed by Nick Maludy (Encore Technologies)
Update various internal dependencies to latest stable versions (apscheduler, pyyaml, kombu,
mongoengine, pytz, stevedore, python-editor, jinja2). #4637
Update logging code so we exclude log messages with log level AUDIT from a default service
log file (e.g. st2api.log). Log messages with level AUDIT are already logged in a
dedicated service audit log file (e.g. st2api.audit.log) so there is no need for them to also
be duplicated and included in regular service log file.

NOTE: To aid with debugging, audit log messages are also included in a regular log file when log
level is set to DEBUG or system.debug config option is set to True.

Reported by Nick Maludy. (improvement) #4538 #4502 #4621
Add missing --user argument to st2 execution list CLI command. (improvement) #4632

Contributed by Tristan Struthers (@trstruth).
Update decrypt_kv Jinja template filter so it to throws a more user-friendly error message
when decryption fails because the variable references a datastore value which doesn't exist.
(improvement) #4634
Updated orquesta to v0.5. (improvement)

Fixed

Refactored orquesta execution graph to fix performance issue for workflows with many references
to non-join tasks. st2workflowengine and DB models are refactored accordingly. (improvement)
StackStorm/orquesta#122.
Fix orquesta workflow stuck in running status when one or more items failed execution for a with
items task. (bug fix) #4523
Fix orquesta workflow bug where context variables are being overwritten on task join. (bug fix)
StackStorm/orquesta#112
Fix orquesta with items task performance issue. Workflow runtime increase significantly when a
with items task has many items and result in many retries on write conflicts. A distributed lock
is acquired before write operations to avoid write conflicts. (bug fix) Stackstorm/orquesta#125
Fix a bug with some API endpoints returning 500 internal server error when an exception contained
unicode data. (bug fix) #4598
Fix the st2 workflow inspect command so it correctly passes authentication token. (bug fix)
#4615
Fix an issue with new line characters (\n) being converted to \r\n in remote shell
command and script actions which use sudo. (bug fix) #4623
Update service bootstrap and st2-register-content script code so non-fatal errors are
suppressed by default and only logged under DEBUG log level. (bug fix) #3933 #4626 #4630
Fix a bug with not being able to decrypt user-scoped datastore values inside Jinja expressions
using decrypt_kv Jinja filter. (bug fix) #4634

Contributed by Hiroyasu OHYAMA (@userlocalhost).
Fix a bug with user-scoped datastore values not working inside action-chain workflows. (bug fix)
#4634
Added missing parameter types to linux.wait_for_ssh action metadata. (bug fix) #4611
Fix HTTP runner (http-request) so it works correctly with unicode (non-ascii) body payloads.
(bug fix) #4601 #4599

Reported by Carlos Santana (@kknyxkk) and Rafael Martins (@rsmartins78).
Fix st2-self-check so it sets correct permissions on pack directories which it copies over
to /opt/stackstorm/packs. (bug fix) #4645
Fix POST /v1/actions API endpoint to throw a more user-friendly error when writing data file
to disk fails because of incorrect permissions. (bug fix) #4645

st2 - v2.10.4

Published by Kami over 5 years ago

Fixed

Fix inadvertent regression in notifier service which would cause generic action trigger to only
be dispatched for completed states even if custom states were specified using
action_sensor.emit_when config option. (bug fix)
Reported by Shu Sugimoto (@shusugmt). #4591
Make sure we don't log auth token and api key inside st2api log file if those values are provided
via query parameter and not header (?x-auth-token=foo, ?st2-api-key=bar). (bug fix) #4592
#4589
Fix rendering of {{ config_context. }} in orquesta task that references action from a
different pack (bug fix) #4570 #4567
Add missing default config location (/etc/st2/st2.conf) to the following services:
st2actionrunner, st2scheduler, st2workflowengine. (bug fix) #4596
Update statsd metrics driver so any exception thrown by statsd library is treated as non fatal.

Previously there was an edge case if user used a hostname instead of an IP address for metrics
backend server address. In such scenario, if hostname DNS resolution failed, statsd driver would
throw the exception which would propagate all the way up and break the application. (bug fix) #4597

Reported by Chris McKenzie.

st2 - v2.10.3

Published by Kami over 5 years ago

Fixed

Fix improper CORS where request from an origin not listed in allowed_origins will be responded
with null for the Access-Control-Allow-Origin header. The fix returns the first of our
allowed origins if the requesting origin is not a supported origin. Reported by Barak Tawily.
(bug fix)

st2 - v2.9.3

Published by m4dcoder over 5 years ago

Fixed

Fix improper CORS where request from an origin not listed in allowed_origins will be responded
with null for the Access-Control-Allow-Origin header. The fix returns the first of our
allowed origins if the requesting origin is not a supported origin. Reported by Barak Tawily.
(bug fix)

st2 - v2.10.2

Published by blag over 5 years ago

Added

Add support for various new SSL / TLS related config options (ssl_keyfile, ssl_certfile,
ssl_ca_certs, ssl_certfile, authentication_mechanism) to the messaging section in
st2.conf config file.

With those config options, user can configure things such as client based certificate
authentication, client side verification of a server certificate against a specific CA bundle, etc.

NOTE: Those options are only supported when using a default and officially supported AMQP backend
with RabbitMQ server. (new feature) #4541
Add metrics instrumentation to the st2notifier service. For the available / exposed metrics,
please refer to https://docs.stackstorm.com/reference/metrics.html. (improvement) #4536

Changed

Update logging code so we exclude log messages with log level AUDIT from a default service
log file (e.g. st2api.log). Log messages with level AUDIT are already logged in a
dedicated service audit log file (e.g. st2api.audit.log) so there is no need for them to also
be duplicated and included in regular service log file.

NOTE: To aid with debugging, audit log messages are also included in a regular log file when log
level is set to DEBUG or system.debug config option is set to True.

Reported by Nick Maludy. (improvement) #4538 #4502
Update pyyaml dependency to the latest version. This latest version fixes an issue which
could result in a code execution vulnerability if code uses yaml.load in an unsafe manner
on untrusted input.

NOTE: StackStorm platform itself is not affected, because we already used yaml.safe_load
everywhere.

Only custom packs which use yaml.load with non trusted user input could potentially be
affected. (improvement) #4510 #4552 #4554
Update Orquesta to v0.4. #4551

Fixed

Fixed the packs.pack_install / !pack install {{ packs }} action-alias to not have
redundant patterns. Previously this prevented it from being executed via
st2 action-alias execute 'pack install xxx'. #4511

Contributed by Nick Maludy (Encore Technologies)
Fix datastore value encryption and make sure it also works correctly for unicode (non-ascii)
values.

Reported by @dswebbthg, @nickbaum. (bug fix) #4513 #4527 #4528
Fix a bug with action positional parameter serialization used in local and remote script runner
not working correctly with non-ascii (unicode) values.

This would prevent actions such as core.sendmail which utilize positional parameters from
working correctly when a unicode value was provided.

Reported by @johandahlberg (bug fix) #4533
Fix core.sendmail action so it specifies charset=UTF-8 in the Content-Type email
header. This way it works correctly when an email subject and / or body contains unicode data.

Reported by @johandahlberg (bug fix) #4533 4534
Fix CLI st2 apikey load not being idempotent and API endpoint /api/v1/apikeys not
honoring desired ID for the new record creation. #4542
Moved the lock from concurrency policies into the scheduler to fix a race condition when there
are multiple scheduler instances scheduling execution for action with concurrency policies.
#4481 (bug fix)
Add retries to scheduler to handle temporary hiccup in DB connection. Refactor scheduler
service to return proper exit code when there is a failure. #4539 (bug fix)
Update service setup code so we always ignore kombu library heartbeat_tick debug log
messages.

Previously if DEBUG log level was set in service logging config file, but --debug
service CLI flag / system.debug = True config option was not used, those messages were
still logged which caused a lot of noise which made actual useful log messages hard to find.
(improvement) #4557

st2 - v2.10.1

Published by Kami almost 6 years ago

Fixed

Fix an issue with GET /v1/keys API endpoint not correctly handling ?scope=all and
?user=<username> query filter parameter inside the open-source edition. This would allow
user A to retrieve datastore values from user B and similar.

NOTE: Enterprise edition with RBAC was not affected, because in RBAC version, correct check is
in place which only allows users with an admin role to use ?scope=all and retrieve / view
datastore values for arbitrary system users. (security issue bug fix)

st2 - v2.10.0

Published by bigmstone almost 6 years ago

Added

Added notify runner parameter to Orquesta that allows user to specify which task(s) to get
notified on completion.
Add support for task delay in Orquesta workflows. #4459 (new feature)
Add support for task with items in Orquesta workflows. #4400 (new feature)
Add support for workflow output on error in Orquesta workflows. #4436 (new feature)
Added -o and -m CLI options to st2-self-check script, to skip Orquesta and/or Mistral
tests. #4347
Allow user to specify new database.authentication_mechanism config option in
/etc/st2/st2.conf.

By default, SCRAM-SHA-1 is used with MongoDB 3.0 and later and MONGODB-CR (MongoDB Challenge
Response protocol) for older servers.

Contributed by @aduca85 #4373
Add new metadata_file attribute to the following models: Action, Action Alias, Rule, Sensor,
TriggerType. Value of this attribute points to a metadata file for a specific resource (YAML file
which contains actual resource definition). Path is relative to the pack directory (e.g.
actions/my_action1.meta.yaml, aliases/my_alias.yaml, sensors/my_sensor.yaml,
rules/my_rule.yaml, triggers/my_trigger.yaml etc.).

Keep in mind that triggers can be registered in two ways - either via sensor definition file in
sensors/ directory or via trigger definition file in triggers/ directory. If
metadata_file attribute on TriggerTypeDB model points to sensors/ directory it means that
trigger is registered via sensor definition. (new feature) #4445
Add new st2client.executions.get_children method for returning children execution objects for
a specific (parent) execution. (new feature) #4444

Contributed by Tristan Struthers (@trstruth).
Allow user to run a subset of pack tests by utilizing the new -f command line option in the
st2-run-pack-tests script.

For example:
1. Run all tests in a test file (module):
  
  st2-run-pack-tests -j -x -p contrib/packs/ -f test_action_download
2. Run a single test class
  
  st2-run-pack-tests -j -x -p contrib/packs/ -f test_action_download:DownloadGitRepoActionTestCase
3. Run a single test class method
  
  st2-run-pack-tests -j -x -p contrib/packs/ -f test_action_download:DownloadGitRepoActionTestCase.test_run_pack_download
(new feature) #4464

Changed

Redesigned and rewritten the action execution scheduler. Requested executions are put in a
persistent queue for scheduler to process. Architecture is put into place for more complex
execution scheduling. Action execution can be delayed on request. (improvement)
core.http action now supports additional HTTP methods: OPTIONS, TRACE, PATCH, PURGE.

Contributed by @emptywee (improvement) #4379
Runner loading code has been updated so it utilizes new "runner as Python package" functionality
which has been introduced in a previous release. This means that the runner loading is now fully
automatic and dynamic.

All the available / installed runners are automatically loaded and registering on each StackStorm
service startup.

This means that st2ctl reload --register-runners flag is now obsolete because runners are
automatically registered on service start up. In addition to that,
content.system_runners_base_path and content.runners_base_paths config options are now
also deprecated and unused.

For users who wish to develop and user custom action runners, they simply need to ensure they are
packaged as Python packages and available / installed in StackStorm virtual environment
(/opt/stackstorm/st2). (improvement) #4217
Old runner names which have been deprecated in StackStorm v0.9.0 have been removed (run-local,
run-local-script, run-remote, run-remote-script, run-python, http-runner). If you are still using
actions which reference runners using old names, you need to update them to keep it working.
#4217
Update various CLI commands to only retrieve attributes which are displayed in the CLI from the
API (st2 execution list, st2 execution get, st2 action list, st2 rule list,
st2 sensor list). This speeds up run-time and means now those commands now finish faster.

If user wants to retrieve and view all the attributes, they can use --attr all CLI command
argument (same as before). (improvement) #4396
Update various internal dependencies to latest stable versions (greenlet, pymongo, pytz,
stevedore, tooz). #4410
Improve st2.conf migration for the new services by using prod-friendly logging settings by default #4415
Refactor Orquesta workflow to output on error. Depends on PR
https://github.com/StackStorm/orquesta/pull/101 and https://github.com/StackStorm/orquesta/pull/102
(improvement)
Rename st2client.liveactions to st2client.executions. st2client.liveactions already
represented operations on execution objects, but it was incorrectly named.

For backward compatibility reasons, st2client.liveactions will stay as an alias for
st2client.executions and continue to work until it's fully removed in a future release.

Fixed

st2 login CLI commands now exits with non zero exit code when login fails due to invalid
credentials. (improvement) #4338
Fix st2 key load that errors when importing an empty file #43
Fixed warning in st2-run-pack-tests about invalid format for pip list. (bug fix)

Contributed by Nick Maludy (Encore Technologies). #4380
Fix a bug with st2 execution get / st2 run CLI command throwing an exception if the
result field contained a double backslash string which looked like an unicode escape sequence.
CLI incorrectly tried to parse that string as unicode escape sequence.

Reported by James E. King III @jeking3 (bug fix) #4407
Fix a bug so timersengine config section in st2.conf has precedence over timer
section if explicitly specified in the config file.

Also fix a bug with default config values for timer section being used if user only
specified timersengine section in the config. Previously user options were incorrectly
ignored in favor of the default values. (bug fix) #4424
st2 pack install -j now only spits JSON output. Similarly, st2 pack install -y only spits
YAML output. This change would enable the output to be parsed by tools.
The behavior of st2 pack install hasn't changed and is human friendly. If you want to get meta
information about the pack as JSON (count of actions, sensors etc), you should rely on already
existing st2 pack show -j.

Reported by Nick Maludy (improvement) #4260
Fix string operations on unicode data in Orquesta workflows, associated with PR
https://github.com/StackStorm/orquesta/pull/98. (bug fix)
Fix access to st2 and action context in Orquesta workflows, associated with PR
https://github.com/StackStorm/orquesta/pull/104. (bug fix)
st2ctl reload --register-aliases and st2ctl reload --register-all now spits a warning when
trying to register aliases with no corresponding action registered in the db.

Reported by nzlosh (improvement) #4372.

st2 -

Published by Kami almost 6 years ago

Fixed

Fix an issue with GET /v1/keys API endpoint not correctly handling ?scope=all and
?user=<username> query filter parameter inside the open-source edition. This would allow
user A to retrieve datastore values from user B and similar.

NOTE: Enterprise edition with RBAC was not affected, because in RBAC version, correct check is
in place which only allows users with an admin role to use ?scope=all and retrieve / view
datastore values for arbitrary system users. (security issue bug fix)

st2 -

Published by bigmstone about 6 years ago

Changed

Speed up pack registration through the /v1/packs/register API endpoint. (improvement) #4342
Triggertypes API now sorts by trigger ref by default. st2 trigger list will now show a sorted list. (#4348)
Update st2-self-check script to include per-test timing information. (improvement) #4359

Fixed

Update st2sensorcontainer service to throw if user wants to run a sensor from a pack which is using Python 3 virtual environment. We only support running Python runner actions from packs which use mixed Python environments (StackStorm components are running under Python 2 and particular a pack virtual environment is using Python 3). #4354
Update st2-pack-install and st2 pack install command so it works with local git repos (file://) which are in a detached head state (e.g. specific revision is checked out). (improvement) #4366
Fix a race which occurs when there are multiple concurrent requests to resume a workflow. #4369