soketi

Fixed releases for Docker

• Added cluster adapter & rate-limiting for local scaling with PM2/Forks (https://github.com/soketi/soketi/pull/171)
• Added filters in the webhook configuration to filter the webhooks based on channel name (https://github.com/soketi/soketi/pull/223, @stayallive)

Cluster Driver for Scaling & Rate Limiting

With the Cluster driver, deploying multiple node workers with PM2 is going to be easier since Redis will no longer be needed.

Alongside this adapter, you should also decide to use the cluster rate limiter driver in case you have rate limiting in place. This is going to scale the rate-limiting across all known processes.

Refactoring adapter code

The Redis adapter code was really messy, so I decided to come with an approach that will make future horizontal-scaling adapters easier. This required a lot of time, a lot of testing, and ideas to better improve the current code. Squashing ~ 800 lines of code, most of them coming from the code for the Redis adapter.

PM2 Support

With this PR, PM2 is easier to use thanks to the cluster mode.

Graceful Shutdown Time

Graceful Shutdown Time is a fancy term for giving the soketi process enough time to shut down all the current connections and redirect users to other processes, before shutting the process itself. This is useful for horizontal scaling, and this time is arbitrarily given. Default is 3000 ms (3s), but you are now free to customize it by giving more time (or less) for your needs with the SHUTDOWN_GRACE_PERIOD, a value in milliseconds.

Adapters and rate limiters now also got a softer close for the active, like closing the Redis sockets at the very end, when the server is closing.

Small Fixes

• Upon presence leave or channel leave, the internal WebSocket connection's arrays were not updated in the current sockets list.
• When shutting down, the adapter clear() method that makes sure it disconnects every socket from the instance was not a Promise, making it hard to predict whether all the sockets were closed in a soft manner

• Removed obsolete files (including Log4j) from Docker context (https://github.com/soketi/soketi/pull/220)
• • The final Docker image size was shrunken by ~ 48% (from 160.5 MB to 82.7 MB)

Explanation

This Log4j issue is not a security concern.

For DynamoDB testing, soketi uses DynamoDbLocal. This server comes with additional files for it, including Log4j, which recently has been exposed to have an RCE vulnerability. DynamoDBLocal uses Log4j, that is committed to the repo so it could work starting it.

Docker seems to be adding these files, but later on, it removes them in the process of building the final Docker image. However, they still can be tracked in the Docker image layers' trace.

To avoid false positives from automatic RCE checkers, they are removed from being added in the context entirely. With the current fixes, they should be ignored and not take part in the Docker build context, alongside other obsolete files like the benchmark folder, thus lowering the final Docker image too.

• Reverted bullmq to 1.57.3 to fix known issues(https://github.com/soketi/soketi/pull/206, https://github.com/soketi/soketi/issues/205)

• Patched various dev dependencies
• Bumped AWS (https://github.com/soketi/soketi/pull/196)

• Bumped packages
• • yargs (https://github.com/soketi/soketi/pull/188)
• • AWS SDK (https://github.com/soketi/soketi/pull/187)
• • ioredis (https://github.com/soketi/soketi/pull/185)
• • jest-circus (https://github.com/soketi/soketi/pull/184)
• • bullmq (https://github.com/soketi/soketi/pull/182)

• Patched packages (https://github.com/soketi/soketi/pull/190, https://github.com/soketi/soketi/pull/186)

• Added distroless variants for Docker builds (https://github.com/soketi/soketi/pull/178, @nsmith5)

• Fixed a bug where soketi would read the numeric app IDs passed through the environment variables as integer, breaking communications with the clients in unexpected ways (https://github.com/soketi/soketi/pull/97, @mattoz0)
• In case that users with the same ID are joining multiple times in presence channels, member_added is no longer broadcasted multiple times. When leaving, member_removed is triggered only when the last connection of that user closes. Having multiple tabs/devices connected at the same time still count against the maximum connection numbers. You may fix this by using service workers (as explained here) (https://github.com/soketi/soketi/pull/19, @jsandfordhughes, @audiojames)

Uncommitted package-lock.json

This may be a breaking change if you are using Prometheus metrics or you are using the Node.js 14.x image for Docker. (https://github.com/soketi/soketi/pull/157)

Please read the new documentation: https://rennokki.gitbook.io/soketi-docs/advanced-usage/scraping-prometheus-metrics

• Dropped Node.js 14.x build for Docker (there was no reason for 14.x if it ran in 16.x which is LTS)
• @stayallive pointed out in #124 that exposing the /metrics and /usage endpoints to 6001 (which is public) was a bad idea in terms of security. Starting with this version, the server will run the metrics to a different port (9601) which may not be exposed to the internet. This way, you can run the Prometheus Scrape service within your server or in your private network.

• Patched fixes regarding server pending closing (https://github.com/soketi/soketi/pull/170)
• Patching AWS SDK, bullmq, rate limiter (https://github.com/soketi/soketi/pull/161, https://github.com/soketi/soketi/pull/160, https://github.com/soketi/soketi/pull/159, https://github.com/soketi/soketi/pull/167)
• Bumped compiling/dev packages (https://github.com/soketi/soketi/pull/164, https://github.com/soketi/soketi/pull/158, https://github.com/soketi/soketi/pull/162, https://github.com/soketi/soketi/pull/165, https://github.com/soketi/soketi/pull/128, https://github.com/soketi/soketi/pull/126, https://github.com/soketi/soketi/pull/168)

• Added AWS Lambda invocations as a webhook sending strategy (https://github.com/soketi/soketi/pull/147, https://rennokki.gitbook.io/soketi-docs/advanced-usage/app-webhooks/aws-lambda-trigger)

Coming soon:

• Removing secret from the WebSocket instance to make the server get the app secret on the webhook sending process and on HTTP requests instead of passing it to the WebSocket instance (https://github.com/soketi/soketi/pull/153, reverted in https://github.com/soketi/soketi/commit/8148eb9c5476d0ccbdcd6bca6f15e88477078a0a). It was reverted as this can lead to throttles if your app manager is other than array, as it will query the app manager on each webhook and private channel connection. Currently investigating a better way to do this.

• Bumped knex to latest patch (https://github.com/soketi/soketi/pull/152)
• Bumped AWS SDK (https://github.com/soketi/soketi/pull/151)
• Bumped bullmq (https://github.com/soketi/soketi/pull/150)

• Fixed issues with MySQL 8.0+ authentication by optionally specifying APP_MANAGER_MYSQL_USE_V2 environment variable (https://github.com/soketi/soketi/pull/146)
• Bumped bullmq (https://github.com/soketi/soketi/pull/145)

• Bumped patched packages (https://github.com/soketi/soketi/pull/144, https://github.com/soketi/soketi/pull/130)

• Bumped uWebSockets.js for some fixes (https://github.com/soketi/soketi/pull/125)

pWS got rebranded to soketi.

You can still use resources for the pre-0.13 version as they were left intact.

• The Docker repository moved to quay.io/soketi/soketi, and you can only use it with the 0.13+ tag
• Documentation links moved
• The command to start the websockets server is now soketi start
• Environment variables are now pulled if they are prefixed with SOKETI_ (previously: PWS_)
• Prometheus prefixes now with soketi_ (previously: pws_)
• Webhooks are being sent with the SoketiWebhooksAxiosClient/1.0 agent

• Reverted AWS client minification

• Performed additional cleanup on the Docker image release size (https://github.com/soketi/pws/pull/141)
• • Reduced the total size of the Docker image from 246.6 MB to 138.0 MB
• • Performed additional build steps to clean up unwanted files
• • Removed aws-sdk in favor of @aws-sdk/client-dynamodb as a lighter dependency