tags: roadrunner v2.12.1

🇺🇦🇺🇦🇺🇦 StandWithUkraine 🇺🇦🇺🇦🇺🇦

🚀 v2.12.1 🚀

👀 New:

• ✏ RR: Automatically set the GOMAXPROCS to match the container CPU quota.
• ✏ AMQP plugin: implement Status to check the AMQP connection PR.
• ✏ SQS plugin: prefetch option now works as expected. RR will not consume new JOBS when it reaches the prefetch limit, until already accepted messages are not ACK/NACK-ed.
• ✏ JOBS(memory) plugin: prefetch option now works as expected (see SQS). You can now emulate FIFO in memory by setting the prefetch option to 1.

🩹 Fixes:

• 🐛 gRPC plugin: server options are applied only when TLS is set. Discussion.
• 🐛 AMQP plugin: fix a few typos in the configuration.

Thanks to our awesome contributors: @wolfy-j , @butschster , @roxblnfk , @kastahov , @msmakouz, @lyt8384 ❤️

tags: roadrunner v2.12.0

🚀 v2.12.0 🚀

⚠️ websocket and broadcast plugins were replaced by the new centrifuge plugin. How to build RR with these deprecated plugins -> link

⚠️ All plugins, sdk and api, updated to v3. There are no breaking changes; we moved all Go code from the api to sdk.

👀 New:

RPC PLUGIN

• ✏ New API to get the RR version and configuration in the JSON format -> rpc.Version, rpc.Config.

  

SERVICES PLUGIN

• ✏ New API to get the correct number of the running services, including statistics about the processes -> server.Statuses.

  

• ✏ New option to show the service name in the logs:

  # Show the name of the service in logs (e.g. service.some_service_1)
  #
  # Default: false
  service_name_in_log: false
  

Birddog: link

METRICS PLUGIN

• ✏ New API unregister previously added collector -> metrics.Unregister.

AMQP PLUGIN

• ✏ New configuration options:

jobs:
  pipelines:
    example:
      driver: amqp
      config:
        # Durable exchange
        #
        # Default: true
        exchange_durable: true

        # Auto-deleted exchange
        #
        # Default: false
        exchange_auto_deleted: false

        # Auto-deleted queue
        #
        # Default: false
        queue_auto_deleted: false

GO-SDK

• ✏ New option to control the reset_timeout:

pool:
  allocate_timeout: 10s
  reset_timeout: 10s
  destroy_timeout: 10s

CENTRIFUGO PLUGIN

• ✏ New centrifugo plugin.
  Docs: PHP-lib

RoadRunner config:

version: "2.7"

centrifuge:
  # Centrifugo server proxy address (docs: https://centrifugal.dev/docs/server/proxy#grpc-proxy)
  #
  # Optional, default: tcp://127.0.0.1:30000
  proxy_address: "tcp://127.0.0.1:30000"

  # gRPC server API address (docs: https://centrifugal.dev/docs/server/server_api#grpc-api)
  #
  # Optional, default: tcp://127.0.0.1:30000. Centrifugo: `grpc_api` should be set to true and `grpc_port` should be the same as in the RR's config.
  grpc_api_address: tcp://127.0.0.1:30000

  # Use gRPC gzip compressor
  #
  # Optional, default: false
  use_compressor: true

  # Your application version
  #
  # Optional, default: v1.0.0
  version: "v1.0.0"

  # Your application name
  #
  # Optional, default: roadrunner
  name: "roadrunner"

  # TLS configuration
  #
  # Optional, default: null
  tls:
    # TLS key
    #
    # Required
    key: /path/to/key.pem

    # TLS certificate
    #
    # Required
    cert: /path/to/cert.pem


  # Workers pool settings. link: https://github.com/roadrunner-server/roadrunner/blob/master/.rr.yaml#L812
  #
  # Optional, default: null (see default values)
  pool: {}

APP-LOGGER PLUGIN

• ✏ Application logger plugin.
  Docs: PHP-lib

🩹 Fixes:

• 🐛 Headers middleware: Header size is too small
• 🐛 gRPC plugin: Protobuf compiler plugin segfaults on import statements
• 🐛 Service plugin: Get services list via RPC
• 🐛 gRPC plugin: Remote protoc-gen-php-grpc plugin error
• 🐛 HTTP plugin: Fail to upload files when RR's permissions are different from worker's

Thanks to our awesome contributors: @wolfy-j, @andrey-tech, @butschster, @masterjus, @phroggyy, @rapita, @egonbraun, @tungfinblox ❤️

👀 New:

• ✏ All plugins: update to v3. This is done not because of some breaking change but because of the internal update.
• ✏ RPC plugin: add new API to provide a running RR version and configuration in JSON format.
• ✏ AMQP plugin: new configuration options. FR, (thanks @andrey-tech)

jobs:
  pipelines:
    example:
      driver: amqp
      config:
        # Durable exchange
        #
        # Default: true
        exchange_durable: true

        # Auto-deleted exchange
        #
        # Default: false
        exchange_auto_deleted: false

        # Auto-deleted queue
        #
        # Default: false
        queue_auto_deleted: false

• ✏ Workers pool (SDK): New option to control the reset_timeout. Note that the pool.Reset is protected by mutexes, meaning that if you have some requests already in the pool, you'll have to wait for these requests to be processed. The reset_timeout does not count this time.

pool:
  allocate_timeout: 10s
  reset_timeout: 10s
  destroy_timeout: 10s

⚠️ websocket and broadcast plugins were replaced by the new centrifuge plugin.

⚠️ All plugins, sdk and api updated to v3. There are no breaking changes except we moved all Go code from the api to sdk.

👀 New:

• ✏ Centrifugo plugin: New centrifugo plugin. This will replace existing broadcast + websockets plugins. FR.
  Docs: PHP-lib

RoadRunner config:

version: "2.7"

centrifuge:
  # Centrifugo server proxy address (docs: https://centrifugal.dev/docs/server/proxy#grpc-proxy)
  #
  # Optional, default: tcp://127.0.0.1:30000
  proxy_address: "tcp://127.0.0.1:30000"

  # gRPC server API address (docs: https://centrifugal.dev/docs/server/server_api#grpc-api)
  #
  # Optional, default: 127.0.0.1:30000. Centrifugo: `grpc_api` should be set to true and `grpc_port` should be the same as in the RR's config.
  grpc_api_address: 127.0.0.1:30000

  # Use gRPC gzip compressor
  #
  # Optional, default: false
  use_compressor: true

  # Your application version
  #
  # Optional, default: v1.0.0
  version: "v1.0.0"

  # Your application name
  #
  # Optional, default: roadrunner
  name: "roadrunner"

  # TLS configuration
  #
  # Optional, default: null
  tls:
    # TLS key
    #
    # Required
    key: /path/to/key.pem

    # TLS certificate
    #
    # Required
    cert: /path/to/cert.pem


  # Workers pool settings. link: https://github.com/roadrunner-server/roadrunner/blob/master/.rr.yaml#L812
  #
  # Optional, default: null (see default values)
  pool: {}

• ✏ App logger plugin: Application logger plugin, FR (thanks @wolfy-j)
  Docs: PHP-lib

🩹 Fixes:

• 🐛 Headers middleware: Header size is too small, BUG (thanks @masterjus)
• 🐛 gRPC plugin: Protobuf compiler plugin segfaults on import statements, BUG (thanks @phroggyy)
• 🐛 Service plugin: Get services list via RPC, BUG (thanks @butschster)
• 🐛 gRPC plugin: Remote protoc-gen-php-grpc plugin error, BUG (thanks @rapita)
• 🐛 HTTP plugin: Fail to upload files when RR's permissions are different from worker's, BUG (thanks @egonbraun)

👀 New:

• ✏ API v3: remove ALL Go code from the API repository. Now, API contains only the RR protobuf API. You don't need to import an interface from the API (it's not a Go way) now. You may declare the interface where you use it (have a look at this PR for more info).
• ✏ SDK v3: refactor all packages, merge worker + sync worker (since there were no differences), pool + sync pool - now, if a user uses supervisor, pool automatically turns on supervisor under the hood.

🩹 Fixes:

• 🐛 http plugin: correctly set permissions for the uploads (multipart-form files), BUG, (thanks @egonbraun)

🔧 Maintenance:

• 🔧 All plugins update to the v3. They don't use API interfaces anymore.

👀 New:

• ✏ Temporal plugin: Support for the SearchAttributes. FR, (thanks @cv65kr).

Docs: link
Samples: link

🔧 Maintenance:

• roadrunner-temporal plugin updated to: 1.7.0
• http plugin updated to: 2.23.5
• sqs plugin updated to: 2.20.4
• config plugin updated to: 2.16.5
• grpc plugin updated to: 2.23.3
• nats plugin updated to: 2.17.3
• jobs plugin updated to: 2.18.4
• server plugin updated to: 2.16.4
• tcp plugin updated to: 2.15.4
• websockets plugin updated to: 2.16.5
• otel plugin updated to: 2.5.6
• kafka plugin updated to: 2.2.3

👀 New:

• ✏ [ALPHA] gRPC plugin: buf remote plugins support for the protoc-gen-php-grpc plugin. FR, (thanks @rauanmayemir)
• ✏ Temporal plugin: mTLS support. FR, (thanks @seregazhuk)

Configuration sample:

temporal:
  address: 127.0.0.1:7233
  cache_size: 100000
  activities:
    num_workers: 4

  tls:
    key: client.key
    cert: client.pem
    root_ca: ca.cert
    client_auth_type: require_and_verify_client_cert
    server_name: "tls-sample"

🩹 Fixes:

• 🐛 Config plugin: properly replace environment variables for the array yaml values. BUG, (thanks @lyt8384)

🧹 Chore:

• 🧑‍🏭: [ALPHA] gRPC plugin: base64 decoder for the google's ErrorProto structure. FR, (thanks @rauanmayemir)

👀 New:

• ✏ Kafka plugin: [ ⚠️ EXPERIMENTAL OPTION ⚠️ ] Kafka plugin now waits for 1 minute (automatically) for the broker to be available, FR, (thanks @Baiquette)

• ✏ Internal: PHP Worker now uses an FSM to transition between states (working, ready, invalid, etc).

• ✏ Internal: ./rr reset now works in parallel. All workers will be restarted simultaneously instead of a one-by-one sync approach.

• ✏ Internal: ./rr reset and destroy (when stopping RR) now gracefully stop the workers (giving a chance for the finalizers to work). If the worker doesn't respond in 10 seconds, it'll be killed.

🩹 Fixes:

• 🐛 SQS plugin: Incorrect detection of the AWS IMDSv2 instances, BUG (thanks @paulermo)
• 🐛 Temporal plugin: Segmentation violation when using TLS, BUG, (thanks @seregazhuk)
• 🐛 NATS plugin: Properly check the stream not found error from NATS, BUG, (thanks @pjtuxe)

🧹 Chore:

• 🧑‍🏭: Temporal plugin: Support for the statsd daemon for stats aggregation, FR, (thanks @cv65kr)
  Configuration stays the same (no breaking changes), but additionally, you may specify a driver:

Prometheus:

temporal:
  address: "127.0.0.1:7233"
  metrics:
    driver: prometheus # <---- prometheus used by default (you may omit the driver in this case)
    address: "127.0.0.1:9095"
    prefix: "samples"
    type: "summary"
  activities:
    num_workers: 4

Statsd:

temporal:
  address: "127.0.0.1:7233"
  metrics:
    driver: statsd # <---- Should be specified to use a statsd driver
    host_port: "127.0.0.1:8125"
    prefix: "samples"
    flush_interval: 1s
    flush_bytes: 512
    tags:
      - foo: bar
  activities:
    num_workers: 4

Detailed description is here: link

👀 New:

• ✏ http plugin: Send raw body (unescaped) to the PHP worker for the application/x-www-form-urlencoded content type. FR1, FR2, (thanks @ekisu, @rlantingmove4mobile)
  Configuration:

http:
  raw_body: true/false (by default)

• ✏ temporal plugin: Overwrite client-name and client-version in Go client to represent PHP-SDK, FR, (thanks, @wolfy-j)

🧹 Chore:

• 🧑‍🏭: Autocomplete .rr.yaml configuration for the cache plugin. link

⚠️ NewRelic middleware was removed. Please, use OTEL middleware instead

⚠️ In 2.12.0 we plan to replace websockets and broadcast plugins with the centrifuge plugin. However, if you still need a RR with these deprecated plugins, you may use Velox to build your custom build.

👀 New:

• ✏️ [BETA]: RoadRunner: Can now be embedded in other go programs. PR, (thanks @khepin)
• ✏️ gRPC Plugin: Implement Google's gRPC errors API. The exception might be passed as a Status structure in the Metadata (key - error) to be parsed and returned to the user w/o worker restart. NOTE: Status structure should be marshaled via proto marshaller, not json. FR
• ✏️ Logger Plugin: Get rid of the context deadline exceeded error on worker's allocation. We updated the error message with the link to the docs with the most common causes for the worker allocation failed error: https://roadrunner.dev/docs/known-issues-allocate-timeout/2.x/en. Feel free to add your cases here :)
• ✏️ CLI: New CLI command to pause, resume, destroy and list Jobs. FR, (thanks @hustlahusky)
• ✏️ Velox: New configuration option: folder, which can be used to specify the folder with the plugin in the repository. (thanks, @darkweak)
• ✏️ Velox: Velox now respects the plugin's replace directives. (thanks, @darkweak)
• ✏️ Cache plugin: RR now uses a great cache (RFC 7234) plugin made by @darkweak
• ✏️ [BETA] Kafka plugin: New Kafka driver for the Jobs plugin. FR, (thanks, @Smolevich, @Baiquette)
• ✏️ Temporal plugin: Temporal now uses a new reset mechanism to prevent WF worker restarts on activity worker failure
• ✏️ Temporal plugin: Temporal plugin now supports a TSL-based authentication with the key and certificate.

Configuration:

temporal:
  tls:
    key: path/to/key
    cert: path/to/cert
  # other options

🩹 Fixes:

• 🐛 Server plugin: use the allocate_timeout from the pool to wait for the tcp/unix socket connection from the PHP worker. BUG, (thanks @Warxcell)
• 🐛 Velox: Fix panic when no github option is specified in the configuration.
• 🐛 SDK: Use pool.allocate_timeout for the sockets/tcp relays instead of silently used of relay_timeout.

🧹 Chore:

• 🧽 Logger plugin: use the parsable timestamp format for the raw logger mode. CH, (thanks @ilsenem)

🔧 Maintenance:

• Temporal GO-SDK and API updated to the latest versions.
• All plugins, including RR, now use Go 1.19

👀 New:

• ✏️ OTEL Middleware: Support for the jaeger_agent exporter - FR, (thanks @L3tum)

📦 Packages:

• 📦 OTEL SDK updated to v1.8.0
• 📦 google.golang.org/grpc updated to v1.48.0
• 📦 logger plugin updated to v2.13.5
• 📦 Go updated to v1.18.4

🩹 Fixes:

• 🐛 SDK: In some cases, worker watcher might freeze if the user kills the worker right after allocation but before wait4 syscall and become a zombie. BUG
• 🐛 AMQP Plugin: Ignored prefetch option when dynamically creating a pipeline. BUG (thanks @rauanmayemir)

👀 New:

• ✏️ VELOX: Velox now supports environment variables for the version, buildtime, GitHub, and GitLab tokens in the velox.toml.
• ✏️ Logger: Say bye-bye to the CRC verification failed error. Starting from the v2.10.6, RR will show user-friendly message with the link to our docs on the most common causes for this type of error. Docs

🩹 Fixes:

• 🐛 SDK: Increase stderr buffer size from 32kb to 64kb, man7-pipe_capacity, BUG, (thanks @7krasov)
• 🐛 AMQP Plugin: Fix incorrect queue binding to the default routing key. BUG, (thanks @rauanmayemir)
• 🐛 HTTP Plugin: Fix x-www-form-urlencoded requests 10Mb limit. BUG (thanks @StreetYo)

🩹 Fixes:

• 🐛 Fix: incorrect reset behavior for the workers' pool.
• 🐛 Fix: correct reset order for the RR and Temporal workers.

👀 New:

• ✏️ CLI: rr stop command. rr stop will read the .pid file to send a graceful-stop signal to the primary RR process (SIGTERM). To create a .pid file, add -p to the serve command: rr serve -p. Docs: link, FR (thanks @Baiquette)

🩹 Fixes:

• 🐛 Fix: incorrect reset behavior for the temporal plugin. BUG, BUG, BUG. (thanks @dmitry-pilipenko, @mzavatsky)

👀 New:

• ✏️ WORKER: Starting from this release, RR is able to show full error messages sent to the STDOUT during the worker bootstrap. FR (thanks @ykweb)

• ✏️ HTTP: Connection might be upgraded from the http/1.1 to h2c: rfc7540
  Headers, which should be sent to upgrade connection:

  1. Upgrade: h2c
  2. Connection: HTTP2-Settings
  3. Connection: Upgrade
  4. HTTP2-Settings: AAMAAABkAARAAAAAAAIAAAAA RFC

• ✏️ VELOX: Add GitLab support. Starting from the beta.2 you may use GitHub and GitLab plugins together. Configuration updated. Keep in mind, that until the stable release 1.0.0 configuration might be changed with breaking changes.

• ✏️ protoc-gen-php-grpc: Use of fully qualified names in place of imports. PR (thanks @ryanjcohen)

👀 New:

• ✏️ Jobs (queues) plugin now can consume any payload from the queue. If RR fails in converting payload into the Job structure, it'll create and fill all available fields manually. To turn on this feature, use consume_all: true in the driver configuration, e.g.:
  Supported drivers: amqp, sqs,beanstalk,nats.

jobs:
  num_pollers: 10
  pipeline_size: 100000
  pool:
    num_workers: 10

  pipelines:
    test-raw:
      driver: sqs
      config:
        consume_all: true # <------- NEW OPTION

  consume: [ "test-raw" ]

• ✏️ SQS Jobs driver now can skip queue declaration in favor of getting queue URL instead. To use this feature, use skip_queue_declaration: true sqs driver option. FR, (thanks @sergey-telpuk)

jobs:
  num_pollers: 10
  pipeline_size: 100000
  pool:
    num_workers: 10

  pipelines:
    test-2:
      driver: sqs
      config:
        skip_queue_declaration: true # <----- NEW OPTION

  consume: [ "test-2" ]

• ✏️ OpenTelemetry middleware now supports Jaeger exporter and propagator.

http:
  address: 127.0.0.1:43239
  max_request_size: 1024
  middleware: [gzip, otel]
  pool:
    num_workers: 2
    max_jobs: 0
    allocate_timeout: 60s
    destroy_timeout: 60s

otel:
  exporter: jaeger # <----- NEW OPTION

• ✏️ HTTP Plugin now supports mTLS authentication. Possible values for the client_auth_type are the same as for the gRPC (no_client_cert, request_client_cert,require_any_client_cert,verify_client_cert_if_given,require_and_verify_client_cert) FR, (thanks @fwolfsjaeger)

version: '2.7'

server:
  command: "php ../../php_test_files/http/client.php echo pipes"
  relay: "pipes"
  relay_timeout: "20s"

http:
  address: :8085
  max_request_size: 1024
  middleware: [ ]
  pool:
    num_workers: 1
    max_jobs: 0
    allocate_timeout: 60s
    destroy_timeout: 60s
  ssl:
    address: :8895
    key: "key"
    cert: "cert"
    root_ca: "rootCA.pem" # <---- REQUIRED to use mTLS
    client_auth_type: require_and_verify_client_cert # <---- NEW OPTION
logs:
  mode: development
  level: error

🩹 Fixes:

• 🐛 Fix: HTTP plugin: non-documented behavior on non-standard (but valid) http codes. BUG, (thanks, @Meroje)
• 🐛 Fix: SQS driver: rr_auto_ack attribute won't fail the existing messages.

🧳 Deprecated:

• 📦 new_relic http middleware. Please, consider using the new OpenTelemetry middleware: otel.

👀 New:

• ✏️ Documentation update: link.
• ✏️ RoadRunner-Temporal plugin now supports local activities. Here is the brief overview: link.
• ✏️ Add Debian amd64 releases. FR
• ✏️ Add signed releases. Starting from the v2.10.0, you can check every released binary with a provided *.asc key. For example:

$ gpg --verify rr.asc

The openPGP key can be verified here: keyserver

• ✏️ All proto api for the Go programming language located here: link. To use it, just import the latest stable version go.buf.build/protocolbuffers/go/roadrunner-server/api latest.
• ✏️ Service plugin now supports auto-reload. It can be added to the reload plugin targets and on change, it'll reload all underlying processes.
• ✏️ AutoAck jobs option. For the messages (jobs), which are acceptable to lose. Or which execution can lead to a worker's stop (for example - OOM). FR, (thanks @hustlahusky)
• ✏️ [BETA] OpenTelemetry support. Starting from now, the new_relic middleware is deprecated, it'll receive only dependency updates and will be removed from the RR bundle in the v2.12.0. (thanks @brettmc)
  OpenTelemetry plugin supports the following exporters:
  1. OTLP (open telemetry protocol): datadog, new relic.
  2. zipkin
  3. stdout
     All these exporters can send their data via http or grpc clients.

Configuration sample (stdout exporter):

http:
  address: 127.0.0.1:43239
  max_request_size: 1024
  middleware: [gzip, otel]
  pool:
    num_workers: 2
    max_jobs: 0
    allocate_timeout: 60s
    destroy_timeout: 60s

otel:
  insecure: false
  compress: true
  exporter: stdout
  service_name: rr_test
  service_version: 1.0.0

New Relic exporter via http client: link

http:
    address: 127.0.0.1:43239
    max_request_size: 1024
    middleware: [gzip, otel]
    pool:
        num_workers: 2
        max_jobs: 0
        allocate_timeout: 60s
        destroy_timeout: 60s
otel:
  insecure: false
  compress: true
  client: http
  exporter: stdout
  custom_url: ""
  service_name: rr_test
  service_version: 1.0.0
  endpoint: otlp.eu01.nr-data.net:4318
  headers:
    - api-key: xxx # your api key here

PHP worker can access tracing data via w3c headers.

• ✏️ protoc-gen-php-grpc now supports optional fields. (thanks @genhoi)

🧹 Chore:

• 🧑‍🏭: All spaces and new lines from the Service plugin output will be automatically trimmed. CHORE, (thanks, @OO00O0O)

Full Changelog: https://github.com/roadrunner-server/roadrunner/compare/v2.10.0-rc.6...v2.10.0-rc.7

Full Changelog: https://github.com/roadrunner-server/roadrunner/compare/v2.10.0-rc.5...v2.10.0-rc.6