kibana-custom-nodejs-builds

Stars
2
Committers
5

kibana-custom-nodejs-builds

Contains configuration and sources to build node.js

The main use case right now is building node.js@18+ with [email protected], which is required for some older platforms. (More context: https://github.com/nodejs/unofficial-builds/pull/69). Aside from that, and more recently, a new usage for that work appeared and it is now also supporting our efforts of using the pointer compression feature of node.js for our serverless docker image.

How to generate a new build?

We have a kibana-flavored buildkite job that does most of the work for building these distributables, by running the scripts in this repo. You can find the job here: https://buildkite.com/elastic/kibana-custom-node-dot-js-builds - configured here.

Build on a branch

You can either create a new branch, where you set the new default value for the Node, and run a new build on that branch (or get the PR merged, and run a build on main). See this PR for reference.

Build with override parameters

Or you can set override parameters on buildkite when starting the job:

OVERRIDE_TARGET_VERSION=18.17.0

Warning ⚠️

If there are native modules going into production along with Kibana, each of them should be built custom along the builds we provide on this repo both for glic-217 and pointer-compression modes.

What happens after?

You should see the job starting up two tasks in parallel. Those are the (glibc-217 + pointerc compression) builds per platform. The arm64 cross-compilation takes a while, but once they're done, we also create a SHASUMS256.txt based on the original version's hashes, but updating the entries for our custom-built versions.

All the artifacts will be uploaded to this bucket: kibana-custom-node-artifacts

How to use these builds in Kibana?

Running locally

You can run most scripts locally on Mac/Linux. You'll need a few of the build/infra tools:

  • Docker
  • node.js
  • gsutil (brew install google-cloud-sdk)

Export some env variables required for the builds

  export TARGET_ARCH="arm64" # or amd64
  export TARGET_VERSION="18.17.0"

Then run any of the individual scripts locally:

  ./scripts/create_build_images.sh
  ./scripts/build_nodejs.sh
  # ./scripts/...

Keep in mind, individual steps might have a dependency on other steps. For reference, check the call order in pipeline.yml.

Docker image for node.js builds

One of the main components we need to create in this step is a docker image for an environment that's set up for building node.js.

The bits for this component are in the build-image-config folder.

The docker image uses mounted directories as working directories, as well as outputting the artifacts in these directories.

Scripts for running the builds

Most of the buildkite logic is sheltered in the scripts directory.

Context

During development, we found some more information that can be helpful as context, should anyone find this repo again

  • This repository is only needed...
  • The created Docker images needn't be pushed
    • they can be used once for the build, then rebuilt in case we need to run it again
  • I decided to remove the VARIATION attribute on the node.js:
    • build would result in the variation showing up in the file and folder names, making the logistics more difficult, if we want to keep this mostly transparent for Kibana