Harp is for Harpocrates (Ancient Greek: Ἁρποκράτης) the god of silence, secrets and confidentiality in the Hellenistic religion. - Wikipedia
Harp is a tool set to operate secret management by contract. The objective is
to reduce the value centric management by handling secret data in a
reproducible
way by providing a technical stack to describe how your value is
managed by contracts and pipelines. Furthermore, we know that naming thing is hard
,
as a consequence a secret could be consistently
associated to a predictable
secret identifier used as a key to refer to the secret value. Finally, the secret
can hold additional metadata (ownership, rotation period, leak severity, etc.)
which can be consumed during the pipeline executions.
These key/value associations (path ⇒ value) form a Bundle
stored in an
immutable file named a Container
. This Container
acts as a pivot format to
allow Harp commands to communicate and create data management pipelines.
In addition to that, it provides a template engine
used to generate various
confidence values (password, passphrase, crypto keys, etc.) and allow more
sophisticated rendering operations (configuration files, etc.).
Finally, it provides a SDK
to allow developers to integrate Harp
features
in their products, and/or extend the Harp pipeline features by creating new
plugins.
Developers
should negotiate secret value for the secret consumer they areSecret Operators
use different set of tools to achieve secretsingle secret value
and you are asking yourselfhow to generate a strong password
- Harp has a template engine with secretthousands secrets
to handle to deploy your platform/customerson multiple cloud providers
with different secret storages
- Harp will help youephemeral secret storage
to bootstrap
your long term cloudmigrate massively
your secrets from one secret storage toalter/modifiy
a secret (rotation/deprecation/renewal) - HarpGitOps-able
secret storage agnostic operation set
, so that youharp
allows you to handle secrets using deterministic pipelines expressed
using an atomic series of CLI operations applied to a commonly shared container
immutable and standalone file system used to store secret collection (Bundle)
generated from a template engine via user specification, or external secret
value coming from files or external secret storage.
These pipelines use the immutable container file system as a data exchange
protocol and could be extended for new input, intermediary operation or output
via plugins created with the harp
SDK.
Each applied transformation creates a container with transformed data inside. This will enforce container reproducibility by eliminating cumulative side effects applied to the same container.
The container handles for you the confidentiality and integrity protection applied to the secret collection stored inside and manipulated by copy during the pipeline execution.
New to harp, let's start with onboarding tutorial ! TL;DR - Features overview
Harp provides :
github.com/elastic/harp/pkg/container
;github.com/elastic/harp/pkg/bundle
;on-steroid
template engine exposed as github.com/elastic/harp/pkg/template
github.com/elastic/harp/pkg/cso
And allows :
BundleTemplate
) to describe secret and their usages;Is it used internally at Elastic? - Yes. It is used to generate bootstrap secrets used to bootstrap the new region infrastructure components. #ChickenEggProblem
Harp is only supporting Vault
? - No, it has been published with only vault
support built-in, but it supports many other secret storage implementations via
plugins.
What's the difference with Vault
? - HashiCorp Vault is an encrypted highly
available K/V store with advanced authorization engine, it doesn't handle
secret provisioning for you. You can't ask Vault to generate secrets for your
application and store them using a defined logic. Harp is filling this
requirement.
harp
artifacts and source code is released under Apache 2.0 Software License.
Download a release or build from source.
For stable version
brew tap elastic/harp
brew install elastic/harp/harp
Download a release or build from source.
$ git clone [email protected]:elastic/harp.git
$ export HARP_REPOSITORY=$(pwd)/harp
Only last 2 minor versions of a major are supported.
Harp
is compiled with :
$ go version
go version go1.17.8 linux/amd64
Simple go version manager - https://github.com/stefanmaric/g
Go 1.18 compilation is enabled for testing purpose and golangci-lint
looks to
hang, so it has been disabled for the moment.
Mage is an alternative to Make where language used is Go. You can install it using 2 different methods.
# Install mage
git clone https://github.com/magefile/mage
cd mage
go run bootstrap.go
export PATH=$HARP_REPOSITORY/tools/bin:$PATH
# Build harp in bin folder
mage
Install nix
on your system, if not already installed.
$ sudo install -d -m755 -o $(id -u) -g $(id -g) /nix
$ curl -L https://nixos.org/nix/install | sh
More information? - https://nixos.wiki/wiki/Nix_Installation_Guide
$ cd $HARP_REPOSITORY
$ nix-shell
# Go to tools submodule
cd $HARP_REPOSITORY/tools
# Resolve dependencies
go mod tidy
go mod vendor
# Pull tools sources, compile them and install executable in tools/bin
mage
You have to build this image once before executing artifact pipelines.
mage docker:tools
Or you can download harp-tools
from GitHub registry
# Standard usecase
$ docker pull ghcr.io/elastic/harp/harp-tools:latest
# FIPS compliant go toolchain
$ docker pull ghcr.io/elastic/harp/harp-tools-fips:latest
Check image integrity with cosign
and the public key build/artifact/cosign.pub
cosign verify --key build/artifact/cosign.pub ghcr.io/elastic/harp/harp-tools:latest
Verification for ghcr.io/elastic/harp/harp-tools:latest --
The following checks were performed on each of these signatures:
- The cosign claims were validated
- The signatures were verified against the specified public key
- Any certificates were verified against the Fulcio roots.
[{"critical":{"identity":{"docker-reference":"ghcr.io/elastic/harp/harp-tools"},"image":{"docker-manifest-digest":"sha256:1be31528e7b00c9e836479aadfdf49319f3b4d7916e705c43ffd0b14965763a8"},"type":"cosign container image signature"},"optional":{"ref":"40714fef947d018e6053991f5ddb54283f466b04","repo":"elastic/harp","workflow":"Build and push docker tools"}}]
# or docker image [distroless:static, rootless, noshell]
mage docker:harp
# To execute in the container
docker run --rm -ti --read-only elastic/harp:<version>
You can find more Harp feature extensions - https://github.com/elastic/harp-plugins
Here is the list of external projects used as inspiration :