Remove legacy unstrict yaml parser

Use yaml strict unmarshal everywhere

• Update flant/helm to the version with strict unmarshal.
• Update gopkg/yaml to v3.
• Update github.com/ghodss/yaml to v1.0.
• Use yaml.UnmarshalStrict everywhere in the werf codebase.

This change may broke existing 1.0 installations with bad yaml-configs.

Temporarily disable stages cleanup by default due to the bug

Dry-run mode is forced by default for stages-cleanup procedure in commands: werf stages cleanup, werf cleanup.

Images cleanup is not affected and enabled as always in werf cleanup.

To enable stages-cleanup forcefully set: WERF_STAGES_CLEANUP_ENABLED=1.

Dry-run by default will be disabled when stages-cleanup procedure bug is fixed.

Installation

Linux amd64

Darwin amd64

Windows amd64

Docs

[docs] Fixed broken links. Links autotest.
[docs] Releases page refactoring. Added buttons to switch between maj.min versions and update channels.
[docs] Dropdown version menu on the sidebar shows version, grouped by maj.min
[docs] Updated footer. Fixed heart icon :)

Bugfixes

Fix werf helm render prints wrong data when printf-like format strings used in the helm templates text.

Werf CI and tests

[ci] Documentation validation workflow: tiny improvements
[ci] Refresh site deployment workflow: remove unused events

[tests] Fix python_encoding_repo/yarn_repo not a git repository
Remove _repo suffixed dir before test. This dir may exist if previous tests has been killed for some reason.
[tests] Fix 'find: unrecognized: -xtype' in stapel_image/git tests: use ubuntu base image instead of alpine
[tests] Enable docs suite for macOS
[tests] Enable resources adopter tests
[tests] Disable a few git suite tests for windows

Installation

Linux amd64

Darwin amd64

Windows amd64

Fix release

• Fix ssh-agent not adding default keys and keys passed by the --ssh-key option.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (17)

• Optimize more stage dependencies checksum calculation (update go-git library).

• Cleanup legacy code that breaks backward compatibility, update library dependencies:

  • Eliminate werf.io/recreate annotation, use helm.sh/hook-delete-policy=before-hook-creation by default.
  • Remove git-mapping legacy cwd param, change paramshash.
    Git-mapping paramshash include 'add' parameter as is without leading slash sign. This will change signature for git-archive stage and need cache rebuild.
  • Remove "legacy" comments for code that is not really legacy.
  • Remove --insecure-repo legacy option: only --skip-tls-verify-registry is available.
  • [werf secret key] Remove dapp secret key legacy support.
  • [go.mod] Update flant/helm, kubernetes lib dependencies to v0.16.7, logboek=origin/debug_functions and kubedog

• Content based tagging by signatures fix, param renamed: werf publish --tag-by-stages-signature and werf ci-env --tagging-strategy=stages-signature.

Installation

Linux amd64

Darwin amd64

Windows amd64

Git stage dependencies optimization

Calculate stage dependencies checksum with git ls-tree.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (13)

• fix gitCache and user-stages with git should select suitable images based on git ancestry;
• publish with optimistic locks and content based tagging
  • werf ci-env --tagging-strategy=content-based option value, which is default;
  • gitLatestPatch stage signature does not include current git-commit, but include a patch content up to current git-commit.
  • werf publish --tag-by-signatures=true|false option to tag images by last stage signatures, which is current method of content-based tagging.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (12)

• Refactor signature calculation, stage signature should depend on stage name.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (11)

Fix panic

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (10)

• Fix invalid build cache: fix dockerInstructions signature does not depend on prev stage signature.
• Optimization: do not create excess git archives before signatures calculation.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (9)

• Fix merge-base error handling for 'Not a valid commit name'.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (8)

• Fix panic when discarding newly built image.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (7)

Fix case when stages storage cache has been invalidated:

• By manual or accidental cleaning of docker image from stages storage while this image exists in stages storage cache.
  Werf will refresh this stage cache from stages storage in this case.

• Always refresh stage cache in stages storage cache when this stage has been newly built.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (6)

• Coherent stages storage cache implemented.
• werf --debug option to print debug logs.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (5)

• Log all stages storage operations with time spent.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (4)

• Select stages from stages storage by label with signature.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (3)

• Fix cannot build dockerfile image.
• Fix panic when specific images for cli cmd has been used.
• Reset image context in build-phase when processing next image.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework (2)

• Commit affects next stages signatures.
• Optimistic locking when saving stage into stages storage: discard already built temporary image when saving into stages storage if there is existing suitable image for this signature.
• New build conveyor architecture.
• Generate image id as: <sha-3-224>-<timestamp-up-to-millisecs>.
• --stages-storage-cache renamed to --stages-storage-lock, only :local value is supported for now, :local is default value.

Installation

Linux amd64

Darwin amd64

Windows amd64

Isolate v1.0 and v1.1 cache for cleanup commands

werf purge, werf cleanup, werf stages purge, werf stages cleanup commands will delete images only by current build cache version.

Installation

Linux amd64

Darwin amd64

Windows amd64

Minor changes and fixes

• Retry docker pull and push if timeout error has occurred.
• Retry docker push when Client.Timeout exceeded while awaiting headers error occurred.
• Rename werf helm ls command to werf helm list.
• Fix werf.io/failures-allowed-per-replica does not affect Job tracking during deploy.
  fixes https://github.com/flant/werf/issues/2071
  refs https://github.com/flant/werf/pull/2072

Docs

• Added two publications.
• Code inspection, spell checking and site fixes.
• Bump anchorjs version and fix its load. Adding HTML anchors on a page was too late and didn't work in Chrome and some other browsers
• Code inspection and spell checking.

Other

[tests] Refuse to get free TCP host port manually.
[tests] Own docker registry image without host volume.
[tests] Rename suites.
[tests] Add package name to temporary test project name.
[tests] Use flant/werf-test images in tests.
[tests] Fix git suite cleanup procedure after test.
[tests] Enable auto gc on macOS.
[ci] Fix repository_dispatch event for deploying site.
[ci] Add scripts to send repository dispatch events.
[ci] Retry docker login 5 times if error occurred.

Werf ci and tests changes.

Installation

Linux amd64

Darwin amd64

Windows amd64

Experimental stages cache rework

• New stage name generation rule. Every build of the stage generates uniq stage-cache image name which consists of 2 parts: old signature and unique identifier.
• New stages storage mechanics. There may be multiple built images for a single signature.
  Select the oldest available with related with the git commit which is an ancestor of current commit.
• Removed git fsck usage to detected unreachable commits.
• --stages-storage-cache param for future usage.

Installation

Linux amd64

Darwin amd64

Windows amd64