Docs fixes

[web] Fix 404 page (#3410)
[web] Change 404 logic (#3397)

[docs] Replace Werf with werf
[docs] Fix style, grammar, and typos

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

[v1.1][GC] Fix StagesStorage ShouldFetch method: perform fresh docker-inspect, do not use cached inspection result from the previous inspect.

This fix particularly needed for the v1.2 host cleanup procedure which can remove freshly built and saved into the stages storage images. In such case should-fetch and stages-storage-manager fetch method will lock an image and perform standard fetch procedure.

[v1.1] Added stage image locking into stages-storage-manager to prevent v1.2 host cleanup procedure from deleting stages which are in use.

Installation

Linux amd64

Darwin amd64

Windows amd64

[GC] Prevent host cleanup procedure from deleting images which are in use

• Fix StagesStorage ShouldFetch method: perform fresh docker-inspect, do not use cached inspection result from the previous inspect.
• Fix host cleanup procedure: do not delete none:none orphan images created less than 3 hours ago to prevent deletion of freshly built stage images, which are not saved into the stages storage yet.

[GC] Adjust default volume usages: 70% (-5% margin) for docker-storage, 60% (-10% margin) for local-cache

[registry cleanup] Improvements

• Take into account the images used in k8s when applying policies. Previously, these images were not counted and policies excluded images in additio
  n to them.
• Keep the images used in k8s and related to nonexistent commits.
• Improve debugging: log the number of images that are deleted and saved by policies.

Other

Add ability to set werf-stapel image name with WERF_STAPEL_IMAGE_NAME environment variable.

Internals

[ci] Tests Workflow: fix

Installation

Linux amd64

Darwin amd64

Windows amd64

[helm] Added werf-helm-registry command

Installation

Linux amd64

Darwin amd64

Windows amd64

[GC] Host auto cleanup refinements

• Enabled auto host cleanup by default.
• --allowed-docker-storage-volume-usage and --allowed-docker-storage-volume-usage-margin to configure docker-storage host cleanup.
• --allowed-local-cache-volume-usage and --allowed-local-cache-volume-usage-margin to configure local-storage host cleanup (including git-data).

Docs

[web] Fix installation page tab logic (#3395)
[docs] Remove 1.0 version from the installation page (#3396)

Internals

[CI] Add commit and PR tests (#3393)
[CI] Update Certificate template

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

Completely remove redudant stages_and_images read-only shared lock usage to reduce the number of requests to lock-server.

Installation

Linux amd64

Darwin amd64

Windows amd64

[GC] Git-data repos, worktrees, archives and patches GC based on LRU algorithm

• Remove v1.1 git-data if werf v1.1 was not started for 3 days on this build-host.
• Remove *.tmp data from git_repos, which are git-clone temporary dirs accidentally left existing.
• Moved gc and git-data-manager related code into the separate package gitdata inside git_repo package (refactor).
• Fixed new bug introduced by the v1.2.10+fix14 version: assets/setup /.werf/stapel/embedded/bin/xargs: Cannot open input file '/.werf/patch/.werf/patch/0c962ed1755d99a8937abe6fe63953bf818...

Installation

Linux amd64

Darwin amd64

Windows amd64

[cleanup][v1.1] Enable v1.2 werf-cleanup mode for v1.1 by default

--git-history-based-cleanup-v1.2 option has been enabled by default: use git history based cleanup and delete images tags without related image metadata (default $WERF_GIT_HISTORY_BASED_CLEANUP_v1_2).

Installation

Linux amd64

Darwin amd64

Windows amd64

[GC] Add support for v1.1 images into host cleanup procedure

Cleanup only those images/stages, which has been published into REPO stages-storage.

NOTE. V1.2's werf host cleanup procedure will not cleanup --stages-storage=:local stages, because this is primary stages storage data, and it can only be cleaned by the regular per-project werf cleanup command with git-history based algorithm.

Installation

Linux amd64

Darwin amd64

Windows amd64

[GC] Save last/first werf run timestamps for v1.2

This info needed for v1.2's host cleanup procedure.

Installation

Linux amd64

Darwin amd64

Windows amd64

[GC][v1.1] Save last/first werf run timestamps for v1.1

This info needed for v1.2's host cleanup procedure.

Installation

Linux amd64

Darwin amd64

Windows amd64

[GC] Added basic support for cached git-data cleaning into host cleanup procedure

• Remove ~/.werf/local_cache/{git_patches|git_archives|git_worktrees|git_repos} completely when volume usage exceeds allowed level (smart GC is in progress).
• Added last-used-timestamps into git_patches, git_archives, git_worktrees and git_repos caches.
• Refactored git_patches and git_archives files structure in ~/.werf/local_cache to reduce number of files which stored in the cache dir at the same time (use first byte of sha256 hash as a directory).
• Possible fix for permission-denied errors related to git-data

Docs

[docs] Fix broken links and update tests (#3384)

Internals

[ci] Update worflow (#3383)

[web] Fix guides style

Installation

Linux amd64

Darwin amd64

Windows amd64

Garbage collection of images for v1.1 backports

[v1.1] Imported lrumeta package from v1.2 to allow cleaning of images created by the v1.1 in the v1.2 werf version.

Docs

[docs] SEO improvement (#3385)

[docs] Update page templates to improve SEO, fix breadcrumbs

[web] Fix breadcrumbs (#3365)

[docs] Backport the new design (#3364)

[docs] Update the version of werf actions
werf/actions/deploy@master -> werf/actions/[email protected]

Installation

Linux amd64

Darwin amd64

Windows amd64

[cleanup] Fix cleaning images built on a virtual merge commit

[metadata] Add image metadata for the from virtual merge commit

Deprecations

Deprecation warning for the --repo-implementation option

DEPRECATION WARNING: The option --repo-implementation ($WERF_REPO_IMPLEMENTATION) is renamed to --repo-container-registry ($WERF_REPO_CONTAINER_REGISTRY) and will be removed in v1.3!

Internals

[docs][ci] SEO improvement and ci updates (#3373)

[ci] Tests Workflow: drop out of self-hosted runners

[ci] Documentation Validation: fix `chmod: Gemfile.lock: No such file or directory

[ci] Fix tests

```
Run # unit tests binaries
?       github.com/werf/werf/cmd/werf   [no test files]
go: directory integration/ci_suites/default/docs/_fixtures/cli/docs/site/backend is outside main module
Error: Process completed with exit code 1.
```

[ci] Workflows: replace always with success || failure condition
The always function returns true even when canceled, or the previous tasks skipped.

[ci] Documentation Validation: fix chmod: Gemfile.lock: No such file or directory

[ci] Use .Values.werf.env instead of .Values.global.env

[maintenance_helper] Fix error format

• Errorf format %s reads arg #2, but call has 1 arg
• Errorf format %s reads arg #3, but call has 2 args

Installation

Linux amd64

Darwin amd64

Windows amd64

[dockerfile] The BuildKit support with explicit user activation

• By default werf disables BuildKit when building dockerfiles.
• If the user explicitly activates BuildKit (DOCKER_BUILDKIT = 1), werf enables it but without output in the background builds due to https://github.com/docker/cli/issues/2889.

[GC] Rework host cleanup more

• Run auto host cleanup only when --disable-host-cleanup=false (WERF_DISABLE_HOST_CLEANUP=false), cleanup will be enabled by default later.
• Run auto host cleanup could run in werf-build/werf-converge and other commands as previously, BUT it will run in the end of the command.
  • Big note message is printed before running auto host cleanup.
    • This message contains info how to disable auto-host-cleanup.
• Run /tmp/werf-config-render-* and /tmp/werf-project-data-* cleanup in the werf-host-cleanup procedure as well.
• Removed global lock for host cleanup, added individual locks for stage-images and build-containers
• Run auto host cleanup only when 2 hour has passed since first werf run on this host (try to determine that current host is persistent).

Fixes

[status] Fix duplications in the path list

Error: unable to read werf config: the following files must be committed:

 - werf.yaml
 - werf.yaml

Internals

[ci] Delete cleanup job (#3371)

Installation

Linux amd64

Darwin amd64

Windows amd64

[web] Refactoring (#3346)

[docs] Change documentation folders structure, page URLs.
[web] Add some new design
[ci] Refactoring

• Cherry-picked from #3350: Add new highlighter theme
• Cherry-picked from #3344: Add news and updates block

Co-authored-by: Anton Klimov [email protected]

Speeding up git-repo operations

[git_repo] Caching and optimization operations with git objects

Updated docker, buildkit and containerd go-dependencies

• github.com/docker/cli => v20.10.5
• github.com/docker/docker => v20.10.5
• github.com/containerd/containerd => v1.5.0-beta.4
• github.com/moby/buildkit => v0.8.2

Installation

Linux amd64

Darwin amd64

Windows amd64

[GC] Host cleaning GC refinements and fixes

[GC] Host cleaning GC refinements and fixes: correct usage of allowed-volume-usage and allowed-volume-usage-margin

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

[git_repo] Invalidate archives and patches cache

Installation

Linux amd64

Darwin amd64

Windows amd64

[GC] Host cleanup garbage collection refinements

• Added --allowed-volume-usage-margin param which sets target volume usage after cleanup: allowed-volume-usage - allowed-volume-usage-margin.
• Delete 10 images minimal in each GC iteration to speed up cleanup.

Installation

Linux amd64

Darwin amd64

Windows amd64