Internals

[CI] Documentation local development (#3457)

To run documentation locally, use:

cd docs/site &&
werf compose up --follow --docker-compose-command-options="-d"

Open http://localhost:80

To stop:

cd docs/site &&
werf compose down

You should commit main site files (site/*) into git to see changes.

Documentation files changes will be almost immediately available.

Docs

• [docs] cleanup.md - translate and sync to RU version.
• [docs] Advanced/Cleanup: update intro.
• [docs] Reference/werf.yaml: fix broken link.
• [docs] Reference/werf.yaml template engine: fix typo.
• [docs] Fix helm configuration templates page (#3463)
• [docs] Fix markup errors in advanced/helm/configuration/templates.html

Fixes

• [helm] Force helm kube client namespace initialization.
• [cleanup] Ignore blob-unknown errors properly.
  During cleanup operation do not raise should-reset-stages-storage-cache error when get stage manifest operation have failed with blob-unknown error.
• [run] Fix nonexistent option in the error.
• [follow] Fix werf run: changes cause reassembly but are not accounted for.
• [logging] Build: add waiting for background tasks log message.

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

[logging] Build: add waiting for background tasks log message.

[cleanup] Ignore blob-unknown errors properly.
During cleanup operation do not raise should-reset-stages-storage-cache error when get stage manifest operation have failed with
blob-unknown error.

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

[v1.1] Skip blob-unknown images in cleanup properly

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

[v1.1] Skip blob-unknown images in werf-cleanup procedure

Installation

Linux amd64

Darwin amd64

Windows amd64

Docs

[docs, ru] Advanced/Cleanup: actualize
[docs] Internal/Stages and storage: actualize
[docs] What's new in v1.2/Changelog: cleaning commands
[docs] Internal/Build process: fix typo
[docs] Fix broken quickstart macos instructions
[docs] Fix v1.1 to v1.2 instruction: wrong images report usage
[docs] Fix v1.1 to v1.2 instruction: more correct report usage: pull images before usage
[docs] advanced/cleanup: added info about werf host cleanup GC

• Only in russian.
• Also added a helper table with all cleanup commands.
  [docs] Fixed some grammar in v1.1 to v1.2 migration guide (RU)
  [docs] Fix bundles translation to RU

Fixes

[dockerfile] Automatically reset the Dockerfile exception by .dockerignore
file rules

There is no way to ignore the Dockerfile due to docker limitation when building an image for a compressed context that reads fro
m STDIN.

Add an exception and warning message.

[GC] Fix werf-host-cleanup default percentage and margin values (always use 70% and 5% margin).

[helm] Werf-helm fixes

• Fix panic on werf-helm-dep-update when secret-values are used (fixes https://github.com/werf/werf/issues/3443).
• Set .Values.werf.env="" stub werf service value in werf-helm-* commands.

Internals

• [web] Add view page source button (#3428)
  [web] Add page source button
• [ci] Switch to GHCR (#3441).
• Fix README werf logo path.

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

Automatically invalidate corrupted manifests cache json records.

Installation

Linux amd64

Darwin amd64

Windows amd64

Docs

[docs] Fix v1.1 to v1.2 instruction: wrong images report usage
[docs] Fix v1.1 to v1.2 instruction: more correct report usage: pull images before usage

Internals

[ci] Switch to GHCR

Installation

Linux amd64

Darwin amd64

Windows amd64

Docs fixes

[docs] Fix broken quickstart macos instructions

Installation

Linux amd64

Darwin amd64

Windows amd64

Helm update

Builtin helm updated to v3.5.3.

Fixes

[helm] Add support for oci registries in Chart.yaml: fix panic; updated helm

• Panic was due to uninitialized helm registry client.
  Fixes https://github.com/werf/werf/issues/3415

[builder] Perform rejection of blob-unknown stages
Perform rejection of stages for which get-manifest has failed with blob-unknown error.

Internals and site

[tests] Bundle Suite: fix

Bump kramdown from 2.3.0 to 2.3.1 in /docs/documentation

Bumps kramdown from 2.3.0 to 2.3.1.

• Release notes
• Changelog
• Commits

[web] Fix documentation overview assets (#3432)

[web] Use common partials for guides (#3394)

• [web] Use common partials for guides
• [web] Guides tiles fix

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

[buidler] Fix panic docker_registry.IsNameUnknownError

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

[builder] Proper rejection of blob-unknown stages

Do not reject stage when manifest-unknown or name-unknown errors detected. Only reject images with blob-unknown errors.

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

[builder] Perform rejection of stages when get-manifest has failed with blob-unknown/manifest-unknown errors during fetch procedure.

Installation

Linux amd64

Darwin amd64

Windows amd64

[dockerfile] Support dirs in contextAddFiles

• Add an ability to specify directories along with files in contextAddFile(s) directive.
  Directories will be added recursively to the current Docker context.
• Rename contextAddFile to contextAddFiles throughout the code and in werf.yaml, but
  keep contextAddFile supported as an alias for contextAddFiles for backwards compatibility.

Cleanup improvements

[cleaning] More detailed cleaning logging
[cleaning] Remove werf-host-project commands
[cleaning] werf host purge command: add the --project-name option to remove project images
[cleaning] werf host cleanup command: add the stub --project-name option (without implementation)
[cleaning] Remove the local storage support for cleanup and purge commands
[storage] Disable handling of image metadata in the local storage
[storage] Disable handling of managed images in the local storage
Remove the local storage support for managed-images commands

[GC] Host cleanup improvements

• Cleanup service meta images: commit-metadata images, client-id images, managed-images records, import-metadata records, rejected-images records.
• Save "werf" service image label into all service images created by the werf for future usage.
• Calculate image size in cleanup GC algorithm as VIRTUAL_SIZE-SHARED_SIZE.

Builder fixes

• Fix: print rsync server docker run command on failure.
• [stapel] Fix: fetch artifact/image stage before running imports rsync server
  Fetch operation will check that image exists correctly and will reset stages building conveyor to rebuild this stage if needed.
• Support for rejected stages in the stages storage
  Reject stage when docker-pull have failed with the "blob unknown" error.
  Add special record into the stages-storage for such rejected stages.
• Fix saving stages-ids into the stages-storage-cache
  When some stage has appeared in the repository tags list, but stage image is invalid (due to MANIFEST_UNKNOWN or BLOB_UNKNOWN container registry err
  ors), werf ignores such stage. But this stage-id of such broken stage has been saved into the stages-storage-cache anyway, which is wrong.

Docs

[docs] What's new in v1.2/Changelog: herebyIAdmitThatFromLatestMightBreakReproducibility/herebyIAdmitThatBranchMightBreakReproducibility

Internals

[web] Add guides sitemap (#3420)

[tests] Cleanup Suite: remove the local storage tests
[tests] Cleanup Suite: werf host purge command tests
[tests] Replace werf purge after callback to werf host purge --project-name
[tests] container_registry_per_implementation set: the optional local container registry support
[tests] Cleanup Suite: code generalization
[tests] container_registry_per_implementation set: the optional local container registry support

Installation

Linux amd64

Darwin amd64

Windows amd64

[v1.1] Fix saving stages-ids into the stages-storage-cache

When some stage has appeared in the repository tags list, but stage image is invalid (due to MANIFEST_UNKNOWN or BLOB_UNKNOWN container registry errors), werf ignores such stage. But this stage-id of such broken stage has been saved into the stages-storage-cache anyway, which is wrong.

Installation

Linux amd64

Darwin amd64

Windows amd64

[v1.1][stapel] Support for rejected stages in the stages storage

Reject stage when docker-pull have failed with the "blob unknown" error.

Add special record into the stages-storage for such rejected stages.

Other

[v1.1] Add ability to set werf-stapel image name with WERF_STAPEL_IMAGE_NAME environment variable.

Installation

Linux amd64

Darwin amd64

Windows amd64

[stapel] Fix: fetch artifact/image stage before running imports rsync server

Fetch operation will check that image exists correctly and will reset stages building conveyor to rebuild this stage if needed.

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

Fix: print rsync server docker run command on imports server run failure

Installation

Linux amd64

Darwin amd64

Windows amd64

Docs fixes

[web] Fix 404 page (#3410)
[web] Change 404 logic (#3397)

[docs] Replace Werf with werf
[docs] Fix style, grammar, and typos

Installation

Linux amd64

Darwin amd64

Windows amd64

Fixes

[v1.1][GC] Fix StagesStorage ShouldFetch method: perform fresh docker-inspect, do not use cached inspection result from the previous inspect.

This fix particularly needed for the v1.2 host cleanup procedure which can remove freshly built and saved into the stages storage images. In such case should-fetch and stages-storage-manager fetch method will lock an image and perform standard fetch procedure.

[v1.1] Added stage image locking into stages-storage-manager to prevent v1.2 host cleanup procedure from deleting stages which are in use.

Installation

Linux amd64

Darwin amd64

Windows amd64

[GC] Prevent host cleanup procedure from deleting images which are in use

• Fix StagesStorage ShouldFetch method: perform fresh docker-inspect, do not use cached inspection result from the previous inspect.
• Fix host cleanup procedure: do not delete none:none orphan images created less than 3 hours ago to prevent deletion of freshly built stage images, which are not saved into the stages storage yet.

[GC] Adjust default volume usages: 70% (-5% margin) for docker-storage, 60% (-10% margin) for local-cache

[registry cleanup] Improvements

• Take into account the images used in k8s when applying policies. Previously, these images were not counted and policies excluded images in additio
  n to them.
• Keep the images used in k8s and related to nonexistent commits.
• Improve debugging: log the number of images that are deleted and saved by policies.

Other

Add ability to set werf-stapel image name with WERF_STAPEL_IMAGE_NAME environment variable.

Internals

[ci] Tests Workflow: fix

Installation

Linux amd64

Darwin amd64

Windows amd64