What's Changed

🚀 Features & ✨ Enhancements

cargo-make is cached so it can be reused across runs until the
next version is published in the upstream repo.

Full Changelog: https://github.com/microsoft/mu_devops/compare/v6.0.1...v6.1.0

What's Changed

🐛 Bug Fixes

This resolves a problem in commit 69f6e96 that was not found until
after check-in due to the nature of the issue. It is summarized
below.

In the original check-in (69f6e96), the following succeeds and fails:

• Downloading and Caching as an Artifact:
  • ✓ Use GitHub REST API to download release and extract the binaries
  • ✓ Publish the binaries as a pipeline artifact in the projectmu org
• Retrieving the Artifact
  • Note: In all cases, try the DownloadPipelineArtifact@2 and
    DownloadBuildArtifacts@1 tasks
  • ✓ Access in a manually triggered pipeline
  • ✓ Access in a PR triggered pipeline from a branch in the microsoft
    org repo (i.e. not a fork)
  • ✗ Access in a PR triggered pipeline from a branch outside the
    microsoft org repo (i.e. a fork)

To allow testing using pre-existing PR check pipelines (which have
access to branches on the microsoft org, not forks), the last case
was unexpected and not encountered until the PRs completed.

Since the information is readily available using the Azure Pipelines
REST API without authentication, this change replaces the built-in
tasks with calls to download the binaries from the REST API.

There's a few quirks with pipelines that are accounted for:

1. The Python code is inline so it can directly be used as a
   template in other YAML files that are used as a repository resource.
   If in a separate Python file, the mu_devops repo would need to be
   checked out to use access the file. Checking out the repo would
   increase build times and complicate pre-existing logic.
2. The Azure Pipeline InvokeRESTAPI@1 task is not used as it is
   limited to agentless jobs.
3. Conditions are not allowed on templates. Therefore the OS condition
   is moved to a string parameter that is compared in the condition
   on the task in the template. By default, the task will run on all
   operating systems if not specified.

As a follow up, I will explore the Cache@2 task which might simplify
some logic but require changes elsewhere as an alternative to cache
the binaries in the pipelines.

Signed-off-by: Michael Kubacki [email protected]

Full Changelog: https://github.com/microsoft/mu_devops/compare/v6.0.0...v6.0.1

What's Changed

General release info: https://github.com/microsoft/mu_devops/releases

• The ubuntu-22-build container image is also updated to the latest: 9ab29bc.
  • Note: This is the first release with Rust dev support.
• v6.0.0 also brings some new changes in the pipelines for Project Mu repos to build rust in CI.
  
  

🚀 Features & ✨ Enhancements

The changes can be grouped into three high-level parts:

1. Sync Rust environment files to repos
2. Add pipeline support to build and test Rust code
3. Update the build container to incorporate Rust support

These changes extend existing Rust efforts in Project Mu and improve user experience outside the CI environment.

Commit/Change Overview

.azurepipelines: Add YAML to cache Cargo tools

These files are added so dedicated pipelines can download the latest
release from each tools GitHub repository and push the binaries to
the pipeline artifacts.

Project Mu pipelines dependent on Rust tools can access the tools
directly from the Project Mu pipeline artifacts which is faster
than building and more reliable than depending on infrastructure
outside the pipelines.

.sync/Files.yml: Move dev container section

Moves the section higher in the file to maintain alphabetic ordering
of sections.

.sync: Sync common Rust files

Syncs the following files to repos that support Rust builds.

• Makefile.toml - Cargo makefile configuration. Repos will no
  longer extend from MU_BASECORE.
• rustfmt.toml - The Rust formatting settings for the repo.

Jobs/PrGate.yml: Give the linux container a reliable name

Passes the --name parameter as a container creation option in
pipelines so steps that may need to operate on the container instance
have a fixed name to refer to the container by.

The name is: mu_devops_build_container

Steps: Add Rust templates

Adds two new step templates.

1. RustSetupSteps.yml - Prepare Cargo for Rust builds including
   build during the edk build process and perform other actions
   available during pre-firmware build.
2. RustCargoSteps.yml.yml - Steps to run common Cargo commands
   on the workspace. Cargo should be installed and the workspace
   source cloned before running these steps.

.sync/azure_pipelines/MuDevOpsWrapper.yml: Add rust support

Allows repos that extend the YAML template to pass a new parameter
rust_build. By default, the option is false. If true, then
the repo will be set up to build for Rust and Cargo commands will
be run at the workspace level to build and test code.

Dockerfile: Add Rust support

Updates the build container image to be able to build Rust code
(including Rust UEFI code).

The following additional applications are installed to assist with
setting up Rust dependencies:

curl, jq, and unzip

The CARGO_HOME and RUSTUP_HOME environment variables are set in
the image and the cargo directory is added to the system path.

The version of Rust specified in Version.njk is installed alongside
support for other tools used in Project Mu Rust builds such as
cargo-make and cargo-tarpaulin. The latest release binaries are
pulled directly from their GitHub releases to minimze container
size impact.

The resulting container images can be used for local development and
within pipelines.

Notes

• The Rust toolchain version is defined in Version.njk. The version is used in:

  • The Dockerfile to ensure the container version is in sync
  • The YAML pipeline files to ensure (Windows) pipelines are in sync
  • The rust-toolchain.toml file to ensure local builds in repos are in sync

• The Cargo make makefile is no longer defined in MU_BASECORE and extended to repos. It is synced in whole to repos. Extending from MU_BASECORE unnecessarily complicated pipeline logic and required pulling MU_BASECORE through a submodule (or worse, an external git dependency) on an agent increasing build time.

• The container image build is structured to provide a ready-to-go Rust dev environment while also minimizing space. The previous build container size was ~1.94GB. Adding the Rust tools initially increased size to 3.41GB. With a variety of adjustments, the size now is ~2.58GB.

• After this PR completes, a docker container build will happen and a follow up PR will update the Version.njk file to use the new container build. After that PR is completed, repos dependent on the container will be able to access it.

File sync tested in the following PRs:

• mu_basecore
• mu_plus
• mu_devops
  
  

Full Changelog: https://github.com/microsoft/mu_devops/compare/v5.0.10...v6.0.0

What's Changed

This change needs to be broken out because the upcoming change will
refer to this entry in Version.njk in the main branch when testing
the container build in its PR checks.

Full Changelog: https://github.com/microsoft/mu_devops/compare/v5.0.9...v5.0.10

What's Changed

🐛 Bug Fixes

Recently acpica.org has been unavailable, see:
https://github.com/acpica/acpica/issues/888

Project Mu already publishes iasl to a NuGet feed. To fix the
immediate container build issue and reduce external dependencies for
the future, this change gets iasl from the NuGet feed instead of from
acpica.org. A NuGet client application is not involved to keep the
container build steps light.

The executable is moved to /usr/bin which is already on the system
path.

Signed-off-by: Michael Kubacki [email protected]

Full Changelog: https://github.com/microsoft/mu_devops/compare/v5.0.8...v5.0.9

What's Changed

Full Changelog: https://github.com/microsoft/mu_devops/compare/v5.0.7...v5.0.8

What's Changed

The TagGenerator Script provides the following functionality:

• Scans git history for the most recent matching tag, e.g. 202302.5.10
• Generates the new tag version, incrementing the "major" version based on the presence of breaking changes.
• Generates release notes including commits by type, links to Azure Devops PRs, and contributors.
  
  

General release info: https://github.com/microsoft/mu_devops/releases

• The ubuntu-22-build container image is also updated to the latest: 3bf70b5.
  
  

Full Changelog: https://github.com/microsoft/mu_devops/compare/v5.0.6...v5.0.7

What's Changed

Adding a parameter to prevent checkout of the repo again for the code coverage step.

New pipelines created after the [September 2022 Azure DevOps sprint 209 update](https://learn.microsoft.com/en-us/azure/devops/release-notes/2022/sprint-209-update) have Shallow fetch enabled by default and configured with a depth of 1. Previously the default was not to shallow fetch. To check your pipeline, view the Shallow fetch setting in the [pipeline settings UI](https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/azure-repos-git#shallow-fetch).

Adding fetchdepth of 0 to disable default of shallow checkout for new pipelines using mu_devops.

Full Changelog: https://github.com/microsoft/mu_devops/compare/v5.0.5...v5.0.6

What's Changed

fixes #217

  </blockquote>
  <hr>
</details>

General release info: https://github.com/microsoft/mu_devops/releases

• The ubuntu-22-build container image is also updated to the latest: ea6d2e6.
  
  

Full Changelog: https://github.com/microsoft/mu_devops/compare/v5.0.4...v5.0.5

What's Changed

• GitHub Action: Bump actions/stale from 7 to 8 @dependabot (#153)
• GitHub Action: Bump actions/labeler from 4.1.0 to 4.2.0 @dependabot (#210)
• GitHub Action: Bump release-drafter/release-drafter from 5.23.0 to 5.24.0 @dependabot (#211)

Full Changelog: https://github.com/microsoft/mu_devops/compare/v5.0.3...v5.0.4

What's Changed

These will automatically be picked up by the Submoodule Release
Updater action which tracks release information. Dependabot only
looks at raw commit diffs instead of release points.

Added a note to the beginning of the file that this dependabot
configuration expects submodules at these paths relative to the
workspace to ignore them properly.

Full Changelog: https://github.com/microsoft/mu_devops/compare/v5.0.2...v5.0.3

What's Changed

Full Changelog: https://github.com/microsoft/mu_devops/compare/v5.0.1...v5.0.2

What's Changed

🐛 Bug Fixes

Full Changelog: https://github.com/microsoft/mu_devops/compare/v5.0.0...v5.0.1

What's Changed

Note: #201 will also update Mu DevOps repos to v5.0.0.

Repos were moved to v4.0.5 after the release was made (before this release).

General release info: https://github.com/microsoft/mu_devops/releases

In particular, this rolls out an update to the issue labeler action
to bring it up to latest.

⚠️ Breaking Changes

Two commits.

Add N, N-1, and main branch release drafting

Updates the ReleaseDrafter.yml workflow to filter the base ref to
use when composing the release draft based on the GitHub trigger
context. This allows simultaneous release drafting in repos that
use release branches while repos with non-release branches (e.g.
main branch) are not impacted.

Adds a variable to Version.njk to track the previous Mu release
branch name. This files continues to serve as a single location
where the release branches are defined.

Updates the file sync process to sync the release drafter config
file as needed for the corresponding repo.

.sync/Version.njk: Update Mu repos to Mu DevOps v5.0.0

Changes since last release:
https://github.com/microsoft/mu_devops/compare/v4.0.5...v5.0.0

General release info: https://github.com/microsoft/mu_devops/releases

A major release is being made because of a change in the file names of
config files passed to the release drafter workflow.

Note: v5.0.0 is used in anticipation of the version created by this
change (since it will roll the major version).

How This Supports Ongoing Mu Release Branches

As of this PR, the following branches are N and N-1:

• N: release/202302
• N-1: release/202208

Each of those branches in each repo currently has a synced release-drafter.yml
file that sets up a branch push trigger for its branch name. This will cause pushes
(PR completions) to that branch to trigger the ReleaseDrafter.yml workflow here
in Mu DevOps.

This workflow compares the latest values for latest_mu_release_branch and
previous_mu_release_branch in .sync/Version.njk to the triggering branch (github.ref)
to determine if there's a match. If there is, either the release-draft-config-n.yml
or release-draft-config-n-1.yml is respectively used. This means, if a branch
(like release/202208) eventually becomes "N-2", it will simply not match and no
longer have release notes/versioning auto drafted.

Those release drafter config files are specially templatized during syncing to specify
commitish base refs (the branch name) to filter for drafting the notes. This means
a release/202208 submission will filter to release/202208 changes.

Since the file sync is always active for the "current branch" in a repo. This means that
over time, each release branch in a repo will have its own relative N and N-1 release
drafter config files that stay in the branch. ReleaseDrafter.yml uses the default
branch config files so the N and N-1 config files should match the latest and
previous values in .sync/Version.njk.

🚀 Features & ✨ Enhancements

Two commits.

Add N, N-1, and main branch release drafting

Updates the ReleaseDrafter.yml workflow to filter the base ref to
use when composing the release draft based on the GitHub trigger
context. This allows simultaneous release drafting in repos that
use release branches while repos with non-release branches (e.g.
main branch) are not impacted.

Adds a variable to Version.njk to track the previous Mu release
branch name. This files continues to serve as a single location
where the release branches are defined.

Updates the file sync process to sync the release drafter config
file as needed for the corresponding repo.

.sync/Version.njk: Update Mu repos to Mu DevOps v5.0.0

Changes since last release:
https://github.com/microsoft/mu_devops/compare/v4.0.5...v5.0.0

General release info: https://github.com/microsoft/mu_devops/releases

A major release is being made because of a change in the file names of
config files passed to the release drafter workflow.

Note: v5.0.0 is used in anticipation of the version created by this
change (since it will roll the major version).

How This Supports Ongoing Mu Release Branches

As of this PR, the following branches are N and N-1:

• N: release/202302
• N-1: release/202208

Each of those branches in each repo currently has a synced release-drafter.yml
file that sets up a branch push trigger for its branch name. This will cause pushes
(PR completions) to that branch to trigger the ReleaseDrafter.yml workflow here
in Mu DevOps.

This workflow compares the latest values for latest_mu_release_branch and
previous_mu_release_branch in .sync/Version.njk to the triggering branch (github.ref)
to determine if there's a match. If there is, either the release-draft-config-n.yml
or release-draft-config-n-1.yml is respectively used. This means, if a branch
(like release/202208) eventually becomes "N-2", it will simply not match and no
longer have release notes/versioning auto drafted.

Those release drafter config files are specially templatized during syncing to specify
commitish base refs (the branch name) to filter for drafting the notes. This means
a release/202208 submission will filter to release/202208 changes.

Since the file sync is always active for the "current branch" in a repo. This means that
over time, each release branch in a repo will have its own relative N and N-1 release
drafter config files that stay in the branch. ReleaseDrafter.yml uses the default
branch config files so the N and N-1 config files should match the latest and
previous values in .sync/Version.njk.

Full Changelog: https://github.com/microsoft/mu_devops/compare/v4.0.5...v5.0.0

What's Changed

Includes the following fixes (occassionally impacting Mu PRs):

• Issue labels that do not match a regex will no longer be removed
• Support empty body and only remove existing labels

Updates actions/labeler from v4 to v4.1.0.

No major known changes that impact Mu DevOps.

Full Changelog: https://github.com/microsoft/mu_devops/compare/v4.0.4...v4.0.5

What's Changed

Full Changelog: https://github.com/microsoft/mu_devops/compare/v4.0.3...v4.0.4

What's Changed

This change will update the publish coverage job's condition to ignore the skipped conditions, such that it will always run the job as long as there the previous job(s) not failing or being cancelled.

Also explicitly sets the submodule weekly update day to Tuesday so
PIP and submodules updates can both be available Wednesday morning
for review and merge.

Full Changelog: https://github.com/microsoft/mu_devops/compare/v4.0.2...v4.0.3

What's Changed

🐛 Bug Fixes

It also updated the tag version to v4.0.0 to pick up the latest change of supporting self-hosted agents.

Full Changelog: https://github.com/microsoft/mu_devops/compare/v4.0.1...v4.0.2

What's Changed

🐛 Bug Fixes

Full Changelog: https://github.com/microsoft/mu_devops/compare/v4.0.0...v4.0.1

What's Changed

General release info: https://github.com/microsoft/mu_devops/releases

A major release is being made because of a change in the container
images used in Project Mu pipelines.

⚠️ Breaking Changes

The main change is around the flag of installing tools or not. The existing routine will expect the incoming flag being a boolean type, which does not work when this parameter is being populated using a variable (string). This change will update the flag to string type as well.

After changing, the only file being synced to individual repos is MuDevOpsWrapper.yml. And each repo can configure their own build matrix for build target, architecture list, run on selfhost agents or not, etc.

This is considered a breaking change, due to:

1. Steps/PrGate.yml updated the extra_install_steps to be invoked before stuart_setup and stuart_update. But this change does not knowingly break the existing consumers.
2. Jobs/PrGate.yml updated the parameters to accept parameters.build_matrix, which needs the consumers to update top level yaml script to comply with the updated template.

🚀 Features & ✨ Enhancements

The main change is around the flag of installing tools or not. The existing routine will expect the incoming flag being a boolean type, which does not work when this parameter is being populated using a variable (string). This change will update the flag to string type as well.

After changing, the only file being synced to individual repos is MuDevOpsWrapper.yml. And each repo can configure their own build matrix for build target, architecture list, run on selfhost agents or not, etc.

This is considered a breaking change, due to:

1. Steps/PrGate.yml updated the extra_install_steps to be invoked before stuart_setup and stuart_update. But this change does not knowingly break the existing consumers.
2. Jobs/PrGate.yml updated the parameters to accept parameters.build_matrix, which needs the consumers to update top level yaml script to comply with the updated template.

Full Changelog: https://github.com/microsoft/mu_devops/compare/v3.0.0...v4.0.0