autopkg

Automating packaging and software distribution on macOS.

OTHER License

Stars
1.3K

Bot releases are hidden (Show)

autopkg - AutoPkg 2.3 RC 1

Published by homebysix over 3 years ago

This is the release candidate for AutoPkg 2.3.

NEW FEATURES:

AutoPkg now supports recipes in yaml format (#698). Yaml recipes tend to be more human-readable than plist recipes, especially for those who don't work with plists on a daily basis.

AutoPkg can produce new recipes in yaml format using autopkg new-recipe SomeCoolApp.pkg.recipe.yaml and make overrides in yaml format using autopkg make-override --format=yaml SomeCoolApp.pkg. Searching for public yaml recipes on GitHub is also possible using autopkg search.

NOTES FOR RECIPE AUTHORS:

  • Because yaml recipes will require AutoPkg 2.3 or later in order to run, and because some members of the AutoPkg community may still be using AutoPkg 1.x, recipe authors are encouraged to be conservative and keep existing public recipes in their current format for a while.
  • If you have both plist and yaml recipes for the same app in your repo, you may experience unexpected behavior now that AutoPkg detects and uses yaml recipes.

OTHER CHANGES FROM 2.2:

  • Added support for internal GitHub URLs (#649)
  • autopkg make-override no longer creates override for deprecated recipes by default (#685)
  • Typo fixed in the recipe template created by autopkg new-recipe
  • Fixed a bug causing autopkg repo-add and autopkg repo-delete to fail for repos in GitHub organizations with non-alphanumeric characters in their names (#712, #715)
  • CodeSignatureVerifier warns when certain incorrect input variables are detected
  • MunkiImporter now uses consistent pkginfo matching logic (#671)
  • Minor edits to help text
  • Improvements to Versioner processor (#600)
  • Help is now shown for autopkg list-processors --help, matching behavior of most other verbs (#717)
  • The output of autopkg list-recipes --plist is now text instead of binary (this matches previous behavior in AutoPkg 1.x)
  • More output when using autopkg repo-add and autopkg repo-delete (#704)
  • Fixed a bug in MunkiImporter that caused incorrect uninstaller_item_location path (#702)
  • Building a foundation for long term expansion of platform support (#648, #651, #653, #656, #658, #666, #670)

KNOWN ISSUES:

  • #710 is currently affecting some autopkg search results (regardless of whether the recipes are plist or yaml)
autopkg - AutoPkg 2.2

Published by nmcspadden about 4 years ago

2.2 (August 24, 2020)

NEW FEATURES
MunkiImporter now supports Munki repo plugins, thanks to @tboyko. The default behavior
is still to use FileRepo as the default local behavior, so existing behavior is
unchanged. (https://github.com/autopkg/autopkg/pull/654)

CHANGES FROM 2.1:

  • URLDownloader handles Content-Disposition filenames with quotes correctly (https://github.com/autopkg/autopkg/pull/633)
  • README and CONTRIBUTING guides updated with correct Python 3 framework info (https://github.com/autopkg/autopkg/pull/638)
  • PyYAML updated to 5.3.1 to address PyYAML-CVE-2020-1747 (https://github.com/autopkg/autopkg/pull/642)
  • Internal autopkg code structure is being shifted, with more code moving into separate
    shared modules. Lots of various lint fixes, formatting, and safety handling improvements.
  • GitHub API queries no longer fail when searching for recipes with spaces in the name,
    and are now quoted correctly (https://github.com/autopkg/autopkg/pull/664)
  • Processor subclasses now automatically configure an empty dictionary for self.env if
    none is provided. This doesn't have any practical effect, but makes it easier to create
    and use new Processor subclasses in the future.
autopkg - AutoPkg 2.1

Published by nmcspadden over 4 years ago

2.1 (May 19, 2020)

NEW FEATURES
AutoPkg now supports the verbs list-repos and processor-list for convenience (https://github.com/autopkg/autopkg/pull/628)

autopkg info --pull/-p now allows you to fetch all parent repos of a recipe
automatically.

Example:

$ autopkg repo-delete recipes
$ autopkg info -p GoogleChrome.munki
Didn't find a recipe for com.github.autopkg.munki.google-chrome.
Found this recipe in repository: recipes
Attempting git clone...

Adding /Users/nmcspadden/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes to RECIPE_SEARCH_DIRS...
Updated search path:
  '.'
  '~/Library/AutoPkg/Recipes'
  '/Library/AutoPkg/Recipes'
  '/Users/nmcspadden/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes'

Description:         Downloads the latest Google Chrome disk image and imports into Munki.
Identifier:          local.munki.GoogleChrome
Munki import recipe: True
Has check phase:     True
Builds package:      False
Recipe file path:    /Users/nmcspadden/Library/AutoPkg/RecipeOverrides/GoogleChrome.munki.recipe
Parent recipe(s):    /Users/nmcspadden/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.munki.recipe
                     /Users/nmcspadden/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.download.recipe

The automatic fetching works by looking at the parent identifier of a recipe, and
searching GitHub via API for that file. It fetches that parent file from GitHub
directly, and adds the repo that it belongs to. Then it parses its parent, recursively
until it finds a recipe with no parents.

Note that the only verb to support this is autopkg info. You can use this feature to
dynamically fetch parents on-demand, instead of preconfiguring your environment with a
list of known repos.

CHANGES FROM 2.0.2:

autopkg - AutoPkg 2.0.2

Published by nmcspadden over 4 years ago

2.0.2 (February 05, 2020)

AutoPkg 2.0 Release

PYTHON 3

This is the official release of a Python 3-only version of AutoPkg. It is no longer compatible with Python 2, and will encounter syntax errors and failures if ran with Python 2.

The release package has an included Python 3 framework that includes all necessary modules to run everything in AutoPkg core, and all of the recipes in autopkg-recipes.

MAJOR HIGHLIGHTS OF THE PYTHON 3 CODE:

  • FoundationPlist has been retired. plistlib in Python 3 should be used to handle all plist parsing.
  • All Python string interpolation now use Python 3-style f-strings (formatted string literals).
  • All references to unicode vs. string types have been refactored to use Python 3's native byte strings whenever possible.
  • All unit tests were updated to Python 3.
  • All Python code now use a symlink to the embedded Python framework. It is strongly recommended you do not modify or touch anything inside the AutoPkg Python framework (i.e. avoid installing things with pip).

CHANGES FROM RC2:

autopkg - AutoPkg 2.0.1 RC 2

Published by nmcspadden over 4 years ago

2.0 RC2 (January 31, 2020)

CHANGES FROM RC1:

autopkg - AutoPkg 2.0 RC 1

Published by nmcspadden almost 5 years ago

ANNOUNCEMENT:

This is the release candidate for AutoPkg 2.0.

This is the best time and way to test if your recipe list works in Python 3. Barring the discovery of serious problems, we expect to release AutoPkg 2.0 next week.

DEVELOPMENT:

The previous branch for AutoPkg development, "py2-to-3", has been merged into Master.

AutoPkg 1.x has been cut into a new branch ("AutoPkg_1.x"). Short of an emergency scenario, development on AutoPkg 1.x is frozen at the release of 1.4.1.

CHANGES FROM BETA 3:

autopkg - AutoPkg 1.4.1

Published by nmcspadden almost 5 years ago

1.4.1 (December 2, 2019)

FIXES:

  • URLGetter now has a download_to_file(url, filename) function that can be used in
    custom processors. It simply downloads a URL to a specific filename, and raises a
    ProcessorError if it fails for any reason.
autopkg - AutoPkg 1.4

Published by nmcspadden almost 5 years ago

1.4 (November 25, 2019)

FIXES:

ADDITIONS:

  • The new URLGetter base Processor class has been merged in. It provides a new centralized way to handle fetching and downloading things from the web. In the future, more convenience functions will be added to allow any custom processor to easily fetch web resources without having to write their own urllib/web-handling code.
  • Thanks to @MichalMMac's heroic efforts, URLGetter is now much easier for other processors to use. There are now two ways a custom processor can download things without needing to write any urllib logic:
    • URLGetter.download_with_curl(curl_command,) takes a curl command as an argument (a list of strings that is passed to subprocess). You can use this along with the other helper functions to arrange your own curl command with custom headers and arguments, and parse the output.
    • URLGetter.download(url, headers=None) takes a URL (and optional headers) and returns the output of the curl command. You can use this to simply retrieve the results of requesting a web page (such as for URLTextSearcher).
  • All custom processors that need to make a web request of any kind in autopkg/recipes have been switched to using URLGetter's methods. No more urllib in processors!
  • URLDownloader now has a new prefetch_filename option that will allow fetching the original name from the redirected URL (https://github.com/autopkg/autopkg/pull/571)
  • GitHub calls now successfully function through proxies (https://github.com/autopkg/autopkg/issues/416)
autopkg - AutoPkg 2.0 Beta 3

Published by nmcspadden almost 5 years ago

PYTHON 3 ONLY RELEASE

This is the second beta release of a Python 3-only version of AutoPkg. It is no longer compatible with Python 2, and will encounter syntax errors and failures if run with Python 2.

The release package has an included Python 3 framework that includes all necessary modules to run everything in AutoPkg core, and all of the recipes in autopkg-recipes.

This release also coincides with the most recent commit to
autopkg/recipes, so both change notes will be provided here.

CHANGES FROM BETA 2:

  • Thanks to @MichalMMac's heroic efforts, URLGetter is now much easier for other processors to use. There are now two ways a custom processor can download things without needing to write any urllib logic:
    • URLGetter.download_with_curl(curl_command, text=True) takes a curl command as an argument (a list of strings that is passed to subprocess). You can use this along with the other helper functions to arrange your own curl command with custom headers and arguments, and parse the output.
    • URLGetter.download(url, headers=None, text=False) takes a URL (and optional headers) and returns the output of the curl command. You can use this to simply retrieve the results of requesting a web page (such as for URLTextSearcher).
      *In both cases, you can pass text mode to determine if you get straight text output.
  • All custom processors that need to make a web request of any kind in autopkg/recipes have been switched to using URLGetter's methods. No more urllib in processors!
  • Some minor formatting changes in the code itself

CHANGES FROM BETA 1:

  • The new URLGetter base Processor class has been merged in. It provides a new centralized way to handle fetching and downloading things from the web. In the future, more convenience functions will be added to allow any custom processor to easily fetch web resources without having to write their own urllib/web-handling code.
  • Failing to import a processor due to a Python syntax error (such as due to py2 vs. py3 imports) now has a more specific and clear error message (e52ae69)
  • Many, many, many bytes vs. string issues resolved in core processors
  • Copier now has a unit test, and produces some more useful output
  • autopkgserver shouldn't complain about socket descriptors anymore
  • isort now has a seed config that explicitly lists certain third party modules so that they're sorted at the top or bottom of import blocks correctly
  • All custom processors in autopkg-recipes now use certifi to set the base SSL certs so that urllib web requests work; this will be removed in the future once URLGetter's convenience functions are written

CHANGES IN INITIAL 2.0 RELEASE:

  • FoundationPlist has been retired. plistlib in Python 3 should be used to handle all plist parsing.
  • All Python string interpolation should prefer the use of f-strings (formatted string literals).
  • All references to unicode vs. string types have been refactored to use Python 3's native byte strings whenever possible.
  • All unit tests were updated to Python 3.
  • All Python code now use a hardcoded path to the embedded Python framework. This path may change at a later time to incorporate a symlink, for easier cross-platform compatibility.

KNOWN ISSUES:

  • There are likely still edge cases in the autopkg/recipes that slipped through testing, so please file issues if you find recipes that don't work as intended.

HOW TO USE THIS BETA RELEASE:

Installing the release package will get you everything you need to run AutoPkg 2.0.

As of this release, all recipes in autopkg/recipes work directly (with commit https://github.com/autopkg/recipes/commit/25518b9b761f943d4c567fcb0944561b32ca3b89).

You will need to update your trust info as many custom processors have changed:

autopkg update-trust-info Firefox.munki
autopkg update-trust-info MakeCatalogs.munki
autopkg run -vv Firefox.munki MakeCatalogs.munki

HOW TO REPORT ISSUES:
Use the "Beta Bug report" GitHub issue template to specifically label the issue as being
beta only. Please make use of the template to convey all information possible in order
to reproduce or diagnose the issue as clearly as possible.

SETTING UP AUTOPKG MANUALLY:
If you do not want to use the AutoPkg release installer, you can manually set up an
AutoPkg 2.0 environment. Setup and place the AutoPkg files as you normally would:

  1. Create /Library/AutoPkg/
  2. Copy the contents of Code into /Library/AutoPkg/
  3. Ensure correct file modes for the autopkgserver components: sudo chmod -R 755 /Library/AutoPkg/autopkgserver/

Build a relocatable python bundle:

  1. Use the CONTRIBUTING guide's instructions on building a relocatable python bundle
    that uses the requirements.txt file for pip
  2. Move/copy the bundle into /Library/AutoPkg/Python3/Python.framework
autopkg - AutoPkg 1.3.1

Published by nmcspadden almost 5 years ago

1.3.1 (November 06, 2019)

FIXES:

autopkg - AutoPkg 1.3

Published by nmcspadden almost 5 years ago

1.3 (November 04, 2019)

FIXES:

autopkg - AutoPkg 1.2

Published by nmcspadden about 5 years ago

1.2 (September 16, 2019)

FIXES:

ADDITIONS:

IMPROVEMENTS:

autopkg - AutoPkg 1.1 Official Release

Published by gregneagle over 5 years ago

This is the official release of AutoPkg 1.1. It is the same build as 1.1 Release Candidate 2.

1.1 (May 29, 2019)

FIXES:

  • Add --compress option to curl calls in SparkleUpdateInfoProvider and URLTextSearcher
    to work around websites that return compressed results even when request does not indicate
    they will be accepted. (GH-461)
  • URLDownloader: Better handling of more HTTP 3xx redirects (GH-429)
  • Better handling of paths starting with ~/ (GH-437) (https://github.com/autopkg/autopkg/commit/603f2207df3cd88b3a2cb3e59543923648ac6522)
  • generate_processor_docs sorts the sidebar alphabetically (GH-520)

ADDITIONS:

IMPROVEMENTS:

  • Code run through several processors/formatters (flake8, isort, black, python-modernize) to
    prepare for Python 3 compatibility
autopkg - AutoPkg 1.0.4

Published by hjuutilainen over 6 years ago

1.0.4 (March 05, 2018)

FIXES:

  • All GitHub API requests are now performed using curl. This fixes TLS errors with
    GitHubReleasesInfoProvider processor and autopkg search functionality on macOS 10.12
    and earlier. (GH-408)

IMPROVEMENTS:

  • A GitHub token can now be specified in AutoPkg preferences (GITHUB_TOKEN) or in a file: ~/.autopkg_gh_token (Original PR was GH-407, code merged here as part of GH-408: https://github.com/autopkg/autopkg/commit/8e0f19b99ce24311752d1300ed408d90713e144c)
  • In parent trust info, store paths within user home as ~/some/path. When verifying
    trust info, expand ~ to current user home directory.
  • FlatPkgUnpacker, PkgPayloadUnpacker and PkgRootCreator will now create intermediary
    directories (GH-401)
  • URLDownloader and URLTextSearcher now accept curl_opts input variable to provide
    additional arguments to curl (GH-384, GH-386)
autopkg - AutoPkg 1.0.3

Published by gregneagle about 7 years ago

1.0.3 (September 22, 2017)

FIXES:

  • Better handling of bundle items in MunkiImporter (GH-352)
  • Prevent stack trace when parent recipe does not exist (GH-363)

IMPROVEMENTS:

  • DmgCreator now explicitly specifies HFS+ format when creating disk images.
    Avoids an issue where APFS images were created under High Sierra,
    potentially causing issues with machines running older macOS versions.
    (GH-357)
  • Improvements to CodeSignatureVerifier (GH-373)
    • Added strict_verification variable to control whether to pass --strict,
      --no-strict or nothing at all to codesign.
    • Added deep_verification variable to control whether to pass --deep to
      codesign or not. Deep verification was previously on by default (and
      still is) but it can now be explicitly disabled.
    • Added codesign_additional_arguments variable for specifying additional
      arguments for codesign tool.
    • Removed the .app file extension checking and no longer require the input to
      be a specific file type. Only check for .pkg, .mpkg or .xip extensions and
      pass those to pkgutil, everything else should go to codesign.
autopkg - AutoPkg 1.0.2

Published by timsutton over 7 years ago

1.0.2 (April 07, 2017)

FIXES:

  • Trust info is ignored if it is ever present in a recipe which is not an override,
    and warns if any such trust info is found (GH-334)
  • Fix a regression in PlistReader and handling disk images
  • PkgCopier can now mount a diskimage in the source path even if its extension is not
    .dmg (GH-349)

IMPROVEMENTS:

  • If an autopkg run has recipes failing due to trust verification failure, this is
    clarified in the output as the reason for the failure.
  • URLDownloader now handles file sizes in FTP server responses, avoiding repeated
    downloads from ftp:// server URLs (GH-338)
  • FileFinder can now mount DMG files given as part of its input pattern (GH-263)
  • When PkgCreator logs Invalid version, it now includes the offending version string
    in this output (GH-343)
autopkg - AutoPkg 1.0.1

Published by timsutton almost 8 years ago

1.0.1 (November 30, 2016)

FIXES:

  • Fix a crash when parsing a plaintext --recipe-list containing a single item
    (GH-323)
autopkg - AutoPkg 1.0 Official Release

Published by timsutton almost 8 years ago

This is the official release of AutoPkg 1.0. There were no changes from Release Candidate 1.

1.0.0 (November 16, 2016)

ADDITIONS:

  • New audit verb, used to output helpful information about any recipes that:
    • Are missing a CodeSignatureVerifier step
    • Use non-HTTP URLs for downloads
    • Supply their own processors and thus will run code not provided by AutoPkg itself
    • Use processors that may potentially be modifying the original software
      downloaded from the vendor
  • New verify-trust-info and update-trust-info verbs. These can be used to
    add "trust" hash information to a recipe override. If a parent recipe and/or
    its processor(s) is later updated (typically via a third-party recipe repo and
    running autopkg repo-update against this or all recipe repos), this
    trust information will be invalid and prevent the recipe from running
    until the trust information has been updated. Running verify-trust-info with
    additional verbosity will print out full diffs of upstream changes made since
    the last trust information was recorded, and update-trust-info will update
    it to match the current state of parent recipes. This behaviour can be bypassed
    using the FAIL_RECIPES_WITHOUT_TRUST_INFO AutoPkg preference. See the
    wiki article for more information.
  • New AppPkgCreator processor, a single processor replacing the several steps
    previously required for building a package from an application bundle.
  • Support for "rich" recipe lists in property list format, which can specify
    pre/post-processors and additional input variables for that specific run. See the
    Running Multpiple Recipes article for more details.

FIXES:

  • Fix SparkleUpdateInfoProvider ignoring appcast_request_headers argument since
    switching from urllib2 to curl. (GH-277)
  • Miscellaneous fixes to better handle unicode in autopkg message output.
    (GH-299)
  • Fix GitHub API error on autopkg search for a recipe name containing spaces.
    (GH-305)

IMPROVEMENTS:

  • URLDownloader now passes --fail option so that most 400-class error codes
    will result in a failed recipe run. (GH-284)
  • AutoPkg now reports an exit code of 70 if any recipe in an autopkg run
    fails. Eventually other exit codes may later be added to report on other
    specific behavior. (GH-297)
  • MunkiImporter now accepts an uninstaller_pkg_path input variable, used to
    copy Adobe uninstaller packages and set uninstaller_item_location in pkginfos.
  • MunkiImporter should now be able to detect existing Adobe CCP-built package
    items in a Munki repo as generated by makepkginfo in Munki tools version 2.8.0
    and higher.
autopkg - AutoPkg 0.6.1

Published by timsutton over 8 years ago

0.6.1 (March 18, 2016)

FIXES:

  • Fix too-restrictive 600 permissions on files downloaded by curl. This caused an issue
    where a file copied to either a local or remote Munki repo may not be readable by the
    webserver. Modes of downloaded files are now set to 644.
autopkg - AutoPkg 0.6.0

Published by timsutton over 8 years ago

0.6.0 (March 15, 2016)

CHANGES:

  • URLDownloader, URLTextSearcher and SparkleUpdateInfoProvider now all use
    the /usr/bin/curl binary for performing HTTP requests. This resolves
    several ongoing issues with Apple's Python urllib2 module and SSL.
    CURLDownloader and CURLTextSearcher processors refer internally to the same
    processors, and recipes using them can be safely switched back to the
    "standard" versions.
    An alternate cURL binary can be specified using the CURL_PATH input variable.
  • The BrewCaskInfoProvider processor is now deprecated. The Cask DSL has added
    over time logic for specifying URLs that requires the ability to actually invoke Ruby
    code, and this processor was never widely used. It will remain in AutoPkg for
    some time but will not function with all Cask files.
  • CodeSignatureVerifier: the use of expected_authority_names to verify .app
    bundles is now deprecated, and will be removed in a future AutoPkg release. Use
    requirement instead. (GH-256)

FIXES:

  • CodeSignatureVerifier: globbing is performed on all paths, rather than only
    within a disk image path. (GH-252)

IMPROVEMENTS:

  • URLDownloader: support for 'CHECK_FILESIZE_ONLY' input variable,
    which skips checks for Last-Modified and ETag headers when checking whether a
    download has changed on the server, and uses only the file size. This is useful
    for recipes that redirect to various mirrors for downloads, where these server
    header values differ, causing repeated downloads. This can be set in a recipe's
    Input section, or like any other variable it can also be altered on the CLI using
    the '--key/-k' option during any given run, for example:
    autopkg run -k CHECK_FILESIZE_ONLY=true VLC.munki
    • related issue: (GH-219)
  • CodeSignatureVerifier: support for xip archives
  • Unarchiver: support for gzip archives