github-archive

• Expand paths for user-supplied --location strings. This now allows for spaces in paths and proper expansion of home directories (eg: ~)

Fixes PyGithub pinned version to ensure Auth is available

• Allows the tool to be run without passing any authentication flags (previously, to use unauthenticated, you'd have to at least pass the --https flag)
• Removes constraint that required --token and --https to be mutually exclusive (you can now authenticate with other tools such as Git Credential Manager instead of only SSH)

• Drops support for Python 3.7
• Updates dependencies

• Fixes a syntax error that led to not being able to remove the authenticated user from the user's list so their git assets weren't included twice (fixes #55)

• Overhauls subprocess error handling so output is no longer clobbered
• No longer inserts newlines (which were often formatted incorrectly) when a log entry was for a new section or action

• Fixes a bug that tries removing the authenticated user from a list so we don't double dip git operations even when there is no authenticated user (eg: using the --https flag)
• Fixes a bug that allowed users to use both the --token and --https when it should only be one or the other

• Adds a new --fork CLI arg which adds support to fork the repos or gists specified via users, orgs, stars, or gists
• Removed various shorthand CLI args to avoid confusion and improved help message output for CLI args (updated docs as well)
• Namespaces for various functions changed for better project organization. As this project is intended to be used as a CLI tool and not a library, the impact should be minimal

• Clarifies default location of github-archive in help output (closes #48)

• Switches from the threading package to the concurrent.futures package allowing us to return values from each individual thread (in this case, the names of failed git assets)
  • Fixes a long-standing bug where git repos that failed to clone would remain in the archive on Windows. This meant that recloning was impossible due to the repo existing and pulling was not possible because the repo wasn't yet fully initialized. The tool now has a proper cleanup step after everything has run that is compatible with both Unix and Windows environments (closes #38)
• Adds --log_level to allow the user to specify a custom log level
• Small adjustment to include and exclude help verbage used to be more clear they are optional filters

• Adds an --include and --exclude CLI flag that accepts a comma-separated list of repo names to either include or exclude. If neither are passed, no filtering will occur (closes #43)

• Adds the ability to specify a custom base_url for GitHub (useful for enterprise GitHub users with a custom hostname, closes #41)

• Adds mypy and fixes typing errors

• Bumps woodchips to use the new implementation (also fixes a bug where we were creating a new woodchips.Logger class each time we called the logger instead of reusing the same logger instance)
• Added Python type hinting

• Uses woodchips for logging and removes internal logging logic from the package
• Refactors git commands to not change directories but instead run commands with the -C flag to invoke it in the directory we want

• Fixes a bug that wouldn't allow for gist cloning/pulling because of a bad "forks" check on a gist GitHub object
• Adds a missing check to ensure that at least one CLI argument was passed (closes #40)
• No longer invoke a shell while using the subprocess module. Git operations should now be more stable across operating systems

• Adds a new --https flag which will authenticate via HTTPS instead of the default SSH
• Fixes a bug where using the --stars flag would not properly run due to a missing parameter. This parameter wasn't actually being used anymore and has been removed. Tests were beefed up for this function to protect against this happening again

Breaking Changes

• Reworks the entire app config to use CLI flags solely instead of a mix of CLI flags and env variables, additionally, most flags have changed names and functionality. See the README for new usage instructions or run github-archive --help (closes #30)
• Repos or gists that fail to clone or pull will now be completely removed so that they can be retried from scratch on the next run of the tool. This was an especially important change for bad clones as the tool would previously leave an empty initialized git folder even if the clone failed which would not possess the actual git repo yet. In this state, you could not properly pull new changes because the content of the repo hadn't properly been cloned yet
• Bumps required Python version from 3.6 to 3.7

Features

• Adds a new --users flag which can be used to clone or pull git assets for a list of comma separated users (closes #20)
• Adds a new --threads flag which can specify the number of concurrent threads to run at once, default is 10 (closes #22)
• Adds a new --view flag which allows you to "dry run" the application, seeing the entire list of repos and gists based on the input provided (closes #25)
• Adds a new --stars flag which you can pass a comma separated list of users to and GitHub Archive will retrieve all of their starred repos which you can then view, clone, or pull (closes #26)
• Adds a new --forks flag which will include forks for whatever lists and operations you provide, default is False (closes #17))

Fixes

• Removed verbose logging of skipped actions and "Already up to date" messages. Added additional logging related to API calls
• Added proper validation and type checking of variables and environment on startup
• Various code refactors, bug fixes, and optimizations
• Bumped the default git operation timeout from 180 seconds to 300 seconds to assist with cloning or pulling larger repos (closes #22)
• Removes mock library in favor of builtin unittest.mock library

• Removed branch flag and functionality as it was causing issues and inconsistencies when cloning/pulling and branches didn't match up. This became especially prevelant when repos started changing from master to main

• Changed all classmethods to staticmethods
• Corrected a bug where org names may not have had whitespace stripped properly
• CLI arguments now have an explicit default of False, this shouldn't change behavior from previous versions
• CLI default argument for branch has been changed from master to None and is handled via logic now. If no branch is specified, the default repo branch will be used instead of blindly assuming that master is the default branch (closes #18)
• Revamped the entire test suite to use conftest, simplified boilerplate, etc