Export GitHub Issues and Pull Requests to CSV and JSON from your Terminal or Node.js app.
MIT License
github-resource-converter
Convert and export GitHub Issues and Pull Requests to JSON and CSV from a Terminal or Node.js app.
grc.authenticate({token, type, key}): void
grc.getAll({owner, repo}): Promise
grc.issues.getForRepo({owner, repo}): Promise
grc.logger
grc.options
grc.pullRequests.getForRepo({owner, repo}): Promise
grc.save({data, dest}): Promise
grc.toCsv({data=[]}): Promise
github-resource-converter
(alias grc
) exports your GitHub and GitHub Enterprise repositories' Issues and Pull Requests to CSV
and JSON
file formats. It's helpful whenever you need to:
Use spreadsheets to analyze, modify, print, or summarize large amounts of data with Pivot Tables and other important financial or statistical operations
Example:
# Convert all GitHub issues to CSV: $ github-resource-converter \ --owner foo \ --repo bar # => Saved "foo-bar-issues-export.2018-03-20T02_11_04_356Z.csv".
Share data with other tools like GitLab and JIRA.
Example:
# Convert and save all GitHub Enterprise # Pull Requests as JSON (using the grc alias): $ grc \ --owner gregswindle \ --repo github-resource-converter \ --resource-type pr \ --dest './docs/reports/export.json' # Saved "docs/reports/gregswindle-github-resource-converter-pr-export.2018-03-20T02_18_33_682Z.json".
github-resource-converter
is written in JavaScript (CommonJS) forwhich must be installed prior to use. Node.js requires npm, which is used for installing dependencies. (npm installs with Node.js.)
Recommended: To avoid rate-limiting, you should and save your personal access token.
MacOS and Unix
$ mkdir -p /usr/local/etc/github-resource-center/envvars/
$ touch /usr/local/etc/github-resource-center/envvars/.env
$ echo "GITHUB_ACCESS_TOKEN="{your-personal-access-token-value}" > \
/usr/local/etc/github-resource-center/envvars/.env
Windows
> md -p C:\usr\local\etc\github-resource-center\envvars\
> touch C:\usr\local\etc\github-resource-center\envvars\.env
> echo "GITHUB_ACCESS_TOKEN="{your-personal-access-token-value}" >>
C:\usr\local\etc\github-resource-center\envvars\.env
# Install globally to execute from a Terminal/command-line
$ npm i -g github-resource-converter
# Install as a dependency within a Node.js app
$ npm i --save github-resource-converter
The following examples assume that
github-resource-converter
is installed globally and invoked from a Terminal (command-line interface)
--dest export.[csv|json]
You can convert GitHub (Enterprise) Issues and Pull Requests into two file formats:
CSV
andJSON
.
--dest
value with a .json
file extension.--resource-type, -t
Convert and export GitHub (Enterprise) by
--resource-type
:issues
,pull_requests
, or both (all
).
issues
is the default resource-type
.
prs
require a --resource-type
or -t
value of
pr
prs
pull_request
pull_requests
--resource-type all
will export all issues
and prs
into a single file.
Filtering is currently unavailable.
If you're interested in CONTRIBUTING to features like filters--e.g., only select "open" issues--we're happily accepting pull requests!
CSV
GitHub:
# GET https://api.github.com/rrepos/:owner/:repo/issues $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --dest './docs/reports/export.csv'
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/issues $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin
JSON
GitHub:
# GET https://api.github.com/repos/:owner/:repo/issues $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --dest './docs/reports/export.json'
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/issues $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin --dest ./export.json
CSV
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --resource-type pr
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin \ --resource-type pr
JSON
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --resource-type pr --dest './export.json'
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin \ --resource-type pr --dest './export.json'
CSV
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --resource-type all
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin \ --resource-type all
JSON
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls $ github-resource-converter \ --owner gregswindle \ --repo github-resource-converter \ --resource-type all --dest './export.json'
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls $ grc --base-url https://api.ecorp.com/api/v3 \ --owner evilcorp \ --repo ecoin \ --resource-type all --dest './export.json'
Errors are written to the console (stdout
) as JSON:
# Attempt to fetch issues from a repository that doesn't exist
$ grc --owner example --repo error
[2018-03-20T02:31:24.737Z] ERROR: github-resource-converter/48219 on User.router.home: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/issues/#list-issues-for-a-repository"}
HttpError: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/issues/#list-issues-for-a-repository"}
at response.text.then.message (/p/a/t/h/github-resource-converter/node_modules/@octokit/rest/lib/request/request.js:56:19)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:188:7)
The --help
flag displays all options:
$ grc --help
Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON.
Usage
$ grc [options] [info]
$ github-resource-converter [options] [info]
Options
--base-url The GitHub REST API v3 URL origin, or a
GitHub Enterprise URL origin and path-prefix.
[Default: 'https://api.github.com']
--dest, -d The CSV's destination path and file name.
[Default: './resources.csv']
--no-auto-filename Disable automatic file naming.
[Default: false]
--owner, -o The GitHub account name or organization name.
--repo, -r The name of the GitHub (or GitHub enterprise)
repository.
--resource-type, -t "issues", "prs", or "all".
[Default: 'issues']
Info
--help Show this dialog.
--version Display the installed semantic version.
Examples
$ grc --owner github --repo hub
// => Exported CSV to /path/of/cwd/issues.csv.
$ grc --owner github --repo hub -dest './reports/issues/YYYY-MM-DD.csv'
// => Exported CSV to /path/to/reports/issues/YYYY-MM-DD.csv.
$ grc --owner example --repo error
// =>
[2018-03-19T08:04:06.596Z] ERROR: github-resource-converter/00000 on localhost: Cannot destructure property `data` of 'undefined' or 'null'.
TypeError: Cannot destructure property `data` of 'undefined' or 'null'.
at paginate (/p/a/t/h/github-resource-converter/lib/base-resource-converter.js:39:16)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:188:7
Use the --version
flag to see which version you have installed:
$ github-resource-converter --version
# => 1.1.17
.
grc.authenticate({token, type, key}): void
Most GitHub API calls don't require authentication. Rules of thumb:
- If you can see the information by visiting the site without being logged in, you don't have to be authenticated to retrieve the same information through the API.
- If you want to change data, you have to be authenticated.
octokit/rest.js. (2018). GitHub. Retrieved 21 March 2018, from https://github.com/octokit/rest.js#authentication
Name | Type | Description | Notes |
---|---|---|---|
key | String | ||
token | String | ||
type | Enum |
basic , oauth , oauth-key-secret , token , and integration
|
// Token (https://github.com/settings/tokens)
grc.authenticate({
token: 'secrettoken123',
type: 'token'
})
grc.getAll({owner, repo}): Promise
Retrieve all open and closed Issues and Pull Requests from a GitHub project.
/repos/:owner/:repo/issues
/repos/:owner/:repo/pulls
grc.getAll
combines the results ofgrc.issues.getForRepo
andgrc.pullRequests.getForRepo
into a singleArray<object>
.For more information, see
grc.issues.getForRepo
andgrc.pullRequests.getForRepo
below.
grc.issues.getForRepo({owner, repo}): Promise
Retrieve all open and closed issues for a repository with this proxy method for .
/repos/:owner/:repo/issues
async/await:
const result = await grc.issues.getForRepo({ owner: 'gregswindle', repo: 'eslint-plugin-crc' })
Promise:
grc.issues .getForRepo({ owner: 'gregswindle', repo: 'eslint-plugin-crc' }) .then(result => {}) .catch(err => {})
grc.logger
A logger instance that uses the LONG
writeable
stream for output.
bunyan.INFO
is the default log LEVEL.
Logger {
domain: null,
_events: {},
_eventsCount: 0,
_maxListeners: undefined,
_level: 30,
streams:
[{
type: 'stream',
stream: BunyanFormatWritable {
},
closeOnExit: false,
level: 30,
raw: false
}],
serializers: null,
src: false,
fields: {
name: 'github-resource-converter',
hostname: 'Gregorys-MBP-2.fios-router.home',
pid: 76698
},
debug(): void,
error(): void,
fatal(): void,
info(): void,
trace(): void,
warn(): void
}
grc.options
Contains default values for api
, cli
, and meta
data.
{
"api": {
"baseUrl": "https://api.github.com",
"headers": {
"Accept":
"application/vnd.github.v3+json, application/vnd.github.symmetra-preview+json",
"user-agent": "gregswindle/github-resource-converter v1.1.17"
},
"owner": null,
"repo": null,
"requestMedia": "application/vnd.github.v3+json",
"timeout": 0
},
"cli": {
"flags": {
"baseUrl": {
"default": "https://api.github.com",
"type": "string"
},
"dest": {
"alias": "d",
"default": "./export.csv",
"type": "string"
},
"no-auto-filename": {
"default": false,
"type": "boolean"
},
"owner": {
"alias": "o",
"type": "string"
},
"repo": {
"alias": "r",
"type": "string"
},
"resource-type": {
"alias": "t",
"default": "issues",
"type": "string"
}
}
},
"meta": {
"input": [],
"flags": {
"noAutoFilename": false,
"baseUrl": "https://api.github.com",
"dest": "./export.csv",
"d": "./export.csv",
"resourceType": "issues",
"t": "issues"
},
"pkg": {
"name": "github-resource-converter",
"description":
"Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON.",
"version": "1.1.17",
"author": {
"name": "Greg Swindle",
"email": "[email protected]",
"url": "https://github.com/gregswindle"
},
"bin": {
"grc": "lib/cli.js"
},
"bugs": {
"url": "https://github.com/gregswindle/github-resource-converter/issues"
},
"commitplease": {
"style": "angular",
"types": [
"build",
"chore",
"ci",
"docs",
"feat",
"fix",
"perf",
"refactor",
"revert",
"style",
"test"
],
"scope": "\\S+.*"
},
"contributors": [],
"dependencies": {
"@octokit/rest": "15.2.6",
"bunyan": "1.8.12",
"bunyan-format": "0.2.1",
"dotenv": "5.0.1",
"fs-extra": "5.0.0",
"insight": "0.10.1",
"jsonexport": "2.0.11",
"lodash": "4.17.5",
"meow": "4.0.0"
},
"devDependencies": {
"@semantic-release/changelog": "2.0.1",
"@semantic-release/git": "4.0.1",
"@semantic-release/npm": "3.2.4",
"ajv": "6.4.0",
"ajv-keywords": "3.1.0",
"codacy-coverage": "2.1.1",
"commitplease": "3.2.0",
"coveralls": "3.0.0",
"eslint": "4.19.1",
"eslint-config-prettier": "^2.4.0",
"eslint-config-scanjs": "1.0.0-beta4",
"eslint-config-standard": "11.0.0",
"eslint-config-xo": "0.20.1",
"eslint-plugin-import": "2.10.0",
"eslint-plugin-jsdoc": "3.5.0",
"eslint-plugin-json": "1.2.0",
"eslint-plugin-no-unsafe-innerhtml": "1.0.16",
"eslint-plugin-node": "6.0.1",
"eslint-plugin-prettier": "^2.2.0",
"eslint-plugin-promise": "3.7.0",
"eslint-plugin-security": "1.4.0",
"eslint-plugin-standard": "3.0.1",
"eslint-plugin-unicorn": "4.0.2",
"eslint-plugin-xss": "0.1.9",
"fixpack": "2.3.1",
"husky": "^0.14.3",
"jest": "22.4.3",
"jest-runner-eslint": "0.4.0",
"lec": "^1.0.1",
"lint-staged": "7.0.3",
"markdown-magic": "0.1.21",
"markdown-magic-dependency-table": "1.3.2",
"markdown-magic-install-command": "1.3.1",
"markdown-magic-package-scripts": "1.2.1",
"nsp": "^3.2.1",
"prettier": "1.11.1",
"semantic-release": "15.1.5"
},
"engines": {
"node": ">=8.0.0"
},
"eslintIgnore": ["lib/__tests__/coverage/**"],
"files": ["lib"],
"homepage":
"https://github.com/gregswindle/github-resource-converter/#readme",
"jest": {
"automock": false,
"collectCoverage": true,
"coverageDirectory": "lib/__tests__/coverage",
"coverageThreshold": {
"global": {
"branches": 100,
"functions": 100,
"lines": 100,
"statements": 100
}
},
"moduleDirectories": ["node_modules", "lib"],
"testMatch": [
"<rootDir>/lib/__tests__/**/*.test.js",
"<rootDir>/lib/__tests__/*.test.js"
],
"watchman": false
},
"jest-runner-eslint": {
"cliOptions": {
"config": ".github/config/jest-runner-eslint.config.json",
"fix": true
}
},
"keywords": [
"QA",
"convert",
"converter",
"converter",
"csv",
"export",
"github",
"google sheets",
"issue",
"json",
"pr",
"pull request",
"pull-request",
"quality assurance",
"quality-assurance",
"reporting",
"reports",
"save",
"sheets",
"spreadsheets",
"testing"
],
"license": "MIT",
"lint-staged": {
"*.js": ["npm run lint:js", "git add"],
"*.json": ["npm run lint:json", "git add"],
"*.md": ["npm run lint:md", "npm run docs", "git add"]
},
"main": "lib/index.js",
"prettier": {
"semi": false,
"singleQuote": true
},
"release": {
"generateNotes": {
"preset": "angular"
},
"prepare": [
"@semantic-release/changelog",
"@semantic-release/npm",
"@semantic-release/git"
],
"verifyConditions": ["@semantic-release/npm", "@semantic-release/git"]
},
"repository": {
"type": "git",
"url":
"git+https://github.com/gregswindle/github-resource-converter.git"
},
"scripts": {
"docs": "npm run docs:toc",
"docs:toc":
"md-magic --config '.github/config/markdown.config.js' --path '**/*.md' --ignore 'node_modules'",
"lint":
"npm run lint:js && npm run lint:json && npm run lint:manifest && npm run lint:md",
"lint:js":
"node_modules/.bin/eslint -c .github/config/.eslintrc.yml --ext .js . --fix",
"lint:json":
"prettier ./**/*.json --ignore-path '.github/config/.prettierignore' --write",
"lint:manifest": "fixpack",
"lint:md": "prettier ./**/*.md -prose-wrap always --write",
"posttest:ci:coverage:codacy":
"cat ./lib/__tests__/coverage/lcov.info | codacy-coverage",
"precommit": "lint-staged",
"prepare": "npm run lint && npm run security",
"prepublish":
"lec lib/cli.js -c LF && npm run security && semantic-release -d",
"prepublishOnly": "npm run prepare",
"pretest": "npm run lint",
"security": "npm run security:nsp:scan",
"security:nsp:scan": "nsp check",
"test": "jest",
"test:config": "jest --showConfig",
"test:watch": "jest ./lib/__tests__/*.test.js --watch",
"test:watch:all": "jest ./lib/__tests__/*.test.js --watchAll"
},
"readme": "ERROR: No README data found!",
"_id": "[email protected]"
},
"help":
"\n Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON.\n\n Usage\n\n $ grc [options] [info]\n $ github-resource-converter [options] [info]\n\n Options\n --base-url The GitHub REST API v3 URL origin, or a\n GitHub Enterprise URL origin and path-prefix.\n [Default: 'https://api.github.com']\n --dest, -d The CSV's destination path and file name.\n [Default: './resources.csv']\n --no-auto-filename Don't append an ISO 8601-like timestamp to the\n output file.\n [Default: false]\n --owner, -o The GitHub account name or organization name.\n --repo, -r The name of the GitHub (or GitHub enterprise)\n repository.\n --resource-type, -t \"issues\", \"prs\", or \"all\".\n [Default: 'issues']\n\n Info\n\n --help Show this dialog.\n --version Display the installed semantic version.\n\n Examples\n\n $ grc --owner github --repo hub\n // => Exported CSV to /path/of/cwd/issues.csv.\n\n $ grc --owner github --repo hub -dest './reports/issues/YYYY-MM-DD.csv'\n // => Exported CSV to /path/to/reports/issues/YYYY-MM-DD.csv.\n\n $ grc --owner example --repo error\n // =>\n [2018-03-19T08:04:06.596Z] ERROR: github-resource-converter/00000 on localhost: Cannot destructure property `data` of 'undefined' or 'null'.\n TypeError: Cannot destructure property `data` of 'undefined' or 'null'.\n at paginate (/p/a/t/h/github-resource-converter/lib/base-resource-converter.js:39:16)\n at <anonymous>\n at process._tickCallback (internal/process/next_tick.js:188:7)\n"
}
}
grc.pullRequests.getForRepo({owner, repo}): Promise
Retrieve an array of all open and closed pull requests for a GitHub or GitHub Enterprise repository with this proxy method for .
/repos/:owner/:repo/pulls
async/await:
const grc = require('github-resource-coverter') const getAllPullRequests = async (params) = { try { const WHITESPACE = 2 const prs = await grc.pullRequests.getForRepo(params) grc.logger.info(JSON.stringify(prs, null, WHITESPACE)) } catch (err) { grc.logger.error(err) } } await getAllPullRequests({ owner: 'democracy-ia', repo: 'govinfo-link-js' })
Promises:
const grc = require('github-resource-coverter') grc.pullRequests .getForRepo({ owner: 'democracy-ia', repo: 'govinfo-link-js' }) .then(prs => { const WHITESPACE = 2 grc.logger.info(JSON.stringify(prs, null, WHITESPACE)) }) .catch(err => { grc.logger.error(err) })
grc.save({data, dest}): Promise
Export a collection of Issues or Pull Requests to your local filesystem.
async/await:
// Save as JSON const result = await grc.save({ data, dest: './export.json' })
Promise:
// Save as CSV grc .save({ data, dest: './export.csv' }) .then(result => {}) .catch((err = {}))
grc.toCsv({data=[]}): Promise
Converts (deeply) nested JSON into CSV format, returning a Promise<string>
.
async/await:
const result = await grc.toCsv(data)
Promise:
grc .toCsv(data) .then(result => {}) .catch((err = {}))
We welcome contributions with GitHub issues and pull requests.
Before embarking on a significant change, please adhere to the following guidelines:
Create an issue—e.g., a defect ("bug") report or a feature request—to propose changes.
Exceptions:
If you're working on documentation and fixing something simple like a typo or an easy bug, go ahead and make a pull request.
Follow the CONTRIBUTING guidelines.
Why:
Standards and guidelines make communication easier. If you're willing and able to program—or want to learn how— following the guidelines will increase the likelihood of adding your changes to the software product.
Why:
It's more fun when everybody's friendly and respectful.
Make a pull request when you're ready for other to review your changes (or you get stuck somewhere).
PR novices:
🙋 Never created a pull request? No problem. . (It even covers most of the conventions in the CONTRIBUTING guidelines!)
MIT © Greg Swindle.
Read the for all third-party software that github-resource-converter
uses.