Mirror of https://gerrit.wikimedia.org/g/operations/software/puppet-compiler See https://www.mediawiki.org/wiki/Developer_access for contributing
OTHER License
Use the following command to test locally
$ CHANGE=731113 NODES=cloudservices1003.wikimedia.org BUILD_NUMBER=1 python3 -m puppet_compiler.cli --debug --force
Or to debug a specific host failing
$ python3 -m puppet_compiler.debug_host -c 650494 sretest1001.eqiad.wmnet
The following instructions are pretty rough but should enable on to set up an environment that allows some local hacking. however please keep in mind that these instructions don't set up or configure the puppetdb server. As such you want be able to test code which makes uses of exported resources or puppetdb queries
puppet-compiler.conf
file and place it in a valid configuration location. In order of precedence, this is:
PC_CONFIG
environment variableXDG_CONFIG_HOME
A sample configuration file, changing PATH_TO_REPO
for the full path to the root of the git repository:
base: "<PATH_TO_REPO>/.workspace/jenkins-workspace"
puppet_src: "<PATH_TO_REPO>/.workspace/catalog-differ/production"
puppet_private: "<PATH_TO_REPO>/.workspace/catalog-differ/private"
puppet_netbox: "<PATH_TO_REPO>/.workspace/catalog-differ/netbox-hiera"
http_url: "http://localhost"
puppet_var: "<PATH_TO_REPO>/.workspace/catalog-differ/puppet"
store_configs: False
The puppet compile have a few Python dependencies, which can be installed using:
$ python3 -mvenv .venv
$ pip install -r requirements.txt
This creates a new virtualenv and installes the required dependencies. Alternatively these can be installed using apt. You can also use tools like pyenv or Conda rather than virtualenv if that is your preferred way of managing Python dependencies.
You can easily run the same tests that CI is running using the script:
If you would also like to test cloud hosts then you will need access to the openstack api for both the enc and custom hiera backend. For this you will need a copy of
* /usr/local/bin/puppet-enc
* /etc/puppet-enc.yaml
and also set up a tunnle to the cloud puppetmaster (notice the different port number used to bypass nginx)
* ssh -N -L8100:localhost:8101 cloud-puppetmaster-03.cloudinfra.eqiad.wmflabs
Finaly add the following to /etc/hosts
* 127.0.0.1 puppetmaster.cloudinfra.wmflabs.org
The compilation output adds a presentation layer suitable for humans consumption. You can trigger a dummy rendering to a "tmpdir" directory (which will be deleted) using:
$ python3 -m puppet_compiler.debug_presentation -o tmpdir --force
...
Rendered files:
tmpdir/output/1911/42/srv001.example.org/index.html
tmpdir/output/1911/42/srv001.example.org/fulldiff.html
tmpdir/output/1911/42/srv001.example.org/corediff.html
The rendering comes from Jinja2 templates in ./puppet_compiler/templates
.