E2E tests suite for the Newspack project
GPL-2.0 License
node
, npm
, docker
, docker-compose
installed.
scripts/.secrets
file, based on secrets/.secrets-sample
newspack-e2e.com
to hosts file (add a line with 127.0.0.1 newspack-e2e.com
)$ docker-compose up --build
$ npm run wp:setup
$ npm run test:ci
$ npm run test
Run $ npm run wp:reset
to reset the WP DB and re-run setup script.
For a real hard reset, stop the docker-compose
process, remove containers (docker-compose down --volumes
), and repeat the initial setup form above.
./wordpress/wp-content/plugins/newspack-plugin
$ rsync -a --exclude-from='.distignore' . /path/to/newspack-e2e-tests/wordpress/wp-content/plugins/newspack-plugin
Pro tip: use chokidar to sync local repository to the Docker machine, by running this in the plugin folder:
$ chokidar "." -c "rsync -a --exclude-from='.distignore' . /path/to/newspack-e2e-tests/wordpress/wp-content/plugins/newspack-plugin"
There are three "testing channels" available, meaning three sources for the Newspack plugins:
master
branches channel – versions built from master
branchesalpha
branches channel – versions currently distributed on Github as pre-releasesThe CI for this project should be configured to run tests periodically – after release days for the stable & alpha
channels, and daily for master
branches channel.
The stable channel is the default. To use a different channel while developing tests locally, add TEST_CHANNEL
variable in scripts/.env
.
Not ready yet for running on CI. Due to font differences the results fail.
After adding a new visual regression test, run $ npm run test:visual:setup
to create the base images.
To check for visual regressions, run $ npm run test:visual:check
.
The images are stored in cypress/snapshots/base
. After a test run, the image diffs will be stored in cypress/snapshots/diff
- in case of tests failing on visual check, inspect those for the regressions.
The visual regression testing is set up to run in Electron browser in order to achieve same results locally and on CI (where Electron is the only available option).
Note that Cypress UI will disappear momentarily when taking a screenshot.
artifacts/video
docker exec -it wordpress_local /bin/bash
- the WP CLI will be installed.
wp
commands have to be ran with --allow-root
optionhttps://newspack-e2e.com
If SLACK_CHANNEL_ID
and SLACK_AUTH_TOKEN
environment variables are set on the CI environment, the selected Slack channel will be updated whenever a test suite fails.