🐳 📡 Docker way of running SonarQube + any DB
LGPL-3.0 License
A complete guide to running SonarQube with any DB in Docker.
Orchestrating Docker with compose sounds easy, but there are a few pitfalls in practice. Read on to learn about the whole story, or if you just wanna run it, jump directly to Getting Started.
When running SonarQube and database containers together by compose for the first time, you may encounter errors like this:
Can not connect to database. Please check connectivity and settings (see the properties prefixed by 'sonar.jdbc.').
Its because database initialization process takes a bit longer than SonarQubes boot time, especially when there is no persistent database.
So, how to detect the readiness state of the database connection?
What failed:
depends_on
option: You can specify this option in the docker-compose.yml file to start services in dependency order, but it won't wait for the dependent service to be ready.
wait script: The wait-for-it script recommended in Docker's Controlling startup order in Compose article can be used to check the availability of the database port and wait. Unfortunately, this doesn't help either. The reason is that the port will be available right after the database container starts, but that doesnt mean the database connection is ready. Just forget about nc -v -n -z -w1 $HOST $PORT
.
HEALTHCHECK
instruction: This new feature is available for Dockerfiles since version 1.12, but not yet for docker-compose (now it is there). Usage: HEALTHCHECK [OPTIONS] CMD command
. This sounds promising, but you still have to write the command on your own, to tell Docker what to check.
Database command: How about running mysql -e "select 1"
to check the database availability? Yep - but wait a second - the SonarQube container doesn't have a mysql client installed, and we have no control over the official SonarQube docker image.
Web Server: Yet another hack - what if we set up a minimal (one-liner) web server in the MySQL container that responds with the database status? Something like while true; do echo -e "HTTP/1.1 200 OK\r\n\r\n$(db_status)" | nc -l -q 0 -p 9999; done
. Unfortunately again, netcat is usually not part of the database image.
Database logs: MySQL writes its readiness status to the logs, so maybe we could try searching there with grep 'ready for connections'
. Normally, the logs are only accessible within the MySQL container, or from the host machine, but not from the SonarQube container. Perhaps we could try to persist MySQL logs to the host directory by adding command: bash -c "mkdir -p /var/log/mysql && mysqld 2>&1 | tee /var/log/mysql/mysql.log"
and volumes: ./data/mysql:/var/log/mysql
. Then we could mount the volume to share it with the SonarQube container, so that it would be available there. But do we really want to mess with adding command
and volumes
configurations on both services' sides?
There has to be a better way
What worked:
Starting with SonarQube v7.9, it will no longer support MySQL. For more information, please visit End of Life of MySQL Support. As a result, for any paragraph below mentioning MySQL, please ignore. Unless you still wanna use MySQL with SonarQube v7.7 (or earlier version).
Make sure that you've cloned the whole project, particularly the Detector.java - for checking the readiness of database.
Pull the desired version of docker images for SonarQube and database (e.g. MySQL or PostgreSQL):
# pull SonarQube image
docker pull sonarqube[:TAG]
# pull database image
docker pull mysql[:TAG]
# or
docker pull postgres[:TAG]
Heads-up: It's NOT a good idea to directly use the latest version of a database without checking the SonarQube requirements (prerequisite for the latest SonarQube version, or docs for previous versions). For instance, SonarQube 6.3 only supports MySQL 5.6 & 5.7. And if you spin up SonarQube 6.3 with MySQL 8.0, an exception would be thrown:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server.
(Optional - MySQL & macOS only) There is a permission problem when mount a host directory in MySQL container using boot2docker
.
[ERROR] InnoDB: Operating system error number 13 in a file operation.
[ERROR] InnoDB: The error means mysqld does not have the access rights to the directory.
Solution:
Build a custom MySQL image for macOS (don't forget to change the latest
tag in mysql_mac/Dockerfile
):
docker build -t mysql_mac[:TAG] [PATH_OF_THIS_REPO_ON_YOUR_DISK]/mysql_mac/
Edit docker-compose-mysql.yml, replace image: mysql
by image: mysql_mac
.
In order to persist data, you need to setup mounting data volumes: replace two mounting points under volumes in docker-compose-<db>.yml file.
- [PATH_TO_PERSIST_SONAR_DATA_ON_HOST]:/opt/sonarqube/extensions
- [PATH_TO_PERSIST_DATABASE_DATA_ON_HOST]:[DATABASE_VOLUMES]
Note: the path to persist data on host could be a relative path, e.g.: ./data/xyz
Instead of using default empty tag or dynamic latest
tag, please alter them in Dockerfile
or docker-compose
file with more specific tags. Because latest
can lead to unpredictable and unrepeatable image builds.
docker-compose -f [PATH_OF_THIS_REPO_ON_YOUR_DISK]/docker-compose-<db>.yml up
SonarQube's plugins.
/opt/sonarqube/extensions
All historical analysis data, imported rules, changed settings are saved in database.
/var/lib/mysql
# or
/var/lib/postgresql
Don't persist ElasticSearch indices (which is located at /opt/sonarqube/data/es
), let it rebuild by itself, otherwise it could cause problem during upgrading. And any ungraceful shutdown (such as crash) of SonarQube could lead to out-of-sync indices.
Always keep a backed up database in case upgrade fails and roll back is needed.
Perform a logical backup on the old version of MySQL
mysqldump -u sonar -p --opt sonar > [PATH_TO_MYSQL_BACKUP]/sonar.sql
Start a MySQL docker container (new version of MySQL)
docker run -i -t -v [PATH_TO_MYSQL_BACKUP]:/tmp -v [PATH_TO_PERSIST_DB]:/var/lib/mysql mysql /bin/bash
Start MySQL server
/etc/init.d/mysql start
Start MySQL client
mysql
Create and use the database
create database sonar;
use sonar;
Grant privileges to user
grant all on sonar.* to 'sonar'@'%' identified by 'sonar';
grant all on sonar.* to 'sonar'@'localhost' identified by 'sonar';
grant usage on *.* to sonar@localhost identified by 'sonar';
grant all privileges on sonar.* to sonar@localhost;
Restore the backup file (by executing SQL script)
source /tmp/sonar.sql
Quit MySQL client
exit
Stop MySQL server
/etc/init.d/mysql stop
Now you have successfully restored the database on the new version of MySQL. The database data are stored in path_to_persist_db of your host.
SonarQube Server upgrade process is automated, you have nothing to manually change in the SonarQube Database.
Migration path: [YOUR_VERSION] -> LTS (if exists) -> [EXPECTED_VERSION]
Upgrading guide by SonarQube (just for reference, please follow below steps.)
Don't try to stop the SonarQube server, if you kill the process, the SonarQube container exits immediately. So you can't really upgrade SonarQube by hand within its container. Don't worry, just try below steps.
Steps:
docker-compose -f [PATH_OF_THIS_REPO_ON_YOUR_DISK]/docker-compose-<db>.yml up
;For big SonarQube upgrading, it also requires database upgrading, but this happens automatically.
After the new SonarQube container is up, open its web page, you'll be redirected to a maintenance page;
sonarqube_1 | 2099.12.31 12:00:00 WARN web[o.s.s.p.DatabaseServerCompatibility] Database must be upgraded. Please backup database and browse /setup
sonarqube_1 | 2099.12.31 12:00:00 INFO web[o.s.s.p.Platform] DB needs migration, entering safe mode
Click the Upgrade button.
The database upgrade can take several minutes. When the DB migration ends successfully, the page will display "Database is up-to-date", then redirect you to home page.
Copyright (c) 2016-2019 Jing Li. SonarOnDocker is released under the GNU Lesser General Public License, Version 3.0. See the LICENSE file for details.