release-1.13

This release updates the SSL-expiration test, to provide clarity when an SSL certificate is close to expiry the CommonName of the certificate will be reported. This allows a user to see if it is actually their certificate which is close to expiration, or any intermediary certificate. This was reported in #49, and closed in #50.

Some dockerfiles were also added to the repository, but these are primarily for the author's benefit at the moment.

release-1.12

This release updates the processing of tests in the event of a DNS-failure.

Since we're multi-protocol (supporting both IPv4 and IPv6) whenever we encounter a test we must result the names involved, to discover the target host(s) which should be probed. It is possible a user might say "Ignore IPv4", or "Ignore IPv6", so we need this step before we actually carry out the test.

For example given the following input we must lookup the IP addresses of mail.example.com (both A and AAAA records) to know what target(s) we must run our test against:

mail.example.com must run imaps on 587 with username 'bob' with password 'hello.world'

Once we have identified the IP address(es) to probe we then run the test:

• If the test-passes, because an IMAP login with TLS succeeds, we write the result to redis.
  • We censor the password from this result before inserting the result.
• If the test-fails, because the login fails, the system is unreachable, or SSL validation fails, etc, we write a failure-result to redis
  • We censor the password from this result before inserting the result.

There was one case not considered; if the initial DNS lookup fails we would record that in redis without censoring the password. That's a rare case, in practice, but it would have caused a minor leak of credentials. We've never explicitly defined a threat-model, or promised security in this case, so I think it's tolerable, but still not ideal.

This release fixes the issue anyway, which was logged as #47

release-1.11

This release updates our code by "stealing" inspiration from one of our more advanced forces, as per #46

• We now setup an explicit (5second) timeout for our redis connections.
• We now allow multiple HTTP-status-codes to be regarded as a pass.
  • For example "https://steve.fi/ must run http with status 200,301,302".
  • I don't have a particular use-case for this, but it seems like a good improvement regardless.

release-1.10

This release allows the number of members in a given redis set, or list, to be tested and an alert triggered if that number is above a particular threshold. This was implemented in #44

We also updated our release process as part of #42 to improve our CI tests, which should ensure regressions are easily found.

This release contains two changes from @cmaster11 :

• Allow running tests from STDIN
  • #39
• Allow a test to have a custome number of retries.
  • #40

Otherwise I've updated the github actions to the new syntax, although due to github oddities this did not complete successfully.

This release is being made solely to transition the testing and release process from TravisCI to github actions:

• github-action-tester
  • Used to run tests.
• github-action-publish-binaries
  • Used to make releases.

This release was made solely to publish (trusted) binary releases for the ARM platform, something I find myself using more nowadays.

There are only minor changes compared to the previous one:

• #37 - Contributed by @pbeckmann
• Minor comment updates in various files.

This release makes a couple of minor internal cleanups to fix go lint and go vet warnings.

The only new feature is the ability to apply a tag to all result-objects, which are stored in our redis-queue. Simply add -tag=$(hostname) to your worker invokation and you can determine whether a failure is local or global by comparing against other hosts. (Assuming you have N-workers all on different hosts.)

This release improves the handling of the IMAP & IMAPS probes, to ensure that once a login-test has been made a logout operation also occurs. This was a grievous omission which caused false-failures when probing remote dovecot servers.

Assuming a test every minute then after ten minutes the probe would start failing with this error message:

• Maximum number of connections from user+IP exceeded (mail_max_userip_connections=10)

Adding the logout operation, and ensuring that the connection is closed properly removes this false-positive.

This release updates the SMTP-probe, such that it now supports optional username & password arguments, for example:

 mail.example.com must run smtp with port 587 \
     with username '[email protected]' \
     with password 's3cr3t'

If a username and password are specified then a login attempt will be made to the remote SMTP-server.

NOTE: To avoid submitting login-credentials in plaintext the login attempt mandates the use of STARTTLS. If your TLS setup is self-signed, or otherwise mismatches the hostname used then errors can be ignored via with tls insecure:

 mail.example.com must run smtp with port 587 \
     with username '[email protected]' \
     with password 's3cr3t' \
     with tls insecure

This release makes a minor update to the parser to allow lines to be continued - by ending them with a trailing \ character.

Allowing line-continuation makes expressing longer tests neater, for example:

   NAMESERVERS are rachel.ns.cloudflare.com, clark.ns.cloudflare.com

   NAMESERVERS must run dns with lookup steve.fi \
       with type A with result '176.9.183.100'
   NAMESERVERS must run dns with lookup steve.fi \
       with type AAAA with result '2a01:4f8:151:6083::100'

We can also breakup HTTP tests nicely:

     https://steve.fi/Security/XSS/Tutorial/filter.cgi must run http \
           with content "test me" \
           with data "text=test%20me" \
           with username `moi` \
           with password `secret`

There are no other changes in this release.

This release features two main changes:

• Be less strict in the SSH-testing, which allows you to probe SSH servers that arent' necessarily openssh.
  • (Specifically this change was made to allow the built in SSH-server of gitbucket to be tested.)
• Collect and submit metrics about test execution.
  • Also system-metrics via golang-metrics

Finally this release is the first to include binary-versions of the bridges we supply.

Note release-1.1 was withdrawn as the .travis.yml file was broken.

This release fixes a minor issue with the timeout-handling for our protocol-tests, to ensure that we use the literal value specified by the worker:

  $ overseer worker -verbose -timeout=15s

(The default timeout for each test is 10 seconds)

The only other improvement in this release, beyond minor documentation fixes and internal-cleanup, is that the FTP protocol-handler has been improved such that it now allows you to actually retrieve a file via FTP, and validate the content returned. This is documented in the handler itself, and details can be read with:

 $ overseer examples ftp

This release continues the trend of improving the internal organization of our code, refactoring some internal details and generally improving readability and reliability.

Each protocol-handler contains an embedded sample/example section, (visible via "overseer examples [filter]"), and now the arguments listed beneath this output are shown in sorted order.

The retry-behaviour has been updated to add a small delay before retrying failing tests. This allows the smoothing to mask failures caused as services go through daily-restarts, for example.

The HTTP-probe has been updated to correctly handle testing upon non-standard ports, allowing the following test to be executed for the first time:

  http://frodo.home:32400/web/index.html must run http with content 'Plex'

• New protocol-tests:
  • finger
    • This allows content-matching too, much like the HTTP-probe.
  • telnet

This release changes the architecture of the project, by removing the need to run an MQ service. In the past there were two queuing-stages:

• Tests to be executed were added to a (redis) queue
  • This would then be emptied by the overseer worker sub-command.
• The result of executing tests were posted in a message-queue
  • From where notifiers could parse them, and let humans know of failure

Although redis and MQ are both excellent pieces of software, and both have their appeal, it seemed a little crazy to use two distinct queuing solutions! On the basis that more developers are probably familiar with redis we've chosen to use that universally.

• New protocol-tests:
  • None

If you've built your own notifier that will need to be updated to use blpop from redis, instead of an MQ-topic subscription, but the changes should be minimal.

This release improves the abilities of the http and the tcp checks, and uses MQ solely for notifications.

The HTTP check now allows you to specify more things in the checks it makes:

• The User-Agent submitted with all tests.
• The HTTP-verb used in the check, "GET", "HEAD", "POST", "PATCH", "DELETE", etc.
• In addition to looking for a literal string in the response of a submission it is now possible to test for a match of a given regular expression.

The general-purpose TCP-check allows you to match a remote greeting the host might send, via a regular expression.

The examples sections of both checks has been updated, and can be viewed with:

  $ overseer examples http
  $ overseer examples tcp

The biggest change is that notifications are now submitted via MQ. There are examples included in the source repository for listening to MQ and announcing via IRC/Purppura, but the use of MQ itself is now mandatory, and the integrated notifiers have been removed.

Using a queue allows less decoupling between the testing system, overseer, and the notification system - which might be pager-duty, pushover, or something else in-house.

• New protocol-tests:
  • None

This release improves user-friendliness by improving validation. Tests which are parsed now have their optional arguments validated by regular-expression, allowing errors to be identified before tests are executed.

The main application has a new sub-command examples which gives an example for every protocol test, and allows filtering. To see all examples:

  $ overseer examples | less

To see documentation for a single test-type:

  $ overseer examples http

The SSL expiration test has been updated to allow explicit units to be specified (both "(h)ours" and "(d)ays").

Internally our comments have been overhauled, or coding style improved, and minor changes made.

• New notifiers:
  • None
• New protocol-tests:
  • None

This release updates the API of the notifiers, to allow persistant notifier-objects. (i.e. In the past the IRC notifier would connect to the IRC network, join a channel, post a message, then disconnect for each notification. Now a single connection is made, and the messages are posted using that connection when appropriate. The same applies to our MQ-based notifier.)

Each of the protocol-probes has been updated to allow it to declare the optional-arguments it supports, which allows our parser to raise an error if an unknown option is configured/used/set.

The HTTP-probe will now raise alerts if it is testing an SSL-site and the SSL certificate is due to expire within the next two weeks - this behaviour can be disabled via the expiration option. For example to disable this:

  https://steve.fi/ must run http with expiration any

Or to change the alerting to only consider a site failed if the certficate is 5 days from expiration:

  https://steve.fi/ must run http with expiration 5

• New notifiers:
  • None
• New protocol-tests:
  • Generic TCP probe
  • VNC probe

The 0.3 release of overseer adds a new notification plugin, which communicates the results of tests via an MQ topic. This means it is easy to add your own notifications as an external process which listens to that same MQ - with no more tight coupling required.

The HTTP-tester has been updated to support basic authentication, which can be used like so:

 https://example.com/ must run http with username 'stats' with password 'secret'

The HTTP-tester has also been updated to support making HTTP POST requests, in addition to the default HTTP GETs. To use this you define the data you wish to POST:

 https://example.com/ must run http with data 'foo=bar;bar=baz'

NOTE: You should URL-encode your data.

• New notifiers:
  • An MQ-based notifier.
• New protocol-tests:
  • None.

This new release improves usability by adding the ability to specify configuration options in a (JSON) configuration-file, pointed to by the OVERSEER environmental-variable.

Signification documentation updates have been made, and the code has been refactored and cleaned up.

In terms of user-visible changes we've added:

• New notifiers:
  • An IRC notifier to announce test-failures to an IRC channel.
• New protocol-tests:
  • A protocol test for the Postgres database.