A golang-based remote protocol tester for testing sites & service availability
GPL-2.0 License
Bot releases are visible (Hide)
This release updates the SSL-expiration test, to provide clarity when an SSL certificate is close to expiry the CommonName of the certificate will be reported. This allows a user to see if it is actually their certificate which is close to expiration, or any intermediary certificate. This was reported in #49, and closed in #50.
Some dockerfiles were also added to the repository, but these are primarily for the author's benefit at the moment.
Published by skx over 4 years ago
This release updates the processing of tests in the event of a DNS-failure.
Since we're multi-protocol (supporting both IPv4 and IPv6) whenever we encounter a test we must result the names involved, to discover the target host(s) which should be probed. It is possible a user might say "Ignore IPv4", or "Ignore IPv6", so we need this step before we actually carry out the test.
For example given the following input we must lookup the IP addresses of mail.example.com
(both A
and AAAA
records) to know what target(s) we must run our test against:
mail.example.com must run imaps on 587 with username 'bob' with password 'hello.world'
Once we have identified the IP address(es) to probe we then run the test:
There was one case not considered; if the initial DNS lookup fails we would record that in redis without censoring the password. That's a rare case, in practice, but it would have caused a minor leak of credentials. We've never explicitly defined a threat-model, or promised security in this case, so I think it's tolerable, but still not ideal.
This release fixes the issue anyway, which was logged as #47
Published by skx almost 5 years ago
This release updates our code by "stealing" inspiration from one of our more advanced forces, as per #46
https://steve.fi/ must run http with status 200,301,302
".Published by skx almost 5 years ago
This release allows the number of members in a given redis set, or list, to be tested and an alert triggered if that number is above a particular threshold. This was implemented in #44
We also updated our release process as part of #42 to improve our CI tests, which should ensure regressions are easily found.
Published by skx about 5 years ago
This release contains two changes from @cmaster11 :
Otherwise I've updated the github actions to the new syntax, although due to github oddities this did not complete successfully.
Published by skx over 5 years ago
This release is being made solely to transition the testing and release process from TravisCI to github actions:
Published by skx over 5 years ago
This release was made solely to publish (trusted) binary releases for the ARM platform, something I find myself using more nowadays.
There are only minor changes compared to the previous one:
Published by skx about 6 years ago
This release makes a couple of minor internal cleanups to fix go lint
and go vet
warnings.
The only new feature is the ability to apply a tag to all result-objects, which are stored in our redis-queue. Simply add -tag=$(hostname)
to your worker invokation and you can determine whether a failure is local or global by comparing against other hosts. (Assuming you have N-workers all on different hosts.)
Published by skx over 6 years ago
This release improves the handling of the IMAP & IMAPS probes, to ensure that once a login-test has been made a logout operation also occurs. This was a grievous omission which caused false-failures when probing remote dovecot servers.
Assuming a test every minute then after ten minutes the probe would start failing with this error message:
Adding the logout operation, and ensuring that the connection is closed properly removes this false-positive.
Published by skx over 6 years ago
This release updates the SMTP-probe, such that it now supports optional username & password arguments, for example:
mail.example.com must run smtp with port 587 \
with username '[email protected]' \
with password 's3cr3t'
If a username and password are specified then a login attempt will be made to the remote SMTP-server.
NOTE: To avoid submitting login-credentials in plaintext the login attempt mandates the use of STARTTLS
. If your TLS setup is self-signed, or otherwise mismatches the hostname used then errors can be ignored via with tls insecure
:
mail.example.com must run smtp with port 587 \
with username '[email protected]' \
with password 's3cr3t' \
with tls insecure
Published by skx over 6 years ago
This release makes a minor update to the parser to allow lines to be continued - by ending them with a trailing \
character.
Allowing line-continuation makes expressing longer tests neater, for example:
NAMESERVERS are rachel.ns.cloudflare.com, clark.ns.cloudflare.com
NAMESERVERS must run dns with lookup steve.fi \
with type A with result '176.9.183.100'
NAMESERVERS must run dns with lookup steve.fi \
with type AAAA with result '2a01:4f8:151:6083::100'
We can also breakup HTTP tests nicely:
https://steve.fi/Security/XSS/Tutorial/filter.cgi must run http \
with content "test me" \
with data "text=test%20me" \
with username `moi` \
with password `secret`
There are no other changes in this release.
Published by skx over 6 years ago
This release features two main changes:
Finally this release is the first to include binary-versions of the bridges we supply.
Note release-1.1 was withdrawn as the .travis.yml file was broken.
Published by skx over 6 years ago
This release fixes a minor issue with the timeout-handling for our protocol-tests, to ensure that we use the literal value specified by the worker:
$ overseer worker -verbose -timeout=15s
(The default timeout for each test is 10 seconds)
The only other improvement in this release, beyond minor documentation fixes and internal-cleanup, is that the FTP protocol-handler has been improved such that it now allows you to actually retrieve a file via FTP, and validate the content returned. This is documented in the handler itself, and details can be read with:
$ overseer examples ftp
Published by skx over 6 years ago
This release continues the trend of improving the internal organization of our code, refactoring some internal details and generally improving readability and reliability.
Each protocol-handler contains an embedded sample/example section, (visible via "overseer examples [filter]
"), and now the arguments listed beneath this output are shown in sorted order.
The retry-behaviour has been updated to add a small delay before retrying failing tests. This allows the smoothing to mask failures caused as services go through daily-restarts, for example.
The HTTP-probe has been updated to correctly handle testing upon non-standard ports, allowing the following test to be executed for the first time:
http://frodo.home:32400/web/index.html must run http with content 'Plex'
Published by skx over 6 years ago
This release changes the architecture of the project, by removing the need to run an MQ service. In the past there were two queuing-stages:
overseer worker
sub-command.Although redis and MQ are both excellent pieces of software, and both have their appeal, it seemed a little crazy to use two distinct queuing solutions! On the basis that more developers are probably familiar with redis we've chosen to use that universally.
If you've built your own notifier that will need to be updated to use blpop
from redis, instead of an MQ-topic subscription, but the changes should be minimal.
Published by skx over 6 years ago
This release improves the abilities of the http
and the tcp
checks, and uses MQ solely for notifications.
The HTTP check now allows you to specify more things in the checks it makes:
The general-purpose TCP-check allows you to match a remote greeting the host might send, via a regular expression.
The examples sections of both checks has been updated, and can be viewed with:
$ overseer examples http
$ overseer examples tcp
The biggest change is that notifications are now submitted via MQ. There are examples included in the source repository for listening to MQ and announcing via IRC/Purppura, but the use of MQ itself is now mandatory, and the integrated notifiers have been removed.
Using a queue allows less decoupling between the testing system, overseer, and the notification system - which might be pager-duty, pushover, or something else in-house.
Published by skx over 6 years ago
This release improves user-friendliness by improving validation. Tests which are parsed now have their optional arguments validated by regular-expression, allowing errors to be identified before tests are executed.
The main application has a new sub-command examples
which gives an example for every protocol test, and allows filtering. To see all examples:
$ overseer examples | less
To see documentation for a single test-type:
$ overseer examples http
The SSL expiration test has been updated to allow explicit units to be specified (both "(h)ours" and "(d)ays").
Internally our comments have been overhauled, or coding style improved, and minor changes made.
Published by skx over 6 years ago
This release updates the API of the notifiers, to allow persistant notifier-objects. (i.e. In the past the IRC notifier would connect to the IRC network, join a channel, post a message, then disconnect for each notification. Now a single connection is made, and the messages are posted using that connection when appropriate. The same applies to our MQ-based notifier.)
Each of the protocol-probes has been updated to allow it to declare the optional-arguments it supports, which allows our parser to raise an error if an unknown option is configured/used/set.
The HTTP-probe will now raise alerts if it is testing an SSL-site and the SSL certificate is due to expire within the next two weeks - this behaviour can be disabled via the expiration
option. For example to disable this:
https://steve.fi/ must run http with expiration any
Or to change the alerting to only consider a site failed if the certficate is 5 days from expiration:
https://steve.fi/ must run http with expiration 5
Published by skx over 6 years ago
The 0.3 release of overseer adds a new notification plugin, which communicates the results of tests via an MQ topic. This means it is easy to add your own notifications as an external process which listens to that same MQ - with no more tight coupling required.
The HTTP-tester has been updated to support basic authentication, which can be used like so:
https://example.com/ must run http with username 'stats' with password 'secret'
The HTTP-tester has also been updated to support making HTTP POST requests, in addition to the default HTTP GETs. To use this you define the data you wish to POST:
https://example.com/ must run http with data 'foo=bar;bar=baz'
NOTE: You should URL-encode your data.
Published by skx over 6 years ago
This new release improves usability by adding the ability to specify configuration options in a (JSON) configuration-file, pointed to by the OVERSEER
environmental-variable.
Signification documentation updates have been made, and the code has been refactored and cleaned up.
In terms of user-visible changes we've added: