hctl

Ultimate tool to manage (transform, merge, remove duplicates, validate) hostlists written in rust!

BSD-3-CLAUSE License

Stars
1
Committers
1
hctl - Breaking changes, new features, dependencies security fixes Latest Release

Published by michalszmidt 11 months ago

Important dependencies updates:

Breaking Changes and new features

  • NEW: Now can read as input format additionally: dnsmasq, bind, hostperm1
  • BREAKING Because of ^ new obligatory arg was added (or in config) --pattern to specify input format of list
  • BREAKING: config file scheme unmarshall has changed, make sure you updated it, see example_config.yml on master branch

Other changes

  • BUILDS: Mac OS X: silicon is on sonoma, intel is on stable (will automatically bump from ventura when gh sets is as -latest)
  • BUILDS: FreeBSD: is on 14.0-RELEASE-p1
  • BUILDS: OpenBSD 7.4 Added! Built with rust from repo, without cargo auditable
  • NEW: mode=url now you can fetch directly one list from web without using yaml file

In Future

  • Remote config file for instance --mode config --config https://my.server/hostlists/hctl.yml will be useful if you want to modify config without logging into server
  • More input formats
  • Automatic input format recogintion (lot of work, needs help)
  • Function unification
  • In-code documentation
  • Proper error handling.
  • More unit-tests.
  • Code coverage report.
  • Automatic testing in gh actions
  • Void Linux package addition still pending
  • homebrew, only personal tap
  • crates.io

Needs feedback

  • My guess: NO: Should I care about DragonflyBSD, Solaris, Illumos, Haiku, Zircon etc. ?
  • My guess: NO: Are you using hctl as critical mission software? Should I care about backward compatibility when it comes to old config schemes?
hctl - Breaking changes, new features, dependencies security fixes

Published by michalszmidt 11 months ago

Important dependencies updates:

Breaking Changes and new features

  • NEW: Now can read as input format additionally: dnsmasq, bind, hostperm1
  • BREAKING Because of ^ new obligatory arg was added (or in config) --pattern to specify input format of list
  • BREAKING: config file scheme unmarshall has changed, make sure you updated it, see example_config.yml on master branch

Other changes

  • BUILDS: Mac OS X: silicon is on sonoma, intel is on stable (will automatically bump from ventura when gh sets is as -latest)
  • BUILDS: FreeBSD: is on 14.0-RELEASE-p1
  • BUILDS: OpenBSD 7.4 Added! Built with rust from repo, without cargo auditable
  • NEW: mode=url now you can fetch directly one list from web without using yaml file

In Future

  • Remote config file for instance --mode config --config https://my.server/hostlists/hctl.yml will be useful if you want to modify config without logging into server
  • More input formats
  • Automatic input format recogintion (lot of work, needs help)
  • Function unification
  • In-code documentation
  • Proper error handling.
  • More unit-tests.
  • Code coverage report.
  • Automatic testing in gh actions
  • Void Linux package addition still pending
  • homebrew, only personal tap
  • crates.io

Needs feedback

  • My guess: NO: Should I care about DragonflyBSD, Solaris, Illumos, Haiku, Zircon etc. ?
  • My guess: NO: Are you using hctl as critical mission software? Should I care about backward compatibility when it comes to old config schemes?
hctl - TODO

Published by michalszmidt 11 months ago

TODO

hctl - TODO

Published by michalszmidt 11 months ago

TODO

hctl - New feature: DNS validation

Published by michalszmidt about 1 year ago

Hi folks!

As promised full DNS-enabled release contains:

  • User defined resolver, see example_config.yml, controllable only in mode=config, in other modes it will use hardcoded uncensored DOT resolvers.
  • New -t --validate arg, that will just output true or false for each domain (in parallel). Works with grep.
  • Since now, intro contains UTC timestamp
  • MAINTENANCE: Added parsing + query unit-test for custom resolvers.
  • MAINTENANCE: I've set binary embedded information using cargo-auditable
  • MAINTENANCE: Changed default branch to master due to condflict with LICENSE amend rebase on other branch.
  • fixed typo vesrion -> version in serde dependency, no more warnings!

Note: that:

  • IPv6-only custom resolver may fail? I don't have one, please report.
  • LICENSE is now just BSD-3-Clause
  • IMPORTANT if you use config mode, you need to add (even empty) key resolvers: below, otherwise yaml unmarshall will fail.

News:

  • hctl has been accepted 🎉 to the alpine repo testing (edge) and in future will be moved to the community (probably at the end of the year), read aports package request for more details.
  • I've set up public home repo on SUSE OBS so you can just type opi hctl to get it on *SUSE
  • hctl compiles on every architecture that SUSE TW is available, even riscv64

Future:

  • Void linux package request pending
  • Resolver from preset (to get yaml readable).
  • Reduce blind .unwrap() in other words - proper error handling.
  • More unit-tests.
  • Code coverage report.
  • Fedora, AUR, brew, FreeBSD ports
hctl - New feature: DNS validation

Published by michalszmidt about 1 year ago

Hi folks!

As promised full DNS-enabled release contains:

  • User defined resolver, see example_config.yml, controllable only in mode=config, in other modes it will use hardcoded uncensored DOT resolvers.
  • New -t --validate arg, that will just output true or false for each domain (in parallel). Works with grep.
  • Since now, intro contains UTC timestamp
  • MAINTENANCE: Added parsing + query unit-test for custom resolvers.
  • MAINTENANCE: I've set binary embedded information using cargo-auditable
  • MAINTENANCE: Changed default branch to master due to condflict with LICENSE amend rebase on other branch.

Note: that:

  • IPv6-only custom resolver may fail? I don't have one, please report.
  • LICENSE is now just BSD-3-Clause

News:

  • hctl has been accepted 🎉 to the alpine repo testing (edge) and in future will be moved to the community (probably at the end of the year), read aports package request for more details.
  • I've set up public home repo on SUSE OBS so you can just type opi hctl to get it on *SUSE
  • hctl compiles on every architecture that SUSE TW is available, even riscv64

Future:

  • Void linux package request pending
  • Resolver from preset (to get yaml readable).
  • Reduce blind .unwrap() in other words - proper error handling.
  • More unit-tests.
  • Code coverage report.
  • Fedora, AUR, brew, FreeBSD ports
hctl - DNS validation

Published by michalszmidt about 1 year ago

Hi folks!

As promised full DNS-enabled release contains:

  • User defined resolver, see example_config.yml, controllable only in mode=config, in other modes it will use hardcoded uncensored DOT resolvers.
  • New -t --validate arg, that will just output true or false for each domain (in parallel). Works with grep.
  • Since now, intro contains UTC timestamp
  • MAINTENANCE: Added parsing + query unit-test for custom resolvers.
  • MAINTENANCE: I've set binary embedded information using cargo-auditable
  • MAINTENANCE: Changed default branch to master due to condflict with LICENSE amend rebase on other branch.

Note: that:

  • IPv6-only custom resolver may fail? I don't have one, please report.
  • LICENSE is now just BSD-3-Clause

News:

  • hctl has been accepted 🎉 to the alpine repo testing (edge) and in future will be moved to the community (probably at the end of the year), read aports package request for more details.
  • I've set up public home repo on SUSE OBS so you can just type opi hctl to get it on *SUSE
  • hctl compiles on every architecture that SUSE TW is available, even riscv64

Future:

  • Void linux package request pending
  • Resolver from preset (to get yaml readable).
  • Reduce blind .unwrap() in other words - proper error handling.
  • More unit-tests.
  • Code coverage report.
  • Fedora, AUR, brew, FreeBSD ports
hctl - BETA: new feature: dns-over-tls record validation

Published by michalszmidt about 1 year ago

Hi folks!

This is pre-release because not all features for dns-over-tls has been implemented. Stay tuned!

What actually changed:

  • this and future releases by default requires openssl, but you can get rid of it
  • small fix for parallel iterators - removed one collect. Should be faster then.
  • validate your records for ip or cname by dns-over-tls with dnseec, controlled by cli arg --dns [yes/no] default no

Note that:

  • For some reason dot queries are slow... Like ~600 records per 1.5 minute. Needs investigation
  • There are inbuilt non-logging, uncensored, privacy friendly dot servers +(cloudflare and quad9), 16 in total.
  • /etc/resolv.conf or any other system dns conf is not respected
  • Regression with DragonflyBSD builds, as trust-dns-resolver bumped msrv

Full release will contain:

  • User-defined dot servers
  • Query speedup (hopefully)
  • New option for only log-generating while dns validation, without rejecting.
hctl - BETA: new feature: dns-over-tls record validation

Published by michalszmidt about 1 year ago

Hi folks!

This is pre-release because not all features for dns-over-tls has been implemented. Stay tuned!

What actually changed:

  • this and future releases by default requires openssl, but you can get rid of it
  • small fix for parallel iterators - removed one collect. Should be faster then.
  • validate your records for ip or cname by dns-over-tls with dnseec, controlled by cli arg --dns [yes/no] default no

Note that:

  • For some reason dot queries are slow... Like ~600 records per 1.5 minute. Needs investigation
  • There are inbuilt non-logging, uncensored, privacy friendly dot servers +(cloudflare and quad9), 16 in total.
  • /etc/resolv.conf or any other system dns conf is not respected
  • Regression with DragonflyBSD builds, as trust-dns-resolver bumped msrv

Full release will contain:

  • User-defined dot servers
  • Query speedup (hopefully)
  • New option for only log-generating while dns validation, without rejecting.
hctl - BETA: new feature: dns-over-tls record validation

Published by michalszmidt about 1 year ago

hctl - New Output formats, Fix NetBSD build

Published by michalszmidt about 1 year ago

Hi folks!

Just a small release

Done:

  • Added 7 new output formats, documented on wiki
  • Fixed CI for DragonFlyBSD and NetBSD Release
  • You can watch detailed hctl download stats here on wiki

Remaining:

  • If anyone could help with OpenBSD VM issue 12 this will unblock OpenBSD releases
  • See also Solaris VM issue 34 if interested in Illumos Builds
  • If anyone could help with FreeBSD NetBSD OpenBSD rust crosscompilation, please open new issue

Deleted first release, as accidentally created tag 2.1 instead of 0.2.1

hctl - Implemented whitelists with subdomain matching

Published by michalszmidt about 1 year ago

hctl - Testing new CI config

Published by michalszmidt over 1 year ago

hctl - Moving forward to 1.0.0 but not yet

Published by michalszmidt over 1 year ago

hctl - Mostly fixes, still not there.

Published by michalszmidt over 1 year ago

hctl - Preview

Published by michalszmidt over 1 year ago

Just a preview, not tested, regex malfunctions in some rare cases.

Package Rankings
Top 19.71% on Alpine-edge
Badges
Extracted from project README
Packaging status Build Status Build Status Build Status Build Status Build Status Build Status Build Status Build Status Build Status Build Status Build Status
Related Projects