site-inspector

• Rubocop fixes
• Wappalyzer
• WHOIS

• Documentation improvements (#76, props @joelcollinsdc)
• Allow Typhoeus options to be overwritable (#77, props @joelcollinsdc)
• Make google lookup case insensitive (#79)
• Better pa11y invocation (#80)
• Added Rubocop (performance and style improvements)

• Significant performance improvements (#74)
  • Prefetch endpoints in parallel when calculating the canonical endpoint
  • Redirects to other domain endpoints now return the cached endpoint object
  • Calls to the root of an endpoint now have an explicit trailing slash to facilitate caching and prevent a potential redirect depending on the domain configuration
  • resolves_to now makes a head request, and returns the redirected endpoint if the endpoint returns a non-30x response code, increasing the likelihood of a cache hit on internal redirects
  • Make checks threadsafe, parallelize checks (#70)
  • Default to HEAD requests for when we don't need the response body (#73)
• ColdFusion detection (#67)
• Ability to programmatically enable/disable checks (#62)
• Return error when domain points to localhost (#69)
• Added debug mode via DEBUG ENV var which forces Ethon and Typhoeus into verbose mode

New

• Added Accessibility (§ 508) checks, props @adelevie (#39)
• Check that endpoint properly returns 404s for missing urls (#34)
• Detect if a website is using an open source framework (#59)
• Check for humans.txt files (#33)

Improvements

• Improved framework sniffing (#57) (breaking change)
• up? now includes sites with 30x response codes
• Update user agent string (#43)
• Better cookie detection and parsing (#58)

Special thanks to @konklone and @adelevie for their contributions.

A backwards-compatible release on the 1.0.x branch that implements @konklone's improvements via https://github.com/benbalter/site-inspector/pull/24 for ease of comparison with the 2.0.x branch.

• Logic completely rewritten. Site Inspector now revolves around two concepts, a domain and an endpoint, with each domain having four potential endpoints (HTTP/HTTPS, WWW/root). Once the canonical endpoint is determined, the original tests are run against that endpoint. (props @konklone)
• Expanded HSTS and HTTPS tests (props @konklone)
• Better command-line interface
• HTTP calls are now performed in parallel
• Disk and Rails caches now supported
• Always use Google DNS
• Better handling of DNS errors
• Improved documentation
• Demo is now hosted at https://site-inspector.herokuapp.com

• Fix for running site-inspector in parallel (e.g. a Rails seed task, certain rspec conditions)
• Move load_data to a class method
• secure_cookies? now always returns a boolean
• Fix for headers erroring when no response was received

• Removed has_cookies? method (#11, props @konklone)
• Added JSON command line interface (#13)
• Fix DNSSEC detection (#16)
• Smarter header and cookie detection (#17)
• Better tests (props @konklone)

Loosen header checks for case-insensitivity, #10, props @konklone

Bug Fixes

Initial release