rita-legacy

What's Changed

• Fix install error (https://github.com/activecm/rita/issues/821) due to Zeek configuration incompatibility (https://github.com/activecm/rita/pull/820)

What's Changed

Improvements:

• Change show-long-connections to sort by total duration instead of longest duration by @Zalgo2462 in https://github.com/activecm/rita/pull/790
• Removal of connection count portion of beacon scoring and adjustment of skew by @lisaSW in https://github.com/activecm/rita/pull/792
• Duration Scoring Update by @lisaSW in https://github.com/activecm/rita/pull/793
• Update to bimodal portion of the histogram score by @lisaSW in https://github.com/activecm/rita/pull/794

Bug Fixes:

• Improve useragent aggregation runtime for datasets with many useragents by @Zalgo2462 in https://github.com/activecm/rita/pull/785
• Fix SSL and DNS log filtering by @Zalgo2462 in https://github.com/activecm/rita/pull/788
• Prevent crashing due to malformed IP addresses in Zeek logs by @lisaSW in https://github.com/activecm/rita/pull/791
• Don't filter internal -> internal DNS traffic by @Zalgo2462 in https://github.com/activecm/rita/pull/797
• Disable SNI connection analysis if SNI beacon analysis is disabled by @Zalgo2462 in https://github.com/activecm/rita/pull/798
• Only maintain one cid's worth of max scores in the host collection by @Zalgo2462 in https://github.com/activecm/rita/pull/801

Full Changelog: https://github.com/activecm/rita/compare/v4.7.0...v4.8.0

Changes:

• Improved beacon scoring algorithms by filtering out bursty connections (https://github.com/activecm/rita/pull/773, https://github.com/activecm/rita/pull/774)
• Deployed the beaconing algorithm introduced in the IP beacons module in v4.6.0 to the Web beacons module (https://github.com/activecm/rita/pull/774)
• Deployed the beaconing algorithm introduced in the IP beacons module in v4.6.0 to the Proxy beacons module (#778)
• Added filter to drop proxied traffic which is entirely on the internal network (https://github.com/activecm/rita/pull/765)
• Added rita clean command to remove RITA datasets without MetaDB entries (https://github.com/activecm/rita/pull/763, #780)
• Removed FQDN Beacons module due to poor performance (https://github.com/activecm/rita/pull/771)
• Removed per-host DNS command and control analysis due to overflowing document sizes (https://github.com/activecm/rita/pull/762)
• Added better error reporting to the install script. Removed support for Ubuntu 18 and Debian 10. (#776)

Bug Fixes:

• Stop host aggregation phase if there aren't any local hosts (https://github.com/activecm/rita/pull/761)
• Check if a max analysis subdocument has already been inserted into the target host's dat collection before updating or inserting (https://github.com/activecm/rita/pull/764)
• Fix strobes from overflooding database documents when strobing is cumulative (https://github.com/activecm/rita/pull/767)
• Ensure bulk writes don't break 16MB limit (https://github.com/activecm/rita/pull/770)

Changes:

• Add support for Ubuntu 20.04 to the installer (#732, #734)
• Write DB Updates in Bulk; Summarize Internal Hosts After Analysis; Documentation Updates (#737)
• Implement FQDN Beaconing using TLS SNI and HTTP Host (#739)
• Change host summarizer to record max total duration instead of max individual duration found in the uconn collection (#741)
• Implement new IP beacon scoring algorithm (#742, #743, #745)
• Store all connection timestamps. Do not de-duplicate connections happening in the same second (#744, #749)
• Remove MalwareDomains as a threat intel source (#746)
• Filter external to internal traffic by default (#753)

Changes:

• Add support for Debian to the installer (#718)

Changes:

• Update Docker GoLang version to 1.17 (#712 )

Bug Fixes:

• Fixed issue where import would freeze on FQDN Beacon analysis if there were no DNS records present (#700)
• Fixed issue in Proxy Beacon analysis where traffic was filtered in the case of an internal system communicating through an internal proxy server (#706)

Changes:

• Add timestamp to HTML report templates (#662)
• Use the past 24 hours of data to analyze proxy beacons rather than just the last hour (#690)
• The RITA parser has been updated with a number of performance tweaks (#654, #695)
• Gather IPs for FQDN beacon analysis using DNS lookups from the past 24 hours of data rather than just the last hour (#676, #700)
• Drop stobe limit down to 86400 (#697)
• Add option to configuration file which filters out connections from external hosts to internal hosts (#655)

Bug Fixes:

• Add unique indexes to beaconFQDN and beaconProxy collections (#689)
• Add additional indexes to host collection (#687)
• Prevented duplicate threat intel records from being created in the host collection (#683)
• Fixed a bug where threat intel records in the host collection were not being updated when using rolling imports (#683)
• Fixed a bug where the max beacon score listed in the host collection for a pair of hosts would never decrease when using rolling imports (#683)
• Fixed a bug where rare signature entries might not be added to the host collection due to a race condition (#683)
• Fixed a bug where the connection counts for each host in the host collection were under-counted when using rolling imports (#683)
• Removed unused/ broken code in max duration analysis (#683)

Changes:

• Extend Zeek TCP inactivity timeout (#660)
• Remove Need for Users to Specify Proxy Servers, Fix Filter Bugs (#665)

Dev changes:

• Clean up TODO and NOTE markers. Remove old ip index in host collection. (#622)
• Update references from Mongo 3.6 to 4.2 (#661)

Changes in v4.3.0

• Handle Processing Long Connections that Haven't Closed (#647)
• Update Mongo Version to 4.2 (#652)

Bug Fixes:

• Fixed missing </td> in report-beacons.go and report-beaconsfqdn.go (#644)
• Speed up beaconFQDN analysis (#638)

Documentation:

• Fixed typo in docker compose documentation (#650)

Changes from v4.2.1 (pre-release):

• Make --config a global option on rita command (#631)
• Add support for detecting beacons behind HTTP proxies (#632)

Bug Fixes:

• Remove invalid certificates from old chunks when using the rolling importer (#634)

Changes:

• Make --config a global option on rita command (#631)
• Add support for detecting beacons behind HTTP proxies (#632)

Bug Fixes:

• Remove invalid certificates from old chunks when using the rolling importer (#634)

Changes:

• Added TotalBytes to show-beacons and html-report (#625)
• Add Indices to Quickly Search for Hosts which Contacted BL Hosts (#627)
• Add no-browser flag to prevent html-report from auto-launching the browser (#630)

Bug Fixes:

• Remove old fqdn beacon info when rolling imports roll over (#621)

Changes:

• Beacon Detection by FQDN (#604, #615, #616, #619, #621)
  • Adds a new command show-beacons-fqdn which reports beaconing activity to groups of external IP addresses based on domain names
• Run exploded dns analysis for the set of domains queried by each host (#608, #610, #613)
  • Adds new data to the host collection for scoring an individual host
    Domain filtering and fqdn threshold hotfix #619

Changes:

• Replace reflect with type assertions in import (#586)
  • Speeds up the import.
• Update threat intel feeds (#581)
  • Reduces false positives in threat intel/blacklist results.
• Support Parsing Zeek Logs Collected By Multiple Remote Agents (e.g. Sysmon) (#591)
  • Allows integrating with Sysmon logs through espy.

This release includes breaking changes. There may be incorrect results or errors if you try to use RITA v4 to read a v3 database or vice versa.

Changes:

• Always Update Custom Blacklists (#575)
• Update installer to v3.3.1 (#579)

Changes:

• Fixed empty log handling and error messages (#555)
• Batch Files During Import To Lower RAM Usage / Break Up Importing for Datasets Larger than 2GB (#560)
• Remove error printed on every incompatible file (#563)
• Specify Output Delimiter with CLI Flag (#573)

Documentation:

• Updating usage docs to make rolling import use cases more clear. (#557)
• Escape % symbols in cron example (#570)

Development:

• Switch to Go modules (#564)

Bugfixes:

• Fixed RITA misspelling (#551)

Installer:

• Use ACM managed Bro repos; Install bro 2.5.5 for Ubuntu Xenial (#554)
• Update installer to v3.2.1 (#558)

Documentation:

• Update zeek links in install documentation (#552)

Changes:

• Add RFC1918 as default subnets (#515)
• Add support for Zeek JSON logs (#513)
• Wrap long domains in human readable exploded-dns output (#535)
• Human readable duration for show-long-connections output (#536)

Bugfixes:

• Allow html report to be created when there are no results for some modules (#527)
• Distinguish empty User Agent strings from empty JA3 hashes (#539)

Installer changes and fixes:

• Pin ja3 download commit to pre-zeek renaming (#523)
• Add identifier so we support RHEL workstation as well as RHEL server (#528)
• Support /var/log/bro/ as log location (#531)
• Prevent Installation Errors When Default Ubuntu Bro Package is Installed (#530)
• Removed unneeded workaround for Bro install on CentOS (#480)
• Don't run gen-node-cfg in noexec temp dir (#541)
• Update installer to rita 3.2.0 (#547)

Documentation:

• Gittiquete summary fix (#534)
• Updating contributing documentation to align with current workflow (#537)
• Update readme to reflect json import (#540)

Changes:

• Update installer to v3.1.1 (#518)

Bugfixes:

• Fixed maxdur to include incoming connections (#517)

Development changes:

• Fix test workflow to accept files in subdirectories (#519)

Changes:

• Force rita build even if it is up to date (#507)
• Add install.sh support for Ubuntu 18.04 (#510)
• Add --delete flag to import to allow re-import (#511)
• Revise install documentation (#502)
• Update installer to version 3.1.0 (#514)

Bugfixes:

• Invalid certificate bug fix (#506)
• Fix to keep track of max duration in hosts (#512)

Changes:

• Update Security Onion link in documentation (#494)
• Update installer to 3.0.6 (#499)

Bugfixes:

• Fix if InternalSubnets is updated (#496)

Development:

• Initial Github action workflows (#497)