Changes:

• Configurable limits on show-* commands (#471)
• Allowing databases to increase the number of chunks (#488)
• Update installer to v3.0.5 (#489)

Changes:

• Removing Bro config section (#465)
• Flags for database deletion (#470)
• Enhance command line rolling params and allow import of files (#474)

Backend changes:

• Update test runners and update Readme (#468)
• Adding checks to ensure index keys aren't too large causing issues with MongoDB (#473)
• Adding an additional warning if no uconn data found (#476)
• Fix certificate count missing check if exists (#478)

Installer changes:

• Add Redhat Enterprise identifiers so the install can continue on RHEL. (#467)
• Updating installer for v3.0.4 (#479)

Important Notes:

• The import command's --chunk|--CC parameter previously accepted values 1 <= chunk <= numchunks (or 1 based indexing). This has been changed to 0 <= chunk < numchunks (or 0 based indexing). If you have a script that uses this parameter please update it accordingly. If you specify chunk such that it is equal to numchunk you will get an error.

Changes:

• Updated to allow disk use in all pipe queries (#460)

Changes:

• Remove DBName, ImportDirectory Config Settings (#438)
• Create Database Only After Valid Files Are Found (#442)
• Removing code for Ubuntu 14.04 (#457)
• Avoid downloading executable script to /tmp during bro install (#458)

Changes:

• Store the dns client IPs for each queried hostname (#436)
• Remove unused Logmover code (#445)
• Converted print statements to logs (#446)

Bugfixes:

• Fix Typos For Rolling Imports (#444)
• Remove Tags From Bro Log Types (Recognize Security Onion http log) (#439)

See the v3.0.0-beta1 release notes for a list of changes.

Since v3.0.0-beta1 there was a small bug fix and documentation updates for v3.

Bugfix:

• Threading issue with certificate importing (#435)

Changes:

• Significant changes to the analysis engine.
• Import and analyze are combined in a single step.
• Introduced a rolling feature that allows continually importing new data into a dataset that keeps a fixed 24 hour view.
• No longer store the original conn, dns, or http logs. This drastically reduces the size of the stored databases.
• Added ssl and x509 parsing (#369)
• Added support for ja3 hashes as a client identifier.
• Added ssl/tls certificate analysis.

Already in master

• Install ja3 module into Bro as part of the Rita installer. (#384)
• Add a --disable-rita command line option. (#392)
• Enable SSL certificate logging (#393)

Still to be done:

• Update documentation
• More testing to ensure this is stable

Changes:

• Added bro to path by default (no prompt) (#321)
• Implement default config values (#329)
• Move hard-coded connection limit to config file (#311)
• Added strobes display to command line and html reporting (#320)
• Update blacklisted analysis (#310)
• Made blacklist database configurable (#310)
• Updated analysis, reset, and delete commands (#324)
• Added NeverInclude to Filtering config section which allows for whitelisting (#328)
• Enabling NeverInclude values by default (#336)
• Change Logging directory structure (#339)
• Create config options for disabling modules (#342)
• Refuse to run import if InternalSubnets is not configured (#341)
• InternalSubnets & Upgrading Documentation (#373)
• Setting local_ Bro values based on InternalSubnets (#350)

Bugfixes:

• Prevent freqConn collection from being reset (#323)
• Added total duration field into uconns (#318)
• Fixed show databases issue (#326)

Config file changes:

• Added Enabled flags to each section to allow turning analysis modules on or off individually. All are enabled by default.
• Filtering section added to defaults.
• Filtering: NeverInclude section added and initialized to safe universal values.
• Filtering: InternalSubnets section commented out by default. ❗ IMPORTANT ❗ This config section must be filled out before RITA will process new data.

General Notes:
This release includes new aliases and flags to commands to help streamline workflow.

• reset-analysis -> reset. Added flag -f|--force to bypass prompt.
• analyze. Added flag -r|--reset to automatically perform reset without prompting followed by analyze.
• delete-database -> delete. Added -f|--force flag to bypass prompt.

This version makes significant changes to the modules that are run. It removes a couple of low-value, high-cost analysis modules, which should greatly improve performance for large datasets. In order to use this version of RITA with older datasets they will require a re-analysis (rita reset-analysis <dataset> && rita analyze <dataset>

Removed:

• Removed scans module from analysis, reporting, and config (#281)
• Removed blacklisted urls and safebrowsing analysis, reporting, and config (#279)
• Removed long urls analysis and reporting (#283)
• Removed http sanitization (#283)
• Removed IPv4 and IPv6 collections and combine into host (#294) (#285)
• Removed crossref analysis (#303)

Changes:

• Stored connection count and average bytes in beacons collection (#297) (#285)
• Stored longest duration in uconn and host collections (#298) (#285)
• Stored several new beacon/blacklist metrics in the host collection (#300) (#285)
• If connections between two hosts are over 250k, all are removed at import time (#291)
• Filter internal-to-internal and external-to-external traffic with exclusions (#301)

Bugfixes:

• Prevent rare case of MetaDatabase state causing crash (#287)

Config file changes:

• Removed Scanning section
• Removed Blacklisted: SafeBrowsing subsection
• Optional Filtering section added (but not included by default)

Known Issues:

• The show-databases command does not work in some cases (#319)
• If InternalSubnets is not configured (as is the default) RITA will filter all connections (#341)

Changes:

• Make some commands periodically check for program updates #255
• Update Mongo version to 3.6 #248
• Add TravisCI test automation #250
• Updating manual install documentation #265

Config file:

• UserConfig section added to config file. This controls how often RITA checks for updates. In older versions where it doesn't exist it will default to 14 days.

Changes:

• Activate bash tab autocomplete (#259)
• Adding error message if there's a problem with the RITA version number (#253)
• Allow Analysis While Importing Separate Data (for IPFIX ingest) (#260)

Changes:

• Install script now configures Bro, starts Bro & Mongo, and configures Bro & Mongo to start at boot #245
• Corrected several spelling errors #246
• Removed unnecessary dependencies from install.sh #242

Bug Fixes

• Resolved issue with printing ports in scan results #209

Changes

• Check for Mongo version >= 3.2 and < 3.7 #221
• Remove a feature that is incompatible with Mongo 3.7 #222
• Lower default import buffer to help with memory consumption when batch processing multiple datasets #220
• Added unit tests #214
• Switched out deprecated go-mgo/mgo package for globalsign/mgo #226
• Filter out beacons with fewer than 3 packets (e.g. prevent port scans from showing up as beacons) #231
• The installer will only install one specific version of RITA instead of getting the latest version #235

This release is mainly an update to documentation and a change to the way the installer works.

Instead of installing Go and compiling RITA from scratch, the installer will pull a precompiled binary from Github as part of the install. This reduces a lot of the complexity and avoids having to install a development environment just to use RITA.

Because of this, you no longer need to clone the entire RITA repository. You can instead download the install.sh file from this release and run it. The script will take care of everything else.

The installer will also now avoid overwriting an existing configuration file. The new file will be saved next to it as config.yaml.new so that a user can manually migrate it over if needed.

Changelog

Improved Functionality

• Better error reporting
• Better support for parsing bro logs as they are normally created
  • Now, logs in the ImportDirectory will be placed in DBRoot
  • Logs in subdirectories of the ImportDirectory will be placed in "<DBRoot>-<subdir>"

New Functionality

• New data size metrics for beaconing
• Better blacklist support through rita-bl
  • Support for custom blacklists
• Support TLS and Authentication for MongoDB

Removed Functionality

• Removed UseDates / log splitting

Configuration Updates

• Removed several configuration values for MongoDB collections (table.yaml)
• Removed the DirectoryMap in the Bro config section
• Configuration now lies in /etc/rita
• Runtime files now lie in /var/lib/rita

Installer Updates

• New installer which should handle various edge cases
  • Install to /etc/rita, /var/lib/rita, and /usr/local/bin/rita
• Support installation on CentOS 7

Documentation

• Added a documentation folder for living documentation

This beta release contains many breaking changes from previous RITA versions. This release should be feature stable for our upcoming v1.0.0 release. We've worked hard to combine all breaking changes into one release with the intention of keeping RITA more stable going forward. We highly recommend running the RITA installation on a fresh install of Ubuntu 16.04.

Why Alpha-2?

We are consistently rolling out new features, squashing bugs, and planning the future of RITA. Currently, we are rapidly iterating on the framework. Due to this rapid development, breaking changes are constantly rolling out. Once the framework settles, version 1.0.0 will be released and RITA will follow semantic versioning.

Installation

From Source

• Follow these instructions
• Before running make install, run git checkout tags/v1.0.0-alpha2

Binary

The attached binary is built for AMD64 Linux.

How to install RITA using the binary.

• Download the binary
• chmod +x rita
• mkdir ~/.rita
• Download the config.yaml file
• mv config.yaml ~/.rita
• Edit the config file according to the README
• Ensure MongoDB is running

Example Run

NAME:
   rita - Look for evil needles in big haystacks.

USAGE:
   rita [global options] command [command options] [arguments...]

VERSION:
   v1.0.0-alpha2-0-g5321fb6

COMMANDS:
     analyze                 Analyze imported databases, if no [database,d] flag is specified will attempt all
     delete-database         Delete an imported database
     import                  Import bro logs into the database
     html-report             Write analysis information to html output
     reset-analysis          Reset analysis of one or more databases
     show-beacons            Print beacon information to standard out
     show-blacklisted        Print blacklisted information to standard out
     show-databases          Print the databases currently stored
     show-exploded-dns       Print dns analysis. Exposes covert dns channels.
     show-long-connections   Print long connections and relevant information
     show-scans              Print scanning information
     show-long-urls          Print the longest urls
     show-most-visited-urls  Print the most visited urls
     show-user-agents        Print user agent information
     test-config             Check the configuration file for validity
     help, h                 Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h     show help
   --version, -v  print the version

Calling this release alpha because we still have some new features to incorporate into version1.x.x.