spamscope

Highlights

• Upgraded requirements: new version of mail-parser.
• Fixed bugs: now more stable.
• CLI tool: added cli command to get payload from Elastsearch.
• Reformat VirusTotal report.
• Added Travis CI.

Highlights

• Upgraded requirements: new versions tika-app, and mail-parser.
• New cli commands: new cli commands for SpamScope management spamscope-elasticsearch and spamscope-topology.

Highlights

• Upgraded requirements: new versions of Thug, Streamparse, and mail-parser.
• Code cleaning: code more clean and easy to manage.
• Fixed many bugs
• Performance improvement: new versions of key functions. New class to manage the mail attachments. Use of @lru_cache for heavy functions. SpamScope uses functions where there is no need of internal state.
• First step to Python 3: SpamScope doesn't work with Python 3, but the core does work.

Highlights

• Thug processing: it's possible to analyze the attachments with Thug.
• Sender IP address: there is the sender IP address in output results.
• Redis output: updated Redis output and added new topology for Redis.

⚠️ Breaking Changes ⚠️

• From version v1.3rc4 you can set main configuration file path with environment variable SPAMSCOPE_CONF_FILE or use default /etc/spamscope/spamscope.yml.
• New fields in output-redis to split results in mails (queue_mails) and attachments (queue_attachments)

⚠️ New configuration sections ⚠️

• Added thug section in attachments bolt.

⚠️ Breaking Changes ⚠️

• From version v1.3rc4 you can set main configuration file path with environment variable SPAMSCOPE_CONF_FILE or use default /etc/spamscope/spamscope.yml.

Highlights

• Thug processing: it's possible to analyze the attachments with Thug.
• Sender IP address: there is the sender IP address in output results.

⚠️ New configuration sections ⚠️

• Added thug section in attachments bolt.

Highlights

• Migration to Elasticsearch 5.0.0
• Added custom Elasticsearch template to improve indexing and search
• Fixed bug in files_mails
• Others minors change

Highlights

• Added filter function. Now it's possible to filter mails/attachments already analyzed, without using any database, but only the RAM. More performance.

⚠️ Breaking Changes ⚠️

• tokenizer configuration section: added tokenizer section for enable filter function.
• tika configuration: replaced content_types_details in valid_content_types.
• new output elasticsearch configuration: splitted mails and attachments in two different indexes.

Highlights

- Apache Storm 1.0.2
- Streamparse 3.1.1

⚠️ Breaking Changes ⚠️

- Topologies are now specified via a Python Topology DSL instead of the Clojure Topology DSL.
- An unique configuration file `spamscope.conf` for both: spouts and bolts.
- Submit configuration file from command line: `-o "spamscope_conf=/etc/spamscope/spamscope.yml`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    

Implemented enhancements:

• Parser mail is now an external library (https://github.com/SpamScope/mail-parser).
• Getting RFC malformed parts of mail.

Fixed bugs:

• Minor bugfix.

Implemented enhancements:

• New output bolt for Elasticsearch (Issue #2).
• Added analisys_date field in mail analysis.

Fixed bugs:

• Reformat urls output to avoid key fields with dot (.). These fields are not accept from Elasticsearch.