xelogstash

This includes Darwin and Linux support. The code is in the "linux" branch. The ZIP file holds all three executables. I'll see about better archives for each OS next.

The Windows support is good but Darwin and Linux are experimental.

• Bug fixes

• Supports adding filters (see Filters in the Application Settings)
• Parse Error Log events into individual fields
• Configuration files are split into two. Sources can now be split out into a separate file named sqlxewriter_sources.toml. NOTE: This requires that configuration files are named sqlxewriter.toml and sqlxewriter_sources.toml.

• Coalesce multiple file change events into a single event. This behaves much better if you're editing the file while it's running. I still don't suggest this though.
• Better handle errors where logstash stops responding

• Hot reload the TOML files on file save
• Bug fixes

1. sqlxewriter.exe runs as a service
2. Added metrics page to see a count of events read and written
3. Log memory use every 24 hours
4. Added retry logic for sinks
5. Logging to a JSON file suitable for Filebeat

• Convert to JSON for application logging
• Removed the applog section and functionality. Use FileBeat to write the JSON application log into ELK.
• FileSink now write to a single shared file for all targets that is rolled hourly. This is designed to be processed by FileBeat.

• Reduced the default worker count to the core count
• Better logging of the application version inside the app
• Added two sample extended event sessions

• Added sample extended event sessions and cleaned up sample TOML files
• Cleaned up logging
• Improved reliability of sending to logstash using TCP

Include support for sp_server_diagnostics_component_result

The beta release has a number of significant changes:

• BREAKING: The TOML file format has changed. Please review the README.
• Adds support for writing events to files so they can be processed by FileBeat
• Improves support for writing directly to Elastic Search
• Adds an mssql_product_level field with the full build number.

• Better logging
• Remove lock file later in program
• Explicit shutdown of HTTP server
• Fix TCP sends to avoid too many outbound connections. This causes issues when a session gets behind and it fails with a "bind" error when trying to catch up.

• Includes early support for writing directly to Elastic
• Added a look_back field to only process the most recent events
• Improved build versioning

This is mostly extra logging in bad situations and error handling to avoid those situation.

Added lock file to prevent the application from running twice. The lock file is based on the TOML file name.

• Added an exposed metric for the total number of events read even if they aren't processed. This helps to check if the application is making progress.
• Added an option to exclude SQL Server 17830 errors

• Added per session counts to the exposed metrics
• Added NOLOCK to the job history query
• Added a sequence_value to the application log. This is useful for sorting the application log messages.
• Switched query_hash to a string as it was overflowing
• Only lower-case the substitutions (ex. $(EXE)) rather than all adds, copies, moves so we can better pass tokens.
• Added utilities to testing sending a JSON file and test sending a small JSON snippet
• Added options to start and stop event processing at specific times. This is mostly useful for testing.

• Added more debug level logging
• blocked process event is now a warning
• Added flag to ignore sessions for a source so we can just grab agent jobs for SQL Server 2008 servers
• Add option to expose a web interface while running to see internal metrics
• The default config file name is now based on the name of the executable

• Non-existent sessions no longer issue a warning
• Handle sessions that aren't stored in the SQL Server LOG directory
• Add a command line /log parameter to write to a local log file
• Improved error handling and logging
• Moved the "status" files to the "xestate" directory. PLEASE test this carefully.
• Added a /debug flag that writes additional information.
• Increased the default workers from the number of cores to 4 * the number of cores.

• Agent jobs have more granularity in how you import them. Instead of true/false, you can now import "all", "none", or "failed". If you choose all, they now import progress reports too.
• The logic to determine error vs. info for Agent jobs is improved.
• Better error handling if you create a source without an "fqdn".
• Better error handling if you enter a non-existent SQL Server.