View change log.

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.15.0/zed-v1.15.0.linux-amd64.tar.gz
tar xzvf zed-v1.15.0.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• Improve the error message when reading NDJSON data with an incomplete last line (#5055)
• Improve the error message when math operators are applied to complex types (e.g., attempting to concatenate arrays with +) (#5059)
• Improve the error message when zq is called with a single argument that's neither a file nor a valid Zed query (#5053)
• Increase the buffer size for line input format to 25 MB (#5045, #5048)
• The body of a user-defined operator is now a scope that can include locally-defined statements (#5074)
• The nameof() function now supports type values (#5081)
• Fix an issue where yield inside a switch inside an over caused a panic (#5058)
• Fix an issue where accessing a variable inside a switch inside an over inside a switch caused a panic (#5080)
• Fix an issue where using over in an expression context caused a panic (#5079)
• Fix an issue where math and join matches involving float16 and float32 types could yield incorrect results (#5086)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.14.0/zed-v1.14.0.linux-amd64.tar.gz
tar xzvf zed-v1.14.0.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• Add the -manage flag to zed serve to have the Zed service process initiate maintenance tasks on a regular interval (#5017)
• Fix an issue where the Python client would not allow loading to a pool with / in its name (#5020)
• Fix an issue where pools with KSUID-like names could not be accessed by name (#5019)
• Fix a reference counting issue that could cause a Zed service panic (#5029, #5030)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.13.0/zed-v1.13.0.linux-amd64.tar.gz
tar xzvf zed-v1.13.0.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• Improve the error message when zed manage -monitor is attempted on a local lake (#4979)
• The zed serve log now includes version, storage root, and auth info at startup (#4988)
• Add docs for the zed manage command to compact data for improved performance (#4961)
• Add the ability to cast to Zed's type type (#4980, #4985)
• Add the ability to yield a Zed error literal (#4998)
• Fix an issue with accessing values inside complex literals (#4953)
• Fix an issue where cast of an empty string to a duration value incorrectly yielded 0s (#4965)
• Fix an issue where a zed vacuum on a large amount of data could crash the Zed service (#4974)
• Fix an issue where some IPv6 values of Zed's net type were not parsed correctly in Zed queries (#4992)
• Fix an issue where output of certain union-typed values was not consistent (#4995)
• Fix an issue where parsing of type literals inside of type literals was incorrectly permitted (#4996)
• Fix an issue where the grok() function would panic when encountering optional fields (#5010)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.12.0/zed-v1.12.0.linux-amd64.tar.gz
tar xzvf zed-v1.12.0.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• Zed CSV output now handles records with the same field names but different types (#4889)
• The grep() function now accepts variables as pattern arguments (#4887)
• Prevent the creation of multi-key pools in Zed lakes (support for this will be added later) (#4903)
• Zed CLI help can now be invoked via --help and is printed to stdout instead of stderr (#4907)
• Symbols (e.g., const and type) may no longer be redefined within the same scope (#4930)
• Set types can now be sliced (#4939)
• Canceled client requests to the Zed lake API are now logged at info level without stack traces (#4947)
• Add support for TSV input/output (useful for spreadsheet data) (#4891, #4913, #4912)
• Add a grok() function for parsing text lines (such as logs) into records (#4827)
• Fix an issue where switch operators could not be chained back-to-back in a Zed pipeline (#4926)
• Fix a panic that occurred if the from operator was attempted in Zed run with zq (#4936)
• Fix an "invalid union tag" panic that could occur when using the collect() aggregate function (#4932)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.11.1/zed-v1.11.1.linux-amd64.tar.gz
tar xzvf zed-v1.11.1.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• Fix an issue where loading and querying certain data caused a panic (#4877)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.11.0/zed-v1.11.0.linux-amd64.tar.gz
tar xzvf zed-v1.11.0.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• Introduce new logic for the zed CLI command to locate the lake (#4758, #4787, #4811)
• Cast expressions now behave more like function calls (#4805)
• Reduce the amount of memory needed to store a Zed value (#4812)
• Add support for unicode in keywords and identifiers (#4799, #4796)
• Add support for dynamic field paths in "left-hand side" contexts with put, cut, and rename operators (#4795, #4802, #4832)
• Add a map() function that applies a function to each element of an array or set. To avoid a naming clash, the aggregate function previously called map() is now called collect_map() (#4806, #4856, #4860)
• Fix an issue where a record expression with duplicate fields caused a panic (#4789)
• Fix an issue where valid ZNG input might not be read successfully via auto-detection (#4790)
• Fix an issue where null values could cause join to produce incorrect output (#4801)
• Fix a zq issue where a call to a user-defined operator included via -I could fail (#4808)
• Fix an issue where running zed commands on a system with many CPU cores caused an internal error (#4826)
• Fix an encoding issue that caused different map values to be rendered the same in Zui (#4853)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.10.0/zed-v1.10.0.linux-amd64.tar.gz
tar xzvf zed-v1.10.0.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• Sorting is now performed automatically on join inputs when needed (explicit sort no longer required) (#4770)
• Various query performance improvements (#4736, #4737, #4739, #4740, #4783, #4785)
• join now works correctly when data inputs are sorted in descending order (#4767)
• Reduce memory consumption during delete -where operations (#4734)
• Fix a null-handling issue that caused incorrect query results after pool compaction (#4735, #4753)
• Allow writing of vectors when compacting objects in a pool (#4756, #4757)
• Ensure query runtime errors are logged and made available through a new Query Status lake API endpoint (#4763, #4765, #4766, #4769)
• Add an example to the where docs showing inverse containment logic (#4761)
• Add an example to the cut docs that includes setting a default value for a field (#4773, #4776)
• Boolean not and ! now both work the same in expressions and search expressions (#4768)
• The zed command now returns a hint mentioning init if no lake exists at the expected path (#4786)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.9.0/zed-v1.9.0.linux-amd64.tar.gz
tar xzvf zed-v1.9.0.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• The Zed Language Overview docs have been split into multiple sections (#4576)
• Add support for user-defined operators (#4417, #4635, #4646, #4644, #4663, #4674, #4698, #4702, #4716)
• Add experimental support to the get operator for customized methods, headers, and body (#4572)
• Allow float decorators on integers in ZSON (#4654)
• The shaping docs have been expanded with a new section on error handling (#4686)
• zq no longer attaches positional command line file inputs directly to join inputs (use file within a Zed program instead) (#4689)
• Zeek-related docs have been moved to the Integrations area of the Zed docs site (#4694, #4696)
• zed create now has a -use flag to set the newly-created pool as the default pool for future operations (#4656)
• Fix an issue where the Zed Python client was incorrectly returning False for all bool values (#4706)
• Fix an issue where the != operator was not returning correct results when comparing certain types (#4704)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.8.1/zed-v1.8.1.linux-amd64.tar.gz
tar xzvf zed-v1.8.1.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• Send an HTTP 400 response instead of HTTP 500 for attempted deletes that find nothing to delete (#4618)
• Send an HTTP 400 response instead of HTTP 500 for queries that parse ok but fail to compile, such as searches lacking a leading from (#4620)
• Fix an issue where some regexps were not matching pool names when used in a from (#4619)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.8.0/zed-v1.8.0.linux-amd64.tar.gz
tar xzvf zed-v1.8.0.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• Improve sort performance for duration and time types (#4469)
• Improve performance and reduce memory used by zed load and sort on multi-GB inputs (#4476, #4484)
• Fix an issue where meta-queries were incorrectly returning results (#4474)
• The join operator now has an additional syntax that uses subqueries, which is more reminiscent of SQL (#4467, #4473, #4492, #4502)
• Improve performance when a Zed lake scan is not order sensitive (#4526)
• The lake API documentation now includes both request & response MIME types (#4512)
• Add more usage examples for the pass operator in documentation (#4541)
• cast() errors are now structured and contain more detail (#4548)
• A different Parquet library is now used, which allows for reading more dialects (#4547)
• Fix an issue where reading certain ZNG files via auto-detect caused a panic (#4569)
• Add a load operator that can be invoked in a Zed pipeline to commit data to a pool (#4471)
• Fix an issue where certain ZNG files could not be read and caused a control error (#4579)
• Fix an issue where zed serve would exit if it tried to write to a closed socket (#4587)
• Improve JSON output for Zed maps (#4589)
• Add the zed vacuum command (#4577, #4598, #4600)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.7.0/zed-v1.7.0.linux-amd64.tar.gz
tar xzvf zed-v1.7.0.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• Add regexp_replace() function for replacing regular expression matches in a string (#4435, #4449)
• Add documentation showing how to configure Auth0 for authenticated access to a Zed lake service (#4439)
• Fix an issue where elements of map could not be accessed if the key was of a union type (#4447)
• Allow head operator to accept an expression (#4451)
• Allow tail operator to accept an expression (#4464)

Note: Changes to the Zed lake storage format

This release includes a change to the Zed lake storage format that is not backward compatible. To migrate your pools to the new format there is a Zed lake migration kit and specific guidance for users of the Zed CLI tools.

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.6.0/zed-v1.6.0.linux-amd64.tar.gz
tar xzvf zed-v1.6.0.linux-amd64.tar.gz

If you've installed the Zui desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Zui docs article for their location.

Release Notes

• Zed lake storage format is now at version 3 (#4386, #4415)
• Allow loading and responses in VNG format over the lake API (#4345)
• Fix an issue where record spread expressions could cause a crash (#4359)
• Fix an issue where the Zed service /version endpoint returned "unknown" if it had been built via go install (#4371)
• Branch-level meta-queries on the main branch no longer require an explicit @main reference (#4377, #4394)
• Add -defaultfmt flag to zed serve to specify the lake API's default response format (#4379, #4396)
• Zed queries now appear in the lake log when zed serve is run at -log.level debug (#4385)
• Fix an issue where elements of complex named types could not be accessed (#4391)
• Add docs for the pass operator and an example of join on more than two inputs (#4403)
• When the result of summarize is a single value and there is no field name specified, the output is now that single value rather than a single-field record containing that value (#4420)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.5.0/zed-v1.5.0.linux-amd64.tar.gz
tar xzvf zed-v1.5.0.linux-amd64.tar.gz

If you've installed the Brim desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Brim wiki article for their location.

Release Notes

• Add float16 primitive type (#4301)
• Add segment compression to the VNG format (#4299)
• Add -unbuffered flag to zed and zq (#4320)
• Add -csv.delim flag to zed and zq for reading CSV with non-comma delimiter (#4325)
• Add csv.delim query parameter to lake API for reading CSV with non-comma delimiter (#4333)
• Add -cors.origin flag to zed serve, which defaults to a wildcard (#4334)
• Use zng. prefix for ZNG-related flags in zed and zq (#4336)
• Fix an issue where type decorators of union values were leaking into CSV output (#4338)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.4.0/zed-v1.4.0.linux-amd64.tar.gz
tar xzvf zed-v1.4.0.linux-amd64.tar.gz

If you've installed the Brim desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Brim wiki article for their location.

Release Notes

• The ZST format is now called VNG (#4256)
• Allow loading of "line" format over the lake API (#4229)
• Allow loading of Parquet format over the lake API (#4235)
• Allow loading of Zeek TSV format over the lake API (#4246)
• Auto-detect Parquet and VNG formats (#4270)
• Add support for Arrow IPC stream format (#4252)
• Add pool level configuration to zed manage (#4234)
• The mask argument of the network_of() function now accepts an ip-type netmask instead of a net type (#4247, #4253)
• Drop support for the older "zng21" format (#4255)
• Add support for user-defined functions (UDFs) to the Zed language (#4264)
• Add journal snapshots, significantly improving performance when a pool contains many small commits (#4274)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.3.0/zed-v1.3.0.linux-amd64.tar.gz
tar xzvf zed-v1.3.0.linux-amd64.tar.gz

If you've installed the Brim desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Brim wiki article for their location.

Release Notes

• Optimize performance of filters on pool key fields (#3930)
• Improve handling of errors during shaping (#4067, #4069)
• Allow use of a pool name regexp/glob pattern with the from operator (#4072, #4075)
• Add levenshtein() function for fuzzy string matching (#4104)
• Allow use of any filter with zed delete -where (#4100, #4124, #4126, #4125, #4127)
• Add regexp() function for regular expression searches and capture groups (#4145, #4158)
• Add coalesce() function for locating non-null/non-error values (#4172)
• Add line format for sourcing newline-delimited input as strings (#4175)
• Add map() aggregation function for constructing maps (#4173)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.2.0/zed-v1.2.0.linux-amd64.tar.gz
tar xzvf zed-v1.2.0.linux-amd64.tar.gz

If you've installed the Brim desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Brim wiki article for their location.

Release Notes

• Compress index values (#3974)
• Replace -znglz4blocksize flag with -zngcompress and -zngframethresh for zed and zq (#3967)
• Add -splitsize flag to zed and zq (#3957)
• Add -aggmem flag to zed and zq (#3921)
• Add zed compact (#3940)
• Remove -repl flag from zc (#3945)
• Always default to ZNG output for -split flag to zed and zq (#3938)
• Number -split flag files sequentially for zed and zq (#3937)
• Remove index support for relative comparisons (#3932)
• Support ... spread operator in array expressions and set expressions (#3910)
• Python client: rework zed.Client.query_raw() to handle any format (#3911)
• Add support for Parquet API responses (#3909)
• Support the indexing operation on sets (#3908)
• Add nullsMax argument to compare() function (#3898)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.1.0/zed-v1.1.0.linux-amd64.tar.gz
tar xzvf zed-v1.1.0.linux-amd64.tar.gz

If you've installed the Brim desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Brim wiki article for their location.

Release Notes

• Allow -configdir, -lake, -q, and -use flags before zed subcommands (#3852)
• Add assert operator (#3858)
• Honor from operator's format option (#3845)
• Add compare() function (#3832)
• Pretty-print ZSON type values (#3819)
• Allow conversion of time values to other numeric types (#3816)
• Remove scaling from duration and time conversions (#3809)
• Add over expressions (#3797)
• Add -where flag to zed delete (#3791)
• Allow base62 object IDs in lake API request bodies (#3783)
• Remove let operator and over operator's as clause (#3785)

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a TGZ/ZIP file of command line executables for each supported platform. To install, unpack with with the appropriate utility for your platform. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v1.0.0/zed-v1.0.0.linux-amd64.tar.gz
tar xzvf zed-v1.0.0.linux-amd64.tar.gz

If you've installed the Brim desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Brim wiki article for their location.

Release Notes

• Comprehensive documentation
• Substantial improvements to the Zed language
• Revamped zed command
• New Zed lake format (see #3634 for a migration script)
• New version of the ZNG format (with read-only support for the previous version)
• New version of the ZSON format

Quick Start

Visit the Brim Data download page to find the package for your platform.

Zed is distributed as a ZIP file of command line executables for each supported platform. To install, unpack with your platform's unzip utility. For example, on Linux:

curl -OL https://github.com/brimdata/zed/releases/download/v0.33.0/zed-v0.33.0.linux-amd64.zip
unzip zed-v0.33.0.linux-amd64.zip

If you've installed the Brim desktop app and want to work with its Zed lake from the command line, we recommend using the Zed executables included with the app to ensure compatibility. See this Brim wiki article for their location.

Release Notes

• zapi: Rename the ZED_LAKE_HOST environment variable to ZED_LAKE and rename the -host flag to -lake (#3280)
• zq: Improve ZNG read performance when the command line includes multiple input files (#3282)
• zed lake serve: Add the -rootcontentfile flag (#3283)
• Python client: Improve error messages (#3279)
• Python client: Fix Zed bytes decoding (#3278)
• Detect CSV input (#3277)
• zed lake serve: Fix an issue where POST /pool/{}/branch/{} format detection errors caused a 500 response (#3272)
• Fix an issue where the ZSON parser failed to normalize maps and sets (#3273)
• Python client: Add authentication (#3270)
• Python client: Handle query errors (#3269)
• Remove support for the TZNG format (#3263)
• zapi, zed lake serve: Add authentication with Auth0 (#3266)
• Fix an issue preventing casting from ip to ip (#3259)
• zed lake serve: Respect the Accept request header for GET /events (#3246)
• Add function documentation (#3215)
• zed lake serve: Change the default response content encoding to ZSON (#3242)
• zapi load, zed lake load: Add the -meta flag to embed custom metadata in commits (#3237)