Bot releases are visible (Hide)
Published by lmorg over 1 year ago
The previous releases have brought significant advancements to Murex's syntax but at the cost of longer gaps between releases. So the 4.1.x versions will be shorter releases but focusing on bug fixes. The 4.1.x release notes will be appended to murex.rocks changelog and available on Github releases too
None
readline: blinking cursor added to fuzzy find. This is just a very minor UI change to make it easier to tell where keyboard input would be updating
git
autocomplete now includes git aliases
merge operator (<~
) should unmarshal strings
scalars available in all assigns (eg +=
, -=
, etc)
lazy assigns allowed when strict types is set to false
. Meaning merge operations and +=
(et al) will work even if the left scalar is unset
suppress !!!
style warnings upon ^C
strip :
colon from command name when command string is passed from the pre-parser (#601)
Published by lmorg over 1 year ago
The previous releases have brought significant advancements to Murex's syntax but at the cost of longer gaps between releases. So the 4.1.x versions will be shorter releases but focusing on bug fixes. The 4.1.x release notes will be appended to murex.rocks changelog and available on Github releases too
None
Merge into operator added: ~>
f
: new flag i
-- deprecates ?
flag due to ?
being a glob pattern. However ?
will remain supported
$PREFIX
variable passed to Dynamic and DynamicDesc functions in autocomplete
-- includes the partial term (read more)
user@
completion support for ssh
Updated autocomplete
docs (read more)
command line is included in error message when error running inside another module
errors within other modules now include the command line string to make debugging even easier
lambdas success/fail criteria is now driven by the exitnum rather than stdout data type. This allows it to be much more robust. In addition $.
is now mutable
f
improved help message (previously some items displayed code numbers instead of descriptions)
readline: hint text flicker removed
Published by lmorg over 1 year ago
The previous releases have brought significant advancements to Murex's syntax but at the cost of longer gaps between releases. So the 4.1.x versions will be shorter releases but focusing on bug fixes. The 4.1.x release notes will be appended to murex.rocks changelog and available on Github releases too
None
<~
readline: fixed out of bounds errors when calculating unicode cell positions
readline: fixed panic raised when vim key p
is used on an empty buffer
readline: fixed bug where vim mode became unavailable / esc
didn't work as expected. This was due to autocompletion find (^f
, ^r
) being launched before tab
completions and state not being handled correctly
Published by lmorg over 1 year ago
The previous releases have brought significant advancements to Murex's syntax but at the cost of longer gaps between releases. So the 4.1.x versions will be shorter releases but focusing on bug fixes. The 4.1.x release notes will be appended to murex.rocks changelog and available on Github releases too
None
lambda support added to piped indexes, ie example -> [{ lambda }]
, to mirror existing @example[{ lambda }]
functionality
--autocomplete
flag added to read
application specific environmental variables (eg <env:FOO=BAR>
) were not getting set
cast
now checks if it is a method. Previously was piping regardless
fixed by where nested maps would overwrite when altered with --merge
. Now they merge as expected
alter
panics are caught issue (#536)
Published by lmorg over 1 year ago
The previous releases have brought significant advancements to Murex's syntax but at the cost of longer gaps between releases. So the 4.1.x versions will be shorter releases but focusing on bug fixes. The 4.1.x release notes will be appended to murex.rocks changelog and available on Github releases too
formap
values to be strings despite that value being a different primitive from the originating object. However this type of code would be an anti-pattern so we shouldn't expect much, if any, code written like thisformap
values are now data type aware. You no longer need to cast values. Keys are still string only and the intention is to keep them this wayimproved error messages around a few internal APIs
micro optimisations in string parsing (minor order to operations, now converting points at a slightly later stage. This adds zero impact on code complexity and almost zero code refactoring but results in slightly less memory getting copied about)
Published by lmorg over 1 year ago
The previous releases have brought significant advancements to Murex's syntax but at the cost of longer gaps between releases. So the 4.1.x versions will be shorter releases but focusing on bug fixes. The 4.1.x release notes will be appended to murex.rocks changelog and available on Github releases too
None
No new features
code references in error messages are cropped to terminal width
error handling around multiple chained but incompatible tokens (#566)
Published by lmorg over 1 year ago
Minor release: improved error messages raised when using variables
Published by lmorg over 1 year ago
This release sees significant improvements for use with non-latin characters in both the interactive prompt and shell scripting. It introduces new syntax to make working with structured data even easier than before. As well as new data types and smoother user experience.
Breaking Changes:
$PWD
is no longer a str
. If you need to parse $PWD
as a string then you should out: $PWD
or use ${pwd}
.
is now a valid bareword character for variables. This will break any instances in code that contain something like out $hello.
however you can now use out $(hello).
syntax for safer string infixing
Features:
dot notation added for variables which allow accessing and writing to object properties. eg $json.array.index
added support variable encapsulation via $()
to avoid variable barewords from leaking into other values when infixed. eg $(foobar)
is the same as $foobar
variable names can now be Unicode, eg $(比如) = "举手之劳就可以使办公室更加环保,比如,使用再生纸。"
Unicode wide characters are properly handled in readline
new data type path
that is an object
new data type paths
support for lambdas added
readline [ctrl]
+[f]
now supports word matching as well as regexp
readline [ctrl]
+[f]
now supports glob filters (eg *.txt
) too
$ENV
and $GLOBAL
objects (deprecates the need for global
and export
)
improved error messages in expressions
scalars can be used as assignees in expressions, eg $foo = "bar"
added -i
flag to ssh autocompletion
readline prompts are now buffered to reduce cursor flickering
Non-User Facing / Maintenance Changes:
Bug Fixes:
better wide character support in auto completion suggestions
function parameters default and comment fields are no longer order dependent
better string parsing for barewords inside arrays and objects
ahead of time hint caching now done on demand per hint page rather than against every executable upon murex start up
new lines should be ignored in %{}
bareword true
and false
should be converted to boolean in %{}
autocomplete suggestions are now dedup'ed when delayed results return similar suggestions to fast results
various preview pane bug fixes
support for ANSI escape sequences in experimental (disabled by default) TTY buffer
correction to count
docs
Published by lmorg over 1 year ago
This release includes mostly bug fixes and new experimental features which are opt into. To enable all experimental features, set the environmental variable MUREX_EXPERIMENTAL
to any value. Or you can enable specific features individually via config
Breaking Changes:
Features:
default
statement added to switch
(this will deprecate catch
, albeit catch
will remain supported for compatibility)
Option to change start directory: config: set shell start-directory $path
(where $path is the working directory you wish to start the shell in)
onCommandCompletion
event no spawns a pty. This allows capturing of stdout while still presenting a tty to the executing process
EXPERIMENTAL: preview files. This can be enabled via config: set shell preview-enabled true
EXPERIMENTAL: preview images. This can be enabled via config: set shell preview-images true
EXPERIMENTAL: tty buffer. This can be enabled via config: set shell tty-buffer-enabled true
ctrl
+r
has been tweaked to make it more user friendly
Long overdue SIGWINCH signal handler to reflow after terminal resized
Lots of improvements to documentation
Non-User Facing / Maintenance Changes
Bug Fixes:
re-added columns
data-type to fix jobs
builtin
added bang alternatives to the glob exception list
added missing support for &&
tokens in expressions
![]
can now produce empty maps
fixed regression bug in file pipe mutex
fixed missing bounds check in statement parser
Published by lmorg almost 2 years ago
This is a major release that brings a significant number of changes and improvements, including a complete overhaul of the parser. Backwards compatibility is a high priority however these new features bring greater readability and consistency to shell scripting. So while the older syntax remains for compatibility, it is worth migrating over to the newer syntax for all new code being written
Breaking Changes:
Optional builtin removed: bson
. This was disabled by default and likely never used. So it has been removed to reduce murex's testing footprint. It can easily be re-added if anyone does actually use it
Optional builtin removed: coreutils
. This was actually enabled by default for Windows builds. However rewriting Linux/UNIX coreutils for Windows support is a massive project in its own right and with the maturity of WSL there's almost no reason to run murex on "native Windows". So the coreutils
builtin has been dropped to allow us to focus on the responsibilities of the shell
Features:
Support for expressions, eg 5 * 5
or foobar = $foo + "bar"
, etc. This syntax can be used directly or specified specifically via the expr
builtin
New syntax sugar for creating JSON objects: %{ foo: bar }
New syntax sugar for creating JSON arrays: %[ foo bar ]
New syntax sugar for creating strings: %()
(this is preferred over the, now deprecated, feature of parenthesis quotes)
Ranges supported in []
(@[
is now deprecated)
Support for multiline comments: /# comment #/
. It is unfortunate this differs from C-style comments (/* comment */
) but this has to be the case because /*
is ambiguous for barewords in shells: is it a path and glob or a comment? Where as /#
isn't a common term since #
is already a comment
If any processes pass null
as a data type across the pipe, it will be ignored. This solves the problem where functions that don't write to STDOUT would still define the data type
Config option auto-glob renamed to expand-glob, and now enabled by default
Globbing exclusion list. This allows you to avoid annoying prompts when parameters shouldn't be expanded as globs by the shell (eg when using regular expressions). This can be managed via shell expand-glob-unsafe-commands option in config
@g
removed. It is no longer needed with expand-glob enabled by default
New builtin: continue
: skip subsequent processes in an iteration block and continue to next iteration
New builtin: break
: exit out of a block of code (eg in an iteration loop)
Additional syntax for index ([
): *1
: 1st row, *A
: 1st column
New alias: help
-> murex-docs
. This brings murex a little more inline with Bash et al
pre-cache-hint-summaries now enabled by default after testing has demonstrated it doesn't have nearly as expensive footprint as first assumed
Hitting TAB when nothing has been typed in the REPL will suggest past command lines
^
autocompletion added
getfile
writes to disk if STDOUT is a TTY
mkarray (eg ja
) now writes an integer array if range is integers. eg ja: [1..3]
. This change wouldn't affect a
since that outputs as list of strings (for streaming performance reasons) rather than a data type aware document
debug
(method) output tweaked
Improved error messages in a number places
Revamped README / website landing page
Non-User Facing / Maintenance Changes
Minimum Go version supported is now 1.17.x
Main parser completely rewritten
ReadArray
API now requires a context.Context
egrep
references changed to grep -E
to work around GNU grep deprecating support for egrep
Added marshallers for boolean
, null
Variables.GetValue()
now errors instead of returns nil
when no variable set
Additional tests. So many new tests added
Lots of code refactoring
Bug Fixes:
regexp
wasn't erroring if nothing was matched
readline: fixed deadlock
append
and prepend
now type aware (no longer converts all arrays into string arrays)
foreach
was setting variables as strings rather than honoring their original data type
yarn
autocompletion errors should be suppressed
spellcheck missing break
causing more occasionally incorrect instances of underlined words
config
wasn't passing data type when executing blocks via eval
debug
wasn't setting data type when used as a function
macro variables don't re-prompt when the same variable is used multiple times
Published by lmorg about 2 years ago
This is a bug fix release:
Published by lmorg about 2 years ago
This is a bug fix release:
Published by lmorg about 2 years ago
This release mainly focuses on refinements in performance and usability, rather than introducing new features
Breaking Changes:
Features:
File index caching for faster completions (eg on mechanical drives). This happens upon shell start up (except on Darwin for security reasons) and after every cd
Relaxed alias
syntax. It now supports whitespace between the =
statement (like set
and let
do)
New aliases added for common methods which work against lists/arrays. Type list.{TAB}
to see the complete list of methods
Added support for alternative separator values in struct-keys
Hint summaries are now cached, to improve performance on slower machines
Opt in support to pre-cache hint summaries. This will also auto-disable appending colon to command suggestions in autocomplete. This is not enabled by default because it could negatively affect performance on some slower machines and is more of an aesthetic improvement. You can enable it by adding config: set shell pre-cache-hint-summaries true
to your murex profile
New autocompletions for yay
(ArchLinux)
Some minor performance improvements
website: updates to docs
Bug Fixes:
readline: max field size adjusted for autocompletes so more completions are visible at one time
readline: all cropped autocompletions now provide an uncropped version in the hint text field
readline: fix bug with history not getting written
OS signals (^c et al) are now re-registered after each exec (fixes bug where some programs would "steal" ^c, breaking support for those signals upon subsequent execs)
Fixed regression bug in man page parsing
Added check for zero-length string used as a process name (this shouldn't ever happen under normal circumstances however closing this bug allows for a managed error rather than the code panicking)
readline: Prompt environment reset on each loop (this change is to future-proof against potential bugs)
Fixed race condition bug when some of the tests are run concurrently (this bug only existed in the unit tests but sometimes caused those tests to fail)
Added CI tests for Go v19.x
Published by lmorg about 2 years ago
Bug fix for range variables in v2.10
Published by lmorg about 2 years ago
This release brings a few minor improvements and bug fixes rather than big new headline features.
Breaking Changes:
Features:
f: +f -> set v
echo: @v[2..]
Complete re-write of f
. It is now the tool I had always intended when I first created that builtin
g
now supports being run as a method (eg to filter items from a file list)
rx
now supports being run as a method (eg to filter items from a file list)
Updated autocompletes for git
Updated autocompletes for terraform
Minor Changes:
Added support for profiling
Removed unused data types
Dropped profile/
prefix from the module string for modules and profiles (as seen in FileRef
)
Bug Fixes:
Lots of improvements to how ||
and &&
are used in other runmode
's, try
, and trypipe
autoglob prompt now has a mutex wrapped around it to prevent concurrent processes sending the prompt haywire
Spellchecker no longer underlines partial words
Better removal of \x16 and other non-printable characters leaking into readline
Published by lmorg over 2 years ago
This release focuses on testing and REPL usability improvements but also
includes updates several new run modes to make error handling easier in
larger scripts.
Breaking Changes:
Features:
>>
redirect (append to file) operator added. This is just syntactic sugar for -> >>
. To redirect and overwrite/truncate a file use |>
rather than >
Support for automatic globbing in the commandline. This has to be enabled via config: set shell auto-glob true
and you will get prompted for each glob. Any globs inside blocks will be ignored and this only works on the interactive prompt (ie not in scripts or background processes) -- these limitations are not a bug but by design
Dynamic autocomplete suggestions are now cached for 5 seconds by default. This cache can be changed via the CacheTTL directive in autocomplete
Additional scoping for try
and trypipe
. You can define the scope as being an entire function or a module. If set for an entire module then any functions, autocompletions, etc inside that module will inherit that try
or trypipe
mode automatically. These are set via the new builtin runmode
New reserved variable $PWD
(this was just a global variable previously)
Additional flags added to range, @[
New builtin, man-get-flags
. This exposes murex's internal man page parser to the command line scripts
Binaries now produced for ARM64 Windows
Minor Changes:
The order in which namespaces are looked up has been altered since private
s should be resolved before function
s
Updated autocomplete for homebrew, brew
New autocomplete for AWS CLI, aws
New autocomplete for yarn, yarn
murex unit tests should report progressively rather than everything upon completion
Minimum recommended version of Go has been raised to 1.15. This is due to some bugs in dependencies when running tests against v1.13 and v1.14. murex will technically compile and work on them but it not worth my time debugging why the CircleCI tests fail for those versions given their age
Vendor directory removed. This isn't really needed now support is focused on newer versions of Go
Website: New 404 error
Website: search bar added
The start of a massive push towards upping the test coverage
Lots of improvements to the docs
Bug Fixes:
murex/shell
to bring it inline with the naming convention of other modulesPublished by lmorg over 2 years ago
This release comes with a number of experimental but stable features that might eventually become standard practice. The features are there to use if you with but adjacent from the older code so there is zero risk in updating to this version.
Breaking Changes:
Features:
New data-type jsonc
(Concatenated JSON). This is an experimental parser for reading multiple JSON documents in a single file or stream. The expectation is that it will eventually replace jsonl
(jsonlines) and possibly even the default json
data-type
Autocomplete can now split on =
. However this is an experimental feature
New autocomplete
directive: Goto
. This is an experimental way to add a little more structure to autocomplete definitions without resorting to using dynamic code everywhere
autocomplete
directive FlagValues
now supports two values: "*"
and ""
. These define default properties regardless of the flag used
Minor Changes:
Unmatched records in index ([
) when parsing a table now produces the same configurable behavior (erroring by default but which can be disabled) as when using index against an object
Autocompletion suggestions for yarn
Data-type field in function
is now optional
Lots of improvements to the docs
Bug Fixes:
$MyVar[[/path/to/element]]
now behaves the same when used infixed as it did when used as a functionPublished by lmorg over 2 years ago
Breaking Changes:
zero length arrays returned from subshells (eg echo @{g this-file-does-not-exist}
) should fail by default, like unset variables. This is enabled by default but can be disabled via config: set proc strict-arrays false
autoglob should fail if it doesn't match any results. eg @g echo this-file-does-not-exist.*
Subshells should fail parent command when used inside try
and trypipe
blocks. eg try { echo ${false} }
Features:
function
now supports defining parameters
Added support fro &&
eg do-something && do-something-else
for chaining successful commands
Added support for ||
eg do-something || do-something-else
for chaining unsuccessful commands
Added support for writing to the terminal emulator's titlebar via config: set shell titlebar-func { block }
titlebar-func
can also be written to your tmux
window title via config: set shell tmux-echo true
.
New reserved variable: $HOSTNAME
New reserved variables: $1
(and upwards) which correlates to the scope's parameter index. $1
is the functions first parameter. $2
is the second. $13
is the thirteenth.
New reserved variable: $0
which contains the function name
New event: onCommandCompletion
(this is experimental and subject to change in the next release)
Macro variables. eg echo Hello ^$name
will prompt the user to enter a name. Macro variables are only support in the REPL
read
now supports flags (eg default option, etc) to allow for a better experience in shell scripting
Minor Changes:
You can now overwrite onKeyPress
events. This no longer produces an error forcing you to remove the old event before adding the new one
Autocompletion suggestions shouldn't be sorted is results include paths (improves the ordering of autocompletion suggestions)
Autocompletion suggestions for openagent
builtin
Autocompletion suggestions for hashtags
Test counts re-added to website
Windows should show file extensions by default in autocompletion results
Bug Fixes:
Fix inverted logic on forceTTY
: config: get proc force-tty false
set by default, which then proxies STDERR and prints them in red
ctrl+c (^c) now currectly sends SIGTERM rather than just ending the child process
Better handling of SIGTERM
SIGTSTP isn't working. Switched to SIGSTOP when (^z) doesn't
Fix panic in event onFilesystemChange
where fileRef is not getting passed correctly
Fix panic in event onFilesystemChange
where path is zero length string
Some improvements to variable previews in the REPL
count
should check if it is a method
AST cache now checked more regukarly. This is to reduce the risk of memory leaks during fuzz or load testing
murex-docs
still referred to len
builtin. That should be changed to count
Lots of fuzzing added -- a few edge case bugs discovered
Published by lmorg over 2 years ago
Breaking Change:
strict-vars
is enabled by default. This means if a variable doesn't exist / hasn't been instantiated then your command will fail. You can disable this by putting config: set proc strict-vars false
inside any affected function
s and/or in your ~/.murex_profile
Features:
select
builtin now added to the default build. This built in will stream tabulated data into an in memory sqlite3 database and take SQL arguments to query that data. Thus allowing you to inline SQL queries as part of your native command line. This addition will add complications for anyone building from source on Windows. If you are a Windows user you are recommended to use WSL or, if you desperately need to use murex natively on Windows, download the appropriate per-compiled binary from the downloads page$PATH
will now be subject to the same inclusion rules as Windows file systems when murex is used in Windows. This means only executables (ie files with .exe
, amongst other, extensions are present). This has dramatically cleaned up autocomplete suggestions when running murex in WSL.autocomplete
directive: FileRegexp
. Use this if you want to return only files where the file name matches a defined pattern (eg only .txt files: \.txt$
)get
, getfile
, post
, open
, etc) supports additional logic with dynamically guessing the correct data type to assign to STDOUT where the Content-Type
hasn't been mapped. Any Content-Type
with a +json
suffix will be assumed to be json
data typegetfile
output has been revamped. If STDOUT is a TTY then you'll now see a progress bar.tfstate
(Terraform state files) have been added to supported JSON extensions~/.ssh/config
hosts have been added to SSH (et al)'s autocomplete suggestionsmurex-package: reload
now only reloads packages, not .murex_preload nor
.murex_profile`Non-user facing changes (internal changes to the murex code base):
Bug fixes:
get
, getfile
, post
, open
, etc) wasn't breaking down the Content-Type
header correctly. This only affected unknown MIME types where murex had to guess the correct data type to assign to STDOUT. This is an edge case bug.while
, when used without a separate parameter for the conditional logic, was buffering the STDOUT for each loop and outputting it after that iteration had ended. while
now uses a "tee" stdio writer which means it can stream STDOUT while the iteration is in progressPublished by lmorg over 2 years ago
Features:
alter
now supports --sum
where structures are merged and numeric values are added together
New builtin count
. This has deprecated len
however len
will stick around as an alias for backwards compatibility
New operators added to let
: +=
, -=
, /=
, *=
New builtin addheading
for adding headings to lists
Compiled profile will now always execute even if murex flags set to ignore the modules/user profile. This is so that aliases like len
get set and thus murex can still move forward with changes to builtins but without breaking backwards compatibility
autocomplete
now passes ISMETHOD
variable to dynamic completions so those dynamic completions are aware if a command requesting auto-completion suggestions is being invoked as a method (mid-pipeline) or function (start of a pipeline)
Index, [
, now supports inlining element, [[
, lookups. eg [ a b [/map/c] ]
Dynamic auto-completions that include @IncFiles
or @IncDirs
will now automatically append files and/or directories to their auto-completion suggestions
New autocomplete
directives: IncExeAll
(like IncExePath
but includes builtins, functions, aliases), IncManPage
(include results from the man
page parser -- usually suppressed when autocomplete
config is set)
Disabled 'Loading default profile' message -- this was always pretty redundant but now that the compiled profile is being loaded all the time (eg murex -c 'command'
or when called in a shebang), it's also now ugly too
select
now supports passing a file in the FROM syntax. eg select * FROM ./example.csv
. The caveat here is this breaks currently auto-complete on column names
select
now supports multiple tables using either named pipes (eg select * FROM <table1>, <table2>
) or variables (eg select * FROM \$table1, \$table2
) passed in the FROM syntax. Variables should be escaped and you cannot mix and match between named pipes, file names nor variables. You can use any number of tables from 1 to 2^63-1 (64bit systems) or 1 to 2^31-1 (32bit systems). Which should be more than enough ;)
config
option for select
to define default output data type where multiple tables are imported
Lots of new and updated documentation!
Non-user facing changes (internal changes to the murex code base):
open
functions can now be called by other functions to take advantage of auto-typing and auto gunzip etc.
tmp.Close()
should return err
. This isn't a bug but it might catch future bugs
LazyLogging
created to speed up writing tests against data structures
utils/List
package created to handle list / array / map functions. Also makes testing more complex routines easier
Bug fixes:
Regression bug fixed where prepend
was invoking append
streams.ReadCloser
not setting context
parameters.StringArray()
should copy values instead of a pointer to ensure the underlying parameters are immutable