ngs

New features

• Add Iter(SubSeq)
• Add =~(Eachable1, Pfx, MatchContext)
• Add DecodeHints
• Add =~(DecodeHints, ProcessesPipeline, MatchContext)
• Add AtPath pattern
• Add experimental ResetPath pattern

Fixes and improvements

• Add default =~(Any, Any, MatchContext) which returns false
• Improve decode() implementation for aws command
• Improve =~(Str, SubSeq, MatchContext)
• Fix MatchContext._path when matching elements in =~(Any, Arr, MatchContext)
• Fix in(Int, NumRange) which was causing for i in 0 ... to run once instead of not running

New features

• Add assert(Fun, Type, Str) for asserting that the code throws the given exception. The type must be subtype of Exception.
• Add namespace net
• Add networking c_sockaddr, c_sockaddr_un, c_sockaddr_in
• Add networking functions: socket(), bind(), listen(), accept(), recvfrom(), send()
• Add networking constants: SOCK_STREAM, SOCK_DGRAM, SOCK_RAW, MSG_OOB, MSG_PEEK, MSG_WAITALL (all prefixed with C_)
• Add Not type and support in pattern matching
• Add env: option to pass environment when running external programs
• Add ensure(Int, NumRange)
• Add last(Eachable1, Any), the counterpart of first()
• Add skip(Iter, Any) - skip elements matching the pattern
• Add skip(Iter, Int) - skip given number of elements
• When a function is called from CLI, the output is now displayed using experimental echo_cli()
• Add AWS2::regions(CommandsPipeline, Arr)
• Add IfExists pattern for optionally present fields
• Add before_first(Str, Str) - get the piece before first occurrence of the delimiter
• Add after_last(Str, Str) - get the piece after the last occurrence of the delimiter

Fixes and improvements

• Add UndefinedUpVar exception, thrown when accessing undefined "upvar"
• Following types now inherit from Error, not Exception: NotImplemented, ReadFail, RequireFail, MainFail
• Experimental syntax - allow many additional elements x.HERE and x::HERE
• Fix and improve documentation
• Improve inspect()
• Add regs parameter to AWS2::regions(Fun)
• $(log: ...) can now take Str to prefix the message
• Improve retry() logging
• Improve Time(Str)

Deprecated

• is_subtype(Type, Type) is now deprecated, use <= operator, example: T1 <= T2. Solves #391.
• typeof(Any) is now deprecated, use Type(Any) instead.
• x \ f syntax is now deprecated, use x.f() and x.{...} instead.

Misc

• GCC 9 on macOS is not supported anymore - it's broken

New features

• Parsing of curl -i ... command into {"code": Int, "message": Str, "headers": Hash, "headers_arr": Hash, "body": Str}
• Add decode(Str, p), where p is a Path or its subtype.
• Add assert(Path)
• Add Bool(Real)
• Add experimental native_ll_maybe_wrap
• Add TmpDir
• Add replace(File, Any, Any) for in-place replacement

Fixes and improvements

• Fix #451 - decode(Bool) is not strict enough
• group(Eachable1, Fun) now has additional parameter v: group(e:Eachable1, k:Fun, v:Fun=...). v tells how to process values before adding to the resulting multi-value-map.
• The ? operator now takes Any instead of Fun as the right argument.
• Fix tr exception when pattern is a Hash and it should match not at the top level.
• Argv now accepts Repeat keys: {Repeat('--repeat'): my_arr}
• Support my_arr[idx] = val for negative idxes.
• c_pcre_exec now behaves correctly when too many captures are specified
• Fix crash when accessing RegExp#options.
• Improve GitHub build action
• Add builds: Arch Linux, centos 7 and 8, fedora 34 and 35, Amazon Linux 2
• Better exception message when redirecting to >${null}.
• Website update & dark theme support
• ($()) is now a valid syntax for the identifier $()

Deprecated

• Argv - ['--repeat'] is deprecated in favor of Repeat('--repeat')
• attrs() is deprecated, use meta() instead

Breaking changes

• Str(Path) is now returning .path, not <Path path=...>
• the_one() is now fixed and will throw exception instead of returning null when appropriate found_none or found_more is not specified.

New features

• Add Hash(Hash) (a no-op)
• Add experimental PatternAction type for combining a pattern and an action to take when there is a match
• Add experimental -> syntax for constructing PatternAction
• Add experimental =~(x, PatternAction) which checks the pattern and conditionally runs the action
• Add experimental MatchContext type
• Add experimental Deep type for deep data structure matching
• Add Hash(Eachable1) - Issue #436 - Add Hash(array_of_key_value_pairs)
• Add exec() methods - Issue #459
• partial() is now deprecated because it's not used
• Add Lit type to convey literal value, devoid of any "magical" meaning
• Add collector(Stats, Fun)
• debug(Str, Fun) is now supported
• Add ensure(Arr, T) for T being subtype of ArrLike
• Add fork(), higher level wrapper around c_fork()
• Add Str(ArrLike)

Fixes and improvements

• inspect() fix for partially applied functions: ngs -pi 'X*2' does not fail anymore.
• =~(x, Hash) now supports patterns for keys
• Documentation generation now handles well ENV as default value for a parameter.
• Reorganize stdlib to eleminate multiple inherit() calls
• Remove unused most_by_cb()
• Booleans now compare correctly when used as Hash keys
• exit_hook now removes hooks in child fork()
• It is now possible to redirect to File without using .path - f=TmpFile(); $(echo a > $f) - Issue #490
• test.ngs in the root folder now accepts optional file names; defaults to lang-tests.ngs, stdlib.ngs, and all *.ngs files in lib/autoload

Breaking changes

• Remove deprecated nofail: option
• write(f:File, s:Str) now returns f

Deprecated

• Deprecate without(Eachable1, Any)
• Deprecate without(Eachable2, Any)

Setup NGS in Github Action support

New features

• Add replace(Str, RegExp, Str)
• Add Bool(UserDefinedMethod)
• Add c_pow(Real, Real), pow(Real, Real), pow(Int, Int)
• Add decode() for JSON Web Token (JWT)
• Add echo(File, Str)
• Add echo(File, Lines)
• Add C_DEFS namespace for C defined constants (currently only with _SC_NPROCESSORS_ONLN but will add more later, issue #113)
• Add c_sysconf(), sysconf()
• Add CleanupPolicy, KeepCleanupPolicy, RemoveCleanupPolicy, KeepOnErrorCleanupPolicy to be used with TmpFile
• TmpFile now accepts optional CleanupPolicy argument (defaults to RemoveCleanupPolicy)
• When running ngs -e ..., ngs -p ..., etc - allow using _ instead of fetch().
• Add "curl" installer.

Fixes and improvements

• Fix Str(File) - now display correct type name for subtypes
• Make decode_base64() more roubst
• pmap(Eachable1, Fun) now limits number of threads (issue #113)
• Add documentation for MaybeFile
• TmpFile is not expirimental anymore
• encode_json() with pretty=true can now handle larger inputs

Breaking changes

• Remove unused defined keyword

New features

• Add write(Str) - write to stdout
• Add del(HashLike, Any) - remove item from HashLike, similar to del(Hash, Any)
• Add trim(Str) - trim whitespace at the beginning and end of the string
• Add -ppj command line switch - print "pretty" JSON. Requires jq program.
• Add assert(val, pattern, msg) - val must match the pattern or AssertFail will be thrown.
• Add Repeat type - Signifies repetition of a pattern (or something else), to be used in patterns (and maybe somewhere else).
• Add group(Eachable1, Str) - group by field
• Add uniq(Eachable1, Str) - uniq by field
• Add duplicates(Eachable1, field_or_callback) - find duplicates

Fixes and improvements

• Introduced =~ and !~ matching operators.
• The Pred machinery replaced with =~ and !~ match/non-match operators for simplification.
• Some guard clauses are now using =~ and as a result are more concise.
• inspect() now shows attributes (including documentation) of methods and types.
• uniq() now works with Eachable1 (previously Arr) and has optional callback.
• reduce(Eachable1, Fun) now does not assume that the given Eachable1 supports Iter()
• encode_json() now supports hints optional prameter (of type Hash). pretty hint with true value will "prettify" the output.
• partition(Eachable1, pattern) now has default pattern (Bool.constructors)
• store() now supports hints optional prameter (of type Hash). pretty hint with true value will "prettify" the output.
• require() now treats ./something.ngs as an absolute path (to fix later: treat as relative to the file)
• AnyOf and AllOf are no more experimental.
• Increase fork-till-exec memory allocation from 1M to 10M (to avoid crashes)
• Str(Set) now works properly.
• Improve exit_hook robustness.
• Fix exception handling when calling map() with invalid arity callback.

Breaking changes

• =~ match against Arr now matches element-wise. Previously only one-element-array was supported as the second argument and it had to match all elements of the first argument.

• Snap packaging
• Improvements in CI/CD
• Documentation fixes and improvements

Detailed list of changes is in CHANGELOG.md

New features

• Homebrew formula and readme instructions (Thanks, @SeekingMeaning)
• Homebrew badge (Thanks, @organom)
• main() can now be defined as part of Namespace of the main file: ns { F main(...) ...}
• Add experimental only(val, predicate, mapper)
• Add realpath(Str)
• Use NGS_PATH environment variable for require(). NGS_DIR is deprecated.

Fixes and improvements

• Github actions instead of Travis (Thanks, @organom)
• $(log ...) now logs i/o redirections
• Remove unused ValueWrapper type
• Bootstrapping - MY_NAMESPACE::main works even if main is not defined in the main file, allowing ngs -e 'require("my_module.ngs") to run its own main()
• filterk(), rejectk(), filterv(), rejectv() - the predicate is now optional and defaults to identity.
• Got rid of xxd build time dependency
• make sys/poll.h dependency optional

Breaking changes

• Remove deprecated n()
• switch and cond are now consistent with if, accepting {...} code blocks for the LHS (switch {a=1; a+a} {...}, cond { {a=b+c; a>0 } ... }).

Notable changes:

• Add NamedInstances type (aka enums)
• Add experimental AllOf()

Detailed list of changes is in CHANGELOG.md

New features

• Concurrency
  • Add c_pthread_cond_* methods
  • Add Cond type that exposes pthread_cond_* functionality
  • Add BlockingList linked list type with bocking shift() and pop() methods
  • Add Executor type
  • Add ThreadedExecutor type for simplified processing in threads
  • Add pmap(Eachable1, Int, Fun) - parallel map using limited numbers of threads
  • Add pfilter(Eachable1, predicate) - parallel filter
  • Add pfilter(Eachable1, Int, predicate) - parallel filter using limited numbers of threads
• Add block IDENTIFIER BODY syntax
• Add section NAME BODY syntax
• Add experimental TmpFile - temporary file that is deleted automatically on exit
• Add experimental Program (allows assert(Program("dd")))
• Add experimental sortv()
• Add experimental OneOf type (allows "abc" ~ OneOf(Int, Str))
• Add List linked list type
• Add JsonData method which converts to JSON-compatible data structures; used by encode_json now.
• Add has_no(container, element)
• Add echo(ProcessesPipeline, Str)
• Automate updating ngs.vim syntax file
• Add lines() - returns all standard input as array of lines
• Add words(Str)
• Add Results type, array-like with Result items
• Add ResultsException type (with .results being Results type)
• Add List type
• Add inspect(Eachable1)
• Add len(Box)
• Add len(NumRange) for Int ranges
• Add RetryBodyMissing exception
• Add Namespace type. It is now returned by ns block.
• Add Pred(NumRange).
• Default code to run when not given on command line is F default_argv_code() { fetch() }() .
• Add Vagrantfile
• Add ~(x, t:Type)
• Add has_index(Eachable1, Int)
• Add len(Int)
• Add experimental Pred(OneOf)
• Add experimental TODO(Str)
• ec2din.ngs now takes expirimental -c COMMAND

Fixes and improvements

• Rare crashes due to GC fixed
• Multi-stage builds for the docker image (thanks, @organom)
• Hash(Arr, Str) is now Hash(Eachable1, Str) (to support ArrLike)
• Hash(Arr, Fun) is now Hash(Eachable1, Fun) (to support ArrLike)
• Add missing brew dependencies to install-mac.sh (thanks for reporting, @zzamboni)
• Add executable to inspect(Process) and to exception reporting.
• basename() is now implemented in stdlib.ngs as opposed to previously using BASENAME(1).
• finished_ok() now checks for known programs by their basename and not absolute paths (@zzamboni reported failing tests; this changes should fix it)
• echo(Int, Lines) is now echo(Any, Lines)
• Fix wait(Process) to behave correctly when exit_code is not set but exit_signal is set.
• Fix $(blah >${true})
• print_exception() - add optional parameter echo
• exception_specific_message() - refactored for reusability
• Improve error messages for syntax errors
• push(e:Enum, name:Str) now returns e
• stat() methods now return timestamps too (access/modify/change)
• pmap() now throws ResultsException if any of the threads fail
• Stats() now works with Eachable1 allowing counting characters in a string for examples
• Fix Pred(SubSeq) and Pred(RegExp) to prevent endless recursion
• Fix $(... >${false}) syntax
• Fix c_strptime()
• Improve rand_uniq implementation
• Improve MatchSuccess - include pattern
• Improve MatchFailure - include data and pattern
• Move get(Arr, Int, Any) from C to stdlib and support negative indexes.
• Merged two first() methods into one
• Fix Hash comparison for null
• + character is now valid "word" symbol - $(dig +short yahoo.com) is now valid syntax

Breaking changes

• Remove Return type and associated machinery
• Remove deprecated filter(Eachable1)
• Remove deprecated to_exit_code()

This release adds new types, adds new methods, and contains various fixes. Listing most important changes:

• $() now returns ProcessesPipeline (a new type). You should check all the places in your code that use CommandsPipeline. Code that handles result of $(...) should be changed to expect ProcessesPipeline, not CommandsPipeline.
• Lines type. Array of strings to be processed as a unit for output purposes (echo, warn, error).
• $(ok_sig: ...) option. If external program terminates by specified signal, NGS will not throw exception.
• $(cd: ...) option. Runs the external program in the specified directory.
• $(log: ...) option. Logs the program to be executed first and then executes the program.
• Argv() - support for arguments that need repeated argument name
• Box() now returns an EmptyBox (thanks @organom !)
• Pred improved so that .filter({"a": 1}) will not throw exception if there is no a field.

Detailed list of changes is in CHANGELOG.md

• Running NGS without arguments no more produces an error but a friendly usage message
• Docker image for each commit; built by Travis
• Add exit() method
• Add myip() method (returns your IP as observable from the Internet)
• Many fixes and improvements

Detailed list of changes is in CHANGELOG.md

• Time
• Set
• Misc fixes
• experimental MapIter, FilterIter, FunIter

Fixes 0.2.2

Fixes to assert_... methods after pipes support

Fixes after pipes support

• Pipes support
• Misc fixes
• write() parameters swap: now - write(destination, data)