yamlpath

3.0.0:
Enhancements:

• Added a new YAML Path Segment Type: *
  This is identical to a Search segment where the search term is [.!=""].
  This translates to "match every Hash key for which its name is not empty and
  every Array element which is not empty". This operator also vertically
  expands results from Collectors, effectively breaking them out from an Array-
  per-line to one-Scalar-per-line. If you place this inside a Collector, the
  results will still be collected into an Array.
• The * character now also serves as a wildcard character for key-names, Hash
  values, and Array value comparisons, converting the segment to a Search. For
  example, a YAML Path like abc.d* becomes abc[.^d], abc.*f becomes
  abc[.$f], and abc.*e* becomes abc[.=~/^.*e.*$/], and so on.
• Added a new YAML Path Segment Type: **
  This new type is a "Traversal" segment which causes YAML Path operations to
  deeply traverse the document from that point. When there are no further
  segments in the YAML Path, every leaf node (Scalar value) is matched. When
  the YAML Path has at least one further segment, it (and all further segments)
  must match subsequent nodes (anywhere deeper than that point in the document)
  or none are matched. Results can be collected.
• The yaml-merge and yaml-get command-line tools now treat the - pseudo-file as
  implicit when NOT specified AND the session is non-TTY. This can be blocked
  with --nostdin|-S. This enables, for example, piping into these commands
  without being forced to specify the - pseudo-file as an argument to them.
• The yaml-merge command now enables users to force the merged document to be
  written out as YAML or JSON via a new --document-format (-D) command-line
  argument. When unset, the format will be based on the file-name extension of
  the --output file when provided, or (last-resort) that of the first document
  (AUTO).
• The yaml-merge command now accepts multi-document YAML files, created when
  the YAML standard-specified End-of-Document, Start-of-Document marker pair
  (... followed by ---) is present, like:

  ---
  document: 1
  ...
  ---
  document: 2
  

• The yaml-merge command now accepts multi-document JSON files, created when
  there are multiple root-level entities, like:

  {"document": 1}
  {"document": 2}
  

• Because any document to yaml-merge can be a multi-document, it no longer
  requires at least 2 YAML_FILEs be supplied on the command-line. If users
  pass only a single file or stream that is not a multi-document file, its
  content will merely be written out without any merging into it. This can be
  useful for trivially converting any file from YAML to JSON or JSON to YAML,
  like yaml-merge --document-format=json file.yaml or
  yaml-merge --document-format=yaml file.json.
• The yaml-set command-line tool can now write changes to empty or minimally-
  viable files, enabling users to build up new data files from scratch. The
  file must already exist, even if completely empty. A non-empty, minimally-
  viable file depends on document type. For example:
  A minimally-viable YAML file:

  ---
  # The triple-dash is required.
  

  Two versions of a minimally-viable JSON file:

  {}
  

  or:

  []
  

  However, minimally-viable structure is necessary only for files with unusual
  file-name extensions. When the file-name extension is one of yaml, yml, or
  json (case-insensitive), the file can be completely empty and still result in
  a YAML or JSON data structure.
• The yaml-set command-line tool now accepts empty-String values.
• The yaml-merge command-line tool now permits overwriting one of its input
  files as long as --overwrite is used instead of --output; these are
  mutually-exclusive options. To help users protect against accidental change,
  a new --backup flag will cause the to-be-overwritten file to be renamed
  with a ".bak" file-name extension. A pre-existing backup file with the same
  name will be unceremoniously replaced.
• The yaml-set command-line tool now accepts an arbitrary set of characters
  from which to derive --random values via a new --random-from argument.
  This is especially useful when you need to limit or expand the characters
  used and when you wish to favor some characters more than others (simply
  repeat the favored characters more than other characters in the argument
  value but do so under caution because doing so reduces randomness).
• The yaml-set command-line tool now accepts documents from STDIN, causing it
  to write the resulting changes to STDOUT. This enables yaml-set to operate
  as a stream editor, like yaml-get and yaml-merge.
• The yaml-paths command-line tool now accepts documents from STDIN. It also
  now accepts multi-document YAML and JSON as file or STDIN input. Because it
  is impossible to determine whether a file or stream contains multi-document
  data without reading through the entire file more than once, output now always
  displays the file-name (or STDIN) and -- new -- the document-index in which
  matches were found. As before, users can turn off file-name display by
  setting --nofile|-F. In previous, single-document versions, the file-name
  display was automatically muted when there was only one YAML_FILE to process.

Bug Fixes:

• Collectors were breaking search nodes with Regular Expressions, making it
  impossible for collected searches to return expected matches.
• Fixed the Known Issue which was logged at version 2.4.0; setting values which
  override aliased key-value pairs now correctly adds the new key-value pair to
  the DOM.
• When the left-most document was JSON, yaml-merge and yaml-set would both
  improperly write out a YAML document start mark (---) and then a hybrid
  JSON/YAML result rather than valid JSON.
• The yaml-merge command would "explode" LHS Anchored Hashes wherever they were
  aliased when the RHS document modified the same Hash into which the alias was
  used.
• Setting a Python-style Boolean value via yaml-set (True or False) without
  setting --format=boolean would cause an error because ruamel.yaml was
  expecting an integer, instead. It is no longer necessary to set --format in
  this case. However, --format=boolean can still be useful to convert more
  "Boolean like" values into true|false, like on, off, yes, no, true, false,
  True, False, 1, 0.

Non-Breaking API Changes:

• The various protected get* methods of Processor were changed to reduce the
  number of positional parameters while also allowing for new special-use
  parameters for future changes. The formerly optional positional parent and
  parentref parameters are now optional keyword arguments by the same names.
  Because these were all protected methods and the affected parameters were
  optional anyway, this is not deemed a breaking change; no one should have
  been directly calling them.
• The get_yaml_editor function now supports several keyword arguments which
  provide for some customization of the returned ruamel.yaml.YAML instance.
  See its documentation for details. This is a non-breaking change as the
  defaults for each new keyword argument set the behavior identical to what it
  was before this change.
• The get_yaml_data function now returns False rather than None when there is
  an issue attempting to load data. This is because an empty-but-viable
  document correctly returns None but there is no valid YAML or JSON document
  which can be comprised only of a Scalar Boolean. This is a non-breaking
  change because None and False are equivalent for code like:

  data = get_yaml_data(get_yaml_editor(), ConsoleLogger(), "file.yaml")
  if not data:
    print("No data")
  

  However, you can now differentiate between "No data" and "Invalid document"
  like so:

  data = get_yaml_data(get_yaml_editor(), ConsoleLogger(), "file.yaml")
  if data is None:
    print("No data")
  elif not data and isinstance(data, bool):
    print("Invalid document")
  else:
    print("Got a non-empty document")
  

• The ConsolePrinter's debug method now prints vastly more detail and allows
  for customization of the output. Read its documentation for details.

From this release forward, the version reported by all command-line tools is
synchronized with the version of the overall yamlpath installation.

Bug Fixes:

• Array-of-Hashes were not being detected for the purpose of applying merge
  rules defined via the INI-style configuration file.
• Array-of-Hashes identity key inference was looking first to the LHS document.
  This was backwards for an RHS-to-LHS merge and has been corrected.

The yaml-merge command now reports version 0.0.4 to reflect these changes.

Enhancements:

• In the INI file's [rules] section, different merge rules can now be applied
  to specific parts -- no matter how deeply nested -- of the same Hash
  structure.

Bug Fixes:

• The 3rd-party Python INI file parser had no way of differentiating between
  the key and value of a YAML Path entry containing an = sign in its key, like
  "/path[.=name]/key = left". This update reconstitutes such lines and
  correctly parses an affected YAML Path from the merge rule.

The yaml-merge command now reports version 0.0.3 to reflect these changes.

Bug Fixes:

• The yaml-merge tool (and underlying Merger class) incorrectly assigned "None"
  Anchors to all floating-point numbers. This preventing all merging when both
  the LHS and RHS documents contained at least one floating-point number, each.
  The yaml-merge command now reports version 0.0.2 to reflect this change.

Enhancements:

• Added new reference command-line tool: yaml-merge. This is a very complex
  tool, so a comprehensive treatise will be added to the project Wiki to
  explore its capabilities. Along with those of its component classes, its
  unit tests also provide many examples of the same.
• YAMLPath instances now support arbitrary changes to seperator.
• YAMLPath instances now support equality testing (against the stored path),
  immune to differences in seperator.
• The get_yaml_data function now supports "-" as a source file. This is
  interpreted as a read from STDIN.
• Due to the change to the get_yaml_data function, the yaml-get reference
  command-line tool now supports retrieving nodes from YAML/Compatible data
  passed to it via STDIN when its YAML_FILE argument is set to -. The new
  yaml-merge reference command-line tool also reads from STDIN when one of its
  YAML_FILE input arguments is -. No other reference command-line tools
  support this change at this time.

Please consider the new yaml-merge (version 0.0.1) a Beta version. Feel free to experiment with it and report any issues at https://github.com/wwkimball/yamlpath/issues!

Bug Fixes:

• Setting negative floats could cause the leading "-" symbol to be replaced
  with an unexpcted "0" when specifying a float format, or crash when using
  the default format.

Bug Fixes:

• When using yaml-set with --format=folded and --eyamlcrypt, the encrypted
  value was being mistakenly appended with a spurious newline character at its
  end. Note that this affected only Puppet's Hiera lookup and not EYAML
  itself; on the command-line, the eyaml command would not show the extra
  newline character. The yaml-get command was also not affected, making it
  very difficult to set up a unit-test for this case.

Bug Fixes:

• Certain YAML constructs trigger AssertionErrors in ruamel.yaml during YAML
  data writes. This was causing yaml-set to generate empty files. Until
  https://sourceforge.net/p/ruamel-yaml/tickets/351/ is fixed, this patch
  will revert the file contents to mitigate data loss under these conditions.
  A specific test has been created to detect when the upstream issue is fixed.

Bug Fixes:

• Minor security patch: Python already makes non-shell subprocess calls safe
  (mitigating shell command injection). This patch makes that already-present
  protection explicit rather than implicit.

Bug Fixes:

• Subtraction Collector math had no effect when the RHS was a list of scalar
  values (because LHS was a list of NodeCoords, so comparison was always
  false). Also reduced O(3N) to O(2N) during Collector subtraction.

Enhancements:

• The console logger's debug method now includes the type of each element in a
  list while it is being dumped.

2.3.2:
Bug Fixes:

• Subtraction collector math crashed when the RHS result was non-iterable.

2.3.1:
Bug Fixes:

• Under certain conditions, some YAML changes were affecting unexpected nodes
  which had identical original values.
• YAMLValueFormats.DEFAULT was identifying all CLI-supplied values as String
  rather than their most-likely data-type.

API COMPATIBILITY WARNING:
Previous versions of the library returned only selected data nodes. Now,
instead of data nodes, NodeCoords objects are returned. This is necessary in
order to resolve Issue #44, which was caused by Python over-aggressively
optimizing memory, making non-String data nodes references to any other node
with the same value. This made changing exactly one of those data nodes
impossible because all nodes with the same original value would change with it.
Now, the NodeCoords class carries DOM tracking data along with each data
node making precise node changes possible. However, your queries will now
return this more complex additional data. In order to evaluate just the data,
please refer to this before-and-after example:

BEFORE:
for node in processor.get_values(yaml_path):
do_something_with(node)

AFTER:
for node_coordinate in processor.get_values(yaml_path):
do_something_with(node_coordinate.node)

If you need to recursively remove DOM tracking data from the results, a new
utility function is available: func.unwrap_node_coords(data). Note however
that you need that tracking data in order to change data within the DOM.

This does not affect the output of the sample command-line utilities.

2.3.0:
Bug Fixes:

• The get_yaml_data helper function now contains ruamel.yaml errors/warnings
  without disrupting calling context handlers.

Enhancements:

• yaml-paths version 0.2.0 now has more detailed output control, trading
  --pathonly for --values, --nofile, --noexpression, and --noyamlpath.

2.2.0:
Bug Fixes:

• YAML construction errors are now caught and more cleanly reported by all
  command-line tools.

Enhancements:

• yaml-paths version 0.1.0 now has more specific flags, enabling:
  • more precise handling of anchors in YAML data, and
  • expanding parent node results to instead return all their child leaf nodes.

Bug Fixes:

• yaml-paths was recursing into nodes for which the name had already matched,
  causing unnecessary search results. Version 0.0.2 fixes this; when a node is
  matched by name, any children are ignored because they will have already been
  yielded as the parent node's value.

Enhancements:

• Added a new yaml-paths command-line tool. In short, it enables searching
  YAML/Compatible files, returning YAML Paths for any matches. As an Alpha-
  grade tool, it is being released at version 0.0.1. Feedback welcome!
• All command-line tools which accept --pathsep now accept symbolic seperators
  rather than only names; so, --pathsep=/ is idental to --pathsep=fslash, etc.
  Minor changes were also made to all command-line tools to consolidate some
  repeat code. Each has a version bump to reflect this minor refactoring
  effort.

Bug Fixes:

• eyaml-rotate-keys was broken by the refactoring for 2.0.0. eyaml-rotate-keys
  v1.0.2 restores functionality.

Enhancements:

• Command-line tools are now managed via pip as entry_points/console_scripts
  rather than external binaries. This enables superior cross-platform
  compatibility as well as unit testing. As such, all of the CLI tools have
  been updated pursuant to (generally trivial, excepting eyaml-rotate-keys)
  issues discovered during their newfound CI tests.

Bug Fixes:

• yaml-set v1.0.4 lost track of EYAML block formatting between read and write,
  causing replacement values to use unexpected formatting. This is fixed in
  yaml-set v.1.0.5.

Enhancements:

• Added Collectors to YAML Path expressions. These take the form of "(YAML
  Path)" -- parenthesis () are used to demarcate each Collector -- resulting in
  a list of zero or more matches for the sub-query. Operators between
  Collectors include + and -, like "(...)+(...)", "(...)-(...)", and nested
  Collectors are possible, like "(...)-((...)+(...))". Collectors may appear
  anywhere within the outer YAML Path, effectively setting the point within the
  data at which each Collector is rooted.
• A major code refactoring was undertaken to break YAMLPath out as its own class
  and improve code quality (per mypy and pylint).

Bug Fixes:

• yaml-set v1.0.4 now implies --mustexist when --saveto is set. Otherwise,
  --saveto was trying to save nothing when --change (without --mustexist)
  pointed nowhere.

Bug Fixes:

• yaml-set v1.0.3 no longer requires --privatekey unless decryption is requested
  via enabling --check. As a side-effect, the script will also no longer ignore
  requests to set the same value as was already set.