hcl

Enhancements

• feat: return an ExprSyntaxError for invalid references that end in a dot (#692)

Enhancements

• Introduce ParseTraversalPartial, which allows traversals that include the splat ([*]) index operator. (#673)
• ext/dynblock: Now accepts marked values in for_each, and will transfer those marks (as much as technically possible) to values in the generated blocks. (#679)

Bugs Fixed

• Expression evaluation will no longer panic if the splat operator is applied to an unknown value that has cty marks. (#678)

Bugs Fixed

• Return ExprSyntaxError when an invalid namespaced function is encountered during parsing (#668)

Internal

• Standardize on only two value dumping/diffing libraries (#669)

What's Changed

• Namespaced functions by @jbardin in https://github.com/hashicorp/hcl/pull/639
• fix: fix NameRange for namespaced functions by @ansgarm in https://github.com/hashicorp/hcl/pull/651
• prefer iterator error over for_each by @DanielMSchmidt in https://github.com/hashicorp/hcl/pull/656
• Create an error type for unknown function diags by @jbardin in https://github.com/hashicorp/hcl/pull/657
• Typo fix in spec.md by @elliotbonneville in https://github.com/hashicorp/hcl/pull/652
• hclwrite: Fix formatting of namespaced functions by @radeksimko in https://github.com/hashicorp/hcl/pull/658
• Run unit tests on macOS by @radeksimko in https://github.com/hashicorp/hcl/pull/660
• Fix license headers by @radeksimko in https://github.com/hashicorp/hcl/pull/659
• Add make targets for common checks + fixes by @radeksimko in https://github.com/hashicorp/hcl/pull/661
• Use new make targets in CI by @radeksimko in https://github.com/hashicorp/hcl/pull/663

New Contributors

• @ansgarm made their first contribution in https://github.com/hashicorp/hcl/pull/651
• @DanielMSchmidt made their first contribution in https://github.com/hashicorp/hcl/pull/656
• @elliotbonneville made their first contribution in https://github.com/hashicorp/hcl/pull/652

Full Changelog: https://github.com/hashicorp/hcl/compare/v2.19.1...v2.20.0

What's Changed

• hcldec must use WithoutOptionalAttributesDeep by @jbardin in https://github.com/hashicorp/hcl/pull/635

Full Changelog: https://github.com/hashicorp/hcl/compare/v2.19.0...v2.19.1

Bugs Fixed

• hclsyntax: Conditional expressions will no longer panic when one or both of their results are "marked", as is the case for situations like how HashiCorp Terraform tracks its concept of "sensitive values". (#630)

Enhancements

• HCL now uses the tables from Unicode 15 when performing string normalization and character segmentation. HCL was previously using the Unicode 13 tables.

  For calling applications where consistent Unicode support is important, consider also upgrading to Go 1.21 at the same time as adopting HCL v2.18.0 so that the standard library unicode tables (used for case folding, etc) will also be from Unicode 15.

Enhancements

• hclsyntax: When evaluating string templates that have a long known constant prefix, HCL will truncate the known prefix to avoid creating excessively-large refinements. String prefix refinements are intended primarily for relatively-short fixed prefixes, such as https:// at the start of a URL known to use that scheme. (#617)
• ext/tryfunc: The "try" and "can" functions now handle unknown values slightly more precisely, and so can return known values in more situations when given expressions referring to unknown symbols. (#622)

Bugs Fixed

• ext/typeexpr: Will no longer try to refine unknown values of unknown type when dealing with a user-specified type constraint containing the any keyword, avoiding an incorrect panic at runtime. (#625)
• ext/typeexpr: Now correctly handles attempts to declare the same object type attribute multiple times by returning an error. Previously this could potentially panic by creating an incoherent internal state. (#624)

Enhancements

• HCL now uses a newer version of the upstream cty library which has improved treatment of unknown values: it can now track additional optional information that reduces the range of an unknown value, which allows some operations against unknown values to return known or partially-known results. (#590)

  Note: This change effectively passes on cty's notion of backward compatibility whereby unknown values can become "more known" in later releases. In particular, if your caller is using cty.Value.RawEquals in its tests against the results of operations with unknown values then you may see those tests begin failing after upgrading, due to the values now being more "refined".

  If so, you should review the refinements with consideration to the cty refinements docs and update your expected results to match only if the reported refinements seem correct for the given situation. The RawEquals method is intended only for making exact value comparisons in test cases, so main application code should not use it; use Equals instead for real logic, which will take refinements into account automatically.

Bugs Fixed

• ext/typeexpr: Verify type assumptions when applying default values, and ignore input values that do not match type assumptions. (#594)

Bugs Fixed

• hclsyntax: Report correct Range.End for FunctionCall with incomplete argument (#588)

Enhancements

• ext/typeexpr: Modify the Defaults functionality to implement additional flexibility. HCL will now upcast lists and sets into tuples, and maps into objects, when applying default values if the applied defaults cause the elements within a target collection to have differing types. Previously, this would have resulted in a panic, now HCL will return a modified overall type. (https://github.com/hashicorp/hcl/pull/574)

  Users should return to the advice provided by v2.14.0, and apply the go-cty convert functionality after setting defaults on a given cty.Value, rather than before.

• hclfmt: Avoid rewriting unchanged files. (https://github.com/hashicorp/hcl/pull/576)

• hclsyntax: Simplify the AST for certain string expressions. (https://github.com/hashicorp/hcl/pull/584)

Bugs Fixed

• hclwrite: Fix data race in formatSpaces. (https://github.com/hashicorp/hcl/pull/511)

Bugs Fixed

• ext/typeexpr: Skip null objects when applying defaults. This prevents crashes when null objects are creating inside collections, and stops incomplete objects being created with only optional attributes set. (#567)
• ext/typeexpr: Ensure default values do not have optional metadata attached. This prevents crashes when default values are inserted into concrete go-cty values that have also been stripped of their optional metadata. (#568)

Enhancements

• ext/typeexpr: With the go-cty upstream depenendency updated to v1.12.0, the Defaults struct and associated functions can apply additional and more flexible 'unsafe' conversions (examples include tuples into collections such as lists and sets, and additional safety around null and dynamic values). (#564)
• ext/typeexpr: With the go-cty upstream depenendency updated to v1.12.0, users should now apply the go-cty convert functionality before setting defaults on a given cty.Value, rather than after, if they require a specific cty.Type. (#564)

Bugs Fixed

• ext/typeexpr: Type convert defaults for optional object attributes when applying them. This prevents crashes in certain cases when the objects in question are part of a collection. (#555)

Enhancements

• ext/typeexpr: Added support for optional object attributes to TypeConstraint. Attributes can be wrapped in the special optional(…) modifier, allowing the attribute to be omitted while still meeting the type constraint. For more information, cty's documentation on conversion between object types. (#549)
• ext/typeexpr: New function: TypeConstraintWithDefaults. In this mode, the optional(…) modifier accepts a second argument which can be used as the default value for omitted object attributes. The function returns both a cty.Type and associated Defaults, the latter of which has an Apply method to apply defaults to a given value. (#549)

Enhancements

• hclsyntax: Evaluation of conditional expressions will now produce more precise error messages about inconsistencies between the types of the true and false result expressions, particularly in cases where both are of the same structural type kind but differ in their nested elements. (#530)
• hclsyntax: The lexer will no longer allocate a small object on the heap for each token. Instead, in that situation it will allocate only when needed to return a diagnostic message with source location information. (#490)
• hclwrite: New functions TokensForTuple, TokensForObject, and TokensForFunctionCall allow for more easily constructing the three constructs which are supported for static analysis and which HCL-based languages typically use in contexts where an expression is used only for its syntax, and not evaluated to produce a real value. For example, these new functions together are sufficient to construct all valid type constraint expressions from the Type Expressions Extension, which is the basis of variable type constraints in the Terraform language at the time of writing. (#502)
• json: New functions IsJSONExpression and IsJSONBody to determine if a given expression or body was created by the JSON syntax parser. In normal situations it's better not to worry about what syntax a particular expression/body originated in, but this can be useful in some trickier cases where an application needs to shim for backwards-compatibility or for static analysis that needs to have special handling of the JSON syntax's embedded expression/template conventions. (#524)

Bugs Fixed

• gohcl: Fix docs about supported types for blocks. (#507)

Bugs Fixed

• hclsyntax: The type for an upgraded unknown value with a splat expression cannot be known (#495)

v2.11.0

Enhancements

• hclsyntax: Various error messages related to unexpectedly reaching end of file while parsing a delimited subtree will now return specialized messages describing the opening tokens as "unclosed", instead of returning a generic diagnostic that just happens to refer to the empty source range at the end of the file. This gives better feedback when error messages are being presented alongside a source code snippet, as is common in HCL-based applications, because it shows which innermost container the parser was working on when it encountered the error. (#492)

Bugs Fixed

• hclsyntax: Upgrading an unknown single value to a list using a splat expression must return unknown (#493)

v2.10.1 (July 21, 2021)

• dynblock: Decode unknown dynamic blocks in order to obtain any diagnostics even though the decoded value is not used (#476)
• hclsyntax: Calling functions is now more robust in the face of an incorrectly-implemented function which returns a function.ArgError whose argument index is out of range for the length of the arguments. Previously this would often lead to a panic, but now it'll return a less-precice error message instead. Functions that return out-of-bounds argument indices still ought to be fixed so that the resulting error diagnostics can be as precise as possible. (#472)
• hclsyntax: Ensure marks on unknown values are maintained when processing string templates. (#478)
• hcl: Improved error messages for various common error situtions in hcl.Index and hcl.GetAttr. These are part of the implementation of indexing and attribute lookup in the native syntax expression language too, so the new error messages will apply to problems using those operators. (#474)

v2.10.0 (April 20, 2021)

Enhancements

• dynblock,hcldec: Using dynblock in conjunction with hcldec can now decode blocks with unknown dynamic for_each arguments as entirely unknown values (#461)
• hclsyntax: Some syntax errors during parsing of the inside of ${ ... } template interpolation sequences will now produce an extra hint message about the need to escape as $${ when trying to include interpolation syntax for other languages like shell scripting, AWS IAM policies, etc. (#462)