fix(hcl): quote string values in HCL output - #2791
Open
ankit090701 wants to merge 1 commit into
Open
Conversation
The HCL encoder used valueNode.Style == 0 as a signal for "this is a
raw, unquoted HCL expression" (an identifier traversal or function
call to preserve on roundtrip, e.g. var.foo or upper(message)).
But Style == 0 is also the default zero-value for any ordinary string
node that isn't explicitly marked otherwise, including:
- strings set via a yq expression against non-HCL input, e.g.
`yq -n -ohcl '.zone = "us-east1-b"'`
- strings decoded from other formats (JSON/YAML) and re-encoded to
HCL, since those formats have no concept of "unquoted expression"
Both are symptoms of the same defect reported in issue mikefarah#2594. As a
result, plain string literals were emitted without quotes, producing
invalid/non-round-trippable HCL.
Fix: introduce EncodeHintRawExpression, an explicit hint set only by
the HCL decoder on the specific node kinds that must stay unquoted to
preserve round trip (ScopeTraversalExpr, FunctionCallExpr, and
unparseable expressions with source text). The encoder now only
treats a string as a raw expression when this hint is present, and
quotes everything else via the existing cty.Value path. This also
removes a Style&LiteralStyle branch and a dead isValidHCLIdentifier
check that suffered from the same Style==0 conflation and could not
produce valid output anyway (tokensForRawHCLExpr doesn't handle
multi-line literal-style content).
Fixes mikefarah#2594
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
-ohcldoesn't quote string values:$ yq -n -ohcl '.zone = "us-east1-b"' zone = us-east1-bThis produces invalid/non-round-trippable HCL (
zone = us-east1-bis a reference to an undefined identifier, not the string"us-east1-b").Root cause
hclEncoder.encodeAttributeusedvalueNode.Style == 0as the signal for "this is a raw, unquoted HCL expression that should be preserved verbatim" (e.g. an identifier traversal likevar.foo, or a function call likeupper(message), round-tripped from HCL source). ButStyle == 0is also just the default zero-value for any ordinary string node that wasn't explicitly given another style, including:-n -ohcl '.zone = "us-east1-b"'kms.tfround-trip example in the comments on HCL output doesn't quote strings #2594)So plain string literals were systematically misidentified as raw expressions and emitted unquoted.
There was also a second, independent branch (
valueNode.Style&LiteralStyle != 0) with the same problem, and a third branch (isValidHCLIdentifier(...) && valueNode.Style == 0) that was actually dead code (unreachable, since theStyle == 0branch above it always returns first) — but would have become live and reintroduced this exact bug for identifier-shaped strings (likeus-east1-b) once the first branch was fixed, so I removed it too.Fix
Introduced
EncodeHintRawExpression, an explicit hint (reusing the existing genericEncodeHintmechanism already used elsewhere in this file for HCL block-encoding decisions) set only by the HCL decoder on the specific expression kinds that must stay unquoted to round-trip correctly:ScopeTraversalExpr(identifier/variable references),FunctionCallExpr, and the fallback case for expressions it can't parse but can still evaluate/preserve as source text.The encoder now only takes the "raw, unquoted" path when this hint is explicitly set. Every other string goes through the existing
cty.Valuepath, which always quotes.Test plan
go build ./...go test ./pkg/yqlib/...— all existing HCL tests pass unchangedhclFormatScenariosinhcl_test.go:web_proxy) set via expression is still quoted (covers the now-removed dead-code branch)var.foo-style variable references still round-trip unquoted through pure HCL→HCL (no regression to the intended raw-expression behavior)kms.tfround-trip via-ojson/-pjsonnow quotesdescription,source, andversioncorrectly (this necessarily also quotessome_attr = var.fooonce it round-trips through JSON, since JSON has no way to represent "this is a raw expression, not a string" — as noted in the issue thread, this is an inherent limitation of that intermediate format, not something this fix can preserve; direct HCL→HCL round-trips ofvar.fooare unaffected)go test ./...suite and found 15 pre-existing failures on my Windows dev machine (symlink test, and severalcmdpackage tests hardcoding/bin/echoetc.) — these reproduce identically on unmodifiedmasterand are unrelated to this change (confirmed viagit stash).Fixes #2594