Skip to content

fix(hcl): quote string values in HCL output - #2791

Open
ankit090701 wants to merge 1 commit into
mikefarah:masterfrom
ankit090701:fix/hcl-encoder-quote-strings
Open

fix(hcl): quote string values in HCL output#2791
ankit090701 wants to merge 1 commit into
mikefarah:masterfrom
ankit090701:fix/hcl-encoder-quote-strings

Conversation

@ankit090701

@ankit090701 ankit090701 commented Jul 31, 2026

Copy link
Copy Markdown

Summary

-ohcl doesn't quote string values:

$ yq -n -ohcl '.zone = "us-east1-b"'
zone = us-east1-b

This produces invalid/non-round-trippable HCL (zone = us-east1-b is a reference to an undefined identifier, not the string "us-east1-b").

Root cause

hclEncoder.encodeAttribute used valueNode.Style == 0 as the signal for "this is a raw, unquoted HCL expression that should be preserved verbatim" (e.g. an identifier traversal like var.foo, or a function call like upper(message), round-tripped from HCL source). But Style == 0 is also just the default zero-value for any ordinary string node that wasn't explicitly given another style, including:

  • strings set via a yq expression against non-HCL input, e.g. -n -ohcl '.zone = "us-east1-b"'
  • strings decoded from JSON/YAML and re-encoded to HCL, since those formats have no notion of "unquoted expression" (see the kms.tf round-trip example in the comments on HCL output doesn't quote strings #2594)

So plain string literals were systematically misidentified as raw expressions and emitted unquoted.

There was also a second, independent branch (valueNode.Style&LiteralStyle != 0) with the same problem, and a third branch (isValidHCLIdentifier(...) && valueNode.Style == 0) that was actually dead code (unreachable, since the Style == 0 branch above it always returns first) — but would have become live and reintroduced this exact bug for identifier-shaped strings (like us-east1-b) once the first branch was fixed, so I removed it too.

Fix

Introduced EncodeHintRawExpression, an explicit hint (reusing the existing generic EncodeHint mechanism already used elsewhere in this file for HCL block-encoding decisions) set only by the HCL decoder on the specific expression kinds that must stay unquoted to round-trip correctly: ScopeTraversalExpr (identifier/variable references), FunctionCallExpr, and the fallback case for expressions it can't parse but can still evaluate/preserve as source text.

The encoder now only takes the "raw, unquoted" path when this hint is explicitly set. Every other string goes through the existing cty.Value path, which always quotes.

Test plan

  • go build ./...
  • go test ./pkg/yqlib/... — all existing HCL tests pass unchanged
  • Added 3 regression cases to hclFormatScenarios in hcl_test.go:
    • a plain string set via expression is quoted (direct repro of HCL output doesn't quote strings #2594)
    • an identifier-shaped string (e.g. web_proxy) set via expression is still quoted (covers the now-removed dead-code branch)
    • var.foo-style variable references still round-trip unquoted through pure HCL→HCL (no regression to the intended raw-expression behavior)
  • Manually verified the exact repros from the issue and its follow-up comment:
    $ yq -n -ohcl '.zone = "us-east1-b"'
    zone = "us-east1-b"
    and the kms.tf round-trip via -ojson/-pjson now quotes description, source, and version correctly (this necessarily also quotes some_attr = var.foo once it round-trips through JSON, since JSON has no way to represent "this is a raw expression, not a string" — as noted in the issue thread, this is an inherent limitation of that intermediate format, not something this fix can preserve; direct HCL→HCL round-trips of var.foo are unaffected)
  • Note: I ran the full go test ./... suite and found 15 pre-existing failures on my Windows dev machine (symlink test, and several cmd package tests hardcoding /bin/echo etc.) — these reproduce identically on unmodified master and are unrelated to this change (confirmed via git stash).

Fixes #2594

The HCL encoder used valueNode.Style == 0 as a signal for "this is a
raw, unquoted HCL expression" (an identifier traversal or function
call to preserve on roundtrip, e.g. var.foo or upper(message)).
But Style == 0 is also the default zero-value for any ordinary string
node that isn't explicitly marked otherwise, including:

  - strings set via a yq expression against non-HCL input, e.g.
    `yq -n -ohcl '.zone = "us-east1-b"'`
  - strings decoded from other formats (JSON/YAML) and re-encoded to
    HCL, since those formats have no concept of "unquoted expression"

Both are symptoms of the same defect reported in issue mikefarah#2594. As a
result, plain string literals were emitted without quotes, producing
invalid/non-round-trippable HCL.

Fix: introduce EncodeHintRawExpression, an explicit hint set only by
the HCL decoder on the specific node kinds that must stay unquoted to
preserve round trip (ScopeTraversalExpr, FunctionCallExpr, and
unparseable expressions with source text). The encoder now only
treats a string as a raw expression when this hint is present, and
quotes everything else via the existing cty.Value path. This also
removes a Style&LiteralStyle branch and a dead isValidHCLIdentifier
check that suffered from the same Style==0 conflation and could not
produce valid output anyway (tokensForRawHCLExpr doesn't handle
multi-line literal-style content).

Fixes mikefarah#2594
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant