### Terraform Version ```shell Terraform v1.18.0-dev on darwin_arm64 ``` (Built from `main` at `f8e7458f5`. The code path is unchanged since the function was introduced, so released versions behave the same.) ### Terraform Configuration Files ```terraform output "encoded" { # A string that contains U+FFFD REPLACEMENT CHARACTER. value = textencodebase64("caf�", "UTF-8") } output "round_trip" { value = textdecodebase64(textencodebase64("caf�", "UTF-8"), "UTF-8") } ``` ### Debug Output Reproduced in `terraform console`, one expression per invocation: ```console $ echo 'textencodebase64("caf�", "UTF-8")' | terraform console "Y2Fm77+9" $ echo 'textdecodebase64("Y2Fm77+9", "UTF-8")' | terraform console ╷ │ Error: Invalid function argument │ │ on line 1: │ (source code not available) │ │ Invalid value for "source" parameter: the given string contains symbols │ that are not defined for UTF-8. ╵ $ echo 'base64decode("Y2Fm77+9")' | terraform console "caf�" ``` ### Expected Behavior `textdecodebase64("Y2Fm77+9", "UTF-8")` returns `"caf�"`. `Y2Fm77+9` is the base64 of `63 61 66 EF BF BD` — four well-formed UTF-8 characters, the last being U+FFFD REPLACEMENT CHARACTER. Nothing about it is undefined for UTF-8, and `textencodebase64` produced that exact string one line earlier. `base64decode`, which takes the same bytes and also requires valid UTF-8, returns the string without complaint. ### Actual Behavior The call fails with *"the given string contains symbols that are not defined for UTF-8"*, so: - `textencodebase64` and `textdecodebase64` do not round-trip for any string that contains U+FFFD; - the two sibling functions disagree on the same input — `base64decode` accepts it, `textdecodebase64` rejects it; - the error message is inaccurate: U+FFFD is defined for UTF-8, and it names the wrong cause, so a config author has nothing to act on. ### Steps to Reproduce 1. `echo 'textdecodebase64("Y2Fm77+9", "UTF-8")' | terraform console` ### Additional Context The cause is in `internal/lang/funcs/encoding.go`: ```go decoder := encoding.NewDecoder() decoded, err := decoder.Bytes(sDec) if err != nil || bytes.ContainsRune(decoded, '�') { return cty.UnknownVal(cty.String), function.NewArgErrorf(0, "the given string contains symbols that are not defined for %s", encName) } ``` The `ContainsRune` check exists for a good reason: `golang.org/x/text` decoders substitute U+FFFD for bytes they cannot decode rather than returning an error, so its presence is the only signal that something was lost. But the check cannot tell a substituted U+FFFD from one the source actually contained, and rejects both. Where this shows up in practice is text that has already been through a lossy decode once — a file read with the wrong encoding, a column of mojibake, captured log output. Passing such a value through `textencodebase64`/`textdecodebase64` fails, and the message points at the encoding rather than at the character. The existing test at `internal/lang/funcs/encoding_test.go` for `gQ==` with `windows-1250` (byte `0x81`, genuinely undefined there) is the case the check is meant to catch, and it should keep failing. A decode is faithful exactly when encoding the result again reproduces the source bytes: a substituted U+FFFD does not reproduce the byte it replaced, while one that was really in the source does. That distinguishes the two without weakening the existing rejection. I have that change working with tests and will open a PR referencing this issue. ### References - `internal/lang/funcs/encoding.go` — `TextDecodeBase64Func` - `internal/lang/funcs/encoding_test.go` — `TestBase64TextDecode` --- Transparency, per the AI Usage section of CONTRIBUTING.md: this report was prepared with AI assistance. The reproduction above was run against a binary built from `main`, and the output is copied verbatim from that run.