Skip to content

fix(resolve): a self-reference with nothing below reads the environment - #31

Merged
lejmr merged 2 commits into
mainfrom
conformance/self-reference-reads-env
Oct 25, 2026
Merged

lejmr merged 2 commits into
mainfrom
conformance/self-reference-reads-env

Conversation

@lejmr

@lejmr lejmr commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

HOME : ${HOME}, with HOME set, was Unresolved. The spec says otherwise:

Here the ${foo} self-reference comes before foo has a value, so it is undefined, exactly as if the substitution referenced a path not found in the document. (examples-of-self-referential-substitutions.6)

A path not found in the document is looked up in the environment (substitution-fallback-to-environment-variables.1, substitutions.13). For ${?foo} the spec says the same thing: "as if it referred to a nonexistent value" (self-referential-substitutions.13). The blocking rule (substitution-fallback-to-environment-variables.4) is about a value the lookup finds, such as HOME : null. A self-reference with nothing below it has no such value. The spec's author said in lightbend/config#652 that it should work, and pyhocon has done it since chimpler/pyhocon#213. typesafe/config reports a cycle, which .7 rules out ("undefined" rather than "intractable cycle").

Behaviour

input, HOME set before after typesafe/config pyhocon
HOME : ${HOME} Unresolved "/home/…" cycle error "/home/…"
HOME : ${?HOME} {} "/home/…" {} "/home/…"
HOME : ${HOME}"/bin" Unresolved "/home/…/bin" cycle error "/home/…/bin"
HOME += x [x] UnsupportedConcatenation [x] "/home/… x"

+= is HOME = ${?HOME} [x], so the variable's text meets a list and the result is an error. A value set earlier, null included, still wins, and with --env=none nothing changes.

Code

  • Found.gone is left for an optional that hit a cycle (env.an optional cycle never reads the variable is unchanged). A self-reference with nothing below is .absent, so it goes on to the variable. With --env=keep it waits for one.
  • A kept self-reference from an include now always falls back to the path as written, so keep.inDocument goes. stringify's row 10 used to be a known loss and now round-trips.
  • The tests that pinned the old rule in env.zig, keep.zig, Include.zig and stringify.zig are rewritten, with the reason in their comments.

Conformance

  • substitution-fallback-to-environment/009 (${X}, java: unsupported) and 010 (${?X}, java: diverges).
  • Measured again on the 478, after a restack onto fix(parse): a value that only starts like a number is text #30: zig-hocon (spec 478, java 419), typesafe/config 1.4.9 (spec 419, java 478), pyhocon (spec 337, java 308). hocon2, pulse-pyhocon, py.hocon and rs.hocon aren't installed here and stay marked as behind.
  • FINDINGS.md, a CHANGELOG line, and "all 478" in the READMEs and MIGRATING.md, pyhocon 337.

Checks: zig build test passes, zig build test-python is at 100 %, zig fmt --check src is clean, and fill-expected.py --check reports 478 cases with 0 sidecars stale.

The case numbers may need to move when #28/#29/#30 land first. None of them touches this section.

lejmr and others added 2 commits October 25, 2026 10:48
HOME : ${HOME} with HOME set was Unresolved. The spec says a self-reference
that comes before its field has a value "is undefined, exactly as if the
substitution referenced a path not found in the document"
(examples-of-self-referential-substitutions.6), and such a path is searched
for in the environment (substitution-fallback-to-environment-variables.1).
The spec's author said so in lightbend/config#652; pyhocon does it since
chimpler/pyhocon#213. typesafe/config reports a cycle, which .7 rules out.

Found.gone is left for an optional that hit a cycle; a self-reference with
nothing below is absent, so it goes on to the variable, and kept with
--env=keep it waits for one. A kept self-reference from an include always
falls back to the path as written, so inDocument goes. The tests that pinned
the old rule are rewritten, the reason in their comments.

Conformance: substitution-fallback-to-environment/009 and 010, Java and
pyhocon measured on the 448.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J2cA6s3DDWV6BtSwCUZMuG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CbBo1aotFTDLY2gery7qYD
@lejmr
lejmr force-pushed the conformance/self-reference-reads-env branch from 06dcf00 to 81ada01 Compare October 25, 2026 09:48
@lejmr
lejmr merged commit 21d66cd into main Oct 25, 2026
42 checks passed
@lejmr
lejmr deleted the conformance/self-reference-reads-env branch October 25, 2026 13:32
lejmr added a commit that referenced this pull request Oct 25, 2026
The tokenizer took every 0 byte for the sentinel past the input. In a comment that ended the file without an error: `a = 1 # note<NUL>` dropped every line below, and a UTF-16 file without a byte order mark read as {}. Only the NUL at input.len ends the input now; one before it is ignored in a comment, kept in unquoted and triple-quoted text, and rejected in a quoted string, as any control character (comments.1, unquoted-strings.14-15, multi-line-strings.1).

- four conformance cases: comments/015, unquoted-strings/031-032, multi-line-strings/010; typesafe/config does all four
- the oracle protocol carries a NUL as \0, since the adapter's shell variable dropped it and the oracles never saw one
- the CHANGELOG drops a sentence #31 left in twice

zig-hocon passes all 482, typesafe/config 423 against the spec, pyhocon 340.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CbBo1aotFTDLY2gery7qYD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant