fix(resolve): a self-reference with nothing below reads the environment

CHANGELOG.md

@@ -16,6 +16,7 @@ What changed for someone using zig-hocon: the Zig package, the `hocon` command l
 
 - An object after a substitution, `top = ${x} { … }`, is a merge, the same as `top = ${x}` and `top { … }` written as two fields. A `+=` inside it appends to the list `${x}` gives (`top = ${x} { arr += four }` was `[four]`, is `[one, four]`), a self-reference looks back at it, and `${top.n}` inside it sees the merged object instead of reporting a cycle. Eight new conformance cases pin it.
 - Layers of a key are merged before they are resolved, as the spec has it: in `p : { a : ${y} }`, `p : ${x}` with `x.a` set, `${y}` is hidden and no longer evaluated (it was an error), and a reference in the earlier object sees the final value. Three more conformance cases, all 457 pass.
+- A field that refers to itself with nothing before it reads the environment variable of its own name, as the spec has it: `HOME : ${HOME}` is the variable `HOME`, where it was `Unresolved`, and `HOME : ${?HOME}` creates the field instead of dropping it, as pyhocon does. A value set earlier still wins, and `HOME += x` with `HOME` set is now an error: `+=` is `HOME = ${?HOME} [x]`, text and a list. Two new conformance cases, all 478 pass. Two new conformance cases pin it.
 
 ## [0.5.2] - 2026-10-23
 

README.md

@@ -48,7 +48,7 @@ The rest of this page reads these two files.
 
 ## Why zig-hocon
 
-- **Correct.** Passes all 476 tests in the [conformance suite](https://htmlpreview.github.io/?https://github.com/lejmr/zig-hocon/blob/main/conformance/report.html), one test for each sentence of the spec.
+- **Correct.** Passes all 478 tests in the [conformance suite](https://htmlpreview.github.io/?https://github.com/lejmr/zig-hocon/blob/main/conformance/report.html), one test for each sentence of the spec.
 - **Your schema is a Zig struct.** HOCON is only what sits on disk; the struct says what it holds, with defaults, optionals, enums, durations and sizes.
 - **Fast.** A 10 MB config in a quarter of a second.\*
 - **Clear errors.** File, line, column and a caret, also for a value of the wrong type.
@@ -210,13 +210,13 @@ b.step("check-config", "Check the config against Config").dependOn(&check.step);
 
 | implementation | against the spec | against typesafe/config 1.4.9 |
 |---|---|---|
-| Java <sub>typesafe/config 1.4.9</sub> | 88.0% (419/476) | 100% (476/476) |
-| hocon2 <sub>hocon2 0.6.3</sub> | 81.1% (386/476) | 75.2% (358/476) |
-| pulse-pyhocon <sub>pulse-pyhocon 0.2.1</sub> | 66.6% (317/476) | 62.8% (299/476) |
-| py.hocon <sub>hocon-parser 1.14.2 (Python)</sub> | 83.6% (398/476) | 81.3% (387/476) |
-| pyhocon <sub>pyhocon 0.3.63</sub> | 70.4% (335/476) | 64.7% (308/476) |
-| rs.hocon <sub>hocon-parser 1.14.2 (Rust)</sub> | 83.6% (398/476) | 80.7% (384/476) |
-| zig-hocon <sub>zig-hocon code@a40d962</sub> | 100% (476/476) | 88.0% (419/476) |
+| Java <sub>typesafe/config 1.4.9</sub> | 87.7% (419/478) | 100% (478/478) |
+| hocon2 <sub>hocon2 0.6.3</sub> | 80.8% (386/478) | 74.9% (358/478) |
+| pulse-pyhocon <sub>pulse-pyhocon 0.2.1</sub> | 66.3% (317/478) | 62.6% (299/478) |
+| py.hocon <sub>hocon-parser 1.14.2 (Python)</sub> | 83.3% (398/478) | 81.0% (387/478) |
+| pyhocon <sub>pyhocon 0.3.63</sub> | 70.5% (337/478) | 64.4% (308/478) |
+| rs.hocon <sub>hocon-parser 1.14.2 (Rust)</sub> | 83.3% (398/478) | 80.3% (384/478) |
+| zig-hocon <sub>zig-hocon code@5bdce02</sub> | 100% (478/478) | 87.7% (419/478) |
 
 The table keeps the best parsers; the others measured, and why they are not in it: [conformance/README.md](conformance/README.md#not-in-the-table).
 

conformance/maintaining/COVERAGE.html

@@ -520,7 +520,7 @@ <h4>Self-Referential Substitutions</h4>
 marker value during resolution.</p>
 <p data-line="714"><mark class="ok" title="self-referential-substitutions.12 — self-referential-examples/025-optional-two-field-cycle-is-undefined.conf">Cycles should be treated the same as a missing value when
 resolving an optional substitution (i.e. the `${?foo}` syntax).</mark>
-<mark class="ok" title="self-referential-substitutions.13 — self-referential-examples/004-optional-self-reference-disappears-silently.conf">If `${?foo}` refers to itself then it&#x27;s as if it referred to a
+<mark class="ok" title="self-referential-substitutions.13 — self-referential-examples/004-optional-self-reference-disappears-silently.conf, substitution-fallback-to-environment/010-optional-self-reference-reads-env.conf">If `${?foo}` refers to itself then it&#x27;s as if it referred to a
 nonexistent value.</mark></p>
 <h4>The `+=` field separator</h4>
 <p data-line="721"><mark class="ok" title="the--field-separator.1 — plus-equals-field-separator/001-plus-equals-desugars-to-optional-self-ref-array.conf">Fields may have `+=` as a separator rather than `:` or `=`.</mark> <mark class="ok" title="the--field-separator.2 — plus-equals-field-separator/005-plus-equals-in-nested-object-with-substitution.conf, plus-equals-field-separator/007-plus-equals-under-a-dotted-key.conf, plus-equals-field-separator/008-plus-equals-with-an-object-containing-a-merge.conf, plus-equals-field-separator/009-appends-through-object-concatenation.conf, plus-equals-field-separator/010-appends-through-chained-object-concatenations.conf, plus-equals-field-separator/011-appends-deep-inside-object-concatenation.conf, plus-equals-field-separator/012-appends-to-a-list-defined-in-object-concatenation.conf, self-referential-examples/023-append-does-not-hide.conf">A
@@ -552,7 +552,7 @@ <h4>Examples of Self-Referential Substitutions</h4>
 <p data-line="761">    foo : ${foo}</p>
 <p data-line="763">and then second:</p>
 <p data-line="765">    foo : { a : 1 }</p>
-<p data-line="767"><mark class="ok" title="examples-of-self-referential-substitutions.6 — self-referential-examples/003-self-reference-before-any-value-is-undefined.conf">Here the `${foo}` self-reference comes before `foo` has a value,
+<p data-line="767"><mark class="ok" title="examples-of-self-referential-substitutions.6 — self-referential-examples/003-self-reference-before-any-value-is-undefined.conf, substitution-fallback-to-environment/009-self-reference-reads-env.conf">Here the `${foo}` self-reference comes before `foo` has a value,
 so it is undefined, exactly as if the substitution referenced a
 path not found in the document.</mark></p>
 <p data-line="771"><mark class="ok" title="examples-of-self-referential-substitutions.7 — self-referential-examples/004-optional-self-reference-disappears-silently.conf">Because `foo : ${foo}` conceptually looks to previous definitions
@@ -1143,7 +1143,7 @@ <h3>Conventional override by system properties</h3>
 settings found in the configuration file. This supports specifying
 config options on the command line.</p>
 <h3>Substitution fallback to environment variables</h3>
-<p data-line="1540"><mark class="ok" title="substitution-fallback-to-environment-variables.1 — substitution-fallback-to-environment/002-required-substitution-resolved-from-env.conf, substitution-fallback-to-environment/003-config-value-wins-over-env.conf">Recall that if a substitution is not present (not even set to
+<p data-line="1540"><mark class="ok" title="substitution-fallback-to-environment-variables.1 — substitution-fallback-to-environment/002-required-substitution-resolved-from-env.conf, substitution-fallback-to-environment/003-config-value-wins-over-env.conf, substitution-fallback-to-environment/009-self-reference-reads-env.conf, substitution-fallback-to-environment/010-optional-self-reference-reads-env.conf">Recall that if a substitution is not present (not even set to
 `null`) within a configuration tree, implementations may search
 for it from external sources.</mark> <mark class="ok" title="substitution-fallback-to-environment-variables.2 — substitution-fallback-to-environment/001-env-fallback-for-undefined-path.conf">One such source could be environment
 variables.</mark></p>

conformance/maintaining/FINDINGS.md

@@ -11,6 +11,7 @@ What the validation pass turned up and did **not** fix. The divergences it was c
 - `plus-equals-field-separator/009`–`012`, `self-referential-examples/027`–`030` — an object concatenated over a substitution, `top : ${x} { ... }`, is a merge (array-and-object-concatenation.6), the same as `top : ${x}` and `top { ... }` written as two fields. So a self-reference inside it, `+=` or the long form, looks back at the field the substitution gives (examples-of-self-referential-substitutions.3), and any other reference to `top.…` inside looks forward and sees the merged object (.14, .16). typesafe/config drops the inherited element of a `+=` (lightbend/config#375, #608; a fix is proposed in #882) and reports the rest as cycles (#586, #728); zig-hocon got the two-field form right and the one-value form wrong.
 - `self-referential-examples/031`–`033` — layers are merged before they are resolved: an object a substitution gives on top hides a field of the object below it, which is then never evaluated (examples-of-self-referential-substitutions.8; lightbend/config#838, and #812 for the same inside `${base} { ... }`), and a reference in the object below looks forward at the merged value (.16). zig-hocon resolved the layer below first: it evaluated the hidden field and saw the old value.
 - Bug reports in lightbend/config and ekrich/sconfig where the spec decides and no case was: an internal error in typesafe/config inside `${a} { ... }` and before a list substitution (`array-and-object-concatenation/020`–`022`), integers past 2^63 as strings and `\u` escapes with a sign or non-ASCII digits accepted (`unchanged-from-json/028`–`030`), and pyhocon's slips: a substituted number's text (`string-value-concatenation/023`), `+=` across an include (`include-merging/016`), an object over a string hidden by a later one (`self-referential-examples/034`), an include two objects deep (`include-substitution/020`), `+=` after `c = ${d}` (`plus-equals-field-separator/013`), and an undefined optional under a path (`substitutions/041`).
+- `substitution-fallback-to-environment/009`–`010` — a self-reference with nothing below it, `HOME : ${HOME}` or `${?HOME}`, reads the environment variable of its own name: it is undefined "exactly as if the substitution referenced a path not found in the document" (examples-of-self-referential-substitutions.6, self-referential-substitutions.13), and such a path is searched for in the environment (substitution-fallback-to-environment-variables.1). The spec's author said as much in lightbend/config#652; pyhocon does it since chimpler/pyhocon#213. typesafe/config reports a cycle for `${HOME}`, which .7 rules out, and drops the field for `${?HOME}`; zig-hocon took the field to be in the document and read nothing.
 
 This pass ran *before* the tie-break rule in `PROCESS.md` was settled, so where a divergence says `expect` was left at java's value, the row is still unresolved — `fill-expected.py` now flags those with `review`.
 

conformance/maintaining/SECTIONS.md

@@ -47,7 +47,7 @@ observable parse behaviour.
 | 1441 | Java properties mapping | — | — JVM-specific |
 | 1498 | Conventional configuration files for JVM apps | — | — JVM-specific |
 | 1528 | Conventional override by system properties | — | — JVM-specific |
-| 1534 | Substitution fallback to environment variables | `substitution-fallback-to-environment` | ✅ 8 cases |
+| 1534 | Substitution fallback to environment variables | `substitution-fallback-to-environment` | ✅ 10 cases |
 | 1567 | hyphen-separated vs. camelCase | — | — naming advice |
 | 1572 | Note on Java properties similarity | — | — |
 | 1598 | Note on Windows and case sensitivity | — | — |

conformance/reports/Java.java.json

@@ -5,10 +5,10 @@
   "mode": "java",
   "java_version": "1.4.9",
   "command": "conformance/adapters/oracle.sh hocon-java",
-  "ran_at": "2026-10-25T07:35:58Z",
-  "suite_digest": "1e463c3b5af96a46",
-  "total": 476,
-  "passed": 476,
+  "ran_at": "2026-10-25T09:44:43Z",
+  "suite_digest": "088483db7c04ff14",
+  "total": 478,
+  "passed": 478,
   "percent": 100.0,
   "sections": {
     "array-and-object-concatenation": {
@@ -120,8 +120,8 @@
       "passed": 19
     },
     "substitution-fallback-to-environment": {
-      "total": 8,
-      "passed": 8
+      "total": 10,
+      "passed": 10
     },
     "substitutions": {
       "total": 38,
@@ -504,6 +504,8 @@
     "substitution-fallback-to-environment/006-empty-env-var-is-empty-string.conf": "pass",
     "substitution-fallback-to-environment/007-empty-env-var-creates-optional-field.conf": "pass",
     "substitution-fallback-to-environment/008-env-vars-always-become-strings.conf": "pass",
+    "substitution-fallback-to-environment/009-self-reference-reads-env.conf": "pass",
+    "substitution-fallback-to-environment/010-optional-self-reference-reads-env.conf": "pass",
     "substitutions/001-basic-substitution.conf": "pass",
     "substitutions/002-optional-substitution-syntax.conf": "pass",
     "substitutions/003-optional-substitution-whitespace-illegal.conf": "pass",
@@ -673,6 +675,8 @@
     "string-value-concatenation/014-array-then-unquoted-text-without-space.conf": "{\"x\":[1]}",
     "string-value-concatenation/015-object-then-unquoted-text-without-space.conf": "{\"x\":{\"a\":1}}",
     "string-value-concatenation/016-text-between-arrays-is-not-dropped.conf": "{\"x\":[1,2]}",
+    "substitution-fallback-to-environment/009-self-reference-reads-env.conf": "rejected",
+    "substitution-fallback-to-environment/010-optional-self-reference-reads-env.conf": "{}",
     "substitutions/039-undefined-optional-with-number-is-a-string.conf": "{\"a\":42}",
     "unchanged-from-json/009-leading-zero-run-is-text.conf": "{\"a\":1}",
     "unchanged-from-json/010-signed-fraction-without-integer-part-is-text.conf": "{\"a\":-0.33}",

conformance/reports/Java.spec.json

@@ -4,11 +4,11 @@
   "version": "typesafe/config 1.4.9",
   "mode": "spec",
   "command": "conformance/adapters/oracle.sh hocon-java",
-  "ran_at": "2026-10-25T07:35:10Z",
-  "suite_digest": "1e463c3b5af96a46",
-  "total": 476,
+  "ran_at": "2026-10-25T09:40:25Z",
+  "suite_digest": "088483db7c04ff14",
+  "total": 478,
   "passed": 419,
-  "percent": 88.0,
+  "percent": 87.7,
   "sections": {
     "array-and-object-concatenation": {
       "total": 22,
@@ -119,7 +119,7 @@
       "passed": 14
     },
     "substitution-fallback-to-environment": {
-      "total": 8,
+      "total": 10,
       "passed": 8
     },
     "substitutions": {
@@ -503,6 +503,8 @@
     "substitution-fallback-to-environment/006-empty-env-var-is-empty-string.conf": "pass",
     "substitution-fallback-to-environment/007-empty-env-var-creates-optional-field.conf": "pass",
     "substitution-fallback-to-environment/008-env-vars-always-become-strings.conf": "pass",
+    "substitution-fallback-to-environment/009-self-reference-reads-env.conf": "fail",
+    "substitution-fallback-to-environment/010-optional-self-reference-reads-env.conf": "fail",
     "substitutions/001-basic-substitution.conf": "pass",
     "substitutions/002-optional-substitution-syntax.conf": "pass",
     "substitutions/003-optional-substitution-whitespace-illegal.conf": "pass",
@@ -672,6 +674,8 @@
     "string-value-concatenation/014-array-then-unquoted-text-without-space.conf": "{\"x\":[1]}",
     "string-value-concatenation/015-object-then-unquoted-text-without-space.conf": "{\"x\":{\"a\":1}}",
     "string-value-concatenation/016-text-between-arrays-is-not-dropped.conf": "{\"x\":[1,2]}",
+    "substitution-fallback-to-environment/009-self-reference-reads-env.conf": "rejected",
+    "substitution-fallback-to-environment/010-optional-self-reference-reads-env.conf": "{}",
     "substitutions/039-undefined-optional-with-number-is-a-string.conf": "{\"a\":42}",
     "unchanged-from-json/009-leading-zero-run-is-text.conf": "{\"a\":1}",
     "unchanged-from-json/010-signed-fraction-without-integer-part-is-text.conf": "{\"a\":-0.33}",
@@ -1524,6 +1528,26 @@
       "got": "{\"x\":[1,2]}",
       "java": "lenient"
     },
+    {
+      "case": "substitution-fallback-to-environment/009-self-reference-reads-env.conf",
+      "spec": "HOCON.md#substitution-fallback-to-environment-variables",
+      "rule": "a self-reference with no earlier value is undefined \"exactly as if the substitution referenced a path not found in the document\", and such a path is searched for in the environment, so the field reads the variable of its own name; typesafe/config reports a cycle, which the spec rules out (\"undefined\" rather than \"intractable cycle\")",
+      "expected": {
+        "CONFORMANCE_HOME": "/from/env"
+      },
+      "got": "rejected (exit 1)",
+      "java": "unsupported"
+    },
+    {
+      "case": "substitution-fallback-to-environment/010-optional-self-reference-reads-env.conf",
+      "spec": "HOCON.md#substitution-fallback-to-environment-variables",
+      "rule": "if ${?foo} refers to itself \"it's as if it referred to a nonexistent value\", and a value not present in the configuration is searched for in the environment, so the field is created from the variable instead of disappearing",
+      "expected": {
+        "CONFORMANCE_HOME": "/from/env"
+      },
+      "got": {},
+      "java": "diverges"
+    },
     {
       "case": "substitutions/039-undefined-optional-with-number-is-a-string.conf",
       "spec": "HOCON.md#substitutions",