Skip to content

Release 8.22.0: every g: component on the primary, database calls that end, host-stall out of service, and app ports proven over UDP - #1886

Merged
Cabecinha84 merged 385 commits into
masterfrom
development
Oct 25, 2026
Merged

Cabecinha84 merged 385 commits into
masterfrom
development

Conversation

@Cabecinha84

@Cabecinha84 Cabecinha84 commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Release PR, development → master. 11 merged PRs since v8.21.1 (#1853), all merged into development directly: #1868, #1867, #1860, #1858, #1881, #1866, #1874, #1876, #1882, #1885, #1887. #1874 → #1876 → #1882 and #1860 → #1858 were stacks, each merged in order. 250 files, +29.7k/−4.3k, of which ZelBack is 76 files, +7.4k/−2.8k. The rest is tests and the harness.

Cut following RELEASING.md.

release step state
master holds no ZelBack changes development is missing yes. master was merged back into development (8a3b5aed8), and that merge changes no files
package.json 8.21.1 → 8.22.0 (9dfef7f4f) done
helpers/hashes.json carries b9bfb6578c581b007f92137826c01cb1 (c10a308a9) done. It is the md5 over ./ZelBack at this tip, checked from a clean git archive
the fingerprint is in the signed document central serves verified at hashes.runonflux.io/hashlist, seq 340
full harness gate at this tip done: 234 suites, 234 pass, 0 fail, at 04ddc7e2ed (#1887's head), whose tree differs from this tip only in helpers/hashes.json

Minor bump: new checks and features, nothing that changes consensus or the message protocol. Several operational changes need attention, listed first.

⚠️ Read before merging

Nodes that will go out of service or change behaviour

API and configuration changes for callers

Shipping order with other components

  • @runonflux/nat-upnp 2.0.5 (chore(upnp): move to @runonflux/nat-upnp 2.0.5 #1860) is the new floor. It is already published.
  • Policy documents cputhrottleexempt, udpportdos and udpinstallrefusaloff can be published before or after this release, because a release ignores names it does not read. udpportdos should stay off until operators have had notice.

Changes

Single-writer (g:) apps

  • fix(election): the primary of a g: app runs every g: component #1867: the primary of a g: app runs every g: component. The election decides the whole app, not its first component. Seeds are decided per app, and a cold start seeds every g: folder from one node. An in-place update restarts through the reconciler, so a standby's g: component stays down.
  • fix(syncthing): a primary that went quiet and then died is taken over, with its data intact #1876: a primary that went quiet and then died is taken over with its data intact. A device stays in an app's folders only while a location lists it. A change to another device's data that no other device holds is never reverted, and reverts act only on the changes they checked. A standby's own changes are kept while no holder runs the app. A cold-start seed waits only on a holder that has not looked, and leaves out a holder that cannot seed (folderStates). The device sweep keeps devices seen in the last 24 h, so a dead primary can still be shown gone.

Database and host health

Network and ports

Apps

Tests

The unit suite passed on each PR. #1876/#1882 were the last full runs: 9,983 passing, 0 failing. Every PR adds harness suites that fail on the tree without their fix: 2241–2256, 2507, 3001–3004, 3301–3320, 3501–3506. The author reported a full gate of 233/0 at an earlier #1882 tip (09780d58e6). The full gate at 04ddc7e2ed, this tree apart from helpers/hashes.json, ran 234 suites: 234 pass, 0 fail.

🤖 Generated with Claude Code

MorningLightMountain713 and others added 30 commits October 19, 2026 18:28
A hard redeploy whose row delete is refused leaves the app held; a second hard redeploy, sent before the removal runs again, removes it and is refused the install, and the node never installs the app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… cannot be read

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…bility check

The availability check maps its test port over TCP alone: its test server
is HTTP. A pass that finds the router has not removed the last test mapping
asks for the removal at once and, while it is refused, makes no new test
mapping and counts as a failed check, so a router that keeps refusing puts
the node in DOS, and the pass after it removes the mapping tests and
releases it.

mapUpnpPort takes the protocols to map, TCP and then UDP unless given.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A removal the router refuses replaces only that protocol's pending removal,
so when it refuses both protocols of a port both are asked again, and a
second removal of a port with nothing mapped since keeps the first's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eck, and UPnP wakes it

upnpService alone asks the router again for a removal it refused, the wait
doubling from 30 s to at most 60 s, and says when a removal becomes pending
with none before ('pending') and when the last goes ('clear').

The availability check reads that state: while any removal is pending it
makes no test mapping and counts each pass as a failed check, so a router
that keeps refusing puts the node in DOS. Either signal starts its next pass
at once, so a refusal is acted on without waiting out a passing test's hour,
and the pass after the router removes the mapping tests and releases the
node. A pass that leaves a removal pending waits no longer than an error's
wait.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s is out of service until it does

The UPnP stub records each request's protocol and description, and a suite
can change how many removals a gateway refuses without resetting its table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
upnpService's routerEvents says 'mappingRefused' each time it counts a
refusal as the router's own, beside 'removalPending' and 'removalsClear',
and each starts the check's next pass at once: a router that starts
refusing is looked at without waiting out a passing test's hour.

A pass that counted a router refusal, saw one during the pass, or left a
removal pending waits no longer than an error's wait (a DOS's while out of
service), so the pass that finds the router working again comes soon. Every
wait between passes is the one a wake ends, and a wake only ends it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The suite is now "a node whose router refuses it is out of service until it
stops": its check waits ten minutes after a passing test and five in DOS, a
full router's refusal of an app's mapping puts it out of service well inside
the first, and it is back at its next pass once the router has room. The
UPnP stub can change a gateway's capacity without resetting its table, and
marks a mapping it refuses for being full.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…timeouts

AbortSignal.timeout throws on a fraction, so a budget that is not a whole
number of milliseconds, or one that halves to a fraction, is rounded down
before the listing's and each removal's timeout are taken from it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…timeouts

AbortSignal.timeout throws on a fraction, so a budget that is not a whole
number of milliseconds, or one that halves to a fraction, is rounded down
before the listing's and each removal's timeout are taken from it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	ZelBack/src/services/upnpService.js
#	tests/unit/upnpService.test.js
chore(upnp): move to @runonflux/nat-upnp 2.0.5
…s height read takes

The rebuild reads the scanned height from mongo before it reaches the
stubbed rebuild, and the tests waited for it a fixed 1000 setImmediate
turns. A read slower than those few milliseconds left the rebuild
unstarted and the next test waiting behind the one still in flight. The
wait now ends when the stub has been called, bounded only by the test's
own timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Before asking the router again to remove a mapping, upnpService asks whether
the router still maps it to this node. One it no longer holds, or holds for
something else, ends the removal with nothing sent; so does a lookup the
router answers with a fault, since a router finds this node's own entries
(no lease) whenever it holds them. Only a lookup the router does not answer
leaves the removal pending, to be asked again. A router that answers a
removal of an entry it no longer holds with anything but 714 can no longer
keep the removal pending, and with it the node out of service.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t with a fault leaves the node in service

The UPnP stub can answer every removal with a fault while still carrying it
out, and a removal of an entry it no longer holds with the same fault rather
than 714.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…is not

A pending removal ends with nothing sent only when the lookup shows the
router holds no entry for the port, or holds it for something else. A
lookup the router answers with a fault, or does not answer, settles
nothing: the mapping is still counted as this node's and the removal is
sent, so a mapping is never left on the router on a lookup that failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he mapping counted as the node's

The UPnP stub can answer every GetSpecificPortMappingEntry with a fault,
whether or not it holds the entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(upnp): a full router table is scored as the router's, and a node still verifies on one
/flux/checkappavailability takes two opt-in fields. `udp: true` probes each
port over UDP once TCP has passed and returns what replied in `udpAnswered`:
the reply, null for silence, the port left out when no probe could be sent.
`everyPort: true`, with `echo`, reads every port in parallel and returns an
answer for each, null for a port that did not answer, instead of ending at the
first failure. A caller sending neither gets the check it always got.

FluxUdpTestServer is the UDP half of a port test: it answers a probe with the
run's token, only to the peer being asked, never with more bytes than the
probe, and at most a fixed number of times.

The test port firewall rule takes its protocol.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every app port is published for TCP and UDP, so the pre-install test now
proves both. Each port gets a UDP test rule and a UDP test server beside the
HTTP one, and the peer is asked to probe UDP.

Our token heard over UDP proves a port, from any peer. A peer hearing nothing
on a port is evidence against it until another peer proves it, and two
distinct peers with standing evidence refuse the install as udpUnreachable.
A peer that does not probe UDP gives no proof and is never a witness, so an
install proven over TCP goes ahead as udpUnproven when no two peers agree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each pass publishes a token on an HTTP and a UDP test server, maps the port
for both protocols, opens a test rule for each, and asks the peer to read the
port and probe it over UDP.

A port the peer read as something other than this node fails, as a port
nothing answered does: an application placed on it here would not be reached.
A port an application at this public address holds, by the network's records
or by a node sharing the address, is skipped untested.

Our token over UDP proves the port. One peer's silence is tested again by
another peer on the next pass, for up to three passes; two peers' silence is
a confirmed UDP failure. It is reported as the `udp` check in /flux/health,
degraded, and counts toward this check's DOS only when the signed policy
document `udpportdos` is `true`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getRunningAppIpList matched every location whose address began with the IP
with its dots read as a pattern, so 1.2.3.4 also took the apps of 1.2.3.45.
The spawner and the availability check read it as the apps holding ports at
this node's public address.

storedAtIp builds the condition once for both queries that read every stored
address at an IP: the escaped IP and the colon after it, every location and
move being stored in ip:port form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r nothing on

Two network shapes put the fault in a node's own namespace: udp-blocked drops
every UDP datagram starting a flow, udp-first-lost drops the first from each
source and lets every later one through. On a fleet of nodes on this branch,
the open node and the first-lost node install with their ports proven, the
blocked node is refused with two distinct peers named, and its availability
check reports udp degraded in /flux/health while it stays in service. On a
fleet whose peers do not probe UDP, a blocked node installs on TCP proof.

The pre-install test publishes ports:proven with the ports and the peer that
proved them, beside ports:notOurs and ports:unproven.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…or TCP and UDP

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… loopback DNS

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… silent ports fit the requester's wait

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ind UPnP, and counts it under policy

The open node runs ufw, so its UDP test rule is what admits the probes, and
no test rule is left once the app installs. udp-every-other drops every other
datagram starting a flow. The every-other node is behind a UPnP gateway, which
is asked for both protocols of the app's port. A fleet whose policy carries
udpportdos takes a node closed over UDP out of service.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MorningLightMountain713 and others added 12 commits October 23, 2026 21:10
…sync limit that keeps the app

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… wrote, refused, cleared or relayed something

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…igns its message

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o its broadcast

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ht, however long it takes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(apps): announcements are judged per entry, and a node keeps its own record of when it took an app
A held write's release started the real write each time it was called, and
the block's afterEach releases every write again after a test has released
its own. The extra writes were not awaited, so one could land while the
outer afterEach removed the temporary directory, failing the rmdir with
ENOTEMPTY.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test(hostStall): a held write is released once
Steps 1 and 2 of RELEASING.md for the development -> master release PR.

package.json moves 8.21.1 -> 8.22.0, which is what release-gate's "version
moves forward" check reads.

helpers/hashes.json gains 0840c013921a322079de7d694417ff42, the md5 over
./ZelBack at this tip. Neither file is inside ZelBack, so committing them does
not move the fingerprint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 95.23449% with 126 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.74%. Comparing base (bb5adc7) to head (9dfef7f).
⚠️ Report is 130 commits behind head on master.

Files with missing lines Patch % Lines
ZelBack/src/services/serviceManager.js 17.85% 46 Missing ⚠️
ZelBack/src/services/explorerService.js 69.33% 23 Missing ⚠️
...ack/src/services/appLifecycle/advancedWorkflows.js 96.46% 7 Missing ⚠️
ZelBack/src/services/appMessaging/messageStore.js 91.13% 7 Missing ⚠️
.../src/services/appMonitoring/availabilityChecker.js 95.10% 7 Missing ⚠️
ZelBack/src/services/dbHelper.js 94.18% 5 Missing ⚠️
ZelBack/src/services/idService.js 93.90% 5 Missing ⚠️
...lBack/src/services/appTamperingDetectionService.js 94.52% 4 Missing ⚠️
...elBack/src/services/appLifecycle/appUninstaller.js 93.75% 3 Missing ⚠️
...ck/src/services/appMonitoring/nodeStatusMonitor.js 96.38% 3 Missing ⚠️
... and 10 more
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1886      +/-   ##
==========================================
+ Coverage   78.75%   81.74%   +2.99%     
==========================================
  Files         223      240      +17     
  Lines       36449    38422    +1973     
==========================================
+ Hits        28704    31409    +2705     
+ Misses       7745     7012     -733     
- Partials        0        1       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@MorningLightMountain713 MorningLightMountain713 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack

@alihm alihm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack

Requests now refuse an app name that does not match the app-name rule and an
owner that is not a Flux ID or an Ethereum address, using the existing
APP_NAME_REGEX, MAX_APP_NAME_LENGTH and signatureVerifier.isValidSigningIdentity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
utils/appIdentity holds the rule: isValidAppNameAndOwner and
verifyAppNameAndOwner, from APP_NAME_REGEX, MAX_APP_NAME_LENGTH and
signatureVerifier.isValidSigningIdentity.

The price, verify, register, update and public-key routes check every spec's
name and owner before reading it. Peer messages and history sync check a
spec's name and owner before reading it; history sync stores a message that
fails the check, as it does any message it cannot read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/apps/appregister is served by registryManager.registerAppGlobalyApi; the copy
in appValidator had no caller. The imports only it used go with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The price, verify, register and update routes checked every spec's name and
owner. Only an enterprise spec (version >= 8 with an enterprise field) is
decrypted, so the check now runs only for those; a non-enterprise spec, a
marketplace price request among them, is no longer refused by it. getpublickey
keeps its check, as every request there is decrypted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
encryptEnterpriseWithAes had no callers. checkAndDecryptAppSpecs computed an
owner it never used; the decrypt reads the spec's own owner.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…enterprise

checkAndDecryptAppSpecs now decrypts only when version >= 8 and an enterprise blob is present, the rule the route checks apply. A spec with a missing or non-numeric version is returned unchanged instead of being decrypted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(apps): validate app name and owner
Step 2 of RELEASING.md for the development -> master release PR.

helpers/hashes.json gains b9bfb6578c581b007f92137826c01cb1, the md5 over
./ZelBack at this tip. The file is outside ZelBack, so committing it does not
move the fingerprint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Cabecinha84
Cabecinha84 merged commit 8dab570 into master Oct 25, 2026
7 checks passed

This branch was successfully deployed

1 active deployment
github-pages — c10a308a Deployed Oct 25, 2026 by MorningLightMountain713 via deploy #3132
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants