Skip to content

Refactor tunnel startup logic similar to WireGuard - #2467

Merged
openipc-ai merged 6 commits into
OpenIPC:masterfrom
usa-:improve-tunnel
Sep 22, 2026
Merged

openipc-ai merged 6 commits into
OpenIPC:masterfrom
usa-:improve-tunnel

Conversation

@usa-

@usa- usa- commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Problem

The /usr/sbin/tunnel startup logic is split between a separate readiness check and the main retry loop, with two independent increasing retry delays.

This change refactors the startup flow to use a single start() function, similar to the recent WireGuard startup changes — PR #2465.

The previous implementation used one increasing delay while waiting for network/DNS readiness and another after failures of setup commands before vtund was started. The latter was intended to reduce repeated failures from flooding the syslog. During the review of PR #2438, it was noted that persistent setup failures could still fill the syslog ring and that retries should be backed off rather than repeated at a fixed short interval. The relevant review discussion is [here](#2438 (review)) (finding 3).

Since the readiness and setup checks are now handled by the same start() function, they can share a single increasing retry delay. This removes the two separate backoff mechanisms while retaining the retry pacing needed for persistent failures.

The initial DNS lookup also uses -timeout=3 -retry=1 to bound the time spent on each failed resolution attempt.

The common logging helpers from /usr/sbin/common are used instead of local copies.

The startup variables were renamed for consistency and clarity, for example vtund_serverserver, vtund_portport, identity_cfgconfig_path, and udhcpc_pidudhcpc_pid_path. The identity-related variables were renamed as well.

The result of the vtund command is not used directly to determine whether startup was successful. With persist yes, vtund is expected to remain running and handle reconnection internally rather than return to the shell when the server is temporarily unavailable, as noted during the review of PR #2438. [Review discussion](#2438 (comment))

To handle fatal startup failures regardless of vtund's exit status, start() measures how long the vtund session remains alive using the monotonic system uptime from /proc/uptime. A session that ends before the 10-second startup threshold is treated as a failed attempt and enters the increasing retry backoff.

Once vtund has been started successfully, start() returns success and the outer loop resumes its normal 10-second interval. The outer loop is retained for now as a precaution; whether it is needed at all will be investigated separately and is outside the scope of this PR.

The Starting the tunnel... log message is emitted immediately before the vtund invocation, so it reflects an actual tunnel start attempt. The corresponding WireGuard startup log message was moved in the same way for consistency.

Hardware tested on

ssc378de

The positive startup scenario was tested successfully with the VTun server address immediately resolvable at camera startup.

The negative paths were also verified on a lab ssc377d using shims:

  • DNS never resolves — retries follow the increasing 10/20/30/40/50/60-second backoff.
  • DNS becomes available after several failures — the tunnel recovers on the next attempt.
  • tunctl -t fails — retries use the increasing backoff.
  • vtund exits immediately with either status 1 or status 0 — retries use the increasing backoff.

The exit-status-0 case is particularly relevant because checking the vtund duration catches critical failures that are not reflected in its exit status.

Evidence

Before:

root@ssc378de-imx415:~# logread | grep -E "vtun|VTun|tunnel"
Sep 21 20:33:16 ssc378de-imx415 daemon.info vtund[830]: VTun client ver 3.X 09/20/2026 started
Sep 21 20:33:16 ssc378de-imx415 daemon.info vtund[830]: Connecting to {hidden by me}
Sep 21 20:33:17 ssc378de-imx415 daemon.debug vtund[830]: Remote Server sends <TeK> .
Sep 21 20:33:17 ssc378de-imx415 daemon.info vtund[830]: Session {hidden by me}[{hidden by me}] opened
Sep 21 20:35:27 ssc378de-imx415 daemon.err vtund[3291]: Can not open /etc/vtund.conf
root@ssc378de-imx415:~# ifconfig tunnel
tunnel    Link encap:Ethernet  HWaddr {hidden by me}
          inet addr:10.20.20.13  Bcast:0.0.0.0  Mask:255.255.255.0
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:122 errors:0 dropped:7 overruns:0 frame:0
          TX packets:209 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:7368 (7.1 KiB)  TX bytes:37548 (36.6 KiB)

root@ssc378de-imx415:~#

After:

root@ssc378de-imx415:~# logread | grep -E "vtun|VTun|tunnel"
Sep 21 20:38:47 ssc378de-imx415 daemon.info tunnel[795]: Starting the tunnel to {hidden by me}
Sep 21 20:38:48 ssc378de-imx415 daemon.info vtund[831]: VTun client ver 3.X 09/20/2026 started
Sep 21 20:38:48 ssc378de-imx415 daemon.info vtund[831]: Connecting to {hidden by me}
Sep 21 20:38:48 ssc378de-imx415 daemon.debug vtund[831]: Remote Server sends <TeK> .
Sep 21 20:38:48 ssc378de-imx415 daemon.info vtund[831]: Session {hidden by me}[{hidden by me}] opened
root@ssc378de-imx415:~# ifconfig tunnel
tunnel    Link encap:Ethernet  HWaddr {hidden by me}
          inet addr:10.20.20.13  Bcast:0.0.0.0  Mask:255.255.255.0
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:87 errors:0 dropped:7 overruns:0 frame:0
          TX packets:63 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:6142 (5.9 KiB)  TX bytes:12088 (11.8 KiB)

root@ssc378de-imx415:~#

The new output shows the tunnel startup message immediately before the vtund launch, followed by a successful VTun connection, and ifconfig confirms that the tunnel interface is up with its DHCP address.

Scope

  • No kernel patches under general/package/all-patches/linux/ (those go to https://github.com/OpenIPC/linux)
  • No files specific to a single retail camera model (those go to https://github.com/OpenIPC/builder)
  • No probing or bring-up tooling (those go to https://github.com/OpenIPC/ipctool)
  • Nothing under general/overlay/ or in a shared load_<vendor> script hardcodes a value specific to my board
  • Package sources come from an OpenIPC repository, and any version bump keeps at least the specificity of the pin it replaces (a new package should pin a full 40-character SHA)
  • No LD_PRELOAD, and no binaries that cannot be rebuilt from source
  • New code is selected by a defconfig, so CI actually builds it

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Unify VTun startup checks with shared retry backoff

🐞 Bug fix ✨ Enhancement 🕐 10-20 Minutes

Grey Divider

AI Description

• Consolidates readiness, setup, and launch checks into one retryable startup function.
• Shares incremental backoff across network, DNS, and tunnel setup failures.
• Bounds DNS attempts and treats persistent VTun startup as successful handoff.
Diagram

graph TD
  A["Startup loop"] --> B["start()"] --> C{"Network ready?"}
  C -- No --> D["Shared backoff"] --> B
  C -- Yes --> E{"Server resolvable?"}
  E -- No --> D
  E -- Yes --> F["Prepare tunnel"] --> G["Write VTun config"] --> H["Launch vtund"] --> A
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Remove the outer startup loop
  • ➕ Avoids precautionary repeated launch attempts
  • ➕ Relies directly on VTun's persistent reconnection behavior
  • ➖ Would not recover if vtund exits unexpectedly
  • ➖ Requires stronger validation of VTun lifecycle behavior across failure modes
2. Use init-level process supervision
  • ➕ Provides explicit daemon lifecycle and restart policies
  • ➕ Separates process recovery from network readiness logic
  • ➖ Requires broader init and packaging changes
  • ➖ May duplicate VTun's built-in persistence behavior

Recommendation: Keep the unified start function and shared backoff in this PR because it matches the established WireGuard pattern and fixes retry pacing with minimal scope. Validate DNS, setup-failure, and process-exit scenarios before separately deciding whether to remove the outer loop or introduce supervision.

Files changed (1) +70 / -77

Refactor (1) +70 / -77
tunnelUnify VTun startup and retry handling +70/-77

Unify VTun startup and retry handling

• Replaces separate readiness and restart delays with one start function and shared incremental backoff. Bounds DNS lookups, consolidates tunnel preparation and configuration, clarifies variable names, and stops using vtund's result to drive retries.

general/package/vtund-openipc/files/tunnel

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Sep 21, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (1) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Tunnel recovery remains unverified 📘 Rule violation ☼ Reliability
Description
The new start() retry loop handles temporary DNS and setup failures, but the PR reports hardware
output only for immediate successful resolution. The description explicitly says temporary DNS
unavailability, setup failures, and recovery after failed attempts still need testing, so these
newly consolidated retry paths have no real-camera result.
Code

general/package/vtund-openipc/files/tunnel[R87-90]

+			while true; do
+				[ $delay -lt $delay_max ] && delay=$(( delay + delay_step ))
+				sleep $delay
+				start && break
Evidence
PR Compliance ID 53 requires real-board results for the relevant behavior of an image-affecting
change. The cited lines introduce the retry-and-recovery path, while the PR description explicitly
states that temporary DNS failure, setup-command failure, and recovery after failed attempts still
need testing.

general/package/vtund-openipc/files/tunnel[87-90]
Best Practice: Repository guidelines

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The refactored tunnel startup consolidates DNS readiness, setup failures, and recovery into one retry loop, but only the immediate-success path was verified on hardware.
## Fix Focus Areas
- general/package/vtund-openipc/files/tunnel[87-90]
## Recommended Fix
Exercise the changed retry behavior on an affected camera by forcing temporary DNS unavailability or a setup-command failure, then restoring the dependency. Add logs showing failed attempts, increasing retry delays, successful recovery, and a working tunnel.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Tunnel crashes retry every ten seconds ✓ Resolved 🐞 Bug ☼ Reliability
Description
start() discards the vtund exit status and unconditionally returns success after the foreground
process terminates. When VTun exits with a fatal local error, the failure bypasses the increasing
backoff and reaches the outer fixed ten-second restart loop across every device using this shared
package.
Code

general/package/vtund-openipc/files/tunnel[R77-79]

+	vtund -n -f "$config_path" "$profile" "$server" >/dev/null 2>&1
+
+	return 0
Evidence
The foreground vtund invocation is immediately followed by unconditional success. The caller
applies increasing delay only when start() returns nonzero; every reported success instead reaches
the fixed ten-second sleep, so any nonzero VTun exit is necessarily retried at that fixed interval.

general/package/vtund-openipc/files/tunnel[77-79]
general/package/vtund-openipc/files/tunnel[84-94]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`start()` always returns success after `vtund` exits, including when it exits with a nonzero status. Those failures therefore retry every ten seconds instead of entering the increasing backoff loop.
## Fix Focus Areas
- general/package/vtund-openipc/files/tunnel[77-94]
## Recommended Fix
Capture the `vtund` exit status and return a nonzero status from `start()` when VTun fails, while retaining the normal ten-second interval only for successful exits. Add coverage for an immediate nonzero VTun exit to verify that retries use the increasing delay.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can route each action level your way: inline, summary, both, or drop

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread general/package/vtund-openipc/files/tunnel
Comment thread general/package/vtund-openipc/files/tunnel Outdated
@usa-

usa- commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

@openipc-ai

Could you please take a look and help verify the negative case, as you have done with my previous PRs?

@openipc-ai openipc-ai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran the negative cases on a lab ssc377d (SigmaStar infinity6c, busybox 1.36.1), driving this branch's tunnel and master's side by side with the same shims so the two are comparable.

The paths you asked about all work.

DNS never resolvestunnel nonexistent-vtun-test.invalid:

07:36:13 daemon.warn tunnel[3320]: Unable to resolve the VTun server hostname
07:36:23   +10
07:36:43   +20
07:37:13   +30
07:37:53   +40
07:38:43   +50
07:39:43   +60

DNS temporarily unavailable, then restored/etc/resolv.conf pointed at an unroutable 192.0.2.1, server openipc.org, resolver restored at 07:42:01:

07:40:22 daemon.warn tunnel[3417]: Unable to resolve the VTun server hostname
07:40:35   +10
07:40:58   +20
07:41:31   +30
07:42:11 daemon.info tunnel[3417]: Starting the tunnel to openipc.org

It recovers on the next rung after DNS returns, and goes on to start vtund normally. -timeout=3 -retry=1 costs about 3 s per failed lookup against one unroutable nameserver, as intended.

Setup command failstunctl -t tunnel forced to exit 1:

07:51:06 daemon.err tunnel[619]: Command `tunctl -t tunnel` finished with code 1.
07:51:16   +10
07:51:36   +20
07:52:06   +30
07:52:46   +40

Same ladder as master on that path.

And your reading of persist yes holds. Real vtund pointed at an unreachable server never came back to the shell — it retried internally for as long as I let it run:

07:44:33 vtund[3650]: Connecting to openipc.org
07:44:33 vtund[3650]: Connect to openipc.org failed. Connection refused(111)
07:44:38 vtund[3650]: Connecting to openipc.org
...
07:45:09 vtund[3650]: Connect to openipc.org failed. Connection refused(111)

One thing needs changing before this goes in, and one thing is worth knowing — both inline.

CI has never run on this branch: all five workflows are sitting at action_required, waiting for a maintainer to approve the fork run. I will approve them once you push, so the matrix runs once on the final head rather than twice.

The repository checks pass on this branch as it stands — test_shell_parse.sh, test_strip_shell_comments.sh and ci-matrix.py --self-test — and the branch is current with master.

Comment thread general/package/vtund-openipc/files/tunnel Outdated
Comment thread general/package/vtund-openipc/files/tunnel
Comment thread general/package/vtund-openipc/files/tunnel Outdated
@usa-

usa- commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

@openipc-ai

Could you please re-review the latest version of the tunnel changes, especially the short-lived vtund session handling and retry/backoff logic?

@usa-
usa- requested a review from openipc-ai September 22, 2026 10:47

@openipc-ai openipc-ai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix verified on the lab ssc377d — the flat 10 s retry is gone and the ladder is back. Details inline.

device_name is restored and the point about vtund's own chatter is understood, so nothing is outstanding from my side. Approving; the two remarks inline are take-it-or-leave-it.

One tidy-up: the description still describes the old behaviour in two places — "The result of the vtund command is no longer used to control the retry loop" and "Once vtund has been started, start() therefore completes successfully". Both are now the opposite of what the code does.

I have approved the five workflow runs on 0e5e2fdb, so the matrix is running on the head you will merge. Let it come back green first.

Comment thread general/package/vtund-openipc/files/tunnel Outdated
@usa-

usa- commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

I also moved the Starting the tunnel to $server log message closer to the actual vtund invocation.

I made the same change for WireGuard for consistency.

@usa-
usa- requested a review from openipc-ai September 22, 2026 13:06

@openipc-ai openipc-ai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving on 43c1cf8. The switch to /proc/uptime and the raised threshold are verified on the lab ssc377d, the matrix is green at 100/100, and both Qodo threads are resolved.

@openipc-ai
openipc-ai merged commit ca38928 into OpenIPC:master Sep 22, 2026
119 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants