Conversation
Bring up an EUI-64 based IPv6 link-local address on the control NIC of system VMs and make sshd listen on it, in addition to the existing 169.254.0.0/16 IPv4 address. Since the address is generated with EUI-64 it can be calculated from the MAC address of the control NIC. listSystemVms now returns it as 'linklocalip6', calculated by the Management Server using NetUtils.ipv6LinkLocal(), and the UI shows it in the System VMs view. On KVM the agent enables IPv6 (link-local only, no RA/SLAAC) on the control bridge (cloud0) when setting up the control network, so the host can reach system VMs over IPv6 once this is used.
The nftables ip6_firewall and ip6_acl tables created on VRs with IPv6 networking have an input hook chain with policy drop, which applies to all interfaces including the control NIC. Accept TCP 3922 between link-local addresses so sshd remains reachable on the IPv6 link-local address of the control interface. Restricting both saddr and daddr to fe80::/10 ensures no global address can reach sshd.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #13795 +/- ##
=========================================
Coverage 19.65% 19.65%
Complexity 19792 19792
=========================================
Files 6368 6368
Lines 574881 574896 +15
Branches 70351 70352 +1
=========================================
+ Hits 112970 112975 +5
- Misses 449639 449648 +9
- Partials 12272 12273 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@blueorangutan package |
|
@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19191 |
| # Only a link-local address is wanted, no SLAAC/RA configuration | ||
| sysctl -w net.ipv6.conf.${eth}.accept_ra=0 | ||
| sysctl -w net.ipv6.conf.${eth}.autoconf=0 | ||
| sysctl -w net.ipv6.conf.${eth}.disable_ipv6=0 |
There was a problem hiding this comment.
tested this on a lab and it gets undone a few seconds later, so the feature doesnt survive a boot.
this sets the runtime value only. the template ships net.ipv6.conf.all.disable_ipv6 = 1 in /etc/sysctl.conf, and bootstrap.sh:104 runs sysctl -p after setup_sshd. boot order on the ssvm:
00:43:15 Configuring sshd
00:43:15 Enabling IPv6 link-local on interface eth0
00:43:17 Interface eth0 has IPv6 link-local address fe80::c00:a9ff:fefe:cadc
00:43:17 Configuring sshd to also listen on fe80::c00:a9ff:fefe:cadc%eth0
00:43:19 Executing cloud-early-config
00:43:20 Bootstrapping systemvm appliance <- sysctl -p in here
end state, address gone and sshd only on v4:
$ ip -6 addr show dev eth0
(nothing)
$ sysctl -n net.ipv6.conf.eth0.disable_ipv6
1
$ ss -tln | grep 3922
LISTEN 0 128 169.254.202.220:3922 0.0.0.0:*
showed it directly on the vm:
$ sysctl -qw net.ipv6.conf.eth0.disable_ipv6=0
$ ip -6 addr show dev eth0 scope link
inet6 fe80::c00:a9ff:fefe:cadc/64 scope link
$ sysctl -p
$ ip -6 addr show dev eth0 scope link
(gone)
so listSystemVms shows a linklocalip6 that nothing answers on. ping6 and ssh to it both time out from the host.
the design is fine by the way, once i left ipv6 on and restarted sshd it worked end to end over fe80::...%cloud0 on 3922.
can we clear the persistent one too? common.sh:126-129 already does that for net.ipv6.conf.all.disable_ipv6 including rewriting sysctl.conf.
There was a problem hiding this comment.
Thanks! I had a manual script checking a few things, didn't notice it was gone right after. Let me look into that, I now have a better lab to test this again.
My goal is to remove IPv4 entirely. We now have a complete allocation mechanism for IPv4 for this control cidr which is a lot of code and database entries which shouldn't be needed.
First step is adding IPv6, making sure its stable and then remove IPv4 in a future PR.
There was a problem hiding this comment.
sounds good, happy to test again once its updated
| enable_ipv6_link_local $eth | ||
| if [ -n "$LINK_LOCAL_IP6" ]; then | ||
| log_it "Configuring sshd to also listen on ${LINK_LOCAL_IP6}%${eth}" | ||
| sed -i -e "/^ListenAddress $ip$/a ListenAddress ${LINK_LOCAL_IP6}%${eth}" /etc/ssh/sshd_config |
There was a problem hiding this comment.
this writes the v6 ListenAddress but nothing rechecks the address is still there when sshd actually starts. on my lab it ended up pointing at an address that no longer existed:
$ grep -i ListenAddress /etc/ssh/sshd_config
ListenAddress 169.254.202.220
ListenAddress fe80::c00:a9ff:fefe:cadc%eth0
$ ss -tln | grep 3922
LISTEN 0 128 169.254.202.220:3922 0.0.0.0:*
sshd was fine with it and just bound v4, so no harm this time. but if it ever refused to start we'd have a systemvm with no way in.
worth only adding the line when the address is actually up at sshd start?
There was a problem hiding this comment.
Same goes for IPv4 ofcourse, we never check if the address exists, we assume it does. This needs fixing somewhere else.
There was a problem hiding this comment.
fair enough, ok to fix that separately
| vmResponse.setLinkLocalMacAddress(singleNicProfile.getMacAddress()); | ||
| vmResponse.setLinkLocalNetmask(singleNicProfile.getIPv4Netmask()); | ||
| if (singleNicProfile.getMacAddress() != null) { | ||
| vmResponse.setLinkLocalIp6(NetUtils.ipv6LinkLocal(singleNicProfile.getMacAddress()).toString()); |
There was a problem hiding this comment.
this is calculated from the mac, it never checks the vm actually has the address. so the field always shows something even when ipv6 is off on the systemvm.
on my lab both systemvms reported a linklocalip6 while nothing was listening on it.
is that the intent, a value you can always compute? if so maybe say so in the field description, otherwise someone will read it as "this address works".
There was a problem hiding this comment.
The calculated address is always the same. It never changes or will be different. This is an RFC for IPv6 where the link-local is persistent. That's the great thing about it.
Calculate instead of store. Saves a lot of code.
There was a problem hiding this comment.
makes sense that it never changes. can we say in the field description that its calculated, so nobody reads it as the address being up?
There was a problem hiding this comment.
Well, isn't this a basic IPv6 knowledge? Link Local IPv6 always exists and you don't need to do anything about it.
wido
changed the title
This branch has not been deployed
4 participants
Description
This PR enables IPv6 link-local addressing on the control (link-local) network of system VMs, alongside the existing 169.254.0.0/16 IPv4 addressing. With IPv6 explicitly enabled on the control network we can move our SSH commands (on at least KVM) to link local IPv6 and remove the 169.254.0.0/16 addressing in the future. That can significantly reduce the code-base inside CloudStack. That's not part of this PR yet.
The existing issue #9957 talks about this and this PR should be groundwork for that to be done in the future.
System VM (
common.sh):addr_gen_mode=0), so the address is deterministic and can be calculated from the NIC's MAC addressaccept_ra=0,autoconf=0); only the link-local address is configuredsetup_sshd()waits for Duplicate Address Detection to complete and adds aListenAddress fe80::...%ethXentry tosshd_config, so sshd listens on both the IPv4 and IPv6 link-local addresses. The sed expressions are idempotent across rebootssetup_sshd()is always handed the control interface (eth0 for CPVM/SSVM on KVM, eth1 for routers)Management Server / API:
listSystemVmsnow returnslinklocalip6: the EUI-64 IPv6 link-local address calculated from the control NIC's MAC address using the existing (unit-tested)NetUtils.ipv6LinkLocal(). This performs the same U/L-bit flip +ff:feexpansion the kernel does, so the reported address always matches what the system VM configuresSystemVmResponse(since 4.23.0)KVM agent:
cloud0) via a sharedenableBridgeIpv6LinkLocal()helper inVifDriverBase, called fromBridgeVifDriver,OvsVifDriverandIvsVifDrivercloud0bridge also get IPv6 enabled on itssh -6 -p 3922 fe80::...%cloud0UI:
This is a first step towards using IPv6 link-local instead of 169.254.0.0/16 on the control network. The Management Server still provisions system VMs over IPv4; nothing changes for existing deployments.
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
How Has This Been Tested?
api,serverandplugins/hypervisors/kvmcompile cleanlybash -noncommon.shNetUtils.ipv6LinkLocal()is covered by existing unit tests inNetUtilsTest(testIpv6LinkLocal), verifying the EUI-64 calculation matches the kernel's