Describe the bug
After upgrading SLES 16.0 nodes from kernel 6.12.0-160000.5-default to 6.12.0-160000.37-default, the nvidia-driver-daemonset (GPU Operator v26.3.3) fails to load the NVIDIA kernel modules. The driver builds successfully, but the final modprobe step fails with:
modprobe: ERROR: could not insert 'nvidia': Permission denied
This is caused by SELinux denial:
type=AVC msg=audit(1787917556.472:1584): avc: denied { module_load } for pid=38113 comm="modprobe" path="/usr/lib/modules/6.12.0-160000.37-default/updates/nvidia.ko.zst" dev="overlay" ino=2768537 scontext=system_u:system_r:unconfined_service_t:s0 tcontext=system_u:object_r:container_file_t:s0 tclass=system permissive=0
type=AVC msg=audit(1787917866.490:1591): avc: denied { module_load } for pid=40474 comm="modprobe" path="/usr/lib/modules/6.12.0-160000.37-default/updates/nvidia.ko.zst" dev="overlay" ino=2768556 scontext=system_u:system_r:unconfined_service_t:s0 tcontext=system_u:object_r:container_file_t:s0 tclass=system permissive=0
The most probable cause why this issue started appearing on newer kernel version a config change (CONFIG_MODULE_COMPRESS_ZSTD=y and CONFIG_MODULE_DECOMPRESS=y). With these options enabled, the kernel uses finit_module() to load compressed modules (*.ko.zst) directly from a file instead of init_module() from an in‑memory buffer, so SELinux now evaluates the module load against the file label.
Adding a policy that grants module_load on container_file_t to the GPU Operator driver container’s domain fixes the issue.
To Reproduce
- Use a SLES 16.0 system with SELinux in enforcing mode and GPU Operator 26.3.3 deployed.
- Start from kernel
6.12.0-160000.5-default.
- Confirm that
nvidia-driver-daemonset works as expected
- Upgrade the kernel to
6.12.0-160000.37-default
- Ensure the
nvidia-driver-daemonset pod is scheduled on the node.
- Check the driver pod logs; observe modprobe failures:
modprobe: ERROR: could not insert 'nvidia': Permission denied
- On the node, inspect SELinux AVCs:
sudo ausearch -m avc -ts today | grep module_load
You should see denials similar to:
type=AVC msg=audit(1787917556.472:1584): avc: denied { module_load } for pid=38113 comm="modprobe" path="/usr/lib/modules/6.12.0-160000.37-default/updates/nvidia.ko.zst" dev="overlay" ino=2768537 scontext=system_u:system_r:unconfined_service_t:s0 tcontext=system_u:object_r:container_file_t:s0 tclass=system permissive=0
Expected behavior
NVIDIA kernel modules should load successfully on SELinux‑enforcing SLES 16.0 systems after a kernel upgrade.
Environment (please provide the following information):
- GPU Operator Version: v26.3.3
- OS: SLES 16.0
- Kernel Version: 6.12.0-160000.37-default
- Container Runtime Version: containerd 2.2.6-k3s1)
- Kubernetes Distro and Version: RKE2, v1.35.7+rke2r1
Workarounds / mitigations observed
These are workarounds we applied to fix this issue on our environment:
- Create nvidia-modload.te file:
module nvidia-modload 1.0;
require {
type container_file_t;
type unconfined_service_t;
class system module_load;
}
#============= unconfined_service_t ==============
allow unconfined_service_t container_file_t:system module_load;
- Build and install policy:
checkmodule -M -m -o nvidia-modload.mod nvidia-modload.te
semodule_package -o nvidia-modload.pp -m nvidia-modload.mod
sudo semodule -i nvidia-modload.pp
nvidia-driver-daemonset-6.12.0-160000.5-default-sles16.0-before_kernel_update.txt
nvidia-driver-daemonset-6.12.0-160000.37-default-sles16.0-after_kernel_update.txt
Describe the bug
After upgrading SLES 16.0 nodes from kernel
6.12.0-160000.5-defaultto6.12.0-160000.37-default, thenvidia-driver-daemonset(GPU Operator v26.3.3) fails to load the NVIDIA kernel modules. The driver builds successfully, but the final modprobe step fails with:This is caused by SELinux denial:
The most probable cause why this issue started appearing on newer kernel version a config change (
CONFIG_MODULE_COMPRESS_ZSTD=yandCONFIG_MODULE_DECOMPRESS=y). With these options enabled, the kernel usesfinit_module()to load compressed modules (*.ko.zst) directly from a file instead ofinit_module()from an in‑memory buffer, so SELinux now evaluates the module load against the file label.Adding a policy that grants
module_loadoncontainer_file_tto the GPU Operator driver container’s domain fixes the issue.To Reproduce
6.12.0-160000.5-default.nvidia-driver-daemonsetworks as expected6.12.0-160000.37-defaultnvidia-driver-daemonsetpod is scheduled on the node.You should see denials similar to:
Expected behavior
NVIDIA kernel modules should load successfully on SELinux‑enforcing SLES 16.0 systems after a kernel upgrade.
Environment (please provide the following information):
Workarounds / mitigations observed
These are workarounds we applied to fix this issue on our environment:
nvidia-driver-daemonset-6.12.0-160000.5-default-sles16.0-before_kernel_update.txt
nvidia-driver-daemonset-6.12.0-160000.37-default-sles16.0-after_kernel_update.txt