Skip to content

Add kubernetes events - #348

Open
yairpod wants to merge 2 commits into
trusted-execution-clusters:mainfrom
yairpod:add_Kubernetes_Events
Open

Add kubernetes events#348
yairpod wants to merge 2 commits into
trusted-execution-clusters:mainfrom
yairpod:add_Kubernetes_Events

Conversation

@yairpod

@yairpod yairpod commented Aug 23, 2026

Copy link
Copy Markdown
Member

Emitting kubernetes events on major registration/attestation flow points.
This will allow cluster admins to follow and debug what happens in confidential clusters.

Summary by Sourcery

Add Kubernetes event reporting throughout the registration, attestation, key management, and reference-value computation flows to improve cluster observability and debugging.

New Features:

  • Emit Kubernetes events for key registration, machine registration, attestation-key approval, key provisioning and revocation, and approved-image computation lifecycle events.
  • Add event-based integration coverage for the attestation workflow.

Enhancements:

  • Centralize Kubernetes event recording and controller recorder configuration across services and operators.
  • Grant the operator permission to create and patch Kubernetes events.

Tests:

  • Add utilities for polling Kubernetes events and verify all major attestation flow events end to end.

Adding Kubernetes events to make attestation results more visable for
admins.

Signed-off-by: Yair Podemsky <ypodemsk@redhat.com>
Assisted-by: AI
@sourcery-ai

sourcery-ai Bot commented Aug 23, 2026

Copy link
Copy Markdown

Reviewer's Guide

This PR adds Kubernetes event emission across the registration, attestation, key provisioning, and reference value computation flows, introduces a shared event-recording helper, wires recorders into controllers and HTTP services, and adds tests and utilities to validate the new events end‑to‑end.

Sequence diagram for machine registration events

sequenceDiagram
    participant User
    participant RegisterServer
    participant KubernetesAPI
    participant Recorder
    participant Machine

    User->>RegisterServer: GET register endpoint
    RegisterServer->>KubernetesAPI: create Machine
    KubernetesAPI-->>RegisterServer: created Machine
    RegisterServer->>Recorder: record_event MachineRegistered
    Recorder->>KubernetesAPI: publish Event for Machine
Loading

Sequence diagram for attestation key registration and approval events

sequenceDiagram
    participant Client
    participant KeyRegister as attestation-key-register
    participant KubernetesAPI
    participant Recorder
    participant AKController as ak-controller
    participant Machine
    participant AttestationKey

    Client->>KeyRegister: PUT attestation key
    KeyRegister->>KubernetesAPI: create AttestationKey
    KubernetesAPI-->>KeyRegister: created AttestationKey
    KeyRegister->>Recorder: record_event AttestationKeyRegistered
    Recorder->>KubernetesAPI: publish Event for AttestationKey
    AKController->>Recorder: record_event AttestationKeyApproved
    Recorder->>KubernetesAPI: publish Event for AttestationKey
    AKController->>Recorder: record_event AttestationKeyApproved
    Recorder->>KubernetesAPI: publish Event for Machine
Loading

Sequence diagram for key provisioning events

sequenceDiagram
    participant MachineController as keygen-controller
    participant KubernetesAPI
    participant Trustee
    participant Recorder
    participant Machine

    MachineController->>Trustee: generate_secret
    MachineController->>Trustee: send_secret
    Trustee-->>MachineController: provisioning result
    alt provisioning succeeds
        MachineController->>Recorder: record_event KeyProvisioned
        Recorder->>KubernetesAPI: publish Event for Machine
    else provisioning fails
        MachineController->>Recorder: record_event KeyProvisioningFailed
        Recorder->>KubernetesAPI: publish Warning Event for Machine
    end
Loading

Sequence diagram for reference value computation events

sequenceDiagram
    participant ImageController as rv-controller
    participant KubernetesAPI
    participant ComputationJob
    participant Recorder
    participant ApprovedImage

    ImageController->>ImageController: handle_new_image
    ImageController->>Recorder: record_event ComputationStarted
    Recorder->>KubernetesAPI: publish Event for ApprovedImage
    ComputationJob->>ImageController: job_reconcile
    ImageController->>KubernetesAPI: delete completed Job
    ImageController->>Recorder: record_event ComputationCompleted
    Recorder->>KubernetesAPI: publish Event for ApprovedImage
Loading

File-Level Changes

Change Details Files
Introduce a reusable helper for publishing Kubernetes events and wire in logging support.
  • Add log as a workspace dependency in the shared library to support event publication warnings.
  • Add record_event async helper wrapping kube runtime event Recorder and k8s_openapi ObjectReference / Event types.
  • Keep helper tolerant of absence of a Recorder (Option) and log a warning on publish failure instead of failing the controller logic.
lib/Cargo.toml
lib/src/lib.rs
Emit events during attestation key registration via the attestation-key-register HTTP service.
  • Introduce AppState to hold both kube Client and Recorder and switch axum handlers to use this state instead of raw Client.
  • Create a Reporter/Recorder for the attestation-key-register service and pass it through Router state.
  • On duplicate-key detection, publish a Warning event 'DuplicateKeyRejected' referencing the existing AttestationKey.
  • On successful AttestationKey creation, publish a Normal event 'AttestationKeyRegistered' referencing the new AttestationKey.
attestation-key-register/src/main.rs
Emit events when machines are registered via the register-server HTTP service.
  • Refactor register-server to use an AppState with Client and Recorder instead of bare Client state.
  • Construct a Reporter/Recorder for the register-server controller and store it in application state.
  • Change create_machine to return the created Machine resource so it can be used as the event’s regarding object.
  • After successfully creating a Machine, publish a Normal 'MachineRegistered' event referencing the Machine.
register-server/src/main.rs
Add event recording to operator controllers for attestation-key approval, key provisioning/revocation, and reference value computation, while standardizing controller context with a Recorder.
  • Extend AkContextData with an optional Recorder, initializing it via a new operator::new_recorder helper.
  • On attestation key approval, emit paired Normal 'AttestationKeyApproved' events: one regarding the AttestationKey and one regarding the Machine.
  • Introduce ControllerContext struct (client + optional Recorder) and use it in reference_values and register_server controllers instead of passing Arc<Client directly.
  • In reference_values job_reconcile, emit Normal 'ComputationCompleted' events regarding ApprovedImage owners once reference values are updated.
  • In reference_values image_add_reconcile, emit Normal 'ComputationStarted' when PCR computation begins and Warning 'ComputationFailed' when it fails, both regarding the ApprovedImage.
  • In register_server keygen_reconcile, emit Normal 'KeyProvisioned' on successful key generation and send, Warning 'KeyProvisioningFailed' on failure, and Normal 'KeyRevoked' when decryption keys are deleted during cleanup.
  • Update controller launch functions (rv_job, rv_image, keygen) to construct ControllerContext with Recorder via new_recorder and pass that context into Controller::run.
  • Adjust tests to use the new ControllerContext abstraction instead of Arc<Client.
operator/src/lib.rs
operator/src/attestation_key_register.rs
operator/src/reference_values.rs
operator/src/register_server.rs
Extend RBAC and test utilities to support and validate event emission, and add an integration test for the complete event flow.
  • Update kubebuilder RBAC annotations to grant create/patch permissions on events.k8s.io events.
  • Add wait_for_event utility using Api, ListParams, and Poller to poll for events by regarding resource name and reason with a timeout.
  • Add attestation integration test that runs a VM, performs attestation, discovers Machine, AttestationKey, and ApprovedImage resources, and waits for eight specific events: ComputationStarted, ComputationCompleted, MachineRegistered, AttestationKeyRegistered, AttestationKeyApproved (on AK and Machine), KeyProvisioned, and KeyRevoked.
  • Use existing wait_for_resource_deleted helper to trigger and then verify KeyRevoked via event after Machine deletion.
api/v1alpha1/crds.go
test_utils/src/lib.rs
tests/attestation.rs

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@openshift-ci

openshift-ci Bot commented Aug 23, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: yairpod

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="api/v1alpha1/crds.go" line_range="36" />
<code_context>
 // +kubebuilder:rbac:groups=trusted-execution-clusters.io,resources=trustedexecutionclusters;machines;approvedimages;attestationkeys,verbs=create;delete;get;list;patch;update;watch
 // +kubebuilder:rbac:groups=trusted-execution-clusters.io,resources=trustedexecutionclusters/finalizers;machines/finalizers;attestationkeys/finalizers;approvedimages/finalizers,verbs=update
 // +kubebuilder:rbac:groups=trusted-execution-clusters.io,resources=trustedexecutionclusters/status;machines/status;approvedimages/status;attestationkeys/status,verbs=get;patch;update
+// +kubebuilder:rbac:groups=events.k8s.io,resources=events,verbs=create;patch

 // TrustedExecutionClusterSpec defines the desired state of TrustedExecutionCluster
</code_context>
<issue_to_address>
**issue (bug_risk):** The event recorder publishes `events.k8s.io` Events, but the checked-in operator RBAC grants `create;patch` only for core `events` (`apiGroups: [""]`), not `events.k8s.io`. Every `record_event` call therefore receives a Kubernetes authorization error in deployed clusters, which is only logged and leaves the new events absent.

**Triggers:** When the checked-in RBAC manifests are deployed without regenerating them to add the `events.k8s.io` rule.

**Suggested fix:** Add `apiGroups: ["events.k8s.io"]` with `resources: ["events"]` and `verbs: ["create", "patch"]` to the deployed operator RBAC, and regenerate all packaged manifests.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 1 finding to address first, and if the event logic is wrong, Kubernetes Event objects can be emitted with incorrect or overly frequent messages and remain after the code is reverted, though they are bounded and can be deleted. The added RBAC grant also changes what these workloads may write in the cluster, but it does not grant access to application data or alter the underlying provisioning decisions.

Blocking findings: api/v1alpha1/crds.go:36


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread api/v1alpha1/crds.go
@yairpod
yairpod force-pushed the add_Kubernetes_Events branch 2 times, most recently from 144447f to 051366e Compare August 23, 2026 14:20
Comment thread operator/src/lib.rs
}

pub struct ControllerContext {
pub client: Client,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use cache(aka AkContextData) here insted of client?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NB this would change a bit with #330

Add a test for the attastation basic events.

Signed-off-by: Yair Podemsky <ypodemsk@redhat.com>
Assisted-by: AI
@yairpod
yairpod force-pushed the add_Kubernetes_Events branch from 051366e to abfe429 Compare August 24, 2026 06:04
@openshift-ci

openshift-ci Bot commented Aug 24, 2026

Copy link
Copy Markdown

@yairpod: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/azure-integration-test abfe429 link false /test azure-integration-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Comment thread operator/src/lib.rs
}

pub struct ControllerContext {
pub client: Client,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NB this would change a bit with #330

Comment thread lib/src/lib.rs
}

pub async fn record_event(
recorder: Option<&Recorder>,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When is this None?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants