fix: set informer handler context before registering handlers. Fixes …

[Megh03]

Fixes #951

Change

Assign resourceEventHandlerCtx before calling AddEventHandler, in jobWatcher, podWatcher and completionsWatcher.

Adds a regression test per watcher that registers against an already started and synced informer. These only fail under -race as, a nil context is harmless to the fake clientset, so tests.sh gains a -race step scoped with -run. The scope is deliberate: the watcher constructors write package-level gauge funcs that parallel tests race on, so the package isn't race-clean yet. Follow-up to come; the filter can be dropped once it is.

Context

On an already-started informer, client-go starts the listener goroutines inside AddEventHandler and then enqueues a synthetic. Add for every cached object, so OnAdd can run before the assignment and read a nil context. That's the normal startup path: the deduper and limiter have already started and synced the shared Jobs informer by the time jobWatcher registers. A finished Job with a job acquisition token Secret then reaches secrets.Get with a nil context and panics inside client-go, crash-looping the controller.

Assigning before AddEventHandler closes this rather than narrowing it . The write is sequenced before the go statement that starts the listener, so the memory model guarantees callbacks observe it. All three watchers share the bug; podWatcher and completionsWatcher are on the Pods informer, and podWatcher registers second whenever the completion watcher is enabled.

Verified against a real cluster with 150 finished annotated Jobs: the controller crashed on startup 5 times out of 5 before the fix (exit 2, stack as in #951), 0 out of 5 after, with the token Secret cleaned up within ~3s.

Affiliation (optional, external contributors)

N/A

Disclosures / Credits

Diagnosis and suggested fix from the reporter in #951 were both correct. Used some assistance from Claude to diagnose the fix. The fix has been verified against a real cluster before submission.