gpu: tier selection — one setting, three values, and off means off

[dpsoft]

Task 11 of the GPU PC sampling plan.

PERFAGENT_GPU_PC_SAMPLING = off (default) | continuous | serialized, with 0/1/2 still accepted on both ends since container specs already carry them. One parser per side — gpu/tier.go and shim/core/pctier.h — with a test that reads the header from Go and pins that both accept the same six spellings. Surfaced as --gpu-pc-sampling on cmd/gpu-cuda-profile.

One field, not two

TimelineConfig.SerializedSampling bool was replaced by TimelineConfig.PCSampling PCSamplingTier, not joined by it: two fields that can disagree about which tier is running is how a profile discloses one thing and does another.

The tier rides out on Snapshot.PCSampling, because "Tier A asked for, no window arrived" and "Tier A never asked for" are different facts — and inferring them from SamplingWindowsReceived == 0 gets them backwards.

Why exclusivity is process-wide, not just forbidden

COLLECTION_MODE is a single per-CUcontext CUPTI attribute. A process could set different modes on different contexts — but which context a kernel lands on is the application's choice, not the profiler's, so a "both" mode would produce a profile whose attribution quality varied along an axis the operator can neither see nor control. Worse than either tier alone.

Refused in three shapes — one value naming both tiers, flag and environment naming different tiers, unknown value — each a startup error that falls closed to off and carries the COLLECTION_MODE reason rather than the rule alone. "Both" is deliberately parseable so the refusal is reachable, and both orderings are tested, since a first-token parser answers differently for each.

The acknowledgement gate and the standing warning

serialized requires --gpu-pc-sampling-acknowledge-perturbation, and the refusal text itself names all three perturbations. joinhealth then prints a four-line standing warning on every render — including on a Tier A snapshot where nothing went wrong, which is exactly the profile whose reader has no other way to learn the tier was on. Counted separately from anomalies, with len(lines)-1 == warnings + anomalies asserted.

All three perturbations, because an operator told only the first will misread the other two: GPU kernel durations inside a burst are inflated and marked gpu_serialized="true"; CPU and off-CPU samples during a burst are distorted and carry no marking at all — serialization inflates precisely the sync wait off-CPU profiling exists to measure; and Tier A is unavailable where CUDA graphs are in use.

Off means off, asserted at the probe site

shim/stub/pc_tier_test.cc patches the four PC-sampling probe nops with int3 — what a uprobe does — and counts traps. Tier off ⇒ zero fires with the stub's PC knobs turned up, while launch/exec probes must still fire so an inert producer cannot pass. The continuous and serialized passes fire those same sites, so the negatives are not vacuous. Mutation-checked: removing the stub's tier gate makes it fail and name the probe.

This required the stub to honour the tier at all — it is now the outer gate over its own knobs, which no existing test sets.

Two defects found in already-merged code

The "unknown" anomaly was guarded by SamplingWindowsReceived > 0, which suppressed it in the worst case: Tier A selected, not one window received, every execution unknown. unknown is unreachable in the other tiers, so the guard only hid the case that needed it most. It now names which cause applies rather than disappearing.

anomalousSnapshot() carried the three serialization counters with PCSampling off — a combination that cannot occur — and would have quietly stopped exercising the warning a real Tier A run carries.

Verification

make -C shim / test (incl. new pctier_test and pc_tier_test 8/8) / check-fpless / check-cubin-defer / nvidia; go build/vet; the four-package test list; -race -count=4; golangci-lint 0 issues. No ABI, BPF or .o churn; Tier A/B behaviour, cubin capture, the CubinView guard and the MODULE_UNLOAD_STARTING drain untouched.

Cannot verify (CapEff: 0, no GPU — no CUpti_ path here has run): the adapter's tier parse ordering relative to CONTEXT_CREATED, g_pc_tier_refused on hardware, and that an explicit =off wins in a real injected process's environ.