diff --git a/.superpowers/sdd/task-10-tier-a-report.md b/.superpowers/sdd/task-10-tier-a-report.md new file mode 100644 index 0000000..a90c216 --- /dev/null +++ b/.superpowers/sdd/task-10-tier-a-report.md @@ -0,0 +1,420 @@ +# Task 10 — Tier A: `KERNEL_SERIALIZED` PC sampling, duty-cycled, and its disclosure + +Branch `feat/tier-a-serialized`, one commit on top of `origin/main` (`965e0a48`, the Task 6 +merge). Tier A is **off by default**: `PERFAGENT_GPU_PC_SAMPLING` is unset, and unset means no +burst controller, no burst timer, no `cuptiPCSamplingStart`, no window on the wire and +`gpu_serialized="false"` unconditionally on every execution. + +Every CUPTI API this task needs exists in CUDA 13.3 exactly as the plan describes: +`CUPTI_PC_SAMPLING_COLLECTION_MODE_KERNEL_SERIALIZED`, +`CUPTI_PC_SAMPLING_CONFIGURATION_ATTR_TYPE_ENABLE_START_STOP_CONTROL` with +`enableStartStopControlData.enableStartStopControl`, `cuptiPCSamplingStart`, +`cuptiPCSamplingStop`. The header also states the flush rule this task turns on: + +> Flushing of GPU PC Sampling data is required at following point to maintain uniqueness of PCs: +> … If configuration option `ENABLE_START_STOP_CONTROL` is enabled, then after every range end +> i.e. `cuptiPCSamplingStop()` + +Nothing was improvised around a missing API. + +--- + +## The burst controller, and its convergence proof + +`shim/core/burst.h` + `shim/core/burst_test.cc`. No CUPTI, no clock, no allocation, no threads: +the caller supplies `now_ns` and the running `(PC, stall)` pair count, so the whole thing runs +against a fake clock on a machine with no NVIDIA hardware. + +**The loop is a pure function of (target rate, observed rate, elapsed):** + +``` +observed_rate = pairs / elapsed pairs per second +ratio = observed_rate / target_rate >1 means too many +cycle = elapsed * ratio the cycle that would have hit target +raw_gap = max(cycle - burst_ns, 0) +gap = prev_gap + gain * (raw_gap - prev_gap) +gap = clamp(gap, min_gap, max_gap) +``` + +`burst_next_gap_ns(cfg, prev_gap, pairs, elapsed)` is a free function and is tested directly, +independently of the state machine. + +**Convergence.** The model, stated in the header rather than left implicit: pairs are produced +only while a burst is open, so pairs-per-burst does not depend on the gap. Under that model +`raw_gap` is a constant `P/target - burst` for a steady workload, and the damped update is a +geometric sequence with ratio `(1 - gain)`. Asserted three ways: + +- the analytic fixed point — 100 pairs per 50 ms burst at a 100/s target wants a 1 s cycle, so + `gap* = 950 ms`. The simulation lands on **949 ms** and an achieved **102.4 pairs/s** over 60 + cycles including the un-converged ramp; +- convergence from both directions — starting at the floor (450 ms) and at the ceiling (10 s), + 40 iterations later the two agree to within 1 ms and both sit in (900 ms, 1000 ms); +- the fixed point is interior to both clamps in that scenario, so the loop has to actually find + it rather than being pinned by a bound. + +**The duty ceiling is a hard bound, not a target.** `min_gap = burst * (1/max_duty - 1)` — 450 ms +for a 50 ms burst at 10% — and `burst_next_gap_ns` clamps to it on every path. The test does not +argue this, it **sweeps** it: eleven pair counts from 0 to `UINT64_MAX` crossed with six previous +gaps, asserting `gap >= min_gap` and `burst/(burst+gap) <= max_duty` on all 66. Degenerate +configuration lands on a clamp rather than on undefined behaviour: `target_rate = 0` (ratio +inf) +→ `max_gap`; `target_rate` NaN → floor; `max_duty = 0` → a positive floor; `elapsed = 0` → hold. +Every comparison is written `!(x > y)` rather than `x <= y` so NaN falls to the safe side. + +**A zero observed rate does not drive the gap to zero.** It drives it to `min_gap` — 450 ms — and +stops: 100 iterations at zero pairs converge on exactly `burst_min_gap_ns(cfg)`, which is `> 0`. +An idle GPU therefore samples at the duty ceiling and no faster, which is the opposite of the +failure the assertion exists to exclude. A whole-run simulation of an idle workload achieves +duty **0.102** against a 0.10 ceiling — the 2% is the burst timer's 5 ms granularity on a 50 ms +burst, asserted with the granularity in it rather than fudged. + +**One design correction the test caught.** The first draft read the pair count inside `poll()` at +stop time. CUPTI hands a burst's PC records over on the flush that *follows* the stop, so the +controller measured every burst as having produced nothing and sat at the duty floor forever — +converged-looking, and wrong. The cycle is now three calls (`poll → kStart`, `poll → kStop`, +`closed(pairs_total)` after the range-end drain), and the reason is written at the class. A caller +that forgets `closed()` degrades rather than breaks: the stop schedules the next burst at the +current gap, so the duty cycle keeps running and only the loop stops adapting. + +--- + +## The three `gpu_serialized` values + +`gpu/serialization.go` holds the evidence and answers the question; `gpu/timeline.go` calls it +once per execution at `Snapshot`; `gpu/projection.go` writes the label. + +**The zero value is `"unknown"`.** `SerializationState` is an integer enum with +`SerializationUnknown = iota`. This is the strongest available form of "unknown must never +degrade to false": a field nobody set, a struct built by a test, a value lost in a copy and a code +path that forgot to classify all land on `"unknown"`, because `"false"` has to be written +deliberately. `TestSerializedLabelIsUnconditionalAndHasThreeValues` includes a view nobody +classified and asserts it renders `"unknown"`. + +**The rule, in order:** + +1. intersects a **closed, kernel-serialized** burst → `"true"`. Definite, and it outranks + `"unknown"` — an execution that provably overlapped a burst is perturbed whatever else is + unknown about the rest of its interval (`TestSerializationTrueOutranksUnknown`); +2. wholly inside a span the store holds an **unbroken** history for, touching no burst → + `"false"`. This is the only branch in the file that returns `"false"`, and it needs positive + evidence on both endpoints; +3. everything else → `"unknown"`. + +**Coverage, and why it is not simply "the earliest to the latest window".** The store tracks a +`coverageStart` that moves **forward only**, on two events: a sequence gap (records were lost, so +nothing before the hole can be shown to be contiguous) and an eviction (the oldest burst left). +`coverageEnd` is the earliest **open** window's start if there is one, otherwise the end of the +last closed burst — deliberately *not* extending past it, because the next burst's open record +may simply not have been drained yet, so the interval after the last known window is not a proven +gap. Executions in that trailing interval read `"unknown"`; on a 60 s profile with a 100 ms drain +that is a small tail, and it is honest. + +**`"unknown"` never becoming `"false"`, pinned.** +`TestSerializationFalseIsOnlyEverReachedFromPositiveEvidence` drives four window histories (none, +one closed burst, two closed bursts, a burst that never closed) against 324 execution intervals +each, and for **every** execution that came out `"false"` asserts containment in the covered span +and non-overlap with every burst. It also asserts the reverse for the two histories that prove no +gap at all: zero `"false"`s. That is the test the plan asks for, and it is a sweep rather than a +sample because the claim is universal. + +Additionally pinned: no windows at all → all `"unknown"` and +`ExecutionsNotSerialized == 0`; outside the covered span → `"unknown"`; a sequence gap restarts +coverage so a previously-provable gap **degrades** to `"unknown"` while the bursts already held +still prove `"true"`; eviction moves answers `"false" → "unknown"` and the test asserts that +direction by classifying the same execution before and after; a window whose mode the producer +left unset is opaque (neither `"true"` nor `"false"`); an inverted window is refused; a PID past +the store's bound reads `"unknown"` rather than inheriting somebody else's history. + +**In Tier B and with sampling off, `"false"` is correct and unconditional.** +`TimelineConfig.SerializedSampling` is the agent's own configuration — Task 11 owns the setting +that flips it — and when it is false the window store is not consulted at all. Windows that arrive +anyway (a leftover producer, a system-wide attach) are still ingested and counted; only the answer +is unconditional (`TestSerializationIgnoresWindowsWhenTierAWasNotSelected`). + +--- + +## The open window, and the two-record protocol it required + +**`end_ns == 0` is open, not zero-length.** Making that reachable took a protocol decision the +plan implies but does not spell out: a burst reaches the wire **twice**. + +- the **open** record goes out the instant `cuptiPCSamplingStart` succeeds, with `end_ns = 0`; +- the **closed** record goes out on the stop, with the same `start_ns` and a real `end_ns`. + +Emitting only on the stop would lose the entire burst on a hard exit, and the executions inside it +would then read `"false"` — "not perturbed" when the truth is "cannot tell", the one answer that +must never be reachable by accident. With the open record already delivered, a `SIGKILL` leaves a +window saying a burst was open from `start_ns` and never closed. + +The consumer's store supersedes one-way: a closed record replaces an open one with the same start, +an open record **never** replaces a closed one. Both delivery orders are tested +(`TestSerializationClosedWindowSupersedesItsOwnOpenRecord`), so a lossy transport cannot leave a +permanently-open window behind an ordinary duty cycle, and `Snapshot.SamplingWindowsOpen` stays a +real signal. + +`TestSerializationOpenWindowMakesEverythingFromItsStartUnknownAndNeverFalse` sets up a completed +burst first — so the store has genuine coverage and a naive implementation would happily answer +`"false"` after it — then opens a burst that never closes, and sweeps 41 executions across the +whole timeline asserting both halves on every one at or after the open start: it **is** +`"unknown"`, and it is **not** `"false"`. + +**`end_ns == 0` means a hard exit specifically.** `at_exit_handler` stops the burst timer first, +then `burst_shutdown()` closes the open window with the exit timestamp, then `on_finalize` runs. +On the CUPTI fatal-error path `on_finalize` calls `g_burst->shutdown()` (which takes only the +controller's own mutex, never `g_pc_mu`, which that path may fail to acquire) and deliberately +does **not** close the window — a CUPTI fatal error *is* the hard case. + +One correction to the `Stats.SamplingWindowsOpen` doc that Task 7 wrote: it said "healthy: zero". +With the open record emitted at every burst start, a healthy Tier A run produces one per burst and +this wire-side counter tracks `SamplingWindowsDecoded / 2`. The anomaly is a window still open +*in the store* at snapshot time, which is `Snapshot.SamplingWindowsOpen`. Both doc comments now +say which is which. + +--- + +## The CUDA-graph refusal + +Structural, in the pure controller, so it is testable with no GPU. `BurstController::poll(now, +graphs_observed)` — once `graphs_observed` reads true, the controller latches `refused_`, returns +`kStop` if a burst is open (so the window closes honestly rather than being abandoned) and +**never returns `kStart` again**. It does not know how to become Tier B, which is the point: a +silent downgrade would leave the operator reading a Tier B profile while believing they asked for +Tier A. + +Two moments in the adapter: + +- **at enable time** — `pc_enable_ctx` refuses outright if `g_exec_from_graph != 0`, logs the + reason at length and counts `g_ctx_enable_failed`. PC sampling is not enabled for that context + at all; +- **on every burst tick** — a process can run for minutes before its first graph launch. When one + arrives the open burst is cut short, `g_tier_a_graph_refused` increments once (not per poll) and + a loud log line says bursts have stopped permanently. Executions already inside a window stay + marked serialized, which is correct: they really were. + +`burst_test.cc` covers both: a graph mid-burst (`kStop`, `last_stop_reason() == kGraph`, the 10 ms +of real perturbation still accounted in `burst_ns()`, then 1000 further polls all `kNone`), and a +graph observed before the first burst (`bursts() == 0`, `burst_ns() == 0`, `graph_refusals() == 1` +— refuses to *start*, distinct from stopping). + +The condition also already rides the wire as `gpu_dropped_v1 / GPU_DROP_CLASS_GRAPH_EXEC`, emitted +before the tier gate in `on_tick` (Task 6). + +--- + +## The sum identity + +`ExecutionsSerialized + ExecutionsNotSerialized + ExecutionsSerializationUnknown == +len(Snapshot.Executions)`, exactly. + +It holds **by construction**: the three counters are incremented where the `ExecutionView` is +built, before any of the join loop's four `continue`s, so every execution is counted exactly once +on every path through `Snapshot`. + +It is asserted in three places: + +- `assertSumIdentity` runs in every one of the 17 tests in `gpu/serialization_test.go`; +- `assertSerializationOutcomesAccounted` is now part of `assertConformanceInvariants`, so it runs + on **every** conformance scenario in the file, including the ones that emit no windows at all — + and it additionally asserts the negative for the default harness (nothing serializes kernels + there, so no execution may read `"true"` or `"unknown"`); +- `joinAnomalies` raises it at runtime, in the same place and with the same wording as the join + outcomes' identity: *"serialization outcomes sum to N but the snapshot holds M executions — some + execution carries no gpu_serialized disclosure at all"*. + +The counters the task named, and where each lives: + +| counter | side | where | +| --- | --- | --- | +| `SamplingBursts` | producer | `g_sampling_bursts`, adapter log `tier A bursts=` | +| `SamplingBurstNs` | producer | `g_sampling_burst_ns`, with `duty=` beside it | +| `SamplingWindowsReceived` | agent | `Snapshot.SamplingWindowsReceived` (cumulative) | +| `ExecutionsSerialized` | agent | `Snapshot`, per snapshot | +| `ExecutionsNotSerialized` | agent | `Snapshot`, per snapshot | +| `ExecutionsSerializationUnknown` | agent | `Snapshot`, per snapshot | + +The producer/consumer split is deliberate: an agent-side `SamplingBursts` would equal +`SamplingWindowsReceived` by construction and could never disagree with it, which is not a +counter. Split across the two ends, the **gap between them is the loss**. Alongside them: +`Snapshot.SamplingWindowsHeld` / `SamplingWindowsOpen` (gauges), +`Dropped.EvictedSamplingWindows`, `SinkStats.SamplingWindows`, and on the shim +`g_windows_emitted`, `g_burst_start_failed`, `g_burst_stop_failed`, `g_tier_a_graph_refused`, +`PCDrainSchedule::range_end()`. + +Every one of them can go non-zero from a test. + +--- + +## What else changed, and what deliberately did not + +**`shim/core/pcdrain.h`** gains `PCDrainReason::kRangeEnd` and its own counter. The range-end +flush is mandatory in this configuration exactly as the module-unload flush is in CONTINUOUS — +missing it does not lose data, it makes two instructions share a PC identity, silently. It also +marks the **shared** schedule's phase, so the 100 ms drain tick coalesces instead of repeating the +pull microseconds later. `pcdrain_test.cc` asserts five range-end drains inside one period, that +they are counted apart from unload and teardown drains, and that the phase moved. + +**A second timer, and this is a deviation from the plan's wording.** The plan says "the existing +drain timer is its natural home". The flush is: it goes through the same `pc_drain_all` and the +same `PCDrainSchedule`. The burst *cycle* is not, and cannot be — the drain tick is 100 ms and a +50 ms burst cannot be expressed on it. Quantizing the burst to the drain period would silently +double the burst length and therefore the duty fraction, which is the one number this tier exists +to bound. So the burst rides a `Drainer` of its own at `burst_ns/5` (10 ms by default), doing an +atomic load and a compare when no transition is due. The reason is written at the declaration. + +**Task 6's `on_tick` hazard was not reintroduced.** The gate at the bottom of `on_tick` was +`if (!g_pc_tier_b) return;` and is now `if (!g_pc_enabled) return;` — a rename of the same +variable, which is now set by `PERFAGENT_GPU_PC_SAMPLING != 0` and so covers both tiers. Nothing +was appended below it. The cubin `drain()` and the `classGraphExec` emission are still **above** +it, with their comments intact; `make -C shim nvidia` and the probe listing confirm all nine +pre-existing probes plus the new one. + +**Nothing weakened.** Tier B's configuration is unchanged (`ENABLE_START_STOP_CONTROL` is still +deliberately off there, and the comment now says why in both directions). The cubin capture, the +`CubinView` guard (`make -C shim check-cubin-defer`: OK, all 5 deferrals still refuse to compile) +and the `MODULE_UNLOAD_STARTING` drain are untouched — `on_resource`'s switch still owns +`MODULE_UNLOAD_STARTING`, still calls `pc_drain_all` and still **blocks** on `g_pc_mu`. + +**Tier selection is not implemented.** `PERFAGENT_GPU_PC_SAMPLING` gained the value `2` for Tier +A, in the same variable as `1` — so the two tiers are mutually exclusive *by construction* rather +than by a check that can be forgotten, which is what Task 11 wants anyway. There is no runtime +switch and no CLI flag; `TimelineConfig.SerializedSampling` defaults false and nothing sets it +yet, so a merge of this branch changes nothing a shipping profiler does. + +**`gpu_serialized` reaches `gpu/projection.go` in this task**, though the plan files the label set +under Task 9. A tri-state that never reaches the profile is exactly the "counter reading green" +shape the constraints warn about, and the addition is three lines plus a comment. Task 9's other +labels are untouched. + +**`notYetFired` is now empty.** `TestEveryProducerProbeHasACookieAndViceVersa` failed on the first +run after the adapter started firing `gpu_sampling_window_v1` and named the fix in its own message, +exactly as it did for `gpu_module_load_v1` in Task 5. Every probe the ABI defines is now fired by a +producer. + +**The stub emits windows.** `PERFAGENT_STUB_SAMPLING_WINDOWS=` cuts the executions' span into +`2n` slices and makes every other one a burst, emitting the open-then-closed pair for each, so +roughly half the executions fall inside a window and half in a proven gap. +`PERFAGENT_STUB_SAMPLING_WINDOW_OPEN=1` leaves the last burst open — the hard-exit shape. With +both unset the stub's wire is byte-for-byte what it was. + +**No ABI change and no BPF change.** `gpu_sampling_window_v1`, `KIND_SAMPLING_WINDOW = 8` and +`REC_SAMPLING_WINDOW` have been in place since Task 2; this task is the first producer. +`git status` shows no `.o` churn. + +--- + +## Verification actually run + +``` +make -C shim exit 0 +make -C shim test exit 0 (burst_test OK, pcdrain_test OK, check-cubin-defer OK, + cubinqueue_test OK, probe_order_test OK, usdt_abi_test OK) +make -C shim check-fpless OK +make -C shim check-cubin-defer OK - the compliant capture compiles, all 5 deferrals do not +make -C shim nvidia exit 0 (CUDA 13.3, real libcupti) +go build ./... && go vet ./... clean +go test ./gpu/ ./gpuprobe/ ./internal/... -count=1 all pass +go test ./gpu/ ./gpuprobe/ -race -count=4 ok gpu 19.7s, gpuprobe 18.8s +~/go/bin/golangci-lint run --timeout=5m 0 issues +``` + +Ten probes in the built adapter, one exported symbol: + +``` +$ readelf -n shim/libperfagent-gpu-nvidia.so | grep -o 'gpu_[a-z_0-9]*' | sort -u +gpu_config_v1 gpu_dropped_v1 gpu_exec_v1 gpu_kernel_name_v1 gpu_launch_sampled_v1 +gpu_launch_v1 gpu_module_load_v1 gpu_pc_sample_batch_v1 gpu_sampling_window_v1 +gpu_stall_reason_map_v1 +$ nm -D --defined-only shim/libperfagent-gpu-nvidia.so +0000000000004c40 T InitializeInjection +``` + +New tests: + +- `shim/core/burst_test.cc` — the duty ceiling swept across the whole observed-rate range, + convergence to the analytic fixed point from both directions, the zero-rate floor, degenerate + configuration, the start/stop state machine over 2,000 ticks, the graph refusal in both its + shapes, and shutdown closing an open burst. +- `shim/core/pcdrain_test.cc` — the range-end drain: mandatory, counted apart, resets the phase. +- `gpu/serialization_test.go` — 17 tests; the three values, the open-window sweep, the supersede + protocol in both delivery orders, the sequence-gap restart, cross-process isolation, eviction's + direction, degenerate records, and the universal "false only from positive evidence" sweep. +- `gpu/projection_test.go` — the label is unconditional, has three values, degrades to `"unknown"` + when unclassified, reaches every PC-derived sample, and beats a forged `Tags` entry. +- `gpu/conformance_test.go` — the sum identity on every scenario. +- `gpu/sink_test.go` — a window draws on the anchor budget, not the data one, so exec volume + cannot starve the disclosure out of the sink. +- `gpu/joinhealth_test.go` — the three outcomes in the summary and the three Tier A anomalies. +- `gpuprobe/consumer_test.go` — the window reaches the sink with its process, the sequence gap + rides the first record of its batch and no other, sequences are per-process, and a refused + window is counted. + +--- + +## Cannot verify — every item needs the RTX 3090 + +Nothing below has been executed. `CapEff: 0`, no GPU on this machine, and **no `CUpti_` code path +added by this commit has ever run**. + +**The plan's four:** + +1. That `correlationId` is non-zero on ≥99% of PC records in `KERNEL_SERIALIZED` mode (the spike + says 1,828/1,828). `Stats.PCSamplesWithoutCorrelation` is the counter that will say; a single + non-zero there breaks Tier A's whole claim. +2. That windows actually bracket the executions that ran in them — i.e. that the mono-clock + timestamps this adapter stamps around `cuptiPCSamplingStart`/`Stop` and the converted activity + timestamps on `gpu_exec_v1` land in the same domain closely enough for the intersection to + mean what it says. **This is the single most load-bearing unverified item in the task**: the + whole disclosure is an interval intersection, and a systematic skew between the two clocks + would mis-mark executions in a way no counter here can detect. +3. **Whether the collection mode can be changed between `Stop` and `Start` without a full + `Disable`/`Enable`.** Undocumented in `cupti_pcsampling.h`, and it decides whether a runtime + tier switch is possible at all. Nothing in this commit depends on the answer — the mode is set + once, at context creation — but Task 11's "switching tiers mid-run is out of scope, gated on + this" cannot be resolved without it. +4. Tier A's overhead (Task 12), which decides whether this tier ships. + +**Discovered while implementing, and added to that list:** + +5. **Whether `ENABLE_START_STOP_CONTROL` is accepted alongside `KERNEL_SERIALIZED` in one + `cuptiPCSamplingSetConfigurationAttribute` call.** Task 6's per-attribute `attributeStatus` + check will catch a refusal loudly (the context disables and `ctx_enable_failed` moves), but + which attributes may be combined is not stated in the header. +6. **Whether `cuptiPCSamplingStart` may be called on a context that has never sampled, and + whether `Stop` on an already-stopped context errors.** The adapter starts and stops every + tracked context on each burst; a context created mid-burst is started on the *next* one, so it + misses part of a window it is nonetheless covered by. `g_burst_start_failed` / + `g_burst_stop_failed` count, and must be 0. +7. **Whether the range-end `cuptiPCSamplingGetData` actually returns the burst's records + synchronously.** The closed-loop controller's yield measurement assumes it does. If CUPTI + defers them to a later flush, the loop sees a lagged pair count — it still converges (the lag + is a constant offset for a steady workload) but the transient after a workload change is one + cycle longer than modelled. Nothing breaks; the settling time is wrong. +8. **The real pairs-per-burst figure**, which decides whether the fixed point is interior to the + clamps at all. The spike saw 352 PC records for ~103k samples in CONTINUOUS over an unknown + interval. If a 50 ms serialized burst yields thousands of pairs, `max_gap_ns` (10 s) binds and + the achieved rate stays above the 100/s target — visible in the adapter's `gap_ns=` reaching + 10000 ms, and the remedy is a longer `max_gap` or a shorter burst. +9. **Whether serialization visibly inflates `gpu_exec_v1` durations**, i.e. whether the + disclosure is measuring something real. The obvious check on hardware: the same workload in + Tier A, comparing the duration distribution of `gpu_serialized="true"` executions against + `"false"` ones **in the same profile**. If they do not separate, either the duty cycle is not + doing what it says or the window intersection is mis-aligned (item 2). +10. **Whether `g_ctx_enabled` is a sufficient guard on the burst timer starting first.** The + timer starts at the end of `InitializeInjection`, before any `CONTEXT_CREATED` callback can + have fired, so `on_burst_tick` refuses to open a burst until at least one context has enabled + — otherwise it would announce a window covering executions nothing was sampling. Whether a + context can *fail* to enable after that (leaving `ctx_enabled` non-zero with no live context) + is unmeasured; if it can, a window would be announced with nothing behind it, over-stating + perturbation for one burst. Safe direction, but wrong. +11. **Whether `cuptiPCSamplingStop` can re-enter our resource callback**, the same hazard Task 6 + listed for `GetData`. `burst_close` holds `g_pc_mu` across `Stop` and then across the drain; + a re-entrant `MODULE_UNLOAD_STARTING` on the same thread would self-deadlock. +12. **The burst timer's real granularity under load.** The 5 ms figure in the test is a fake + clock; a 10 ms `sleep_for` on a loaded machine can overshoot, and every millisecond of + overshoot is duty the ceiling did not authorise. `duty=` in the adapter's report is the + measurement, and it is reported precisely because `max_duty` bounds what the loop may *ask* + for and not what the OS delivers. +13. **Whether `g_exec_from_graph` moves early enough for the refusal to be useful.** It is set + from `CUpti_ActivityKernel12.graphId` on the activity path, which arrives on the drain tick — + up to 100 ms, and one or two bursts, after the first graph kernel actually ran. Those bursts' + windows are emitted and their executions are marked, so nothing is silent; but Tier A does run + briefly in a graph-using process before refusing, and the size of that window is unmeasured. diff --git a/gpu/conformance_test.go b/gpu/conformance_test.go index 4c2626a..694e98e 100644 --- a/gpu/conformance_test.go +++ b/gpu/conformance_test.go @@ -100,6 +100,7 @@ type attemptSink struct { pcAttempts uint64 moduleAttempts uint64 eventAttempts uint64 + windowAttempts uint64 } func newAttemptSink(inner EventSink) *attemptSink { @@ -135,6 +136,11 @@ func (a *attemptSink) EmitEvent(e GPUTimelineEvent) error { return a.inner.EmitEvent(e) } +func (a *attemptSink) EmitSamplingWindow(w GPUSamplingWindow) error { + a.windowAttempts++ + return a.inner.EmitSamplingWindow(w) +} + // conformanceHarness is a fresh Timeline + CountingSink pair, wired the way a // real backend is: producer -> CountingSink (admission control) -> // Timeline (join point). The clock is frozen (never advanced) so the token @@ -254,6 +260,14 @@ func assertConformanceInvariants(t *testing.T, h *conformanceHarness) Snapshot { assertPCSampleLossesAccounted(t, snap, sinkStats, h.attempt.pcAttempts) assertPCAttribAccompaniesSamples(t, snap) + // The serialization disclosure's sum identity, on EVERY scenario in this + // file rather than only on the ones that emit windows. That is the point: + // the three outcomes are exhaustive and mutually exclusive, so they must + // add up in a run with no windows at all exactly as they do in one full of + // them, and an execution that reached the profile carrying no disclosure + // would otherwise be invisible. + assertSerializationOutcomesAccounted(t, snap) + return snap } @@ -288,6 +302,31 @@ func assertPCAttribAccompaniesSamples(t *testing.T, snap Snapshot) { } } +// assertSerializationOutcomesAccounted extends invariant 5's discipline to +// gpu_serialized: every execution in the snapshot takes exactly one of the +// three outcomes, so they sum to len(snap.Executions). +// +// It also asserts the negative that matters more than the sum: with the +// default harness configuration (SerializedSampling unset — nothing is ever +// serialized) NO execution may read "true" or "unknown". A conformance run +// that started reporting perturbation nobody caused would be as wrong as one +// that stopped reporting perturbation that happened. +func assertSerializationOutcomesAccounted(t *testing.T, snap Snapshot) { + t.Helper() + sum := snap.ExecutionsSerialized + snap.ExecutionsNotSerialized + + snap.ExecutionsSerializationUnknown + assert.Equal(t, uint64(len(snap.Executions)), sum, + "the three gpu_serialized outcomes must sum to the executions in the snapshot") + assert.Zero(t, snap.ExecutionsSerialized, + "nothing serializes kernels in this configuration") + assert.Zero(t, snap.ExecutionsSerializationUnknown, + "with no serialized sampling selected, \"false\" is unconditional and correct") + for _, v := range snap.Executions { + assert.Equal(t, SerializationNotSerialized, v.Serialized, + "execution %+v", v.Exec.Correlation) + } +} + // assertNoFabricatedLaunch is invariant 3: every non-nil view.Launch must // carry a Correlation that some emitted launch actually carried, never a // zero-value or otherwise invented value. diff --git a/gpu/joinhealth.go b/gpu/joinhealth.go index b1866bb..81ed5a9 100644 --- a/gpu/joinhealth.go +++ b/gpu/joinhealth.go @@ -147,6 +147,18 @@ func joinSummary(snap Snapshot, anomalies int) string { plural(uint64(snap.PendingModuleGroups), "kernel group", "kernel groups")) } + // The serialization disclosure, and only when there is one to make. With + // PC sampling off or in continuous collection every execution is "false" + // and nothing was ever serialized, so a permanent "0 serialized" clause + // would be exactly the zero-valued noise this format avoids. + if snap.ExecutionsSerialized > 0 || snap.ExecutionsSerializationUnknown > 0 || + snap.SamplingWindowsReceived > 0 { + fmt.Fprintf(&b, "; serialization %d true, %d false, %d unknown over %s", + snap.ExecutionsSerialized, snap.ExecutionsNotSerialized, + snap.ExecutionsSerializationUnknown, + plural(uint64(snap.SamplingWindowsHeld), "burst", "bursts")) + } + switch anomalies { case 0: b.WriteString("; no anomalies") @@ -196,6 +208,48 @@ func joinAnomalies(snap Snapshot, proj ProjectionStats) []string { "disagree with what is actually present; treat every figure below as unreliable", outcomes, execs) } + // The serialization disclosure's own sum identity, and it is raised in the + // same place and for the same reason as the join one above: three + // mutually-exclusive outcomes, every execution takes exactly one, so they + // must add up to what is actually in the snapshot. A shortfall means an + // execution reached the profile with no disclosure at all. + if serialization := snap.ExecutionsSerialized + snap.ExecutionsNotSerialized + + snap.ExecutionsSerializationUnknown; serialization != execs { + add("serialization outcomes sum to %d but the snapshot holds %d executions — some "+ + "execution carries no gpu_serialized disclosure at all; treat every duration "+ + "in this profile as unqualified", + serialization, execs) + } + // The whole point of Tier A's disclosure. Raised BEFORE the join + // anomalies below because it qualifies the durations themselves rather + // than what they were attributed to: a perturbed measurement joined + // perfectly is still a perturbed measurement. + if snap.ExecutionsSerialized > 0 { + add("%d of %d executions ran while GPU kernels were SERIALIZED by the profiler — "+ + "their durations are inflated by the measurement and are marked "+ + "gpu_serialized=\"true\". CPU and off-CPU samples taken during those bursts are "+ + "distorted too and carry no marking at all", + snap.ExecutionsSerialized, execs) + } + if snap.ExecutionsSerializationUnknown > 0 && snap.SamplingWindowsReceived > 0 { + add("%d of %d executions cannot be said to have run unperturbed — no sampling window "+ + "covers them (a dropped batch, a late attach, a sequence gap, or a burst that "+ + "never closed). They are marked gpu_serialized=\"unknown\" and MUST NOT be read "+ + "as \"false\"", + snap.ExecutionsSerializationUnknown, execs) + } + if snap.SamplingWindowsOpen > 0 { + add("%s still open — the producer stopped reporting mid-burst (a hard exit), so the "+ + "end of the burst is unknown and every execution from its start onward is "+ + "gpu_serialized=\"unknown\"", + plural(uint64(snap.SamplingWindowsOpen), "sampling window", "sampling windows")) + } + if dr.EvictedSamplingWindows > 0 { + add("%s evicted from the serialization disclosure store — executions that far back "+ + "degrade from \"false\" to \"unknown\"; raise "+ + "TimelineConfig.MaxSamplingWindowsPerPID or snapshot more often", + plural(dr.EvictedSamplingWindows, "sampling window", "sampling windows")) + } if js.UnmatchedExecutionCount > 0 { add("%d of %d executions unmatched — GPU time arrived with no launch to attach it to; "+ "it is in the profile under %s carrying no CPU stack", diff --git a/gpu/joinhealth_test.go b/gpu/joinhealth_test.go index 3aca945..fd1d1e6 100644 --- a/gpu/joinhealth_test.go +++ b/gpu/joinhealth_test.go @@ -20,6 +20,12 @@ func healthySnapshot() Snapshot { ExactExecutionJoinCount: 512, }, LaunchCache: LaunchCacheStats{Live: 256}, + // PC sampling off, so every execution is "false" unconditionally and + // correctly: nothing was ever serialized. Set here rather than left + // zero because the zero value of SerializationState is "unknown" and + // the three counters must sum to len(Executions) — the same identity + // the join outcomes carry, checked in the same place. + ExecutionsNotSerialized: 512, } } @@ -57,6 +63,16 @@ func anomalousSnapshot() Snapshot { AttributedPCSamples: 900, PendingSamples: 12, PendingCorrelations: 5, + + // Tier A gone wrong in all three ways at once: bursts perturbed some + // executions, an unbroken history proved others were untouched, and a + // window that never closed leaves the rest unplaceable. + SamplingWindowsReceived: 41, + SamplingWindowsHeld: 21, + SamplingWindowsOpen: 1, + ExecutionsSerialized: 120, + ExecutionsNotSerialized: 300, + ExecutionsSerializationUnknown: 92, } } @@ -93,6 +109,10 @@ func TestJoinHealthAnomaliesEachGetTheirOwnLine(t *testing.T) { "11 timeline events evicted", "1 module evicted before this snapshot — kernels from evicted modules resolve", "sink dropped 64 PC samples at admission", + "120 of 512 executions ran while GPU kernels were SERIALIZED", + "92 of 512 executions cannot be said to have run unperturbed", + "1 sampling window still open", + "serialization 120 true, 300 false, 92 unknown over 21 bursts", } { assert.Contains(t, joined, want) } diff --git a/gpu/projection.go b/gpu/projection.go index d178b3e..fe944fe 100644 --- a/gpu/projection.go +++ b/gpu/projection.go @@ -623,6 +623,28 @@ func projectionLabels(view ExecutionView) map[string]string { if view.Ambiguous { labels["gpu_ambiguous"] = "true" } + // gpu_serialized is set UNCONDITIONALLY on every execution, exactly as + // gpu_join is and for exactly the same reason: an absent label would read + // as "not perturbed" to a consumer that does not know to check for its + // absence, and "not perturbed" is the one answer that must never be + // reachable by accident. + // + // It rides on every execution rather than only on PC-bearing ones because + // serialization is a property of the INTERVAL, not of whether a sample + // landed: every kernel that ran inside a burst ran serialized, sampled or + // not (see gpu/serialization.go). + // + // The value comes from SerializationState.String(), whose zero value is + // "unknown" — so a view that never reached the classifier degrades to + // "unknown" here rather than to "false". + // + // LIMITATION, stated rather than left to be discovered: this is the GPU + // projection. On-CPU and off-CPU samples taken during a burst are + // distorted too — serialization inflates precisely the synchronization + // wait that off-CPU profiling exists to measure — and they carry no + // marking at all, because those profilers know nothing about GPUs. + // joinhealth reports it whenever any window was recorded. + labels["gpu_serialized"] = view.Serialized.String() // gpu_sample_period rides only on the population that actually carries a // sampled stack, and only when the producer declared a period. It is the // denominator a consumer needs to extrapolate "sampled GPU time" to "all diff --git a/gpu/projection_test.go b/gpu/projection_test.go index 673da1a..e9afb37 100644 --- a/gpu/projection_test.go +++ b/gpu/projection_test.go @@ -943,3 +943,71 @@ func TestProjectionSourceLabelsCannotBeForgedByAbsence(t *testing.T) { "gpu_src_status is unconditional on PC-DERIVED samples; an execution with none has nothing to say") assert.NotContains(t, samples[2].Labels, "gpu_pc_attrib") } + +// gpu_serialized is on EVERY sample projected from EVERY execution, with one +// of exactly three values. It is not omitempty and there is no "only if +// interesting" branch: an absent label reads as "not perturbed" to a consumer +// that does not know to check for its absence, which is the failure the whole +// disclosure exists to prevent. +func TestSerializedLabelIsUnconditionalAndHasThreeValues(t *testing.T) { + snap := Snapshot{Executions: []ExecutionView{ + {Exec: GPUKernelExec{StartNs: 0, EndNs: 10, KernelName: "kA"}, + Serialized: SerializationSerialized}, + {Exec: GPUKernelExec{StartNs: 0, EndNs: 10, KernelName: "kB"}, + Serialized: SerializationNotSerialized}, + {Exec: GPUKernelExec{StartNs: 0, EndNs: 10, KernelName: "kC"}, + Serialized: SerializationUnknown}, + // A view nobody classified at all. The zero value of + // SerializationState is "unknown", so this degrades to the safe + // answer rather than to "false". + {Exec: GPUKernelExec{StartNs: 0, EndNs: 10, KernelName: "kD"}}, + }} + + samples := ProjectExecutions(snap) + require.Len(t, samples, 4) + assert.Equal(t, "true", samples[0].Labels["gpu_serialized"]) + assert.Equal(t, "false", samples[1].Labels["gpu_serialized"]) + assert.Equal(t, "unknown", samples[2].Labels["gpu_serialized"]) + assert.Equal(t, "unknown", samples[3].Labels["gpu_serialized"], + "an unclassified execution must degrade to \"unknown\", never to \"false\"") + for i, s := range samples { + assert.Contains(t, s.Labels, "gpu_serialized", "sample %d", i) + } +} + +// It rides on the PC-derived samples too — one per PC sample, all carrying the +// execution's own disclosure. Serialization is a property of the interval, not +// of whether a sample landed. +func TestSerializedLabelReachesEveryPCDerivedSample(t *testing.T) { + snap := Snapshot{Executions: []ExecutionView{ + {Exec: GPUKernelExec{StartNs: 0, EndNs: 100, KernelName: "kAdd"}, + Serialized: SerializationSerialized, + PCSamples: []GPUPCSample{ + {PCOffset: 0x10, Count: 1}, {PCOffset: 0x20, Count: 3}}}, + }} + + samples := ProjectExecutions(snap) + require.Len(t, samples, 2) + for i, s := range samples { + assert.Equal(t, "true", s.Labels["gpu_serialized"], "pc sample %d", i) + } +} + +// A producer-supplied tag must not be able to forge the disclosure — this is +// the reserved-name rule, and it matters more here than anywhere else: a tag +// named gpu_serialized set to "false" would claim a perturbed measurement was +// clean. +func TestSerializedLabelBeatsAForgedTag(t *testing.T) { + snap := Snapshot{Executions: []ExecutionView{ + {Exec: GPUKernelExec{StartNs: 0, EndNs: 10, KernelName: "kAdd"}, + Serialized: SerializationSerialized, + Launch: &GPUKernelLaunch{Launch: LaunchContext{ + Tags: map[string]string{"gpu_serialized": "false"}, + }}}, + }} + + samples := ProjectExecutions(snap) + require.Len(t, samples, 1) + assert.Equal(t, "true", samples[0].Labels["gpu_serialized"], + "a tag named gpu_serialized must never override the derived disclosure") +} diff --git a/gpu/serialization.go b/gpu/serialization.go new file mode 100644 index 0000000..67e6722 --- /dev/null +++ b/gpu/serialization.go @@ -0,0 +1,300 @@ +package gpu + +import "sort" + +// The serialization disclosure: which executions ran while GPU kernels were +// being serialized by the profiler, which provably did not, and which cannot +// be said either way. +// +// Why the window is the unit and the sampled kernel is not +// ------------------------------------------------------- +// In CUPTI's kernel-serialized collection the device serializes kernels for as +// long as sampling is enabled. Every kernel that executed inside a burst +// therefore ran perturbed — SAMPLED OR NOT. Marking only the executions that +// received a PC sample would under-report the perturbation by exactly the +// fraction of kernels the sampler missed, which is most of them. +// +// Why "unknown" is a first-class answer +// ------------------------------------- +// A profile that reports "not perturbed" when it means "cannot tell" is the +// failure spec §4 forbids, and it is the exact shape of the gpu_join +// precedent. So "false" is emitted ONLY from positive evidence: an execution +// interval that lies wholly inside a span of time the agent holds an unbroken +// window history for, and that intersects none of the bursts in it. Everything +// else — no windows at all, an execution outside the covered span, a hole in +// the window sequence, a burst that never closed — is "unknown". +// +// Stated as an invariant, because it is the one this file exists to hold: +// SerializationNotSerialized is returned by exactly one branch of exactly one +// function below, and that branch requires containment in a proven interval. +// Every other path falls through to SerializationUnknown, which is also the +// zero value of the type. + +// defaultMaxSamplingWindowsPerPID bounds how many bursts one process's history +// may hold. At the shipped duty cycle (a 50 ms burst at most every 500 ms) a +// process produces at most two bursts a second, so this is roughly half an +// hour of Tier A before the oldest are dropped — and dropping the oldest moves +// the coverage start FORWARD, which turns old executions from "false" into +// "unknown". That is the safe direction and the only direction eviction here +// can move an answer. +const defaultMaxSamplingWindowsPerPID = 4096 + +// defaultMaxSamplingWindowPIDs bounds how many processes' histories are held +// at once. Tier A is opt-in and perturbing, so a machine running it in +// hundreds of processes at once is a misconfiguration rather than a workload; +// a process past the bound gets no windows, and its executions read "unknown". +const defaultMaxSamplingWindowPIDs = 256 + +// samplingWindow is one burst as the store holds it. +type samplingWindow struct { + startNs uint64 + endNs uint64 // 0 = still open when the producer stopped reporting + mode SamplingMode +} + +func (w samplingWindow) open() bool { return w.endNs == 0 } + +// serializes reports whether kernels running inside this window were +// serialized. Only the kernel-serialized mode does. An UNSET mode does not: +// it is a producer that did not say, which is not the same as a producer that +// said no — such a window is handled as opaque by classify below rather than +// being read either way. +func (w samplingWindow) serializes() bool { return w.mode == SamplingModeKernelSerialized } + +// intersects reports whether [startNs, endNs] overlaps the window. An open +// window is [startNs, +inf). +func (w samplingWindow) intersects(startNs, endNs uint64) bool { + if endNs < w.startNs { + return false + } + if w.open() { + return true + } + return startNs <= w.endNs +} + +// windowSet is one process's burst history. +// +// coverageStartNs is the earliest instant this history can speak for. It is +// the first window's start, and it moves FORWARD on two events: a sequence gap +// (records were lost, so nothing before the gap can be shown to be contiguous) +// and an eviction (the oldest window left, so the same is true of it). It +// never moves backward, so an answer can only ever become less certain. +type windowSet struct { + wins []samplingWindow + coverageStartNs uint64 + haveCoverage bool +} + +// windowStore holds every process's burst history and answers the one question +// the disclosure needs. +type windowStore struct { + byPID map[uint32]*windowSet + maxPerPID int + maxPIDs int + + // received counts windows accepted into the store, superseded counts the + // closed records that replaced their own burst's open record (so + // received - superseded is the number of distinct bursts), and the last + // three are the store's own bounded-storage losses. Every one of them is + // assertable from a test. + received uint64 + superseded uint64 + evicted uint64 + refusedPIDs uint64 + unknownMode uint64 +} + +func newWindowStore(maxPerPID, maxPIDs int) *windowStore { + if maxPerPID <= 0 { + maxPerPID = defaultMaxSamplingWindowsPerPID + } + if maxPIDs <= 0 { + maxPIDs = defaultMaxSamplingWindowPIDs + } + return &windowStore{ + byPID: make(map[uint32]*windowSet), + maxPerPID: maxPerPID, + maxPIDs: maxPIDs, + } +} + +// add records one window. +// +// A burst reaches the wire twice: an OPEN record the instant it starts and a +// CLOSED record with the same StartNs when it stops. The pairing is what makes +// a hard exit visible — the open record is already delivered when the process +// dies — so this must not count one burst twice, and it must not let the open +// record win. The rule is one-way: a closed record replaces an open one with +// the same start, and an open record never replaces a closed one, whichever +// order a lossy transport delivers them in. +func (s *windowStore) add(w GPUSamplingWindow) { + if w.EndNs != 0 && w.EndNs < w.StartNs { + // An inverted window is a producer contract violation, not a hole. + // gpuabi.DecodeSamplingWindow already refuses these at the wire + // boundary; refusing again here keeps the store's own invariant + // (endNs == 0 or endNs >= startNs) true for callers that build one + // directly, e.g. a replay fixture. + s.unknownMode++ + return + } + set := s.byPID[w.PID] + if set == nil { + if len(s.byPID) >= s.maxPIDs { + // Refused rather than evicting somebody else's history: dropping + // a live process's windows would turn its executions from a + // proven answer into "unknown", and doing that to an established + // process to make room for a new one trades a good answer for no + // answer. The new process's executions read "unknown", which is + // what they honestly are. + s.refusedPIDs++ + return + } + set = &windowSet{} + s.byPID[w.PID] = set + } + s.received++ + if w.Mode != SamplingModeContinuous && w.Mode != SamplingModeKernelSerialized { + s.unknownMode++ + } + if w.Lost > 0 || !set.haveCoverage { + // Records were lost between the previous window and this one, so the + // history has a hole and nothing before this window can be shown to + // be a gap. Coverage restarts here; the older windows STAY, because + // they are still positive evidence that a burst was open then. + set.coverageStartNs = w.StartNs + set.haveCoverage = true + } + + nw := samplingWindow{startNs: w.StartNs, endNs: w.EndNs, mode: w.Mode} + // Windows arrive in start order from one producer, so the common case is + // an append or a match against the tail. The scan is backwards for that + // reason and is bounded by maxPerPID in the worst case. + for i := len(set.wins) - 1; i >= 0; i-- { + if set.wins[i].startNs != nw.startNs { + continue + } + if set.wins[i].open() && !nw.open() { + set.wins[i] = nw // the close, superseding its own open record + } + s.superseded++ + return + } + set.wins = append(set.wins, nw) + if len(set.wins) > 1 && set.wins[len(set.wins)-2].startNs > nw.startNs { + sort.Slice(set.wins, func(i, j int) bool { return set.wins[i].startNs < set.wins[j].startNs }) + } + if len(set.wins) > s.maxPerPID { + drop := len(set.wins) - s.maxPerPID + s.evicted += uint64(drop) + set.wins = append(set.wins[:0], set.wins[drop:]...) + // The evicted windows were the earliest, so the history no longer + // covers the interval before whatever is now oldest. Forward only. + if len(set.wins) > 0 && set.wins[0].startNs > set.coverageStartNs { + set.coverageStartNs = set.wins[0].startNs + } + } +} + +// coverageEndNs is the last instant this history can speak for. +// +// An OPEN window ends coverage at its own start: the burst was running from +// there and nothing says when — or whether — it stopped. Otherwise coverage +// runs to the end of the last closed burst. It deliberately does NOT extend +// past that: the next burst's open record may simply not have been drained +// yet, so the interval after the last known window is not a proven gap. +func (set *windowSet) coverageEndNs() (uint64, bool) { + if !set.haveCoverage || len(set.wins) == 0 { + return 0, false + } + var end uint64 + var have bool + for _, w := range set.wins { + if w.startNs < set.coverageStartNs { + continue + } + if w.open() { + // The earliest open window at or after the coverage start caps + // everything. wins is start-ordered, so this is the answer. + return w.startNs, true + } + if !have || w.endNs > end { + end, have = w.endNs, true + } + } + return end, have +} + +// classify answers the disclosure for one execution. +// +// The three branches, in the order that matters: +// +// 1. Intersects a closed kernel-serialized burst -> "true". Definite, and it +// wins over everything else: an execution that provably overlapped a burst +// is perturbed whatever else is unknown about the rest of its interval. +// 2. Wholly inside a proven span and intersecting no burst -> "false". This +// is the ONLY branch that returns "false", and it needs positive evidence +// on both endpoints. +// 3. Everything else -> "unknown". +func (s *windowStore) classify(pid uint32, startNs, endNs uint64) SerializationState { + if endNs < startNs { + // A backwards execution cannot be placed against anything. + return SerializationUnknown + } + set := s.byPID[pid] + if set == nil || !set.haveCoverage { + // No window history for this process at all: a dropped batch, a late + // attach, a producer that never fired the probe, or a PID the store + // refused. All of them are "cannot tell". + return SerializationUnknown + } + // "true" outranks "unknown", so the whole set is scanned before an + // unknown is returned: an execution that provably overlapped a closed + // burst is perturbed whatever is unknown about the rest of its interval, + // and reporting "unknown" there would throw away a fact we hold. + opaque := false + for _, w := range set.wins { + if !w.intersects(startNs, endNs) { + continue + } + switch { + case w.open(): + // The burst was running from its start and nothing says when — or + // whether — it stopped. Cannot be placed. + opaque = true + case w.mode == SamplingModeUnset: + // The producer did not say which mode. That is not evidence of + // serialization and it is not evidence of its absence either. + opaque = true + case w.serializes(): + return SerializationSerialized + default: + // A closed continuous-mode burst. Nothing was serialized; keep + // looking, and this interval still counts as covered. + } + } + if opaque { + return SerializationUnknown + } + end, ok := set.coverageEndNs() + if !ok || startNs < set.coverageStartNs || endNs > end { + return SerializationUnknown + } + return SerializationNotSerialized +} + +// windows returns how many bursts are held and how many of them are still +// open. Both are gauges for the operator: a non-zero open count says a burst's +// end is unknown and an unbounded tail of executions cannot be said to have +// run unperturbed. +func (s *windowStore) windows() (held, open int) { + for _, set := range s.byPID { + held += len(set.wins) + for _, w := range set.wins { + if w.open() { + open++ + } + } + } + return held, open +} diff --git a/gpu/serialization_test.go b/gpu/serialization_test.go new file mode 100644 index 0000000..d16ac83 --- /dev/null +++ b/gpu/serialization_test.go @@ -0,0 +1,543 @@ +package gpu + +import ( + "fmt" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// The serialization disclosure, end to end through Timeline. +// +// The assertion these tests exist for is negative and is repeated in several +// shapes on purpose: "unknown" must never come out as "false". Every other +// property here is in service of that one, because a profile that says "this +// duration was not perturbed" when it means "I cannot tell" is precisely what +// spec §4 forbids — and it is the failure mode fifteen defects on this project +// have taken: a counter or a check reading green exactly when things were +// worst. + +const tierAPID = uint32(4242) + +func tierATimeline() *Timeline { + return NewTimeline(TimelineConfig{SerializedSampling: true}) +} + +// serializedExec is an execution from tierAPID over [startNs, endNs]. The PID +// is on the correlation because that is where the disclosure reads it from — +// windows are per-process, and an execution that does not name its process +// cannot be placed against any of them. +func serializedExec(value string, startNs, endNs uint64) GPUKernelExec { + return GPUKernelExec{ + Correlation: CorrelationID{Backend: BackendCUPTI, PID: tierAPID, Value: value}, + KernelName: "k_" + value, + StartNs: startNs, + EndNs: endNs, + } +} + +func burst(pid uint32, startNs, endNs uint64) GPUSamplingWindow { + return GPUSamplingWindow{ + Backend: BackendCUPTI, + PID: pid, + ClockDomain: ClockDomainCPUMonotonic, + StartNs: startNs, + EndNs: endNs, + Mode: SamplingModeKernelSerialized, + } +} + +// emitBurst delivers a burst the way the producer does: an OPEN record the +// instant it starts, a CLOSED record with the same start when it stops. +func emitBurst(t *testing.T, tl *Timeline, pid, startNs, endNs uint64) { + t.Helper() + require.NoError(t, tl.EmitSamplingWindow(burst(uint32(pid), startNs, 0))) + require.NoError(t, tl.EmitSamplingWindow(burst(uint32(pid), startNs, endNs))) +} + +// states pulls the disclosure off a snapshot in execution order. +func states(snap Snapshot) []string { + out := make([]string, len(snap.Executions)) + for i, v := range snap.Executions { + out[i] = v.Serialized.String() + } + return out +} + +// assertSumIdentity is the discipline gpu_join's three outcomes already carry, +// applied to the three the disclosure adds. It is called from every test below +// rather than from one of them: a shortfall means some execution reached the +// profile carrying no disclosure at all, and that is exactly the condition +// that would otherwise be invisible. +func assertSumIdentity(t *testing.T, snap Snapshot) { + t.Helper() + sum := snap.ExecutionsSerialized + snap.ExecutionsNotSerialized + + snap.ExecutionsSerializationUnknown + require.Equal(t, uint64(len(snap.Executions)), sum, + "the three gpu_serialized outcomes must sum to the executions in the snapshot") +} + +// --------------------------------------------------------------------------- +// With Tier A off, "false" is correct and unconditional. + +func TestSerializationIsFalseUnconditionallyWhenTierAWasNotSelected(t *testing.T) { + tl := NewTimeline(TimelineConfig{}) // the default: no serialized sampling + require.NoError(t, tl.EmitExec(serializedExec("a", 100, 200))) + require.NoError(t, tl.EmitExec(serializedExec("b", 300, 400))) + + snap := tl.Snapshot() + assert.Equal(t, []string{"false", "false"}, states(snap)) + assert.Equal(t, uint64(2), snap.ExecutionsNotSerialized) + assert.Zero(t, snap.ExecutionsSerializationUnknown) + assertSumIdentity(t, snap) +} + +// Windows can arrive even with the agent configured for continuous collection +// (a producer left over from another run, a system-wide attach). They are +// ingested and counted — but they cannot make an execution "true", because +// nothing this agent asked for serializes anything. +func TestSerializationIgnoresWindowsWhenTierAWasNotSelected(t *testing.T) { + tl := NewTimeline(TimelineConfig{}) + emitBurst(t, tl, uint64(tierAPID), 100, 200) + require.NoError(t, tl.EmitExec(serializedExec("inside", 120, 180))) + + snap := tl.Snapshot() + assert.Equal(t, []string{"false"}, states(snap)) + assert.Equal(t, uint64(2), snap.SamplingWindowsReceived, + "the window is still ingested and counted; only the answer is unconditional") + assertSumIdentity(t, snap) +} + +// --------------------------------------------------------------------------- +// The three values. + +func TestSerializationMarksExecutionsOverlappingABurst(t *testing.T) { + tl := tierATimeline() + emitBurst(t, tl, uint64(tierAPID), 1000, 2000) + emitBurst(t, tl, uint64(tierAPID), 4000, 5000) + + // Deliberately includes executions that only PARTLY overlap: a kernel + // straddling a burst boundary ran serialized for part of its life, which + // is enough to make its measured duration perturbed. + require.NoError(t, tl.EmitExec(serializedExec("wholly-inside", 1200, 1800))) + require.NoError(t, tl.EmitExec(serializedExec("straddles-start", 900, 1100))) + require.NoError(t, tl.EmitExec(serializedExec("straddles-end", 1900, 2100))) + require.NoError(t, tl.EmitExec(serializedExec("spans-a-burst", 800, 2200))) + require.NoError(t, tl.EmitExec(serializedExec("in-the-gap", 2500, 3500))) + + snap := tl.Snapshot() + assert.Equal(t, []string{"true", "true", "true", "true", "false"}, states(snap)) + assert.Equal(t, uint64(4), snap.ExecutionsSerialized) + assert.Equal(t, uint64(1), snap.ExecutionsNotSerialized) + assert.Zero(t, snap.ExecutionsSerializationUnknown) + assertSumIdentity(t, snap) +} + +// "straddles-start" above begins before the first window and is still "true". +// That is not an accident of the coverage rule — it is the rule that matters, +// so it gets its own assertion: an execution that provably overlapped a burst +// is perturbed whatever else is unknown about the rest of its interval. +func TestSerializationTrueOutranksUnknown(t *testing.T) { + tl := tierATimeline() + // One closed burst, then one that never closed. + emitBurst(t, tl, uint64(tierAPID), 1000, 2000) + require.NoError(t, tl.EmitSamplingWindow(burst(tierAPID, 3000, 0))) + + // This execution overlaps the closed burst AND runs past the open one's + // start. It is definitely perturbed. + require.NoError(t, tl.EmitExec(serializedExec("both", 1500, 3500))) + + snap := tl.Snapshot() + assert.Equal(t, []string{"true"}, states(snap)) + assertSumIdentity(t, snap) +} + +// --------------------------------------------------------------------------- +// "unknown", and the ways it is reached. + +// The plan's first case: Tier A was selected and no window records arrived at +// all — a dropped batch, a late attach, a producer that never fired the probe. +func TestSerializationIsUnknownWhenNoWindowsArrived(t *testing.T) { + tl := tierATimeline() + require.NoError(t, tl.EmitExec(serializedExec("a", 100, 200))) + require.NoError(t, tl.EmitExec(serializedExec("b", 300, 400))) + + snap := tl.Snapshot() + assert.Equal(t, []string{"unknown", "unknown"}, states(snap)) + assert.Equal(t, uint64(2), snap.ExecutionsSerializationUnknown) + assert.Zero(t, snap.ExecutionsNotSerialized, + `"no window arrived" must never be reported as "not serialized"`) + assert.Zero(t, snap.SamplingWindowsReceived) + assertSumIdentity(t, snap) +} + +// An execution outside the span the agent holds an unbroken history for is +// unplaceable in either direction. Before the first window this is not merely +// conservative, it is correct: the shim cannot emit a window before a consumer +// attaches, so everything before the first one it sees is genuinely unknown. +func TestSerializationIsUnknownOutsideTheCoveredSpan(t *testing.T) { + tl := tierATimeline() + emitBurst(t, tl, uint64(tierAPID), 1000, 2000) + + require.NoError(t, tl.EmitExec(serializedExec("before", 100, 200))) + require.NoError(t, tl.EmitExec(serializedExec("inside-coverage", 1000, 2000))) + require.NoError(t, tl.EmitExec(serializedExec("after", 9000, 9500))) + + snap := tl.Snapshot() + assert.Equal(t, []string{"unknown", "true", "unknown"}, states(snap)) + assertSumIdentity(t, snap) +} + +// A gap between two known bursts IS proven, and must read "false" — otherwise +// the tier discloses nothing usable and every execution is "unknown". +func TestSerializationIsFalseInAProvenGap(t *testing.T) { + tl := tierATimeline() + emitBurst(t, tl, uint64(tierAPID), 1000, 2000) + emitBurst(t, tl, uint64(tierAPID), 4000, 5000) + + require.NoError(t, tl.EmitExec(serializedExec("gap", 2500, 3500))) + + snap := tl.Snapshot() + assert.Equal(t, []string{"false"}, states(snap)) + assert.Equal(t, uint64(1), snap.ExecutionsNotSerialized) + assertSumIdentity(t, snap) +} + +// --------------------------------------------------------------------------- +// The open window. This is the test the plan singles out. + +// A window with end_ns == 0 is OPEN, not zero-length. Every execution from its +// start_ns onward is "unknown" and NEVER "false" — treating it as zero-length +// would mark a whole perturbed tail "not serialized". +// +// Swept rather than sampled: the assertion is over every execution position +// from well before the open window to well after it, and the negative half of +// it ("never false") is checked on every single one. +func TestSerializationOpenWindowMakesEverythingFromItsStartUnknownAndNeverFalse(t *testing.T) { + tl := tierATimeline() + // A complete burst first, so the store has genuine coverage and a naive + // implementation would happily answer "false" after it. + emitBurst(t, tl, uint64(tierAPID), 1000, 2000) + // Then a burst that opens and never closes: the hard-exit case. + const openAt = uint64(4000) + require.NoError(t, tl.EmitSamplingWindow(burst(tierAPID, openAt, 0))) + + for start := uint64(0); start <= 10000; start += 250 { + require.NoError(t, tl.EmitExec( + serializedExec(fmt.Sprintf("e%d", start), start, start+100))) + } + + snap := tl.Snapshot() + require.Len(t, snap.Executions, 41) + for _, v := range snap.Executions { + if v.Exec.EndNs >= openAt { + assert.NotEqual(t, SerializationNotSerialized, v.Serialized, + "execution [%d,%d] touches or follows an OPEN window at %d: it must never "+ + "read \"false\"", v.Exec.StartNs, v.Exec.EndNs, openAt) + assert.Equal(t, SerializationUnknown, v.Serialized, + "execution [%d,%d] touches or follows an OPEN window at %d", + v.Exec.StartNs, v.Exec.EndNs, openAt) + } + } + assert.Positive(t, snap.ExecutionsSerializationUnknown) + assert.Equal(t, 1, snap.SamplingWindowsOpen, + "the open burst is a gauge the operator can read, not only an internal state") + assertSumIdentity(t, snap) +} + +// The other half of the open-window contract: the CLOSED record supersedes its +// own burst's open record, so an ordinary duty cycle does not leave a trail of +// permanently-open windows behind it. Both delivery orders are checked, +// because a lossy transport produces both. +func TestSerializationClosedWindowSupersedesItsOwnOpenRecord(t *testing.T) { + for _, tc := range []struct { + name string + order []GPUSamplingWindow + }{ + {"open then closed", []GPUSamplingWindow{ + burst(tierAPID, 1000, 0), burst(tierAPID, 1000, 2000)}}, + {"closed then open", []GPUSamplingWindow{ + burst(tierAPID, 1000, 2000), burst(tierAPID, 1000, 0)}}, + } { + t.Run(tc.name, func(t *testing.T) { + tl := tierATimeline() + for _, w := range tc.order { + require.NoError(t, tl.EmitSamplingWindow(w)) + } + emitBurst(t, tl, uint64(tierAPID), 4000, 5000) + require.NoError(t, tl.EmitExec(serializedExec("gap", 2500, 3500))) + + snap := tl.Snapshot() + assert.Equal(t, 2, snap.SamplingWindowsHeld, "two bursts, not four records") + assert.Zero(t, snap.SamplingWindowsOpen, + "an open record must never survive its own burst's close") + assert.Equal(t, []string{"false"}, states(snap), + "a closed burst either side of the gap is what proves the gap") + assertSumIdentity(t, snap) + }) + } +} + +// --------------------------------------------------------------------------- +// A hole in the window history. + +// A sequence gap means records between the last window and this one were lost, +// so the interval they covered cannot be shown to be a gap. Coverage restarts; +// executions before the hole degrade from "false" to "unknown", and the +// windows already held still prove "true" for anything that overlapped them. +func TestSerializationSequenceGapRestartsCoverageRatherThanSpanningIt(t *testing.T) { + tl := tierATimeline() + emitBurst(t, tl, uint64(tierAPID), 1000, 2000) + emitBurst(t, tl, uint64(tierAPID), 3000, 4000) + // Two records lost, then a burst much later. Nothing says what happened + // in between. + lost := burst(tierAPID, 20000, 0) + lost.Lost = 2 + require.NoError(t, tl.EmitSamplingWindow(lost)) + require.NoError(t, tl.EmitSamplingWindow(burst(tierAPID, 20000, 21000))) + emitBurst(t, tl, uint64(tierAPID), 23000, 24000) + + require.NoError(t, tl.EmitExec(serializedExec("old-gap", 2200, 2800))) + require.NoError(t, tl.EmitExec(serializedExec("old-burst", 1200, 1800))) + require.NoError(t, tl.EmitExec(serializedExec("across-the-hole", 5000, 19000))) + require.NoError(t, tl.EmitExec(serializedExec("new-gap", 21500, 22500))) + + snap := tl.Snapshot() + assert.Equal(t, []string{ + "unknown", // a gap that WAS proven, before the hole: no longer provable + "true", // still positive evidence; the hole does not erase a burst + "unknown", // squarely inside the hole + "false", // after the restart, between two known bursts + }, states(snap)) + assertSumIdentity(t, snap) +} + +// --------------------------------------------------------------------------- +// Process isolation. + +// Windows are per-process, and the PID is IN the key rather than in a check +// performed elsewhere (issue #52's discipline). One process bursting must +// never mark another process's executions perturbed, and must never let +// another process's executions read "false" either. +func TestSerializationWindowsNeverCrossProcesses(t *testing.T) { + tl := tierATimeline() + const other = uint32(777) + emitBurst(t, tl, uint64(tierAPID), 1000, 2000) + emitBurst(t, tl, uint64(tierAPID), 4000, 5000) + + mine := serializedExec("mine", 1200, 1800) + theirs := GPUKernelExec{ + Correlation: CorrelationID{Backend: BackendCUPTI, PID: other, Value: "theirs"}, + KernelName: "k_theirs", + StartNs: 1200, + EndNs: 1800, + } + // An execution that does not name its process at all cannot be placed + // against anybody's windows. + anonymous := GPUKernelExec{ + Correlation: CorrelationID{Backend: BackendCUPTI, Value: "anon"}, + KernelName: "k_anon", + StartNs: 2500, + EndNs: 3500, + } + require.NoError(t, tl.EmitExec(mine)) + require.NoError(t, tl.EmitExec(theirs)) + require.NoError(t, tl.EmitExec(anonymous)) + + snap := tl.Snapshot() + assert.Equal(t, []string{"true", "unknown", "unknown"}, states(snap)) + assertSumIdentity(t, snap) +} + +// --------------------------------------------------------------------------- +// The store's own bounds, and the direction they may move an answer. + +// Eviction drops the OLDEST bursts, which moves the coverage start forward. +// That can only turn "false" into "unknown" — never the reverse — and this +// asserts the direction rather than trusting the comment. +func TestSerializationEvictionDegradesTowardsUnknownNeverTowardsFalse(t *testing.T) { + tl := NewTimeline(TimelineConfig{SerializedSampling: true, MaxSamplingWindowsPerPID: 4}) + // The gap between bursts 1 and 2 is provable while both are held. + emitBurst(t, tl, uint64(tierAPID), 1000, 2000) + emitBurst(t, tl, uint64(tierAPID), 3000, 4000) + require.NoError(t, tl.EmitExec(serializedExec("early-gap", 2200, 2800))) + early := tl.Snapshot() + require.Equal(t, []string{"false"}, states(early)) + + // Six more bursts push the first two out of a four-entry store. + for i := uint64(0); i < 6; i++ { + emitBurst(t, tl, uint64(tierAPID), 10000+i*2000, 11000+i*2000) + } + require.NoError(t, tl.EmitExec(serializedExec("early-gap-again", 2200, 2800))) + + snap := tl.Snapshot() + assert.Equal(t, []string{"unknown"}, states(snap), + "once the bursts either side of it are gone, the gap is no longer proven") + assert.Positive(t, snap.Dropped.EvictedSamplingWindows, + "an eviction that costs certainty must be counted, not silent") + assertSumIdentity(t, snap) +} + +// A process past the PID bound gets no window history, so its executions read +// "unknown". It must not inherit somebody else's. +func TestSerializationRefusedPIDReadsUnknown(t *testing.T) { + tl := NewTimeline(TimelineConfig{SerializedSampling: true, MaxSamplingWindowPIDs: 1}) + emitBurst(t, tl, 1, 1000, 2000) + emitBurst(t, tl, 2, 1000, 2000) // refused: the store already holds one PID + + require.NoError(t, tl.EmitExec(GPUKernelExec{ + Correlation: CorrelationID{Backend: BackendCUPTI, PID: 2, Value: "x"}, + StartNs: 1200, EndNs: 1800, + })) + + snap := tl.Snapshot() + assert.Equal(t, []string{"unknown"}, states(snap)) + assertSumIdentity(t, snap) +} + +// --------------------------------------------------------------------------- +// Degenerate records. + +// A window whose mode the producer left unset says nothing about whether +// kernels were serialized. It is not evidence of perturbation and it is not +// evidence of its absence, so the interval it covers is opaque. +func TestSerializationUnsetModeWindowIsOpaqueNotFalse(t *testing.T) { + tl := tierATimeline() + emitBurst(t, tl, uint64(tierAPID), 1000, 2000) + unset := burst(tierAPID, 3000, 4000) + unset.Mode = SamplingModeUnset + require.NoError(t, tl.EmitSamplingWindow(unset)) + emitBurst(t, tl, uint64(tierAPID), 5000, 6000) + + require.NoError(t, tl.EmitExec(serializedExec("in-the-unset", 3200, 3800))) + require.NoError(t, tl.EmitExec(serializedExec("in-a-real-gap", 4200, 4800))) + + snap := tl.Snapshot() + assert.Equal(t, []string{"unknown", "false"}, states(snap)) + assertSumIdentity(t, snap) +} + +// A continuous-mode window says the producer was reporting but serialized +// nothing. It extends coverage and marks nothing perturbed. +func TestSerializationContinuousModeWindowMarksNothingPerturbed(t *testing.T) { + tl := tierATimeline() + cont := burst(tierAPID, 1000, 5000) + cont.Mode = SamplingModeContinuous + require.NoError(t, tl.EmitSamplingWindow(cont)) + + require.NoError(t, tl.EmitExec(serializedExec("inside", 2000, 3000))) + + snap := tl.Snapshot() + assert.Equal(t, []string{"false"}, states(snap)) + assertSumIdentity(t, snap) +} + +// An inverted window is a producer contract violation. gpuabi refuses it at +// the wire boundary; the store refuses it again so a directly-constructed one +// (a replay fixture, a test) cannot put a negative interval into the evidence. +func TestSerializationInvertedWindowIsRefused(t *testing.T) { + tl := tierATimeline() + require.NoError(t, tl.EmitSamplingWindow(burst(tierAPID, 5000, 1000))) + require.NoError(t, tl.EmitExec(serializedExec("x", 2000, 3000))) + + snap := tl.Snapshot() + assert.Equal(t, []string{"unknown"}, states(snap)) + assert.Zero(t, snap.SamplingWindowsHeld) + assertSumIdentity(t, snap) +} + +// --------------------------------------------------------------------------- +// The invariant, stated as a test. + +// Across every shape this file builds, "false" is reachable ONLY from positive +// evidence. This drives a matrix of window histories against a matrix of +// execution intervals and asserts the negative: wherever the store cannot +// prove containment in an unbroken span with no burst in it, the answer is not +// "false". +func TestSerializationFalseIsOnlyEverReachedFromPositiveEvidence(t *testing.T) { + type histCase struct { + name string + emit func(*testing.T, *Timeline) + // covered is the interval the history can speak for; executions + // wholly inside it and clear of every burst may read "false". + coverStart, coverEnd uint64 + bursts [][2]uint64 + // anyFalse says whether this history leaves a provable gap at all. It + // is false for "no windows" (nothing is provable) and for a single + // burst whose extent IS the whole covered span (there is no gap + // inside it), and those two are exactly the shapes where a "false" + // appearing would be the defect. + anyFalse bool + } + cases := []histCase{ + {name: "no windows at all", emit: func(*testing.T, *Timeline) {}}, + { + name: "one closed burst", + emit: func(t *testing.T, tl *Timeline) { emitBurst(t, tl, uint64(tierAPID), 1000, 2000) }, + coverStart: 1000, coverEnd: 2000, + bursts: [][2]uint64{{1000, 2000}}, + }, + { + name: "two closed bursts", + emit: func(t *testing.T, tl *Timeline) { + emitBurst(t, tl, uint64(tierAPID), 1000, 2000) + emitBurst(t, tl, uint64(tierAPID), 4000, 5000) + }, + coverStart: 1000, coverEnd: 5000, + bursts: [][2]uint64{{1000, 2000}, {4000, 5000}}, + anyFalse: true, + }, + { + name: "a burst that never closed", + emit: func(t *testing.T, tl *Timeline) { + emitBurst(t, tl, uint64(tierAPID), 1000, 2000) + require.NoError(t, tl.EmitSamplingWindow(burst(tierAPID, 4000, 0))) + }, + coverStart: 1000, coverEnd: 4000, + bursts: [][2]uint64{{1000, 2000}}, + anyFalse: true, + }, + } + + for _, hc := range cases { + t.Run(hc.name, func(t *testing.T) { + tl := tierATimeline() + hc.emit(t, tl) + for s := uint64(0); s <= 8000; s += 100 { + for _, d := range []uint64{0, 150, 900, 3000} { + require.NoError(t, tl.EmitExec( + serializedExec(fmt.Sprintf("e_%d_%d", s, d), s, s+d))) + } + } + snap := tl.Snapshot() + assertSumIdentity(t, snap) + require.NotEmpty(t, snap.Executions) + + var falses int + for _, v := range snap.Executions { + if v.Serialized != SerializationNotSerialized { + continue + } + falses++ + s, e := v.Exec.StartNs, v.Exec.EndNs + require.GreaterOrEqual(t, s, hc.coverStart, + `"false" outside the covered span: [%d,%d]`, s, e) + require.LessOrEqual(t, e, hc.coverEnd, + `"false" outside the covered span: [%d,%d]`, s, e) + for _, b := range hc.bursts { + require.False(t, e >= b[0] && s <= b[1], + `"false" for an execution [%d,%d] overlapping the burst [%d,%d]`, + s, e, b[0], b[1]) + } + } + if hc.anyFalse { + assert.Positive(t, falses, + "the tier has to be able to prove SOMETHING, or it discloses nothing usable") + } else { + assert.Zero(t, falses, + `this history proves no gap, so nothing may read "false"`) + } + }) + } +} diff --git a/gpu/sink.go b/gpu/sink.go index f13d2d5..1ec9f1e 100644 --- a/gpu/sink.go +++ b/gpu/sink.go @@ -41,6 +41,12 @@ type SinkStats struct { PCSamples EventKindStats `json:"pc_samples,omitempty"` Modules EventKindStats `json:"modules,omitempty"` Events EventKindStats `json:"events,omitempty"` + // SamplingWindows is the serialization disclosure's ingest record. A + // dropped window does not lose a measurement — it loses the ability to + // say an execution ran unperturbed, which degrades the answer to + // "unknown". Counting it here is what makes that degradation visible + // rather than looking like a quiet run. + SamplingWindows EventKindStats `json:"sampling_windows,omitempty"` } // eventKind identifies which EventKindStats an admission outcome is @@ -56,6 +62,7 @@ const ( kindPCSample kindModule kindEvent + kindSamplingWindow ) // statsFor returns the EventKindStats field to record kind's outcome @@ -70,6 +77,8 @@ func (s *CountingSink) statsFor(kind eventKind) *EventKindStats { return &s.stats.PCSamples case kindModule: return &s.stats.Modules + case kindSamplingWindow: + return &s.stats.SamplingWindows default: return &s.stats.Events } @@ -375,6 +384,37 @@ func (s *CountingSink) EmitEvent(e GPUTimelineEvent) error { return nil } +// EmitSamplingWindow draws on the ANCHOR budget, not the data one, and skips +// the clock-domain check for the same reason EmitModule does — a window is +// two timestamps and a mode, with no per-event domain to validate beyond what +// the wire decoder already enforced. +// +// classAnchor because a window is the anchor of the serialization disclosure +// exactly as a launch is the anchor of a join: bursts are a handful per +// second while executions are thousands, and letting exec volume starve the +// windows out of the sink would turn every execution in the run "unknown" +// while the profile still looked full. +func (s *CountingSink) EmitSamplingWindow(w GPUSamplingWindow) error { + s.mu.Lock() + if err := s.admitCapacity(kindSamplingWindow, classAnchor); err != nil { + s.mu.Unlock() + return err + } + s.mu.Unlock() + + err := s.inner.EmitSamplingWindow(w) + + s.mu.Lock() + defer s.mu.Unlock() + if err != nil { + s.release(classAnchor) + s.statsFor(kindSamplingWindow).DroppedDownstream++ + return err + } + s.statsFor(kindSamplingWindow).Accepted++ + return nil +} + // SnapshotWith calls tl.Snapshot() and embeds s's own SinkStats into the // result - review Important 2: Timeline has no reference to the // CountingSink wrapping it (EventSink is the only contract between them), diff --git a/gpu/sink_test.go b/gpu/sink_test.go index e0283b8..99a41f7 100644 --- a/gpu/sink_test.go +++ b/gpu/sink_test.go @@ -17,6 +17,7 @@ type recordingSink struct { pcSamples int modules int events int + windows int } func (r *recordingSink) EmitLaunch(GPUKernelLaunch) error { r.launches++; return nil } @@ -24,6 +25,10 @@ func (r *recordingSink) EmitExec(GPUKernelExec) error { r.execs++; return ni func (r *recordingSink) EmitPCSample(GPUPCSample) error { r.pcSamples++; return nil } func (r *recordingSink) EmitModule(GPUModule) error { r.modules++; return nil } func (r *recordingSink) EmitEvent(GPUTimelineEvent) error { r.events++; return nil } +func (r *recordingSink) EmitSamplingWindow(GPUSamplingWindow) error { + r.windows++ + return nil +} // erroringSink lets a test control exactly what the downstream sink returns, // to exercise the reserve/delegate/settle path when delivery fails. @@ -31,11 +36,12 @@ type erroringSink struct { err error } -func (e *erroringSink) EmitLaunch(GPUKernelLaunch) error { return e.err } -func (e *erroringSink) EmitExec(GPUKernelExec) error { return e.err } -func (e *erroringSink) EmitPCSample(GPUPCSample) error { return e.err } -func (e *erroringSink) EmitModule(GPUModule) error { return e.err } -func (e *erroringSink) EmitEvent(GPUTimelineEvent) error { return e.err } +func (e *erroringSink) EmitLaunch(GPUKernelLaunch) error { return e.err } +func (e *erroringSink) EmitExec(GPUKernelExec) error { return e.err } +func (e *erroringSink) EmitPCSample(GPUPCSample) error { return e.err } +func (e *erroringSink) EmitModule(GPUModule) error { return e.err } +func (e *erroringSink) EmitEvent(GPUTimelineEvent) error { return e.err } +func (e *erroringSink) EmitSamplingWindow(GPUSamplingWindow) error { return e.err } // atomicSink is a genuinely concurrency-safe EventSink, used only by the // concurrent-access test so a data race in the test double itself can never @@ -44,11 +50,12 @@ type atomicSink struct { launches atomic.Int64 } -func (a *atomicSink) EmitLaunch(GPUKernelLaunch) error { a.launches.Add(1); return nil } -func (a *atomicSink) EmitExec(GPUKernelExec) error { return nil } -func (a *atomicSink) EmitPCSample(GPUPCSample) error { return nil } -func (a *atomicSink) EmitModule(GPUModule) error { return nil } -func (a *atomicSink) EmitEvent(GPUTimelineEvent) error { return nil } +func (a *atomicSink) EmitLaunch(GPUKernelLaunch) error { a.launches.Add(1); return nil } +func (a *atomicSink) EmitExec(GPUKernelExec) error { return nil } +func (a *atomicSink) EmitPCSample(GPUPCSample) error { return nil } +func (a *atomicSink) EmitModule(GPUModule) error { return nil } +func (a *atomicSink) EmitEvent(GPUTimelineEvent) error { return nil } +func (a *atomicSink) EmitSamplingWindow(GPUSamplingWindow) error { return nil } // fakeClock is an injectable, manually-advanced clock so the token-bucket // refill can be tested deterministically instead of racing the wall clock. @@ -229,6 +236,37 @@ func TestCountingSinkEmitModuleForwardsCountsAndEnforcesCapacity(t *testing.T) { assert.Equal(t, uint64(1), s.Stats().Modules.DroppedFull) } +// A sampling window draws on the ANCHOR budget, not the data one. That is the +// whole reason it has its own eventKind: bursts are a handful per second while +// executions are thousands, and if window admission shared the data bucket a +// busy workload could starve the disclosure out of the sink — turning every +// execution in the run gpu_serialized="unknown" while the profile still looked +// full. +func TestCountingSinkEmitSamplingWindowDrawsOnTheAnchorBudget(t *testing.T) { + inner := &recordingSink{} + s := NewCountingSink(inner, 2) + + // Spend the DATA budget entirely; the anchor budget is separate. + require.NoError(t, s.EmitExec(GPUKernelExec{ClockDomain: ClockDomainCPUMonotonic})) + require.NoError(t, s.EmitExec(GPUKernelExec{ClockDomain: ClockDomainCPUMonotonic})) + require.Error(t, s.EmitExec(GPUKernelExec{ClockDomain: ClockDomainCPUMonotonic})) + + require.NoError(t, s.EmitSamplingWindow(GPUSamplingWindow{ + Backend: BackendCUPTI, PID: 7, StartNs: 100, Mode: SamplingModeKernelSerialized, + })) + assert.Equal(t, 1, inner.windows) + assert.Equal(t, uint64(1), s.Stats().SamplingWindows.Accepted) + + // It is still bounded, and the loss is counted against its own kind + // rather than folded into another's. + require.NoError(t, s.EmitSamplingWindow(GPUSamplingWindow{Backend: BackendCUPTI, PID: 7, StartNs: 100, EndNs: 200})) + err := s.EmitSamplingWindow(GPUSamplingWindow{Backend: BackendCUPTI, PID: 7, StartNs: 300}) + require.Error(t, err) + assert.True(t, errors.Is(err, ErrSinkFull)) + assert.Equal(t, uint64(1), s.Stats().SamplingWindows.DroppedFull) + assert.Zero(t, s.Stats().Modules.DroppedFull, "a window's loss must not read as a module's") +} + // TestCountingSinkConcurrentEmitAndStats exercises CountingSink from many // goroutines at once - the point a clean sequential -race run cannot make. // It uses atomicSink so any race reported by -race can only be inside diff --git a/gpu/timeline.go b/gpu/timeline.go index 0cc3f40..271780a 100644 --- a/gpu/timeline.go +++ b/gpu/timeline.go @@ -43,6 +43,18 @@ type ExecutionView struct { // execution. Empty exactly when PCSamples is empty, since there is then // nothing to describe; one of PCAttribs() otherwise. PCAttrib PCAttrib `json:"pc_attrib,omitempty"` + + // Serialized is the gpu_serialized disclosure: whether this execution's + // measured duration was perturbed by the profiler serializing kernels + // around it. Set on EVERY execution, in every tier, by Snapshot — never + // left to a caller and never omitted. + // + // It is NOT `omitempty`, and the JSON tag says so on purpose. An absent + // field would read as "not perturbed" to a consumer that does not know to + // check for its absence, which is the same failure gpu_join's + // unconditional label exists to prevent. The type's zero value is + // "unknown" for the same reason (see SerializationState). + Serialized SerializationState `json:"serialized"` } // TimelineDropStats counts what Timeline's own bounded storage evicted. @@ -73,6 +85,13 @@ type TimelineDropStats struct { // // Zero on a healthy run of either tier. EvictedPendingModuleSamples uint64 `json:"evicted_pending_module_samples,omitempty"` + + // EvictedSamplingWindows counts PC-sampling bursts dropped from the + // bounded serialization-disclosure store (see windowStore). Eviction there + // only ever moves an execution's answer from "false" towards "unknown" — + // never the other way — so a non-zero value here costs certainty, not + // correctness. Zero on any run shorter than the store's bound. + EvictedSamplingWindows uint64 `json:"evicted_sampling_windows,omitempty"` } // Snapshot is a point-in-time, fully-joined view of everything Timeline @@ -142,6 +161,40 @@ type Snapshot struct { // many executions ended up carrying an inferred gpu_pc_attrib. See // PCJoinStats. PCJoin PCJoinStats `json:"pc_join,omitempty"` + // ---- The serialization disclosure (Tier A). + + // SamplingWindowsReceived is the CUMULATIVE number of PC-sampling burst + // records accepted into the disclosure store, and SamplingWindowsHeld / + // SamplingWindowsOpen are gauges of what it holds right now. A burst + // reaches the wire twice — open at its start, closed at its end — so on a + // clean Tier A run Received is about twice the number of bursts and Held + // is the number of distinct bursts. + // + // SamplingWindowsOpen is the one to read: non-zero means a burst's end is + // unknown, which is what a hard exit mid-burst looks like, and every + // execution from that burst's start onward reads "unknown". + // + // Zero everywhere in Tier B and with sampling off, where nothing is ever + // serialized and no window is ever emitted. + SamplingWindowsReceived uint64 `json:"sampling_windows_received,omitempty"` + SamplingWindowsHeld int `json:"sampling_windows_held,omitempty"` + SamplingWindowsOpen int `json:"sampling_windows_open,omitempty"` + + // The three gpu_serialized outcomes for the executions in THIS snapshot. + // + // THEY SUM TO len(Executions), EXACTLY. That identity is the same + // discipline gpu_join's three outcomes carry, and it is what makes the + // disclosure auditable rather than decorative: an execution that fell + // through every branch would show up as a shortfall in the sum instead of + // silently reading as one of the three. gpu/conformance_test.go asserts + // it, including on an empty snapshot. + // + // ExecutionsSerializationUnknown is the one that matters. It must never + // be reported as ExecutionsNotSerialized: "not perturbed" when the truth + // is "cannot tell" is precisely what spec §4 forbids. + ExecutionsSerialized uint64 `json:"executions_serialized,omitempty"` + ExecutionsNotSerialized uint64 `json:"executions_not_serialized,omitempty"` + ExecutionsSerializationUnknown uint64 `json:"executions_serialization_unknown,omitempty"` } // TimelineConfig configures Timeline's storage bounds. @@ -235,6 +288,28 @@ type TimelineConfig struct { // against. It is NOT silently skipped, which would make a missing store // look identical to a healthy run with no PC samples. Modules *ModuleStore + // SerializedSampling says that KERNEL_SERIALIZED PC sampling (Tier A) was + // SELECTED for this run. It is the agent's own configuration, not + // something inferred from the wire, and that is the point: "Tier A was + // asked for and no window arrived" and "Tier A was never asked for" are + // different facts with different answers, and only the agent knows which + // one holds. + // + // FALSE (the default) means every execution is gpu_serialized="false", + // unconditionally and correctly — with sampling off or in continuous + // collection nothing is ever serialized, so there is nothing to be unsure + // about. + // + // TRUE routes every execution through the window store, where the answer + // is "true", "false" or "unknown" depending on the evidence. Task 11 owns + // the setting that flips this; nothing here selects a tier. + SerializedSampling bool + + // MaxSamplingWindowsPerPID and MaxSamplingWindowPIDs bound the disclosure + // store. Zero means defaultMaxSamplingWindowsPerPID / + // defaultMaxSamplingWindowPIDs. + MaxSamplingWindowsPerPID int + MaxSamplingWindowPIDs int } // Timeline is the indexed join point: it ingests launches, executions, PC @@ -430,6 +505,20 @@ type Timeline struct { events *ring[GPUTimelineEvent] modules *ring[GPUModule] + // windows is the serialization disclosure's evidence store: the + // PC-sampling bursts this Timeline has been told about, per process. It is + // consulted at Snapshot rather than at EmitExec because a burst's closing + // record routinely arrives after the executions it covers — the producer + // drains on a timer — so classifying at ingest would mark a burst's own + // executions "unknown" for the very reason the window exists. + // + // serializedSampling is TimelineConfig.SerializedSampling, copied out at + // construction. When it is false this store is never consulted and every + // execution is "false", which is unconditionally correct in that + // configuration. + windows *windowStore + serializedSampling bool + dropped TimelineDropStats } @@ -568,9 +657,28 @@ func NewTimeline(cfg TimelineConfig) *Timeline { modstore: cfg.Modules, devicesByPID: make(map[uint32]processDevices), + + windows: newWindowStore(cfg.MaxSamplingWindowsPerPID, cfg.MaxSamplingWindowPIDs), + serializedSampling: cfg.SerializedSampling, } } +// EmitSamplingWindow records one PC-sampling burst. +// +// It is accepted in EVERY configuration, not only when SerializedSampling is +// set. A producer that is emitting windows is a producer that is bursting, and +// dropping the evidence because the agent's own config disagrees would leave +// the two ends silently out of step. What SerializedSampling gates is the +// ANSWER, not the ingest. +func (t *Timeline) EmitSamplingWindow(w GPUSamplingWindow) error { + t.mu.Lock() + defer t.mu.Unlock() + before := t.windows.evicted + t.windows.add(w) + t.dropped.EvictedSamplingWindows += t.windows.evicted - before + return nil +} + // isPendingLiveLocked answers orderedFIFO's isLive callback for pendingOrder: // a position is live only if t.pending still holds an entry for id stamped // with exactly seq. The caller must hold t.mu. @@ -1155,6 +1263,26 @@ func (t *Timeline) Snapshot() Snapshot { pendingSamples := t.pendingSampleTotal pendingModuleGroups := len(t.pendingModule) pendingModuleSamples := t.pendingModuleSampleTotal + // The disclosure's evidence, classified under the lock rather than copied + // out: the store is not drained by Snapshot (a window covers executions + // that have not arrived yet, and the next snapshot needs it), so the + // alternative would be to clone every burst on every call. + // + // serializedSampling FALSE takes the constant branch. With PC sampling off + // or in continuous collection nothing is ever serialized, so "false" is + // correct and unconditional and no store is consulted at all. + serialization := make([]SerializationState, len(execs)) + if t.serializedSampling { + for i, exec := range execs { + serialization[i] = t.windows.classify(exec.Correlation.PID, exec.StartNs, exec.EndNs) + } + } else { + for i := range serialization { + serialization[i] = SerializationNotSerialized + } + } + windowsReceived := t.windows.received + windowsHeld, windowsOpen := t.windows.windows() t.mu.Unlock() // The heuristic's candidate set is built lazily - only once the loop @@ -1207,8 +1335,14 @@ func (t *Timeline) Snapshot() Snapshot { views := make([]ExecutionView, 0, len(execs)) matched := make(map[CorrelationID]struct{}) + // The three gpu_serialized outcomes, counted where the view is BUILT and + // before any of the loop's `continue`s, so every execution is counted + // exactly once on every path through the join. That is what makes the sum + // identity (the three equal len(Executions)) hold by construction rather + // than by remembering to count in each branch. + var serializedCount, notSerializedCount, serializationUnknownCount uint64 for i, exec := range execs { - view := ExecutionView{Exec: exec, PCSamples: execSamples[i]} + view := ExecutionView{Exec: exec, PCSamples: execSamples[i], Serialized: serialization[i]} // gpu_pc_attrib, decided entirely by which index served this // execution. It is set independently of view.Join and view.Ambiguous // below and never reads or writes either: an execution can be joined @@ -1221,6 +1355,14 @@ func (t *Timeline) Snapshot() Snapshot { case len(view.PCSamples) > 0: view.PCAttrib = pcJoin.attribAt(i) } + switch view.Serialized { + case SerializationSerialized: + serializedCount++ + case SerializationNotSerialized: + notSerializedCount++ + default: + serializationUnknownCount++ + } if exec.Correlation.Present() { if l, ok := t.cache.Get(exec.Correlation); ok { @@ -1329,6 +1471,14 @@ func (t *Timeline) Snapshot() Snapshot { PendingModuleSamples: int(pendingModuleSamples), PendingModuleGroups: pendingModuleGroups, PCJoin: pcJoin.stats, + + SamplingWindowsReceived: windowsReceived, + SamplingWindowsHeld: windowsHeld, + SamplingWindowsOpen: windowsOpen, + + ExecutionsSerialized: serializedCount, + ExecutionsNotSerialized: notSerializedCount, + ExecutionsSerializationUnknown: serializationUnknownCount, } } diff --git a/gpu/types.go b/gpu/types.go index c63c14a..54ff7b9 100644 --- a/gpu/types.go +++ b/gpu/types.go @@ -359,6 +359,123 @@ type GPUPCSample struct { Count uint64 `json:"count"` } +// SamplingMode says which PC-sampling collection mode a GPUSamplingWindow +// describes. It mirrors GPU_SAMPLING_MODE_* in shim/core/usdt_abi.h, and the +// zero value is deliberately not one of the two real modes: a producer that +// left the field unset must not be read as having said "continuous". +type SamplingMode uint8 + +const ( + // SamplingModeUnset is the zero value. A window carrying it says nothing + // about whether kernels were serialized, so the interval it covers is + // "unknown" rather than either answer. + SamplingModeUnset SamplingMode = 0 + // SamplingModeContinuous is Tier B. Nothing is serialized, so a window in + // this mode marks nothing perturbed; it still says the producer was + // reporting over that interval. + SamplingModeContinuous SamplingMode = 1 + // SamplingModeKernelSerialized is Tier A. Every kernel that executed while + // this window was open ran serialized, sampled or not. + SamplingModeKernelSerialized SamplingMode = 2 +) + +func (m SamplingMode) String() string { + switch m { + case SamplingModeContinuous: + return "continuous" + case SamplingModeKernelSerialized: + return "kernel-serialized" + default: + return "unset" + } +} + +// GPUSamplingWindow is one PC-sampling burst: the interval over which the +// producer had PC sampling enabled on a context. +// +// It exists for exactly one reason, and it is not sampling coverage. In +// kernel-serialized collection the GPU serializes kernels while sampling is +// on, so every kernel that executed inside a window ran perturbed — SAMPLED OR +// NOT. The window, not the set of sampled kernels, is therefore the honest +// unit of the disclosure, and an execution is marked from its overlap with a +// window rather than from whether any PC sample was attributed to it. +// +// EndNs == 0 means the window was still OPEN when the producer stopped +// reporting. It is NOT a zero-length window and must never be read as one: the +// producer emits an open record the instant a burst starts and a closed record +// with the same StartNs when it stops, so a zero here means the closed record +// never came — a hard exit mid-burst. Treating it as zero-length would mark a +// whole perturbed tail "not serialized", which is the one answer that must +// never be reachable by accident. +type GPUSamplingWindow struct { + Backend GPUBackendID `json:"backend"` + PID uint32 `json:"pid,omitempty"` + ClockDomain ClockDomain `json:"clock_domain,omitempty"` + StartNs uint64 `json:"start_ns"` + EndNs uint64 `json:"end_ns,omitempty"` + Mode SamplingMode `json:"mode,omitempty"` + // Lost is how many gpu_sampling_window_v1 records the consumer knows were + // dropped between the previous window from this process and this one, from + // the producer's own sequence numbers. It is not decoration: a hole in the + // window history means the intervals either side of it cannot be shown to + // be gaps, so the store uses this to move its coverage start forward + // rather than spanning across the hole and reporting an unknown interval + // as a proven one. + Lost uint64 `json:"lost,omitempty"` +} + +// Open reports whether this window never closed. See the EndNs doc comment: +// an execution at or after an open window's StartNs is "unknown", never "not +// serialized". +func (w GPUSamplingWindow) Open() bool { return w.EndNs == 0 } + +// SerializationState is the gpu_serialized disclosure: whether an execution's +// measured duration was perturbed by kernel serialization. +// +// The ZERO VALUE IS "unknown", and that is the whole design. The failure this +// type exists to make unreachable is a profile that says "not perturbed" when +// it means "cannot tell" — spec §4 forbids exactly that, and it is the shape +// of the gpu_join precedent. Making "unknown" the zero value means a field +// nobody set, a struct built by a test, a value lost in a copy and a code path +// that forgot to classify all degrade to "unknown"; NONE of them can degrade +// to "false", because "false" has to be written deliberately. +type SerializationState uint8 + +const ( + // SerializationUnknown: kernel-serialized sampling was selected but no + // window covering this execution arrived — a dropped batch, a late + // attach, a sequence gap, or a burst that was still open when the + // producer stopped reporting. It must never degrade to + // SerializationNotSerialized. + SerializationUnknown SerializationState = iota + // SerializationSerialized: this execution overlapped a burst. Its + // duration is PERTURBED BY THE MEASUREMENT and must be read as such. + SerializationSerialized + // SerializationNotSerialized: nothing was serialized over this + // execution's interval. Unconditionally correct in continuous collection + // and with PC sampling off — nothing is ever serialized there — and in + // kernel-serialized collection only when the agent holds a proven gap. + SerializationNotSerialized +) + +// String is the label value. "true"/"false"/"unknown" rather than a Go-ish +// spelling because these strings ARE the gpu_serialized pprof label values, +// and the one place they are written is here. +func (s SerializationState) String() string { + switch s { + case SerializationSerialized: + return "true" + case SerializationNotSerialized: + return "false" + default: + return "unknown" + } +} + +// MarshalJSON renders the label value rather than the underlying integer, so a +// serialized Snapshot reads the same way the profile does. +func (s SerializationState) MarshalJSON() ([]byte, error) { return json.Marshal(s.String()) } + // TimelineEventKind classifies a GPUTimelineEvent. type TimelineEventKind string @@ -487,6 +604,12 @@ type EventSink interface { EmitPCSample(GPUPCSample) error EmitModule(GPUModule) error EmitEvent(GPUTimelineEvent) error + // EmitSamplingWindow delivers one PC-sampling burst. It is a method of + // its own rather than a GPUTimelineEvent with attributes because the + // disclosure it carries must not depend on string parsing: an execution + // is marked perturbed from these intervals, and a window that failed to + // be recognised would silently downgrade "unknown" to "false". + EmitSamplingWindow(GPUSamplingWindow) error } // Backend produces normalized GPU events into an EventSink. diff --git a/gpuprobe/attach_test.go b/gpuprobe/attach_test.go index 4fcc925..bc41005 100644 --- a/gpuprobe/attach_test.go +++ b/gpuprobe/attach_test.go @@ -16,11 +16,12 @@ import ( // nothing here is ever called on these error paths. type nopSink struct{} -func (nopSink) EmitLaunch(gpu.GPUKernelLaunch) error { return nil } -func (nopSink) EmitExec(gpu.GPUKernelExec) error { return nil } -func (nopSink) EmitPCSample(gpu.GPUPCSample) error { return nil } -func (nopSink) EmitModule(gpu.GPUModule) error { return nil } -func (nopSink) EmitEvent(gpu.GPUTimelineEvent) error { return nil } +func (nopSink) EmitLaunch(gpu.GPUKernelLaunch) error { return nil } +func (nopSink) EmitExec(gpu.GPUKernelExec) error { return nil } +func (nopSink) EmitPCSample(gpu.GPUPCSample) error { return nil } +func (nopSink) EmitModule(gpu.GPUModule) error { return nil } +func (nopSink) EmitEvent(gpu.GPUTimelineEvent) error { return nil } +func (nopSink) EmitSamplingWindow(gpu.GPUSamplingWindow) error { return nil } // Attach must return an error, never panic, when there is no such file. func TestAttachNonexistentShimReturnsError(t *testing.T) { diff --git a/gpuprobe/consumer.go b/gpuprobe/consumer.go index 4a19e4b..c62a7c2 100644 --- a/gpuprobe/consumer.go +++ b/gpuprobe/consumer.go @@ -1095,30 +1095,38 @@ type Stats struct { // PC sampling, silently absent, if it were not counted. StallNamesMissing uint64 // SamplingWindowsDecoded counts gpu_sampling_window_v1 records read. - // One record is one PC-sampling burst. Tier A duty-cycles, so a Tier A - // run produces many; Tier B does not burst, so a Tier B run produces - // at most one. - // - // The windows' CONTENT is not retained yet - the serialization - // disclosure that consumes it is Task 10 - so this counter is - // deliberately the whole of what the consumer does with them. That - // makes the discard sized and visible rather than silent, which is the - // contract; it is not a claim that the windows have been used. - // - // Healthy: non-zero in Tier A, zero or one in Tier B. Worst: zero in - // Tier A, where nothing would then say which executions ran perturbed. + // A burst reaches the wire TWICE - an open record the instant it starts + // and a closed record with the same start_ns when it stops - so this is + // about twice the number of bursts on a clean Tier A run. Tier B does + // not burst and emits none. + // + // The windows are normalized and handed to the sink, where the + // serialization disclosure consumes them (gpu/serialization.go). This + // counter is the wire-side half of the reconciliation: + // Snapshot.SamplingWindowsReceived is what actually reached the store. + // + // Healthy: non-zero in Tier A, zero in Tier B. Worst: zero in Tier A, + // where nothing would then say which executions ran perturbed and every + // one of them reads serialized="unknown". SamplingWindowsDecoded uint64 // SamplingWindowsOpen is the subset of SamplingWindowsDecoded that // arrived with end_ns == 0, which the ABI defines as "still open when - // the producer stopped reporting" - a hard exit mid-burst, since the - // shim's atexit handler closes the window on the ordinary path. It is - // NOT a zero-length window, and the two must never be conflated: an - // open window means every execution at or after its start_ns is - // serialized="unknown", never "false". - // - // Healthy: zero. Worst: non-zero, meaning a burst's end is unknown and - // an unbounded tail of executions cannot be said to have run - // unperturbed. + // the producer stopped reporting". It is NOT a zero-length window, and + // the two must never be conflated: an open window means every execution + // at or after its start_ns is serialized="unknown", never "false". + // + // It is NOT an anomaly by itself, and this is the counter's one + // subtlety. The producer emits an open record at every burst START + // precisely so that a hard exit leaves the burst visible instead of + // losing it, so a healthy Tier A run produces one of these per burst and + // this counter tracks SamplingWindowsDecoded / 2. What IS an anomaly is + // a window still open once its close should have arrived, which is + // Snapshot.SamplingWindowsOpen - a gauge of the store, not a count of + // records. + // + // Healthy: about half of SamplingWindowsDecoded in Tier A, zero in Tier + // B. Worst: zero in Tier A, meaning the burst-start records are being + // lost and a hard exit would take its whole perturbed tail with it. SamplingWindowsOpen uint64 // ConfigsDecoded counts gpu_config_v1 records read. The producer emits // one per process and replays it on late attach, so this counts @@ -1777,13 +1785,27 @@ func decodeBatch(b []byte) (batch, error) { // noteSeq counts batches lost between the ones that arrived. A gap is loss // the consumer did not observe and must never be silent (spec §6.1). The // stream is identified by (kind, pid): see seqKey. Caller holds mu. -func (c *Consumer) noteSeq(kind, pid uint32, seq uint64) { +// noteSeq records a batch's producer sequence number and returns how many +// records of this kind, from this process, are known to have been lost since +// the previous one. +// +// The return value is not decoration. The serialization disclosure below needs +// it: a hole in the sampling-window history means the intervals either side of +// it cannot be shown to be gaps, so the window store has to restart its +// coverage rather than span across the hole and report an unproven gap as a +// proven one. That is the difference between gpu_serialized="unknown" and +// gpu_serialized="false", and it is the one distinction this tier exists to +// keep. +func (c *Consumer) noteSeq(kind, pid uint32, seq uint64) uint64 { key := seqKey{kind: kind, pid: pid} prev, seen := c.seqByStream[key] + var lost uint64 if seen && seq > prev+1 { - c.stats.SequenceGaps += seq - prev - 1 + lost = seq - prev - 1 + c.stats.SequenceGaps += lost } c.seqByStream[key] = seq + return lost } // correlationOf converts a wire correlation into the core's CorrelationID. @@ -1868,7 +1890,7 @@ func (c *Consumer) applyBatch(b batch) { defer c.mu.Unlock() c.stats.Batches++ - c.noteSeq(b.Kind, b.PID, b.Seq) + lost := c.noteSeq(b.Kind, b.PID, b.Seq) // First sight of a process is what makes it interesting to the walker, // and "first sight" is any batch, not the first sampled launch. // @@ -1976,9 +1998,18 @@ func (c *Consumer) applyBatch(b batch) { c.learnStallNameLocked(s) } case kindSamplingWindow: - for _, w := range b.SamplingWindows { + for i, w := range b.SamplingWindows { c.stats.Records++ - c.noteSamplingWindowLocked(w) + // The loss is attributed to the FIRST window in the batch and to + // no other: `lost` counts records missing before this batch, so + // charging it to every record in it would restart the store's + // coverage once per window and throw away the rest of the batch's + // evidence. + var recLost uint64 + if i == 0 { + recLost = lost + } + c.noteSamplingWindowLocked(b.PID, w, recLost) } case kindConfig: for _, cfg := range b.Configs { @@ -2147,21 +2178,48 @@ func (c *Consumer) learnStallNameLocked(rec gpuabi.StallReason) { } } -// noteSamplingWindowLocked records one PC-sampling burst. Caller holds mu. +// noteSamplingWindowLocked normalizes one PC-sampling burst and hands it to +// the sink. Caller holds mu. // -// The window's content is not retained: the serialization disclosure that -// consumes it - marking which executions ran perturbed - is a later task, -// and building half of it here would leave a store nothing reads. What this -// does is make the discard sized and visible, which is the standing rule: -// counted is not the same as used, and Stats.SamplingWindowsDecoded says so. -func (c *Consumer) noteSamplingWindowLocked(w gpuabi.SamplingWindow) { +// The PID comes from the batch header - the process that fired the probe - and +// travels IN the event rather than being checked somewhere else, the same +// discipline CorrelationID carries. Two processes both running Tier A produce +// interleaved windows on one system-wide attach, and a window store that +// mixed them would mark one process's executions perturbed because the other +// one was bursting. +// +// lost is how many window records from this process are known to have been +// dropped just before this one. It is the difference between "we hold an +// unbroken history and can prove this interval was a gap" and "we cannot", so +// it is carried on the event rather than left as a global counter nobody can +// attribute. +func (c *Consumer) noteSamplingWindowLocked(pid uint32, w gpuabi.SamplingWindow, lost uint64) { c.stats.SamplingWindowsDecoded++ if w.Open() { // end_ns == 0 is the ABI's "still open when the producer stopped // reporting", not a zero-length window. DecodeSamplingWindow has // already refused a genuinely inverted one. + // + // This is NOT an anomaly by itself, and its doc comment on Stats says + // why: the producer emits an open record at every burst START so that + // a hard exit leaves the burst visible rather than losing it, so a + // healthy Tier A run produces one of these per burst. What is an + // anomaly is a window still open at Snapshot time, which + // Snapshot.SamplingWindowsOpen reports. c.stats.SamplingWindowsOpen++ } + ev := gpu.GPUSamplingWindow{ + Backend: c.cfg.Backend, + PID: pid, + ClockDomain: gpu.ClockDomainCPUMonotonic, + StartNs: w.StartNs, + EndNs: w.EndNs, + Mode: gpu.SamplingMode(w.Mode), + Lost: lost, + } + if err := c.cfg.Sink.EmitSamplingWindow(ev); err != nil { + c.stats.SinkRejected++ + } } // noteConfigLocked records the producer's sampling configuration. Caller diff --git a/gpuprobe/consumer_test.go b/gpuprobe/consumer_test.go index 0d49da4..9c487e3 100644 --- a/gpuprobe/consumer_test.go +++ b/gpuprobe/consumer_test.go @@ -57,6 +57,7 @@ type recordingSink struct { execs []gpu.GPUKernelExec pcSamples []gpu.GPUPCSample modules []gpu.GPUModule + windows []gpu.GPUSamplingWindow err error // onEmit, if set, is called after each accepted event. The lifecycle // tests use it to know Run has completed a loop iteration. @@ -115,6 +116,17 @@ func (s *recordingSink) EmitModule(m gpu.GPUModule) error { func (s *recordingSink) EmitEvent(gpu.GPUTimelineEvent) error { return s.errOnly() } +func (s *recordingSink) EmitSamplingWindow(w gpu.GPUSamplingWindow) error { + s.mu.Lock() + defer s.mu.Unlock() + if s.err != nil { + return s.err + } + s.windows = append(s.windows, w) + s.note() + return nil +} + func (s *recordingSink) errOnly() error { s.mu.Lock() defer s.mu.Unlock() @@ -3241,6 +3253,102 @@ func TestSamplingWindowsAreCountedAndTheOpenOneIsSeparate(t *testing.T) { assert.Zero(t, st.Undecoded) } +// samplingWindowBatchFor is samplingWindowBatch with the batch header's pid +// and producer sequence spelled out, because both are load-bearing for the +// serialization disclosure: the pid is what keeps one process's bursts off +// another's executions, and the sequence is what tells the consumer records +// were lost. +func samplingWindowBatchFor(pid uint32, seq uint64, wins ...[3]uint64) []byte { + buf := make([]byte, batchHdrSize+len(wins)*gpuabi.SizeSamplingWindow) + putU32(buf[0:], kindSamplingWindow) + putU32(buf[4:], uint32(len(wins))) + putU64(buf[8:], seq) + putU32(buf[16:], pid) + putU64(buf[24:], uint64(len(wins)*gpuabi.SizeSamplingWindow)) + putU32(buf[32:], ^uint32(0)) + for i, w := range wins { + off := batchHdrSize + i*gpuabi.SizeSamplingWindow + putU64(buf[off:], w[0]) + putU64(buf[off+8:], w[1]) + buf[off+16] = uint8(w[2]) + } + return buf +} + +// The window has to REACH the sink, carrying the producing process. Counting +// it and dropping it would leave the disclosure with no evidence at all and +// every execution reading "unknown" — the exact state Task 7 left this in and +// said so. +func TestSamplingWindowsReachTheSinkWithTheirProcess(t *testing.T) { + sink := &recordingSink{} + c := newTestConsumer(sink) + apply(t, c, samplingWindowBatchFor(4242, 0, + [3]uint64{1_000, 51_000, uint64(gpuabi.SamplingModeKernelSerialized)})) + + require.Len(t, sink.windows, 1) + w := sink.windows[0] + assert.Equal(t, gpu.BackendCUPTI, w.Backend) + assert.Equal(t, uint32(4242), w.PID, + "windows are per-process; the pid travels IN the event, not in a check elsewhere") + assert.Equal(t, gpu.ClockDomainCPUMonotonic, w.ClockDomain) + assert.Equal(t, uint64(1_000), w.StartNs) + assert.Equal(t, uint64(51_000), w.EndNs) + assert.Equal(t, gpu.SamplingModeKernelSerialized, w.Mode) + assert.False(t, w.Open()) + assert.Zero(t, w.Lost) +} + +// A gap in the producer's window sequence is the difference between "we hold +// an unbroken history and can prove this interval was a gap" and "we cannot". +// It is carried on the first window of the batch that noticed it, and on no +// other — charging it to every record would restart the store's coverage once +// per window and throw the rest of the batch's evidence away. +func TestSamplingWindowSequenceGapIsCarriedOnTheFirstWindowOnly(t *testing.T) { + sink := &recordingSink{} + c := newTestConsumer(sink) + apply(t, c, samplingWindowBatchFor(4242, 0, + [3]uint64{1_000, 2_000, uint64(gpuabi.SamplingModeKernelSerialized)})) + // seq jumps 0 -> 3: two records lost. + apply(t, c, samplingWindowBatchFor(4242, 3, + [3]uint64{9_000, 10_000, uint64(gpuabi.SamplingModeKernelSerialized)}, + [3]uint64{11_000, 12_000, uint64(gpuabi.SamplingModeKernelSerialized)})) + + require.Len(t, sink.windows, 3) + assert.Zero(t, sink.windows[0].Lost) + assert.Equal(t, uint64(2), sink.windows[1].Lost) + assert.Zero(t, sink.windows[2].Lost, + "the loss precedes the batch; it belongs to one record in it, not to all of them") + assert.Equal(t, uint64(2), c.Stats().SequenceGaps) +} + +// Another process's sequence is its own. A gap in one must not be charged to +// the other, or one busy producer would permanently degrade a quiet one's +// disclosure to "unknown". +func TestSamplingWindowSequencesArePerProcess(t *testing.T) { + sink := &recordingSink{} + c := newTestConsumer(sink) + apply(t, c, samplingWindowBatchFor(1, 0, [3]uint64{1_000, 2_000, 2})) + apply(t, c, samplingWindowBatchFor(2, 0, [3]uint64{1_000, 2_000, 2})) + apply(t, c, samplingWindowBatchFor(1, 5, [3]uint64{3_000, 4_000, 2})) + apply(t, c, samplingWindowBatchFor(2, 1, [3]uint64{3_000, 4_000, 2})) + + require.Len(t, sink.windows, 4) + assert.Equal(t, uint64(4), sink.windows[2].Lost, "pid 1 lost four") + assert.Zero(t, sink.windows[3].Lost, "pid 2's stream is contiguous") +} + +// A sink that refuses a window is counted, never silent: a refused window is +// evidence the disclosure will not have. +func TestRefusedSamplingWindowIsCounted(t *testing.T) { + sink := &recordingSink{err: errors.New("full")} + c := newTestConsumer(sink) + apply(t, c, samplingWindowBatchFor(4242, 0, [3]uint64{1_000, 2_000, 2})) + + st := c.Stats() + assert.Equal(t, uint64(1), st.SamplingWindowsDecoded) + assert.Equal(t, uint64(1), st.SinkRejected) +} + // An inverted window is a producer contract violation, not a short buffer. // It is refused at the batch boundary so a negative duration can never reach // the serialization disclosure. diff --git a/gpuprobe/kindmax_test.go b/gpuprobe/kindmax_test.go index f3695e9..a43e1be 100644 --- a/gpuprobe/kindmax_test.go +++ b/gpuprobe/kindmax_test.go @@ -128,17 +128,17 @@ func TestEveryProducerProbeHasACookieAndViceVersa(t *testing.T) { "attached, its semaphore never arms, and every record it would carry "+ "is silently never produced", probe) } - // One probe has a cookie and no producer yet, waiting on a task that is - // explicitly not this one. Named rather than skipped: a cookie with no - // producer is dead wire surface that reads as coverage, so the list has to - // shrink to empty and every entry has to say what shrinks it. + // THE LIST IS EMPTY, and that is the state it was built to reach: every + // probe the ABI defines is now fired by a producer, so no cookie names + // dead wire surface that reads as coverage. // - // gpu_module_load_v1 left this list when Task 5 landed: both producers - // fire it now, and the assertion below is what forced the entry out rather - // than letting it sit here claiming a gap that had closed. - notYetFired := map[string]string{ - "gpu_sampling_window_v1": "Tier A, KERNEL_SERIALIZED, duty-cycled (Task 10)", - } + // It emptied one entry at a time and each time because this assertion + // forced it, not because anyone remembered. gpu_module_load_v1 left when + // Task 5 landed; gpu_sampling_window_v1 left when Tier A started emitting + // a window around every burst. A future probe added ahead of its producer + // goes back in here with the task that will fire it named, and comes out + // the same way. + notYetFired := map[string]string{} for _, probe := range knownProbeNames { if _, ok := emitted[probe]; ok { assert.NotContainsf(t, notYetFired, probe, diff --git a/shim/Makefile b/shim/Makefile index a8784d8..9c3e677 100644 --- a/shim/Makefile +++ b/shim/Makefile @@ -228,11 +228,12 @@ check-cubin-defer: core/cubin_defer_test.cc core/cubinqueue.h # _Static_asserts are the test. Compiling it only as C++ leaves the C11 spelling # unguarded -- which is how the header's validity broke earlier in this branch, # when a C++ translation unit included it for the first time. -test: check-cubin-defer core/batch_test.cc core/clock_test.cc core/cubin_test.cc core/cubinqueue_test.cc core/drain_test.cc core/pcdrain_test.cc core/enroll_test.cc core/usdt_abi_test.c core/sampler_test.cc core/probe_args_test.cc stub/probe_order_test.cc stub/stub.cc $(CORE_SRC) +test: check-cubin-defer core/batch_test.cc core/clock_test.cc core/cubin_test.cc core/cubinqueue_test.cc core/drain_test.cc core/pcdrain_test.cc core/burst_test.cc core/enroll_test.cc core/usdt_abi_test.c core/sampler_test.cc core/probe_args_test.cc stub/probe_order_test.cc stub/stub.cc $(CORE_SRC) $(CXX) -std=c++17 -pthread -I core -o /tmp/batch_test core/batch_test.cc core/batch.cc && /tmp/batch_test $(CXX) -std=c++17 -I core -o /tmp/clock_test core/clock_test.cc core/clock.cc && /tmp/clock_test $(CXX) -std=c++17 -pthread -I core -o /tmp/drain_test core/drain_test.cc core/drain.cc && /tmp/drain_test $(CXX) -std=c++17 -pthread -I core -o /tmp/pcdrain_test core/pcdrain_test.cc && /tmp/pcdrain_test + $(CXX) -std=c++17 -Wall -Werror -pthread -I core -o /tmp/burst_test core/burst_test.cc && /tmp/burst_test $(CXX) -std=c++17 -pthread -I core -o /tmp/enroll_test core/enroll_test.cc core/enroll.cc && /tmp/enroll_test $(CXX) -std=c++17 -Wall -Werror -pthread -I core -o /tmp/cubin_test core/cubin_test.cc core/cubin.cc core/enroll.cc && /tmp/cubin_test $(CXX) -std=c++17 -Wall -Werror -pthread -I core -o /tmp/cubinqueue_test core/cubinqueue_test.cc core/cubinqueue.cc && /tmp/cubinqueue_test diff --git a/shim/core/burst.h b/shim/core/burst.h new file mode 100644 index 0000000..5b07a83 --- /dev/null +++ b/shim/core/burst.h @@ -0,0 +1,328 @@ +// Tier A's duty cycle: when a KERNEL_SERIALIZED PC-sampling burst starts, +// when it stops, and how long to wait before the next one. +// +// Why this is a closed loop and not a constant +// -------------------------------------------- +// In CUPTI_PC_SAMPLING_COLLECTION_MODE_KERNEL_SERIALIZED every kernel that +// runs while sampling is on runs serialized. That is a measurable perturbation +// of the workload being measured, so the profiler owes the operator two +// things: a bound on how much of the run was perturbed, and a record of which +// part. This class is the first. The second is gpu_sampling_window_v1, which +// the adapter emits around every burst this class opens. +// +// A fixed gap cannot bound the yield: the number of (PC, stall) pairs a 50 ms +// burst produces depends on the workload's kernel occupancy and on the +// sampling period, and varies by orders of magnitude between an idle process +// and a saturating one. Parca's PC sampler ships ~50 ms bursts aimed at ~100 +// pairs per second; holding a target rate is what makes the wire volume, the +// label cardinality and the perturbation all predictable instead of +// workload-dependent. So the gap is tuned by a controller and the burst length +// is held fixed. +// +// The duty ceiling is a HARD bound, not a target +// ---------------------------------------------- +// The controller can only ever lengthen the gap past the ceiling's minimum; it +// can never shorten it below. burst/(burst+gap) <= max_duty holds for every +// gap this class will ever produce, for every observed rate, including zero, +// including a target rate of zero and including arithmetic that overflows into +// infinity or NaN. That is asserted directly in core/burst_test.cc by sweeping +// the observed rate across its whole range rather than by reasoning about the +// loop, because "the controller would never ask for that" is exactly the +// argument that has been wrong before on this project. +// +// Pure, so it is provable without a GPU +// ------------------------------------- +// Nothing here calls CUPTI, allocates, locks or reads a clock: the caller +// supplies `now_ns` and the running (PC, stall) pair count, and the loop +// itself is a static function of (target rate, observed rate, elapsed). That +// is what lets core/burst_test.cc prove convergence, the duty ceiling and the +// zero-rate behaviour against a fake clock on a machine with no NVIDIA +// hardware at all. +#ifndef PERFAGENT_BURST_H +#define PERFAGENT_BURST_H + +#include +#include +#include + +namespace perfagent { + +// What the burst timer should do at this instant. +enum class BurstAction { + kNone, // stay as you are + kStart, // cuptiPCSamplingStart(): open a window + kStop, // cuptiPCSamplingStop(): close the window, flush, then wait +}; + +// Why a burst stopped. It travels with the stop so the adapter's log and its +// counters can tell an ordinary duty-cycle stop from a refusal. +enum class BurstStopReason { + kDutyCycle, // the burst reached burst_ns + kGraph, // a CUDA-graph execution was observed; Tier A refuses to run + kShutdown, // teardown, finalize or exit +}; + +struct BurstConfig { + // How long one burst samples for. Held fixed; the gap is what moves. + uint64_t burst_ns = 50ull * 1000000ull; // 50 ms + + // The (PC, stall) pairs per second the loop holds. Parca's default. + double target_rate = 100.0; + + // The hard ceiling on burst/(burst+gap). 0.1 means at most a tenth of + // wall-clock time may run serialized. Clamped into (0, 1] on use. + double max_duty = 0.1; + + // The longest the loop may space bursts out. Without it a workload that + // produces a huge pair count in one burst would push the next burst out + // to hours and Tier A would silently stop reporting. + uint64_t max_gap_ns = 10ull * 1000000000ull; // 10 s + + // The loop's gain: how much of the newly computed gap to adopt each + // cycle. 1.0 is deadbeat and chases noise; 0 never moves. 0.5 halves the + // error per cycle, which converges in a handful of bursts and still + // damps a single anomalous burst. + double gain = 0.5; +}; + +// The smallest gap the duty ceiling permits after a burst of cfg.burst_ns. +// +// duty = burst / (burst + gap) <= max_duty <=> gap >= burst * (1/max_duty - 1) +// +// max_duty is clamped into (0, 1] first: a zero or negative ceiling would ask +// for an infinite gap and a ceiling above 1 is not a ceiling at all. +inline uint64_t burst_min_gap_ns(const BurstConfig &cfg) { + double duty = cfg.max_duty; + if (!(duty > 0.0)) duty = 0.001; // also catches NaN + if (duty > 1.0) duty = 1.0; + const double g = (double)cfg.burst_ns * (1.0 / duty - 1.0); + if (!(g > 0.0)) return 0; + if (g >= (double)cfg.max_gap_ns) return cfg.max_gap_ns; + return (uint64_t)g; +} + +// THE LOOP, as a pure function of (target rate, observed rate, elapsed). +// +// observed_rate = pairs / elapsed pairs per second +// ratio = observed_rate / target_rate >1 means too many +// cycle = elapsed * ratio the cycle length that +// would have hit target +// raw_gap = cycle - burst_ns +// gap = prev_gap + gain * (raw_gap - prev_gap) damped +// gap = clamp(gap, min_gap, max_gap) +// +// The model this assumes, stated rather than left implicit: pairs are produced +// only while a burst is open, so the pairs-per-burst figure does not depend on +// the gap. Under that model raw_gap is constant for a steady workload and the +// damped update is a geometric sequence with ratio (1 - gain) — it converges +// monotonically, which core/burst_test.cc asserts by running it. +// +// Every degenerate input lands on a clamp rather than on undefined behaviour: +// elapsed_ns == 0 no information; hold prev_gap (still clamped) +// pairs == 0 ratio 0, raw_gap 0 -> the gap walks DOWN to min_gap. +// Never to zero: min_gap is the duty ceiling's floor and +// is what a workload producing nothing converges on, so +// a quiet GPU samples at the ceiling and no faster. +// target_rate <= 0 ratio is +inf or NaN -> clamped to max_gap +// huge pairs clamped to max_gap +// The comparisons are written so NaN falls through to the safe side; `!(x > +// y)` rather than `x <= y` is deliberate everywhere below. +inline uint64_t burst_next_gap_ns(const BurstConfig &cfg, uint64_t prev_gap_ns, + uint64_t pairs, uint64_t elapsed_ns) { + const uint64_t lo = burst_min_gap_ns(cfg); + const uint64_t hi = cfg.max_gap_ns > lo ? cfg.max_gap_ns : lo; + + double gap; + if (elapsed_ns == 0) { + gap = (double)prev_gap_ns; + } else { + const double observed_rate = (double)pairs * 1e9 / (double)elapsed_ns; + const double ratio = observed_rate / cfg.target_rate; + const double cycle = (double)elapsed_ns * ratio; + double raw_gap = cycle - (double)cfg.burst_ns; + if (!(raw_gap > 0.0)) raw_gap = 0.0; // also catches NaN + double gain = cfg.gain; + if (!(gain > 0.0)) gain = 0.0; + if (gain > 1.0) gain = 1.0; + gap = (double)prev_gap_ns + gain * (raw_gap - (double)prev_gap_ns); + } + + // NaN and -inf land here; so does any value below the duty floor. + if (!(gap > (double)lo)) return lo; + if (gap >= (double)hi) return hi; + return (uint64_t)gap; +} + +// The state machine the adapter's burst timer polls. One mutex, because the +// poll runs on the burst thread while the shutdown path runs on whoever is +// exiting. +// +// The cycle has three calls and they are three because of WHEN the yield is +// known, not for symmetry: +// +// poll() -> kStart open a window, cuptiPCSamplingStart() +// poll() -> kStop cuptiPCSamplingStop(), close the window +// closed() after the range-end flush, once the pairs that +// burst produced are actually on the wire +// +// CUPTI hands the burst's PC records over on the flush that FOLLOWS the stop, +// so a controller that read the pair count at stop time would measure every +// burst as having produced nothing and would sit at the duty floor forever. +// That is not a hypothetical: it is what the first draft of this class did, +// and the convergence test below is what caught it. closed() is therefore a +// separate call made after the drain. +// +// A caller that forgets closed() degrades rather than breaks: stop_locked +// schedules the next burst at the CURRENT gap, so the duty cycle keeps +// running at whatever rate it had last converged on and only the loop stops +// adapting. +class BurstController { +public: + explicit BurstController(BurstConfig cfg) + : cfg_(cfg), gap_ns_(burst_min_gap_ns(cfg)) {} + + // The whole start/stop decision, taken once per burst-timer tick. + // + // graphs_observed is the CUDA-graph refusal, and it is IN this function + // rather than beside it on purpose. A graph launch fires one runtime + // callback for N kernels, so N executions share one correlation and Tier + // A's entire claim --- that a PC sample's correlation names the kernel + // that stalled --- becomes false while still looking exact. The refusal is + // therefore structural: once this reads true the controller never returns + // kStart again, and it returns kStop first if a burst is open, so the + // window closes honestly instead of being abandoned. It is not a downgrade + // to Tier B: this class does not know how to become Tier B, and silently + // becoming it is what the plan forbids. + // + // Side-effecting by design, exactly like PCDrainSchedule::due: it records + // the transition it just returned, so "ask, then forget to act" is not a + // shape this API offers. + BurstAction poll(uint64_t now_ns, bool graphs_observed) { + std::lock_guard g(mu_); + if (graphs_observed && !refused_) { + refused_ = true; + graph_refusals_++; + } + if (sampling_) { + if (refused_) return stop_locked(now_ns, BurstStopReason::kGraph); + if (now_ns - burst_start_ns_ < cfg_.burst_ns) return BurstAction::kNone; + return stop_locked(now_ns, BurstStopReason::kDutyCycle); + } + if (refused_) return BurstAction::kNone; + if (started_ && now_ns < next_start_ns_) return BurstAction::kNone; + sampling_ = true; + if (!started_) { + started_ = true; + first_start_ns_ = now_ns; + } + burst_start_ns_ = now_ns; + bursts_++; + return BurstAction::kStart; + } + + // Called after the range-end flush, with the process-wide RUNNING TOTAL of + // (PC, stall) pairs put on the wire. The controller differences it itself, + // so the caller cannot get the delta wrong by forgetting to reset + // something. + // + // The cycle the loop measures is burst + gap: the pairs a burst produced, + // divided by the wall time the next burst will be spaced out to. That is + // why the fixed point is independent of the current gap and the loop + // converges geometrically rather than oscillating. + void closed(uint64_t pairs_total) { + std::lock_guard g(mu_); + if (!have_open_yield_) return; + have_open_yield_ = false; + // The delta is taken between consecutive closes rather than from the + // burst's own start, because pairs are only produced while a burst is + // open: anything that arrives late still belongs to the burst that + // produced it, and this attributes it there instead of losing it. + const uint64_t pairs = + pairs_total > pairs_at_last_close_ ? pairs_total - pairs_at_last_close_ : 0; + pairs_at_last_close_ = pairs_total; + pairs_ += pairs; + const uint64_t elapsed = last_burst_dur_ + gap_ns_; + gap_ns_ = burst_next_gap_ns(cfg_, gap_ns_, pairs, elapsed); + next_start_ns_ = last_burst_end_ns_ + gap_ns_; + } + + // Teardown. Returns kStop when a burst was open, so the caller closes the + // window with a real end timestamp on the ordinary exit path --- which is + // precisely what makes end_ns == 0 on the wire mean a HARD exit and + // nothing else. + BurstAction shutdown(uint64_t now_ns) { + std::lock_guard g(mu_); + refused_ = true; // no further bursts after teardown begins + if (!sampling_) return BurstAction::kNone; + return stop_locked(now_ns, BurstStopReason::kShutdown); + } + + bool sampling() const { std::lock_guard g(mu_); return sampling_; } + bool refused() const { std::lock_guard g(mu_); return refused_; } + // The start timestamp of the burst that is currently open. Zero when none + // is; the caller checks sampling() rather than testing this for zero. + uint64_t burst_start_ns() const { std::lock_guard g(mu_); return burst_start_ns_; } + BurstStopReason last_stop_reason() const { + std::lock_guard g(mu_); + return last_stop_; + } + + // Every counter here is assertable at a known value from a test, which is + // the standing rule on this pipeline: a duty cycle that stopped firing + // must not look like a workload that stopped stalling. + uint64_t bursts() const { std::lock_guard g(mu_); return bursts_; } + uint64_t burst_ns() const { std::lock_guard g(mu_); return burst_ns_; } + uint64_t pairs_observed() const { std::lock_guard g(mu_); return pairs_; } + uint64_t graph_refusals() const { std::lock_guard g(mu_); return graph_refusals_; } + uint64_t gap_ns() const { std::lock_guard g(mu_); return gap_ns_; } + const BurstConfig &config() const { return cfg_; } + + // The duty fraction actually achieved so far, over the interval since the + // first burst opened. Reported rather than assumed: max_duty bounds what + // the loop may ASK for, and this is what the process actually did. + double duty(uint64_t now_ns) const { + std::lock_guard g(mu_); + if (!started_ || now_ns <= first_start_ns_) return 0.0; + return (double)burst_ns_ / (double)(now_ns - first_start_ns_); + } + +private: + BurstAction stop_locked(uint64_t now_ns, BurstStopReason why) { + sampling_ = false; + last_stop_ = why; + const uint64_t dur = now_ns > burst_start_ns_ ? now_ns - burst_start_ns_ : 0; + burst_ns_ += dur; + last_burst_dur_ = dur; + last_burst_end_ns_ = now_ns; + have_open_yield_ = true; + // Provisional: the next burst is scheduled at the gap already in + // force, so a caller that never calls closed() keeps duty-cycling + // instead of either stalling or free-running. + next_start_ns_ = now_ns + gap_ns_; + return BurstAction::kStop; + } + + mutable std::mutex mu_; + const BurstConfig cfg_; + + bool sampling_ = false; + bool started_ = false; + bool refused_ = false; + bool have_open_yield_ = false; + uint64_t burst_start_ns_ = 0; + uint64_t last_burst_end_ns_ = 0; + uint64_t last_burst_dur_ = 0; + uint64_t first_start_ns_ = 0; + uint64_t next_start_ns_ = 0; + uint64_t pairs_at_last_close_ = 0; + uint64_t gap_ns_ = 0; + uint64_t bursts_ = 0; + uint64_t burst_ns_ = 0; + uint64_t pairs_ = 0; + uint64_t graph_refusals_ = 0; + BurstStopReason last_stop_ = BurstStopReason::kDutyCycle; +}; + +} // namespace perfagent + +#endif diff --git a/shim/core/burst_test.cc b/shim/core/burst_test.cc new file mode 100644 index 0000000..2f03d4e --- /dev/null +++ b/shim/core/burst_test.cc @@ -0,0 +1,303 @@ +// Tier A's duty cycle, against a fake clock. +// +// No GPU, no CUDA toolkit, no real time. The controller takes `now_ns` and a +// running (PC, stall) pair count as parameters precisely so the three +// properties that bound Tier A's perturbation --- it converges to the target +// rate, it never exceeds the duty ceiling whatever the workload does, and a +// zero observed rate does not collapse the gap --- can be PROVEN here instead +// of being inferred from a hardware run that this machine cannot do. +#include "burst.h" + +#include +#include +#include +#include + +using perfagent::BurstAction; +using perfagent::BurstConfig; +using perfagent::BurstController; +using perfagent::BurstStopReason; +using perfagent::burst_min_gap_ns; +using perfagent::burst_next_gap_ns; + +static const uint64_t kMs = 1000000ull; +static const uint64_t kSec = 1000000000ull; + +// Runs the controller against a fake clock, with a workload that produces +// `pairs_per_burst` (PC, stall) pairs for every full burst and nothing during +// a gap. Returns the achieved pair rate over the whole simulated interval. +struct SimResult { + double rate; // pairs per second, over the whole run + double duty; // burst_ns / wall_ns + uint64_t final_gap; // the gap the loop settled on + uint64_t bursts; + uint64_t wall_ns; +}; + +static SimResult simulate(BurstConfig cfg, uint64_t pairs_per_burst, unsigned cycles, + uint64_t tick_ns = 5 * kMs) { + BurstController c(cfg); + uint64_t now = 0, pairs = 0; + uint64_t open_at = 0; + unsigned done = 0; + // Bounded: every cycle is at most burst + max_gap, so this cannot spin. + const uint64_t deadline = (uint64_t)cycles * (cfg.burst_ns + cfg.max_gap_ns) + kSec; + while (done < cycles && now < deadline) { + const BurstAction a = c.poll(now, false); + if (a == BurstAction::kStart) { + open_at = now; + } else if (a == BurstAction::kStop) { + // The pairs land on the range-end FLUSH, which is after the stop: + // that ordering is CUPTI's and it is the reason closed() is a + // separate call. Proportional to how long the burst was actually + // open --- the tick granularity makes a "50 ms" burst 50..55 ms. + const uint64_t dur = now - open_at; + pairs += pairs_per_burst * dur / cfg.burst_ns; + c.closed(pairs); + done++; + } + now += tick_ns; + } + SimResult r{}; + r.wall_ns = now; + r.rate = now ? (double)pairs * 1e9 / (double)now : 0.0; + r.duty = now ? (double)c.burst_ns() / (double)now : 0.0; + r.final_gap = c.gap_ns(); + r.bursts = c.bursts(); + return r; +} + +int main() { + // ---- The duty ceiling, derived rather than asserted by eye. + { + BurstConfig cfg; // 50 ms burst, 10% ceiling + const uint64_t lo = burst_min_gap_ns(cfg); + assert(lo == 450 * kMs); + const double duty = (double)cfg.burst_ns / (double)(cfg.burst_ns + lo); + assert(duty <= cfg.max_duty + 1e-12); + } + + // ---- The loop NEVER produces a gap below the duty floor, whatever the + // observed rate is. This is the property the plan calls out, and it is + // swept rather than argued: the whole point of a hard bound is that it + // does not depend on the controller being reasonable. + { + BurstConfig cfg; + const uint64_t lo = burst_min_gap_ns(cfg); + const uint64_t elapsed = 500 * kMs; + const uint64_t pairs_cases[] = { + 0, 1, 7, 50, 100, 1000, 100000, 10000000, 1000000000ull, + std::numeric_limits::max() / 2, + std::numeric_limits::max(), + }; + const uint64_t prev_cases[] = {0, 1, lo, 2 * lo, cfg.max_gap_ns, cfg.max_gap_ns * 2}; + for (uint64_t p : pairs_cases) { + for (uint64_t prev : prev_cases) { + const uint64_t g = burst_next_gap_ns(cfg, prev, p, elapsed); + assert(g >= lo); + assert(g <= cfg.max_gap_ns); + const double duty = (double)cfg.burst_ns / (double)(cfg.burst_ns + g); + assert(duty <= cfg.max_duty + 1e-12); + } + } + } + + // ---- A zero observed rate does not drive the gap to zero. It walks the + // gap DOWN to the duty floor and stops there: an idle GPU samples at the + // ceiling and no faster, which is the opposite of the failure this test + // exists to exclude (a quiet workload sampled continuously). + { + BurstConfig cfg; + const uint64_t lo = burst_min_gap_ns(cfg); + uint64_t gap = cfg.max_gap_ns; + for (int i = 0; i < 100; i++) gap = burst_next_gap_ns(cfg, gap, 0, 500 * kMs); + assert(gap == lo); + assert(gap > 0); + } + + // ---- Degenerate configuration lands on a clamp, never on zero and never + // on undefined behaviour. + { + BurstConfig cfg; + cfg.target_rate = 0.0; // ratio -> +inf + assert(burst_next_gap_ns(cfg, 0, 10, 500 * kMs) == cfg.max_gap_ns); + cfg.target_rate = -1.0; // ratio -> negative + assert(burst_next_gap_ns(cfg, 0, 10, 500 * kMs) == burst_min_gap_ns(cfg)); + BurstConfig nan_cfg; + nan_cfg.target_rate = std::numeric_limits::quiet_NaN(); + const uint64_t g = burst_next_gap_ns(nan_cfg, 0, 10, 500 * kMs); + assert(g >= burst_min_gap_ns(nan_cfg)); + BurstConfig zero_duty; + zero_duty.max_duty = 0.0; + assert(burst_min_gap_ns(zero_duty) > 0); + BurstConfig over_duty; + over_duty.max_duty = 5.0; // "500%" is not a ceiling + assert(burst_min_gap_ns(over_duty) == 0); + // elapsed 0 carries no information: hold, clamped. + assert(burst_next_gap_ns(cfg, 3 * kSec, 10, 0) >= burst_min_gap_ns(cfg)); + } + + // ---- Convergence. A workload producing 5,000 pairs per 50 ms burst is + // 100,000 pairs/s while sampling; at a 100/s target the loop must space + // bursts out to ~50 s of cycle --- which the 10 s max_gap clamps --- so + // use a gentler workload where the fixed point is inside the clamps. + // + // 100 pairs per burst at a 100/s target wants a 1 s cycle: 50 ms burst + // plus a 950 ms gap. The floor is 450 ms and the ceiling 10 s, so the + // fixed point is interior and the loop must actually find it. + { + BurstConfig cfg; + const SimResult r = simulate(cfg, 100, 60); + // Within 15%: the tick granularity (5 ms on a 50 ms burst) and the + // first few un-converged cycles are both in the average. + assert(r.rate > 85.0 && r.rate < 115.0); + // And the gap landed near the analytic fixed point of 950 ms. + assert(r.final_gap > 850 * kMs && r.final_gap < 1100 * kMs); + assert(r.bursts == 60); + printf("burst_test: converged rate=%.1f/s gap=%llums bursts=%llu\n", + r.rate, (unsigned long long)(r.final_gap / kMs), + (unsigned long long)r.bursts); + } + + // ---- Monotone convergence from both sides: starting far above and far + // below the fixed point both land in the same place. + { + BurstConfig cfg; + uint64_t from_below = burst_min_gap_ns(cfg); + uint64_t from_above = cfg.max_gap_ns; + for (int i = 0; i < 40; i++) { + from_below = burst_next_gap_ns(cfg, from_below, 100, 1 * kSec); + from_above = burst_next_gap_ns(cfg, from_above, 100, 1 * kSec); + } + const uint64_t d = from_below > from_above ? from_below - from_above + : from_above - from_below; + assert(d < kMs); + assert(from_below > 900 * kMs && from_below < 1000 * kMs); + } + + // ---- The duty ceiling holds over a whole simulated run, for a workload + // that produces nothing at all (the case that pushes the gap to its floor + // and therefore the duty to its ceiling). + { + BurstConfig cfg; + const SimResult r = simulate(cfg, 0, 40); + // The tick granularity makes a burst 50..55 ms rather than exactly + // 50, so the achieved duty can exceed the nominal ceiling by that + // ratio and no more. Asserted with the granularity in it rather than + // fudged: the ceiling bounds what the LOOP asks for, and the timer's + // resolution is a separate, stated cost. + assert(r.duty <= cfg.max_duty * 1.15); + assert(r.final_gap == burst_min_gap_ns(cfg)); + printf("burst_test: idle duty=%.3f (ceiling %.2f) gap=%llums\n", + r.duty, cfg.max_duty, (unsigned long long)(r.final_gap / kMs)); + } + + // ---- The state machine: start and stop strictly alternate, the burst is + // at least burst_ns long, and burst_ns() accumulates what was actually + // open rather than what was asked for. + { + BurstConfig cfg; + BurstController c(cfg); + uint64_t now = 0, pairs = 0, opened = 0, closed = 0, open_at = 0; + bool open = false; + for (int i = 0; i < 2000; i++) { + const BurstAction a = c.poll(now, false); + if (a == BurstAction::kStart) { + assert(!open); + open = true; + open_at = now; + opened++; + } else if (a == BurstAction::kStop) { + assert(open); + assert(now - open_at >= cfg.burst_ns); + open = false; + closed++; + pairs += 100; + c.closed(pairs); + } + now += 5 * kMs; + } + assert(opened == closed + (open ? 1 : 0)); + assert(c.bursts() == opened); + assert(c.burst_ns() >= closed * cfg.burst_ns); + assert(c.pairs_observed() == closed * 100); + } + + // ---- The CUDA-graph refusal. Once a graph execution has been observed + // the controller closes an open burst and NEVER starts another. It does + // not become Tier B --- it stops, loudly and counted. + { + BurstConfig cfg; + BurstController c(cfg); + uint64_t now = 0; + assert(c.poll(now, false) == BurstAction::kStart); + now += 10 * kMs; + // The graph arrives mid-burst: the burst is cut short, not abandoned. + assert(c.poll(now, true) == BurstAction::kStop); + c.closed(5); + assert(c.last_stop_reason() == BurstStopReason::kGraph); + assert(c.refused()); + assert(c.graph_refusals() == 1); + // The short burst is still accounted: 10 ms of the workload really + // did run serialized and the window really was open for it. + assert(c.burst_ns() == 10 * kMs); + // And no burst ever starts again, however long the clock runs. + for (int i = 0; i < 1000; i++) { + now += 100 * kMs; + assert(c.poll(now, true) == BurstAction::kNone); + } + assert(c.bursts() == 1); + assert(c.graph_refusals() == 1); // counted once, not per poll + } + + // ---- A graph observed BEFORE the first burst means Tier A never starts + // at all --- the "refuses to start" case, distinct from "stops". + { + BurstConfig cfg; + BurstController c(cfg); + uint64_t now = 0; + for (int i = 0; i < 100; i++) { + assert(c.poll(now, true) == BurstAction::kNone); + now += 100 * kMs; + } + assert(c.bursts() == 0); + assert(c.burst_ns() == 0); + assert(c.refused()); + assert(c.graph_refusals() == 1); + } + + // ---- Shutdown closes an open burst with a real timestamp. This is what + // makes end_ns == 0 on the wire mean a HARD exit and nothing else: on the + // ordinary path the atexit handler comes through here and the window is + // closed. + { + BurstConfig cfg; + BurstController c(cfg); + assert(c.poll(0, false) == BurstAction::kStart); + assert(c.sampling()); + assert(c.burst_start_ns() == 0); + assert(c.shutdown(20 * kMs) == BurstAction::kStop); + assert(!c.sampling()); + assert(c.last_stop_reason() == BurstStopReason::kShutdown); + assert(c.burst_ns() == 20 * kMs); + c.closed(42); + assert(c.pairs_observed() == 42); + // Idempotent, and never reopens. + assert(c.shutdown(30 * kMs) == BurstAction::kNone); + assert(c.poll(10 * kSec, false) == BurstAction::kNone); + assert(c.bursts() == 1); + } + + // ---- Shutdown with no burst open is a no-op, so a caller that always + // calls it does not emit a phantom window. + { + BurstConfig cfg; + BurstController c(cfg); + assert(c.shutdown(kSec) == BurstAction::kNone); + assert(c.bursts() == 0); + assert(c.burst_ns() == 0); + } + + printf("burst_test: OK\n"); + return 0; +} diff --git a/shim/core/pcdrain.h b/shim/core/pcdrain.h index 606d240..f6e2478 100644 --- a/shim/core/pcdrain.h +++ b/shim/core/pcdrain.h @@ -39,6 +39,7 @@ namespace perfagent { enum class PCDrainReason { kPeriodic, // the drain timer, and the period had elapsed kModuleUnload, // CUPTI_CBID_RESOURCE_MODULE_UNLOAD_STARTING + kRangeEnd, // Tier A: immediately after cuptiPCSamplingStop() kTeardown, // context destroy, finalize or exit }; @@ -71,10 +72,10 @@ class PCDrainSchedule { void force(uint64_t now_ns, PCDrainReason reason) { std::lock_guard g(mu_); mark_locked(now_ns); - if (reason == PCDrainReason::kModuleUnload) { - unload_++; - } else { - teardown_++; + switch (reason) { + case PCDrainReason::kModuleUnload: unload_++; break; + case PCDrainReason::kRangeEnd: range_end_++; break; + default: teardown_++; break; } } @@ -85,6 +86,11 @@ class PCDrainSchedule { // otherwise look exactly like a workload that stopped stalling. uint64_t periodic() const { std::lock_guard g(mu_); return periodic_; } uint64_t unload() const { std::lock_guard g(mu_); return unload_; } + // Tier A only. cupti_pcsampling.h requires a PC-data flush "after every + // range end i.e. cuptiPCSamplingStop()" when ENABLE_START_STOP_CONTROL is + // on, so this is not a rate-limited drain either: it must equal the number + // of bursts that closed. Zero in Tier B and with sampling off. + uint64_t range_end() const { std::lock_guard g(mu_); return range_end_; } uint64_t teardown() const { std::lock_guard g(mu_); return teardown_; } // Ticks that found the period had not elapsed, because a forced drain had // just taken the data. Non-zero is healthy on a module-churning process @@ -93,7 +99,7 @@ class PCDrainSchedule { uint64_t coalesced() const { std::lock_guard g(mu_); return coalesced_; } uint64_t total() const { std::lock_guard g(mu_); - return periodic_ + unload_ + teardown_; + return periodic_ + unload_ + range_end_ + teardown_; } private: @@ -108,6 +114,7 @@ class PCDrainSchedule { bool drained_ = false; uint64_t periodic_ = 0; uint64_t unload_ = 0; + uint64_t range_end_ = 0; uint64_t teardown_ = 0; uint64_t coalesced_ = 0; }; diff --git a/shim/core/pcdrain_test.cc b/shim/core/pcdrain_test.cc index 58bf33c..29caec9 100644 --- a/shim/core/pcdrain_test.cc +++ b/shim/core/pcdrain_test.cc @@ -103,6 +103,28 @@ int main() { assert(s.periodic() + s.coalesced() == (uint64_t)kIter); } + // Tier A's range-end drain: mandatory like an unload, counted apart from + // it, and it resets the phase so the next tick does not immediately + // repeat the pull. cupti_pcsampling.h requires a flush after every + // cuptiPCSamplingStop(), so a range-end drain that could be skipped is + // the same class of silent PC-identity corruption the unload drain + // exists to prevent. + { + PCDrainSchedule s(100 * kMs); + assert(s.due(0)); + for (int i = 1; i <= 5; i++) s.force((uint64_t)i * kMs, PCDrainReason::kRangeEnd); + assert(s.range_end() == 5); + assert(s.unload() == 0); + assert(s.teardown() == 0); + assert(s.periodic() == 1); + assert(s.total() == 6); + // The phase moved with the last forced drain, so a tick 50ms later + // coalesces rather than pulling again. + assert(!s.due(55 * kMs)); + assert(s.coalesced() == 1); + assert(s.due(106 * kMs)); + } + printf("pcdrain_test OK\n"); return 0; } diff --git a/shim/nvidia/cupti_adapter.cc b/shim/nvidia/cupti_adapter.cc index 4f9fe32..0f0a770 100644 --- a/shim/nvidia/cupti_adapter.cc +++ b/shim/nvidia/cupti_adapter.cc @@ -10,6 +10,7 @@ // If this file emits what the stub emits, the consumer, the BPF program, the // stack capture and the projection all work unchanged. #include "batch.h" +#include "burst.h" #include "clock.h" #include "cubin.h" #include "cubinqueue.h" @@ -53,12 +54,17 @@ PERFAGENT_USDT_EMITTER(gpu_kernel_name_v1, 272); // the record must be emitted at a very particular instant (see // on_cubin_captured), which a batch's flush would move. PERFAGENT_USDT_EMITTER(gpu_module_load_v1, 40); -// PC sampling (Tier B). All four fire only when PERFAGENT_GPU_PC_SAMPLING is -// set AND a consumer is attached; the semaphore gate is inside the emitter, -// the tier gate is g_pc_tier_b. +// PC sampling. All of these fire only when PERFAGENT_GPU_PC_SAMPLING is set +// AND a consumer is attached; the semaphore gate is inside the emitter, the +// tier gate is g_pc_enabled. PERFAGENT_USDT_EMITTER(gpu_pc_sample_batch_v1, 40); PERFAGENT_USDT_EMITTER(gpu_stall_reason_map_v1, 136); PERFAGENT_USDT_EMITTER(gpu_config_v1, 24); +// Tier A ONLY, and the whole of that tier's honesty obligation: one record +// per PC-sampling burst, so the consumer can say which executions ran while +// kernels were serialized. Never fired in Tier B --- nothing is serialized +// there, so there is no window to disclose. +PERFAGENT_USDT_EMITTER(gpu_sampling_window_v1, 24); // Producer-side loss of every class, including the two PC-sampling omissions // CUPTI documents and cannot recover. PERFAGENT_USDT_EMITTER(gpu_dropped_v1, 16); @@ -347,18 +353,47 @@ void on_module_loaded(const CUpti_ResourceData *rd) { bool check(CUptiResult r, const char *what); unsigned env_uint(const char *name, unsigned dflt); -// ------------------------------------------------- PC sampling (Tier B) +// --------------------------------------------------------- PC sampling // -// CUPTI_PC_SAMPLING_COLLECTION_MODE_CONTINUOUS. Kernels are NOT serialized in -// this mode, which is the only reason it is a candidate for always-on -// profiling; the cost is that every PC record's correlationId is zero, so a -// PC sample joins to a kernel through its module and never to the launch that -// issued it. Tier A (KERNEL_SERIALIZED, duty-cycled) is a separate task and is -// deliberately not implemented here. +// Two collection tiers, mutually exclusive, both OFF BY DEFAULT. Nothing +// below runs, allocates or calls CUPTI unless PERFAGENT_GPU_PC_SAMPLING is +// set, so merging either of them cannot degrade a profiler that is shipping +// today. // -// OFF BY DEFAULT. PERFAGENT_GPU_PC_SAMPLING=1 turns it on. Nothing below runs, -// allocates or calls CUPTI unless that is set, so merging this cannot degrade -// a profiler that is shipping today. +// Tier B --- PERFAGENT_GPU_PC_SAMPLING=1, CUPTI_PC_SAMPLING_COLLECTION_MODE_ +// CONTINUOUS. Kernels are NOT serialized in this mode, which is the only +// reason it is a candidate for always-on profiling; the cost is that every PC +// record's correlationId is zero, so a PC sample joins to a kernel through its +// module and never to the launch that issued it. +// +// Tier A --- PERFAGENT_GPU_PC_SAMPLING=2, CUPTI_PC_SAMPLING_COLLECTION_MODE_ +// KERNEL_SERIALIZED with ENABLE_START_STOP_CONTROL, duty-cycled by +// core/burst.h. CUPTI populates correlationId on every PC record here, so a +// sample joins to a launch --- and therefore to a CPU stack --- exactly. The +// price is that every kernel that runs while a burst is open runs SERIALIZED, +// which perturbs the very durations the profile reports. Three things follow +// and all three are implemented below rather than documented and skipped: +// +// 1. The perturbation is BOUNDED by the duty cycle (core/burst.h), not left +// to run continuously. +// 2. The perturbation is DISCLOSED: every burst emits gpu_sampling_window_v1 +// the moment it opens and again when it closes, and the consumer marks +// every execution overlapping a window gpu_serialized="true". The window, +// not the set of sampled kernels, is the honest unit --- every kernel +// that ran inside a burst ran serialized whether it was sampled or not. +// 3. Tier A REFUSES to run where CUDA graphs have been observed. A graph +// launch fires one runtime callback for N kernels, so N executions share +// one correlation and Tier A's exactness claim becomes false while still +// looking exact. The refusal is loud and counted (g_tier_a_graph_refused), +// never a silent downgrade to Tier B. +// +// Start/Stop, not Enable/Disable. cuptiPCSamplingEnable/Disable tears the +// configuration down and rebuilds it, which is not what the start/stop control +// exists for; Start/Stop is the documented way to duty-cycle a configured +// context. CUPTI additionally requires a PC-data flush "after every range end +// i.e. cuptiPCSamplingStop()" in this configuration, so the stop path drains +// immediately and marks the shared PCDrainSchedule so the periodic tick does +// not redundantly repeat it. // // What CUPTI will not tell us, which the profile must state rather than imply // ------------------------------------------------------------------------- @@ -407,7 +442,11 @@ constexpr size_t kPCDefaultCollectNumPcs = 2048; // else's process; hitting the bound is counted rather than retried forever. constexpr unsigned kPCMaxDrainRounds = 64; -bool g_pc_tier_b = false; // PERFAGENT_GPU_PC_SAMPLING +bool g_pc_enabled = false; // PERFAGENT_GPU_PC_SAMPLING != 0 +// Tier A. Set from the SAME variable as g_pc_enabled (value 2), so the two +// tiers are mutually exclusive by construction rather than by a check that +// can be forgotten --- which is what Task 11's tier selection wants anyway. +bool g_pc_tier_a = false; uint32_t g_pc_period = 0; // the exponent actually in force size_t g_pc_collect_num_pcs = kPCDefaultCollectNumPcs; size_t g_pc_scratch_bytes = 0; // 0 = CUPTI's default @@ -454,6 +493,41 @@ std::atomic g_pc_seq{0}; std::atomic g_stall_seq{0}; std::atomic g_config_seq{0}; std::atomic g_dropped_seq{0}; +std::atomic g_window_seq{0}; + +// ------------------------------------------------------ Tier A duty cycle +// +// The burst controller and its own timer. It is a SECOND timer, not the drain +// tick, and that is a deliberate deviation from the plan's "the existing drain +// timer is its natural home": the drain tick is 100 ms and a 50 ms burst +// cannot be expressed on it. Quantizing the burst to the drain period would +// silently double the burst length and therefore the duty fraction --- the one +// number this tier exists to bound --- so the burst rides a tick of its own +// whose period is a fraction of the burst length. The flush that CUPTI +// requires after every range end runs on the stop, immediately, and marks the +// SHARED PCDrainSchedule so the 100 ms tick coalesces instead of repeating it. +perfagent::BurstController *g_burst = nullptr; +perfagent::Drainer *g_burst_timer = nullptr; +unsigned g_burst_tick_ms = 10; + +// Bursts and their total open time. The pair is what bounds the perturbation: +// burst_ns / wall_ns is the fraction of the run that ran serialized, and it is +// REPORTED rather than assumed to equal the configured ceiling. +std::atomic g_sampling_bursts{0}; +std::atomic g_sampling_burst_ns{0}; +// Windows actually put on the wire. Two per burst on the ordinary path --- one +// open at the start, one closed at the end --- so the consumer sees an open +// window if the process dies mid-burst instead of seeing nothing at all. +std::atomic g_windows_emitted{0}; +// cuptiPCSamplingStart / Stop failures. MUST be 0 on a healthy run: a failed +// start means a window was announced for a burst that never sampled, and a +// failed stop means kernels stayed serialized past the window's end. +std::atomic g_burst_start_failed{0}; +std::atomic g_burst_stop_failed{0}; +// The CUDA-graph refusal. Non-zero means Tier A stopped, permanently, because +// exact launch attribution had become false. It is never zero-and-silent: the +// same condition also rides gpu_dropped_v1 under GPU_DROP_CLASS_GRAPH_EXEC. +std::atomic g_tier_a_graph_refused{0}; // Every one of these is assertable at a known value on a healthy run, which is // the whole point: a context that failed to enable is otherwise a silent hole @@ -633,6 +707,23 @@ void pc_disable_ctx(PCContext *c, const char *why) { // uses. If any step after the enable fails, the enable is undone rather than // left half-applied. void pc_enable_ctx(CUcontext ctx) { + // The CUDA-graph refusal, at its earliest reachable point. If a graph + // execution has already been seen, Tier A's exact-correlation claim is + // already false for this process and the tier must not start at all. It + // does NOT fall back to CONTINUOUS: a silent downgrade would leave the + // operator reading a Tier B profile while believing they asked for Tier A. + if (g_pc_tier_a && g_exec_from_graph.load(std::memory_order_relaxed)) { + if (g_tier_a_graph_refused.fetch_add(1, std::memory_order_relaxed) == 0) { + logf("perfagent-cupti: REFUSING Tier A: %llu CUDA-graph execution(s) already " + "observed. A graph launch fires one callback for N kernels, so N " + "executions share one correlation and Tier A's exact launch " + "attribution would be confidently wrong. PC sampling is NOT enabled " + "for this context; this is not a downgrade to Tier B.\n", + (unsigned long long)g_exec_from_graph.load(std::memory_order_relaxed)); + } + g_ctx_enable_failed.fetch_add(1, std::memory_order_relaxed); + return; + } CUpti_PCSamplingEnableParams en{}; en.size = CUpti_PCSamplingEnableParamsSize; en.ctx = ctx; @@ -653,11 +744,22 @@ void pc_enable_ctx(CUcontext ctx) { } pc_setup_buffer(c); - CUpti_PCSamplingConfigurationInfo info[7]{}; + CUpti_PCSamplingConfigurationInfo info[8]{}; size_t n = 0; info[n].attributeType = CUPTI_PC_SAMPLING_CONFIGURATION_ATTR_TYPE_COLLECTION_MODE; info[n++].attributeData.collectionModeData.collectionMode = - CUPTI_PC_SAMPLING_COLLECTION_MODE_CONTINUOUS; + g_pc_tier_a ? CUPTI_PC_SAMPLING_COLLECTION_MODE_KERNEL_SERIALIZED + : CUPTI_PC_SAMPLING_COLLECTION_MODE_CONTINUOUS; + if (g_pc_tier_a) { + // The duty cycle's mechanism. Without it cuptiPCSamplingStart/Stop + // return an error and the only way to bound the perturbation would be + // Enable/Disable per burst --- which tears the configuration down and + // rebuilds it every 500 ms, re-queries nothing, and is not what the + // start/stop control exists for. + info[n].attributeType = + CUPTI_PC_SAMPLING_CONFIGURATION_ATTR_TYPE_ENABLE_START_STOP_CONTROL; + info[n++].attributeData.enableStartStopControlData.enableStartStopControl = 1; + } // All stall reasons. The label cardinality is device-fixed (38 on GA102), // so collecting all of them costs bytes per PC and nothing that grows with // the length of the run. @@ -681,10 +783,10 @@ void pc_enable_ctx(CUcontext ctx) { info[n].attributeType = CUPTI_PC_SAMPLING_CONFIGURATION_ATTR_TYPE_HARDWARE_BUFFER_SIZE; info[n++].attributeData.hardwareBufferSizeData.hardwareBufferSize = g_pc_hw_buffer_bytes; } - // ENABLE_START_STOP_CONTROL is deliberately left off. It is what Tier A's - // duty cycling needs; in CONTINUOUS mode turning it on would change when a - // flush is required (cupti_pcsampling.h: "after every range end") without - // buying this tier anything. + // In Tier B, ENABLE_START_STOP_CONTROL is deliberately left off: turning + // it on in CONTINUOUS mode would change when a flush is required + // (cupti_pcsampling.h: "after every range end") without buying that tier + // anything. It is set above for Tier A, where it is the whole mechanism. CUpti_PCSamplingConfigurationInfoParams cp{}; cp.size = CUpti_PCSamplingConfigurationInfoParamsSize; @@ -841,6 +943,149 @@ void pc_drain_all(perfagent::PCDrainReason reason) { if (reason != perfagent::PCDrainReason::kPeriodic && g_pcb) g_pcb->flush(); } +// ---------------------------------------------- Tier A: bursts and windows +// +// gpu_sampling_window_v1, twice per burst. +// +// The OPEN record goes out the instant cuptiPCSamplingStart succeeds, with +// end_ns = 0. That is what makes end_ns == 0 mean something on the wire: if +// this process is killed mid-burst, the consumer still holds a record saying a +// burst was open from start_ns and never closed, and every execution from +// start_ns onward is gpu_serialized="unknown". Emitting only on the stop would +// lose the whole burst on a hard exit, and the executions inside it would then +// read "false" --- "not perturbed" when the truth is "cannot tell", which is +// the one answer that must never be reachable by accident. +// +// The CLOSED record goes out on the stop with the real end_ns and the SAME +// start_ns, and supersedes the open one in the consumer's store. A closed +// record never loses to an open one there, so the two orderings a lossy +// transport can produce both end up correct. +void emit_window(uint64_t start_ns, uint64_t end_ns) { + if (!gpu_sampling_window_v1_enabled()) return; + gpu_sampling_window_v1 w{}; + w.start_ns = start_ns; + w.end_ns = end_ns; + w.mode = GPU_SAMPLING_MODE_KERNEL_SERIALIZED; + // UNBATCHED, like gpu_dropped_v1 and for the same reason: two records per + // burst at a few bursts per second is not volume, and a window still + // sitting in a partly filled batch when the process dies is a window that + // never existed --- which would take the hard-exit disclosure with it. + gpu_sampling_window_v1_emit(&w, 1, g_window_seq.fetch_add(1, std::memory_order_relaxed)); + g_windows_emitted.fetch_add(1, std::memory_order_relaxed); +} + +void pc_start_ctxs_locked() { + for (PCContext *c : g_pc_ctxs) { + if (!c->enabled) continue; + CUpti_PCSamplingStartParams sp{}; + sp.size = CUpti_PCSamplingStartParamsSize; + sp.ctx = c->ctx; + if (!check(cuptiPCSamplingStart(&sp), "cuptiPCSamplingStart")) + g_burst_start_failed.fetch_add(1, std::memory_order_relaxed); + } +} + +void pc_stop_ctxs_locked() { + for (PCContext *c : g_pc_ctxs) { + if (!c->enabled) continue; + CUpti_PCSamplingStopParams sp{}; + sp.size = CUpti_PCSamplingStopParamsSize; + sp.ctx = c->ctx; + if (!check(cuptiPCSamplingStop(&sp), "cuptiPCSamplingStop")) + g_burst_stop_failed.fetch_add(1, std::memory_order_relaxed); + } +} + +// Closes an open burst: stop every context, drain (CUPTI requires the flush +// after every range end), close the window, and hand the yield to the loop. +// Caller must NOT hold g_pc_mu. +void burst_close(uint64_t now_ns, uint64_t start_ns) { + { + std::lock_guard g(g_pc_mu); + pc_stop_ctxs_locked(); + } + // The range-end flush. cupti_pcsampling.h: "If configuration option + // ENABLE_START_STOP_CONTROL is enabled, then after every range end i.e. + // cuptiPCSamplingStop()". Missing it does not lose data --- it makes two + // instructions share a PC identity, silently. force() also moves the + // shared schedule's phase so the 100 ms tick coalesces rather than + // repeating the pull microseconds later. + if (g_pc_schedule) g_pc_schedule->force(now_ns, perfagent::PCDrainReason::kRangeEnd); + pc_drain_all(perfagent::PCDrainReason::kRangeEnd); + emit_window(start_ns, now_ns); + g_sampling_burst_ns.fetch_add(now_ns > start_ns ? now_ns - start_ns : 0, + std::memory_order_relaxed); + // AFTER the drain, which is the whole reason BurstController::closed is a + // separate call: the pairs this burst produced only reach g_pc_records on + // the flush that follows the stop, so a loop that read the count at stop + // time would measure every burst as having yielded nothing and would sit + // at the duty floor for the entire run. + if (g_burst) g_burst->closed(g_pc_records.load(std::memory_order_relaxed)); +} + +// The burst timer's tick. Runs on its own thread; see the note on g_burst. +void on_burst_tick() { + if (!g_pc_tier_a || !g_burst) return; + // No enabled context means nothing can be serialized, so there is no + // burst to open and no window to announce. This timer starts at the end + // of InitializeInjection, which is before the first CONTEXT_CREATED + // callback can have fired, so without this the first burst would announce + // a window covering executions that were never perturbed. Over-stating + // perturbation is the safe direction and would not be a defect --- but it + // would be a lie about an interval nothing was sampling, and there is no + // reason to tell it. + if (!g_ctx_enabled.load(std::memory_order_relaxed) && !g_burst->sampling()) return; + const uint64_t now = mono_ns(); + // The graph refusal, re-checked on every tick and not only at enable + // time: a process can run for minutes before its first graph launch, and + // the moment one arrives Tier A's exactness claim stops being true. The + // controller closes any open burst and never starts another. + const bool graphs = g_exec_from_graph.load(std::memory_order_relaxed) != 0; + const bool was_refused = g_burst->refused(); + const uint64_t start_ns = g_burst->burst_start_ns(); + switch (g_burst->poll(now, graphs)) { + case perfagent::BurstAction::kStart: { + { + std::lock_guard g(g_pc_mu); + pc_start_ctxs_locked(); + } + g_sampling_bursts.fetch_add(1, std::memory_order_relaxed); + // The window is announced even if some or all contexts refused to + // start (g_burst_start_failed counts that, and must be 0 on a + // healthy run). Over-stating the perturbation for one burst marks + // executions "true" that were not perturbed, which is the SAFE + // direction: the answer that must never be reachable by accident + // is "false", and no path here can produce it. + emit_window(now, 0); // open; closed by the kStop below + break; + } + case perfagent::BurstAction::kStop: + burst_close(now, start_ns); + break; + case perfagent::BurstAction::kNone: + break; + } + if (!was_refused && g_burst->refused() && graphs) { + g_tier_a_graph_refused.fetch_add(1, std::memory_order_relaxed); + logf("perfagent-cupti: REFUSING Tier A: %llu CUDA-graph execution(s) observed " + "mid-run. N executions share one correlation, so exact launch " + "attribution is false while still looking exact. Bursts have STOPPED " + "permanently; this is not a downgrade to Tier B. Executions already " + "inside a window stay marked serialized.\n", + (unsigned long long)g_exec_from_graph.load(std::memory_order_relaxed)); + } +} + +// Teardown for the duty cycle. Closes an open burst with a real end timestamp, +// which is exactly what makes end_ns == 0 on the wire mean a HARD exit and +// nothing else. +void burst_shutdown() { + if (!g_pc_tier_a || !g_burst) return; + const uint64_t now = mono_ns(); + const uint64_t start_ns = g_burst->burst_start_ns(); + if (g_burst->shutdown(now) == perfagent::BurstAction::kStop) burst_close(now, start_ns); +} + // gpu_config_v1: the sampling configuration in force, emitted once, replayed // on late attach. sampling_factor is "one PC sample per N SM cycles" --- it is // NOT a scale factor and no count is ever multiplied by it. @@ -879,7 +1124,21 @@ void on_finalize(const char *why) { bool expected = false; if (!done.compare_exchange_strong(expected, true)) return; g_finalize_seen.fetch_add(1, std::memory_order_relaxed); - if (!g_pc_tier_b) return; + if (!g_pc_enabled) return; + + // Tier A: stop the duty cycle before anything else, so no burst can open + // against contexts this function is about to disable. Only the + // controller's own mutex is taken here --- deliberately NOT g_pc_mu, which + // this function may fail to acquire below and which the fatal-error + // callback can already be holding. + // + // On the fatal-error path an open window is left OPEN on the wire. That is + // correct and it is the point: a CUPTI fatal error IS the hard case, and + // end_ns == 0 is how the consumer learns that the tail of the run cannot + // be said to have run unperturbed. The ordinary exit path closes it first + // (at_exit_handler -> burst_shutdown), which is what makes a zero here + // mean the hard case specifically. + if (g_burst) g_burst->shutdown(mono_ns()); // try_lock, not lock, and this is the one place that is right. // @@ -1011,7 +1270,7 @@ void on_launch(const CUpti_CallbackData *cb) { // The RESOURCE callbacks PC sampling needs. Everything here is a no-op unless // Tier B is on. void on_resource(CUpti_CallbackId cbid, const CUpti_ResourceData *rd) { - if (!g_pc_tier_b || !rd) return; + if (!g_pc_enabled || !rd) return; switch (cbid) { case CUPTI_CBID_RESOURCE_CONTEXT_CREATED: { g_ctx_seen.fetch_add(1, std::memory_order_relaxed); @@ -1254,7 +1513,7 @@ void report(const char *why) { logf("perfagent-cupti: graph_execs=%llu multi_device=%llu devices=%zu\n", (unsigned long long)g_exec_from_graph.load(), (unsigned long long)g_multi_device.load(), g_devices_seen.size()); - if (!g_pc_tier_b) { + if (!g_pc_enabled) { logf("perfagent-cupti: pc_sampling=off (set PERFAGENT_GPU_PC_SAMPLING=1)\n"); return; } @@ -1263,13 +1522,35 @@ void report(const char *why) { // getdata_failed, drain_rounds_capped, pc_dropped_hw, pc_buffer_full and // multi_device are all zero. pc_non_user is NOT expected to be zero --- it // is the size of a structural omission, not a fault. + if (g_pc_tier_a) { + // The perturbation, reported rather than assumed. bursts x burst_ns is + // how much of this run ran with kernels serialized; duty is that as a + // fraction of the interval since the first burst opened. windows must + // be 2 x bursts on a clean run (one open, one closed per burst) and + // 2 x bursts - 1 when the process died mid-burst. + const uint64_t now = mono_ns(); + logf("perfagent-cupti: tier A bursts=%llu burst_ns=%llu duty=%.4f gap_ns=%llu " + "windows=%llu range_end_drains=%llu start_failed=%llu stop_failed=%llu " + "graph_refused=%llu sampling_now=%d\n", + (unsigned long long)g_sampling_bursts.load(), + (unsigned long long)g_sampling_burst_ns.load(), + g_burst ? g_burst->duty(now) : 0.0, + (unsigned long long)(g_burst ? g_burst->gap_ns() : 0), + (unsigned long long)g_windows_emitted.load(), + (unsigned long long)(g_pc_schedule ? g_pc_schedule->range_end() : 0), + (unsigned long long)g_burst_start_failed.load(), + (unsigned long long)g_burst_stop_failed.load(), + (unsigned long long)g_tier_a_graph_refused.load(), + g_burst && g_burst->sampling() ? 1 : 0); + } logf("perfagent-cupti: pc %s period=%u(=%u cycles) stall_reasons=%zu " "ctx_seen=%llu ctx_enabled=%llu ctx_enable_failed=%llu " "ctx_destroyed=%llu ctx_disable_failed=%llu " "pc_records=%llu pcs=%llu pc_batch_dropped=%llu pc_unattached=%llu " "zero_stall_pairs=%llu getdata=%llu getdata_failed=%llu " "drain_rounds_capped=%llu drains_periodic=%llu drains_unload=%llu " - "drains_teardown=%llu drains_coalesced=%llu module_unload_drains=%llu " + "drains_range_end=%llu drains_teardown=%llu drains_coalesced=%llu " + "module_unload_drains=%llu " "dropped_hw=%llu buffer_full=%llu non_user_samples=%llu " "total_samples=%llu emitted_counts=%llu " "graph_execs=%llu multi_device=%llu finalize_seen=%llu " @@ -1293,6 +1574,7 @@ void report(const char *why) { (unsigned long long)g_pc_drain_rounds_capped.load(), (unsigned long long)(g_pc_schedule ? g_pc_schedule->periodic() : 0), (unsigned long long)(g_pc_schedule ? g_pc_schedule->unload() : 0), + (unsigned long long)(g_pc_schedule ? g_pc_schedule->range_end() : 0), (unsigned long long)(g_pc_schedule ? g_pc_schedule->teardown() : 0), (unsigned long long)(g_pc_schedule ? g_pc_schedule->coalesced() : 0), (unsigned long long)g_module_unload_drains.load(), @@ -1357,7 +1639,7 @@ void on_tick() { // // It MUST stay above the Tier B gate below. Cubin capture is not part of // PC sampling and is on whenever a consumer is attached, so putting this - // after `if (!g_pc_tier_b) return;` would silence the offer half of every + // after `if (!g_pc_enabled) return;` would silence the offer half of every // module capture in the DEFAULT configuration -- with modules_captured // still counting up and cubins_sent stuck at zero. if (g_cubins) g_cubins->drain(perfagent::cubin_offer_to_consumer, g_cubin_timeout_ms); @@ -1375,7 +1657,7 @@ void on_tick() { emit_dropped(graphs - reported, GPU_DROP_CLASS_GRAPH_EXEC); } - if (!g_pc_tier_b) return; + if (!g_pc_enabled) return; // The config record is emitted from the tick rather than from the enable // path, because sm_count and clock_hz come from a DEVICE activity record @@ -1404,7 +1686,14 @@ void on_tick() { // a partly filled CUPTI buffer at exit are lost, and so is whatever is in the // batches. void at_exit_handler() { - // PC sampling first, and specifically before cuptiActivityFlushAll: the + // Tier A's duty cycle first of all. The timer thread is stopped before the + // controller is asked to close, so no tick can open a burst against + // contexts the finalize below is about to disable -- and the open window + // is closed with the exit timestamp, which is precisely what makes + // end_ns == 0 on the wire mean a HARD exit and nothing else. + if (g_burst_timer) g_burst_timer->stop(); + burst_shutdown(); + // PC sampling next, and specifically before cuptiActivityFlushAll: the // finalize handler drains and then disables each context, and // cuptiPCSamplingDisable is what joins CUPTI's PC worker threads. Doing it // after the activity flush would leave those threads running across the @@ -1550,8 +1839,21 @@ extern "C" __attribute__((visibility("default"))) int InitializeInjection(void) // here; tier selection is another. This flag is the whole of the switch // for now, and its default is what keeps a merge from changing what a // shipping profiler does. - g_pc_tier_b = env_uint("PERFAGENT_GPU_PC_SAMPLING", 0) != 0; - if (g_pc_tier_b) { + // 0/unset = off, 1 = Tier B (CONTINUOUS), 2 = Tier A (KERNEL_SERIALIZED, + // duty-cycled). One variable, so the two tiers cannot both be selected -- + // they configure the same per-context CUPTI attribute and "both" would + // produce a profile whose attribution quality varied by an axis the + // operator cannot see. Task 11 replaces this with a named setting and a + // CLI flag; the exclusivity is already structural here. + const unsigned pc_mode = env_uint("PERFAGENT_GPU_PC_SAMPLING", 0); + g_pc_enabled = pc_mode != 0; + g_pc_tier_a = pc_mode == 2; + if (pc_mode > 2) { + logf("perfagent-cupti: PERFAGENT_GPU_PC_SAMPLING=%u is not a tier " + "(1=continuous, 2=serialized); treating it as 1\n", pc_mode); + g_pc_tier_a = false; + } + if (g_pc_enabled) { // 0 = leave CUPTI's own SM-count-derived default, which is then read // back so gpu_config_v1 reports the real period. Anything outside // CUPTI's documented 5..31 is refused here rather than passed through @@ -1585,6 +1887,31 @@ extern "C" __attribute__((visibility("default"))) int InitializeInjection(void) gpu_config_v1_emit(&r, 1, g_config_seq.fetch_add(1, std::memory_order_relaxed)); }); } + if (g_pc_tier_a) { + perfagent::BurstConfig bc; + bc.burst_ns = (uint64_t)env_uint("PERFAGENT_GPU_PC_BURST_MS", 50) * 1000000ull; + bc.target_rate = (double)env_uint("PERFAGENT_GPU_PC_TARGET_RATE", 100); + // Per-mille, so a ceiling can be expressed without a float in the + // environment. 100 = 10%. + bc.max_duty = (double)env_uint("PERFAGENT_GPU_PC_MAX_DUTY_PERMILLE", 100) / 1000.0; + bc.max_gap_ns = (uint64_t)env_uint("PERFAGENT_GPU_PC_MAX_GAP_MS", 10000) * 1000000ull; + g_burst = new perfagent::BurstController(bc); + // The burst timer's period. A fifth of the burst length by default, so + // a "50 ms" burst is 50..60 ms rather than 50..150 ms on the 100 ms + // drain tick. It costs one wakeup per period doing an atomic load and + // a compare when no transition is due. + unsigned burst_tick_ms = env_uint("PERFAGENT_GPU_PC_BURST_TICK_MS", + (unsigned)(bc.burst_ns / 5000000ull)); + if (!burst_tick_ms) burst_tick_ms = 1; + g_burst_timer = new perfagent::Drainer(); + g_burst_timer->on_tick(on_burst_tick); + logf("perfagent-cupti: tier A duty cycle burst_ms=%llu target_rate=%.0f/s " + "max_duty=%.3f min_gap_ms=%llu max_gap_ms=%llu tick_ms=%u\n", + (unsigned long long)(bc.burst_ns / 1000000ull), bc.target_rate, bc.max_duty, + (unsigned long long)(perfagent::burst_min_gap_ns(bc) / 1000000ull), + (unsigned long long)(bc.max_gap_ns / 1000000ull), burst_tick_ms); + g_burst_tick_ms = burst_tick_ms; + } if (!check(cuptiSubscribe(&g_subscriber, (CUpti_CallbackFunc)on_callback, nullptr), "cuptiSubscribe")) @@ -1596,7 +1923,7 @@ extern "C" __attribute__((visibility("default"))) int InitializeInjection(void) "enable RUNTIME_API"); check(cuptiEnableDomain(1, g_subscriber, CUPTI_CB_DOMAIN_RESOURCE), "enable RESOURCE"); - if (g_pc_tier_b) { + if (g_pc_enabled) { // The only notification that CUPTI is about to finalize itself. Not // subscribed when Tier B is off: with no PC sampling enabled there is // nothing for the handler to tear down, and the subscription is not @@ -1609,7 +1936,7 @@ extern "C" __attribute__((visibility("default"))) int InitializeInjection(void) "cuptiActivityRegisterCallbacks"); check(cuptiActivityEnable(CUPTI_ACTIVITY_KIND_CONCURRENT_KERNEL), "enable CONCURRENT_KERNEL"); - if (g_pc_tier_b) { + if (g_pc_enabled) { // sm_count and clock_hz for gpu_config_v1 have no other source: CUPTI // has no device attribute for either and this adapter does not link // libcuda. One record per device, delivered once. @@ -1623,6 +1950,11 @@ extern "C" __attribute__((visibility("default"))) int InitializeInjection(void) g_drainer->on_tick(on_tick); g_drainer->start(drain_ms); + // Started AFTER the drain timer and after atexit is armed: a burst that + // opened before the exit handler existed could not be closed by it, and + // an unclosed window would report a hard exit that did not happen. + if (g_burst_timer) g_burst_timer->start(g_burst_tick_ms); + atexit(at_exit_handler); // The seed is logged because it IS the schedule: with it and the period, @@ -1633,7 +1965,15 @@ extern "C" __attribute__((visibility("default"))) int InitializeInjection(void) // sem_at_init vs sem_after_init is the measurement #49 needed: the first // fix gated the rendezvous on the semaphore and lost the CUDA path. logf("perfagent-cupti: pc_sampling=%s tier=%s\n", - g_pc_tier_b ? "on" : "off", g_pc_tier_b ? "B/continuous" : "none"); + g_pc_enabled ? "on" : "off", + !g_pc_enabled ? "none" : (g_pc_tier_a ? "A/kernel-serialized" : "B/continuous")); + if (g_pc_tier_a) { + logf("perfagent-cupti: WARNING Tier A SERIALIZES GPU kernels while a burst is " + "open. Kernel durations inside a window are inflated by the measurement " + "and are marked gpu_serialized=\"true\"; CPU and off-CPU samples taken " + "during a burst are distorted and carry NO marking at all; and Tier A " + "refuses to run where CUDA graphs are in use.\n"); + } logf("perfagent-cupti: initialized pid=%d sample_period=%u sample_seed=0x%016llx " "drain_ms=%u clock_offset_ns=%lld enroll=%s sem_at_init=%u sem_after_init=%u " "enroll_addr=@%s cubin_addr=@%s cubin_timeout_ms=%u\n", diff --git a/shim/stub/stub.cc b/shim/stub/stub.cc index 371e926..e8388d9 100644 --- a/shim/stub/stub.cc +++ b/shim/stub/stub.cc @@ -53,6 +53,11 @@ PERFAGENT_USDT_EMITTER(gpu_pc_sample_batch_v1, 40); PERFAGENT_USDT_EMITTER(gpu_stall_reason_map_v1, 136); PERFAGENT_USDT_EMITTER(gpu_config_v1, 24); PERFAGENT_USDT_EMITTER(gpu_dropped_v1, 16); +// Tier A's disclosure. PERFAGENT_STUB_SAMPLING_WINDOWS= synthesizes n +// KERNEL_SERIALIZED bursts bracketing the executions this stub emits, so the +// consumer's window -> execution intersection and all three gpu_serialized +// values are reachable on a machine with no GPU. +PERFAGENT_USDT_EMITTER(gpu_sampling_window_v1, 24); // The synthetic stall table. Real names from GA102 rather than invented ones: // a consumer that renders these into gpu_stall label values should show what a @@ -218,8 +223,24 @@ perfagent_stub_run(unsigned launches, unsigned period_us, unsigned sample_period unsigned long stall_seq = 0; unsigned long config_seq = 0; unsigned long dropped_seq = 0; + unsigned long window_seq = 0; bool names_was_attached = false; + // Tier A, synthesized. n bursts spread across the executions below, each + // emitting an OPEN record (end_ns = 0) and then a CLOSED one with the same + // start_ns -- the same two-record shape the CUPTI adapter uses, because + // that is what makes a hard exit mid-burst visible instead of losing the + // window entirely. + // + // PERFAGENT_STUB_SAMPLING_WINDOW_OPEN=1 leaves the LAST burst open: the + // hard-exit case, where every execution from that start_ns onward is + // gpu_serialized="unknown" and must never read "false". + const char *winenv = getenv("PERFAGENT_STUB_SAMPLING_WINDOWS"); + const unsigned sampling_windows = (winenv && *winenv) ? (unsigned)atoi(winenv) : 0; + const char *openenv = getenv("PERFAGENT_STUB_SAMPLING_WINDOW_OPEN"); + const bool leave_last_open = openenv && *openenv && atoi(openenv) != 0; + uint64_t first_exec_ns = 0, last_exec_ns = 0; + // The same queue the CUPTI adapter runs, wired the same way: capture on // the caller's thread, offer on the drain thread. perfagent::CubinQueue cubins; @@ -344,6 +365,8 @@ perfagent_stub_run(unsigned launches, unsigned period_us, unsigned sample_period e.device_id = 0; e.start_ns = now + 10000; // 10us after the launch e.end_ns = now + 10000 + 50000; // 50us on device + if (!first_exec_ns) first_exec_ns = e.start_ns; + last_exec_ns = e.end_ns; eb.add(e); if (period_us) std::this_thread::sleep_for(std::chrono::microseconds(period_us)); @@ -352,6 +375,33 @@ perfagent_stub_run(unsigned launches, unsigned period_us, unsigned sample_period lb.flush(); eb.flush(); + // ---- Tier A, synthesized: the serialization windows. + // + // The span the executions occupy is cut into 2n slices and every other one + // is a burst, so roughly half the executions fall inside a window and half + // fall in a gap. That is the shape the consumer has to get right: an + // execution intersecting a window is "true", one in a proven gap is + // "false", and one outside the covered span entirely is "unknown". + if (sampling_windows && last_exec_ns > first_exec_ns) { + const uint64_t span = last_exec_ns - first_exec_ns; + const uint64_t slice = span / (2ull * sampling_windows); + for (unsigned i = 0; i < sampling_windows && slice; i++) { + const uint64_t ws = first_exec_ns + (uint64_t)(2 * i) * slice; + const uint64_t we = ws + slice; + gpu_sampling_window_v1 w{}; + w.start_ns = ws; + w.mode = GPU_SAMPLING_MODE_KERNEL_SERIALIZED; + // Open first, always: the consumer must supersede it with the + // closed record rather than double-count the burst. + if (gpu_sampling_window_v1_enabled()) + gpu_sampling_window_v1_emit(&w, 1, window_seq++); + if (leave_last_open && i + 1 == sampling_windows) break; + w.end_ns = we; + if (gpu_sampling_window_v1_enabled()) + gpu_sampling_window_v1_emit(&w, 1, window_seq++); + } + } + // ---- Tier B, synthesized. // // PC samples FIRST and the stall map after them, deliberately: a stall @@ -456,6 +506,11 @@ perfagent_stub_run(unsigned launches, unsigned period_us, unsigned sample_period (unsigned long long)lb.dropped(), (unsigned long long)eb.dropped(), perfagent::enroll_result_name(enrolled), sem_at_enroll, gpu_launch_sampled_v1_semaphore_count(), enroll_name); + if (sampling_windows) + fprintf(stderr, "stub: sampling_windows=%u records=%lu last_open=%d " + "exec_span=[%llu,%llu]\n", + sampling_windows, window_seq, leave_last_open ? 1 : 0, + (unsigned long long)first_exec_ns, (unsigned long long)last_exec_ns); if (pc_samples) fprintf(stderr, "stub: pc_samples=%u stall_reasons=%u cubins=2 functions=4 " "drop_classes=4 replays=%llu\n",