Skip to content

Commit a9ae75f

Browse files
committed
[lang][nfc] Add execution and metaprogramming documentation
Signed-off-by: Bryce Adelstein Lelbach <brycelelbach@gmail.com>
1 parent 9c3ee3c commit a9ae75f

4 files changed

Lines changed: 165 additions & 0 deletions

File tree

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
.. SPDX-FileCopyrightText: Copyright (c) <2026> NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
..
3+
.. SPDX-License-Identifier: Apache-2.0
4+
5+
.. currentmodule:: cuda.lang
6+
7+
.. _execution-execution-model:
8+
9+
Execution Model
10+
===============
11+
12+
Abstract Machine
13+
----------------
14+
15+
.. _thread:
16+
.. _block:
17+
.. _grid:
18+
19+
A *SIMT kernel* is executed using the following thread hierarchy:
20+
21+
- A *grid* is a 1D, 2D, or 3D collection of thread blocks.
22+
- A *thread-block cluster* is an optional 1D, 2D, or 3D group of blocks within
23+
the grid.
24+
- A *thread block* is a 1D, 2D, or 3D collection of threads.
25+
- A *warp* groups threads within a block for SIMT execution.
26+
- A *thread* executes the body of the |kernel|.
27+
28+
A thread can query its position in the thread, block, warp, and cluster
29+
hierarchy using the functions listed under
30+
:ref:`SIMT Model <operations-simt-model>`.
31+
32+
Threads in a block can communicate through shared memory and synchronize
33+
explicitly. Blocks in the same cluster can map each other's shared memory and
34+
synchronize at cluster scope. Global memory is accessible to all blocks,
35+
subject to CUDA synchronization and memory-ordering rules. See the
36+
:ref:`Data Model <data-data-model>` for the corresponding memory spaces.
37+
38+
39+
.. _execution-execution-spaces:
40+
41+
Execution Spaces
42+
----------------
43+
44+
|cuda lang| code runs on one or more *targets*. A target is an execution
45+
environment defined by its hardware resources and programming model.
46+
47+
.. _host code:
48+
.. _SIMT code:
49+
50+
The set of targets where a construct can be used is called its *execution
51+
space*. |cuda lang| defines two execution spaces:
52+
53+
- *Host code* --- Python code that prepares data and launches kernels on a CPU.
54+
- *SIMT code* --- code compiled for CUDA SIMT targets and executed by GPU
55+
threads.
56+
57+
Some constructs span multiple execution spaces. For example, :func:`cdiv` is
58+
usable in both host code and SIMT code.
59+
60+
A function whose decorator explicitly specifies its execution space is called
61+
an *annotated function*.
62+
63+
64+
.. _execution-simt-functions:
65+
66+
SIMT Functions
67+
--------------
68+
69+
A *SIMT function* is a helper function usable from SIMT code. The
70+
``@cl.function`` decorator explicitly marks such a function. An undecorated
71+
Python function called from a kernel or SIMT function is compiled recursively
72+
as SIMT code, so an explicit decorator is not required.
73+
74+
75+
.. _execution-simt-kernels:
76+
77+
SIMT Kernels
78+
------------
79+
80+
A *SIMT kernel* is an entry point executed by each thread in each block in a
81+
grid. Kernels cannot be called directly. Use :func:`launch` to queue a kernel
82+
for execution with explicit grid and block dimensions.
83+
84+
85+
Python Subset
86+
-------------
87+
88+
|SIMT code| supports a subset of the Python language. There is no Python
89+
runtime within SIMT code. Features such as exceptions and coroutines are not
90+
supported today.
91+
92+
Object Model & Lifetimes
93+
~~~~~~~~~~~~~~~~~~~~~~~~
94+
95+
Arrays and pointers are views of memory and may alias or update the same
96+
storage. Their visibility and lifetime depend on their CUDA memory space.
97+
Because a launch is asynchronous with respect to the host, kernel arguments
98+
must remain valid until the kernel completes.
99+
100+
Control Flow
101+
~~~~~~~~~~~~
102+
103+
Supported Python control flow statements include ``if``, ``while``, and
104+
range-based ``for`` loops.
105+
106+
Current limitations
107+
^^^^^^^^^^^^^^^^^^^
108+
109+
The ``step`` of a ``range`` must be strictly positive. Negative-step ranges
110+
such as ``range(10, 0, -1)`` are not supported today.

‎experimental/cuda-lang/docs/source/index.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,8 @@ memory:
7171
:maxdepth: 2
7272
:hidden:
7373

74+
execution
75+
metaprogramming
7476
data
7577
operations
7678
private
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
.. SPDX-FileCopyrightText: Copyright (c) <2026> NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
..
3+
.. SPDX-License-Identifier: Apache-2.0
4+
5+
.. currentmodule:: cuda.lang
6+
7+
.. _metaprogramming-metaprogramming:
8+
9+
Metaprogramming
10+
===============
11+
12+
|cuda lang| uses compile-time values and Python evaluation to specialize
13+
generated SIMT code.
14+
15+
Compile-Time Constants
16+
----------------------
17+
18+
Some facilities require values that are known at compile time. Literals and
19+
expressions derived entirely from compile-time constants remain known to the
20+
compiler. A value that depends on run-time thread state is dynamic.
21+
22+
Annotate a kernel parameter with ``cl.Constant[T]`` when its launch-time value
23+
must be embedded in the compiled kernel. A distinct kernel is compiled for each
24+
unique value of a constant-embedded parameter.
25+
26+
Static Evaluation
27+
-----------------
28+
29+
``cl.static_eval`` evaluates an expression with Python semantics during
30+
compilation. It can inspect compile-time properties of dynamic values and
31+
select among symbolic expressions, but it cannot perform run-time GPU
32+
operations.
33+
34+
Static Iteration
35+
----------------
36+
37+
``cl.static_iter`` evaluates an iterable during compilation and expands the
38+
loop body once for each item.
39+
40+
Static Assertions
41+
-----------------
42+
43+
``cl.static_assert`` checks a condition during compilation.
44+
``cl.ensure_constant`` requires a value to be known at compile time and returns
45+
that value.
46+
47+
Static Functions
48+
----------------
49+
50+
The ``@cl.static_def`` decorator marks a reusable helper function to be
51+
evaluated with static-evaluation semantics.

‎experimental/cuda-lang/docs/source/operations.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,8 @@ Type Casts
7373
bitcast
7474

7575

76+
.. _operations-simt-model:
77+
7678
SIMT Model
7779
----------
7880
.. autosummary::

0 commit comments

Comments
 (0)