Skip to content

uploads: use fewer statements and bounded concurrency - #41

Merged
Uzaaft merged 6 commits into
masterfrom
perf/uploads
Nov 18, 2026
Merged

Uzaaft merged 6 commits into
masterfrom
perf/uploads

Conversation

@Uzaaft

@Uzaaft Uzaaft commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

No description provided.

Every single-session writer (reserve, adopt, seal, abort, candidate
allocation) and the batched part reservation tried the upload advisory
locks in one statement and then locked the session rows in a second.

Do both in one statement. The row locks are a subquery correlated on the
aggregate of the try-locks, so PostgreSQL has tried every advisory lock,
in the same sorted order, before it requests any row lock, and requests
none when one was held. The modes, the ID order of the row locks, the
errors, and the reload that follows under a fresh snapshot are unchanged;
a regression test pins that a refused writer holds no row lock.
A part reservation read the parts already reserved and then the
organization's staging total, and inserted the new parts and then added
their sizes to the sessions: four statements. A batch then created each
session's provider upload and adopted it, one session after another.

Read both values in one statement and write both tables in another. The
session locks the callers hold keep the reserved parts still, so one
snapshot sees what two consecutive ones did, and the total stays the soft
limit it was. Neither write reads the other's table, and neither table
has a trigger. Create and adopt the provider uploads of a batch four at a
time, as preparation already runs: the sessions are distinct, each
adoption remains its own single-session transaction, and results keep
request order.
Sealing a session for admission read the session's batch, then
share-locked the offered batch, appended the member, and reset a
capacity refusal, each in its own statement, and every seal read the
batch a third time for its claim. A publication re-read and re-locked
the same token for each member of its transaction, and checked each
member's batch in two statements.

Read the session's batch and lock the offered one in one statement: the
lock is still taken only by a session with no batch, and the session's
locks hold its membership still. Append and retry in one statement: an
appended row has no result to reset, and an existing row is not
appended. Return the claim from the delegated authority check, which
share-locks the batch it reads, so the later unlocked read could only
see the same value; the claim of a completed session or of an admission
was never used and is no longer read. In a publication, read the token
once: its share lock, the transaction's fixed NOW(), and the key share
locks on the group's caches leave a second read nothing new to see. The
claim is compared in the statement that locks the batch. Each seal and
each publication still locks the token in its own transaction, so
revocation is ordered against them exactly as before.
After verification, an inline batch allocated each member's candidate,
wrote its bytes to storage, and confirmed it, one member after another,
so a batch waited for every storage write in turn.

Run up to four members at once, the bound `prepare` already uses for
multipart members. The sessions of a batch are distinct and each member
still allocates and confirms its own candidate in its own
single-session transaction, so no lock order or isolation changes. The
members reach publication in request order as before, which is what
decides path winners, and a preparation-only member still wakes the
job's worker as soon as its own candidate is confirmed.
The statements that take a session's locks, read a group's reserved
parts, authorize a finalization, and read a session's batch membership
were bound through runtime query functions and decoded into tuples by
position. They now use the query macros, which verify each statement
and its column types against the schema at build time and name the
columns the callers read.
The authority check copied its checked row into a positional tuple so
the existing pattern matches kept working, which discarded the column
names the query macro had just provided. The checks now read the
authorized and claimed columns directly and return the row's work
claim.
@Uzaaft
Uzaaft merged commit 77ffffd into master Nov 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant