Repository navigation
uploads: use fewer statements and bounded concurrency - #41
Merged
Merged
Conversation
Every single-session writer (reserve, adopt, seal, abort, candidate allocation) and the batched part reservation tried the upload advisory locks in one statement and then locked the session rows in a second. Do both in one statement. The row locks are a subquery correlated on the aggregate of the try-locks, so PostgreSQL has tried every advisory lock, in the same sorted order, before it requests any row lock, and requests none when one was held. The modes, the ID order of the row locks, the errors, and the reload that follows under a fresh snapshot are unchanged; a regression test pins that a refused writer holds no row lock.
A part reservation read the parts already reserved and then the organization's staging total, and inserted the new parts and then added their sizes to the sessions: four statements. A batch then created each session's provider upload and adopted it, one session after another. Read both values in one statement and write both tables in another. The session locks the callers hold keep the reserved parts still, so one snapshot sees what two consecutive ones did, and the total stays the soft limit it was. Neither write reads the other's table, and neither table has a trigger. Create and adopt the provider uploads of a batch four at a time, as preparation already runs: the sessions are distinct, each adoption remains its own single-session transaction, and results keep request order.
Sealing a session for admission read the session's batch, then share-locked the offered batch, appended the member, and reset a capacity refusal, each in its own statement, and every seal read the batch a third time for its claim. A publication re-read and re-locked the same token for each member of its transaction, and checked each member's batch in two statements. Read the session's batch and lock the offered one in one statement: the lock is still taken only by a session with no batch, and the session's locks hold its membership still. Append and retry in one statement: an appended row has no result to reset, and an existing row is not appended. Return the claim from the delegated authority check, which share-locks the batch it reads, so the later unlocked read could only see the same value; the claim of a completed session or of an admission was never used and is no longer read. In a publication, read the token once: its share lock, the transaction's fixed NOW(), and the key share locks on the group's caches leave a second read nothing new to see. The claim is compared in the statement that locks the batch. Each seal and each publication still locks the token in its own transaction, so revocation is ordered against them exactly as before.
After verification, an inline batch allocated each member's candidate, wrote its bytes to storage, and confirmed it, one member after another, so a batch waited for every storage write in turn. Run up to four members at once, the bound `prepare` already uses for multipart members. The sessions of a batch are distinct and each member still allocates and confirms its own candidate in its own single-session transaction, so no lock order or isolation changes. The members reach publication in request order as before, which is what decides path winners, and a preparation-only member still wakes the job's worker as soon as its own candidate is confirmed.
The statements that take a session's locks, read a group's reserved parts, authorize a finalization, and read a session's batch membership were bound through runtime query functions and decoded into tuples by position. They now use the query macros, which verify each statement and its column types against the schema at build time and name the columns the callers read.
The authority check copied its checked row into a positional tuple so the existing pattern matches kept working, which discarded the column names the query macro had just provided. The checks now read the authorized and claimed columns directly and return the row's work claim.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.