Repository navigation
Keep the session maps unlocked across network I/O - #140
Conversation
SessionManager and SftpManager each kept one map lock held across every network await: a keystroke send, a disconnect, a directory listing, and in SFTP's case an entire file transfer. One peer that stopped reading, or a large download, therefore stalled every other session, and close_all could not drain the map. The vault reset gives close_all five seconds and then drops it, so an SFTP session busy in a transfer or an endless listing stayed reachable after the reset that was meant to cut it off. Entries are now Arc'd and looked up under a short lock; the I/O runs after the guard is released. Writes to one SSH session still go out in order through a per-session lock. close and close_all remove entries first and bound the goodbye. SFTP listings get a 30 second overall deadline, since russh-sftp reads until the server says end-of-file with no total limit. The SSH reader also removes its own entry and disconnects when the remote shell exits, instead of leaving the connection and its keepalives up until the pane is closed, and it no longer spins on a closed resize channel while a close is in flight.
PR Summary by QodoKeep SSH and SFTP session maps unlocked during network I/O
AI Description
Diagram
High-Level Assessment
Files changed (4)
|
Code Review by Qodo |
Taking an entry out of the map stopped new calls from reaching a session, but a call already holding the entry kept it alive: a keystroke send stuck behind a peer that stopped reading, or an SFTP listing or transfer still running, held the connection and the session id until it finished on its own. Each SSH session and SFTP connection now carries a closed signal that close, close_all and the reader's own cleanup flip. Writes, listings and transfers race against it and return an error once it fires. The goodbye sent after a close holds only the connection handle, so it no longer keeps the session id claimed while a stalled peer fails to answer.
Listings and transfers already stopped when their connection closed, but mkdir, remove, rename, canonicalize and stat did not: a request issued just before a close or a vault reset could still complete and report success after teardown began. Every SftpManager operation now runs under the connection's closed signal.
A close timed the goodbye out after 5 s and dropped it. russh stops reading its outbound queue while a channel waits on the peer's window, and the reader's channel kept its session task alive, so a close on a stalled peer left the connection authenticated for good: the id stayed claimed, the close was never announced, and a vault reset left the shell open even after the peer recovered. - connect() wraps the TCP stream in a Transport that can be cut. The shared goodbye queues the disconnect, waits up to 5 s for russh to end the session, then cuts the stream, which ends the session task and every channel on it. SSH and SFTP both close through it. - The SSH reader keeps only the read half and never waits on outbound messages. Window changes go out from their own task, and resize() replaces the pending size instead of queueing on a bounded channel that a stalled peer filled. close() drops the entry before the goodbye. - SFTP listings run on a second, raw SFTP channel: page by page under the per-request timeout, capped at 200k entries / 32 MiB of names, and the directory handle is closed however the listing ends. That replaces the 30 s total cap that refused large directories on slow links. A server that will not open the channel falls back to read_dir. - An upload without overwrite creates the file exclusively, so of two uploads to one path only one creates it now that SFTP calls no longer serialize on the map lock. - One request helper for the single-request SFTP calls; one CloseSignal type for both managers; session_write/session_resize look the id up once. New tests cover a close on a peer that never reads again and on one that recovers after the goodbye timed out, resizes on a stalled peer, the listing cap and handle close, and two racing exclusive uploads. All five fail without this change.
# Conflicts: # core/src/connection.rs
|
@codex review the PR |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f68bdb419f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
With the map lock gone, SFTP calls on one session run side by side, and two things it used to serialize no longer were. - Two overwrite uploads to the same remote path truncated and wrote the file at once, leaving a mix of both (and zero-filled gaps) while both reported success. Uploads and write_file now take a per-connection lock on their destination path, so the second waits for the first and then replaces a complete file. Locks are dropped from the map once nothing holds or waits on them. - A slow listing could answer after a newer one and move the browser back to a directory the user had left. The store now publishes only the latest listing of the current session; a superseded listing's result, error and loading state are ignored, and the refresh after an upload goes through the same path.
|
@codex review the PR |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ad94fb4ce9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let mut local = open_download_target(local_path, overwrite).await?; | ||
|
|
||
| tokio::io::copy(&mut remote, &mut local).await?; | ||
| local.flush().await?; |
There was a problem hiding this comment.
Serialize downloads that write to the same local destination
When two downloads target the same local file, removing the map-wide lock allows both to truncate and write it concurrently. In the inspected desktop flow, SftpBrowser.vue leaves the Download context-menu action enabled during transfers, and sftp_transfer.rs::sftp_download_to_local always passes overwrite = true after the save dialog. A user can therefore start a second download to the same destination while the first is running; both can report success while leaving mixed contents or trailing bytes from the larger transfer. Unlike uploads, downloads have no destination lock. Serialize downloads per local destination across the manager, or write to separate temporary files and atomically publish completed downloads.
Useful? React with 👍 / 👎.
Downloads no longer serialize on the map lock, and the desktop always downloads with overwrite once the save dialog has confirmed it. Two downloads to the same local file, from one session or two, truncated and wrote it together, leaving a mix of both while both reported success. A download now takes a lock on its local destination, held in the manager since the local disk is shared by every session, so the second waits for the first and then replaces a complete file. The key resolves the file's directory, so a path through a symlink or `..` is still the same file. The destination lock is the one uploads use, made generic over its key.
|
@codex review the PR |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
computerbox124
left a comment
There was a problem hiding this comment.
@Sadykhzadeh thank you!
#140 landed on main as one squashed commit, while this branch still carried its earlier commits, so session.rs, commands.rs, sftp.rs and session_isolation.rs conflicted. Each takes main's version, which is #140's final state, and this branch's own changes are carried over on top of it. download_to_local keeps main's per-destination lock, so of two downloads to one local file the one started last is left in place, and claims it and opens the remote file under until_closed. The copy then runs through write_download against the connection's CloseSignal, so a close stops the download without cutting its cleanup short. a_download_waits_for_one_already_writing_the_same_local_file watched the target grow, which a staged download never shows until it is complete. It now watches the staging file, checks the first download is still running when the second starts, and that no staging file is left.
Closes #139.
What this changes
Map locks are never held across network I/O
Both managers now keep
Arc'd entries. A call takes the lock, clones the entry and releases it before awaiting anything on the network:writeslock, so keystrokes reach the channel in order. A peer that stops reading now holds up only its own input. The russhHandleno longer sits behind a mutex, sincedataanddisconnecttake&self.Arc<SftpConnection>. A multi-GB download no longer blocks listing, renaming or closing on another host.close/close_all. These remove entries first and then say goodbye (below).close_allsends its goodbyes in parallel.This restores what
contain_open_sessionsrelies on: after a vault reset, no SSH or SFTP session can still be addressed, whatever was running on it.A close always ends the connection
russh has no call that stops a session.
Handle::disconnectonly queues a message, and russh stops reading that queue while a channel waits on the peer's window, so a goodbye to a peer that stopped reading never goes out. Dropping every handle doesn't end the session either while a channel half is still alive. Without a way to force it, a close on a stalled peer left the connection authenticated for good, kept the id claimed and never announced the close, even after the peer recovered.connection::connectopens the TCP stream itself, wraps it in aTransport(newcore/src/transport.rs) and returnsConnection { handle, transport }.goodbye: queue the disconnect, wait up to 5 s for russh to end the session, then cut the stream. A cut fails every pending read and write, so russh ends the session and every channel on it.A close ends what is already running
Each SSH session and SFTP connection carries a
CloseSignal, raised byclose,close_alland the reader's own cleanup. SSH writes and every SFTP call (listings, transfers and single requests) race against it. A call that already held the entry returns an error right away rather than when the cut lands.SSH session lifecycle
resizereplaces the pending size (awatch) and never waits. On a stalled peer, a bounded resize queue would fill up and every later resize would hang.closedrops the entry before awaiting the goodbye, so the entry's share of the id goes at once.session_writeandsession_resizelook the id up once and fall through to local terminals onSessionNotFound.SFTP listings
SftpSession::read_dircollects a whole directory before returning anything, so it can't cap a listing or close the directory handle when one is abandoned. Listings now run on a second SFTP channel per connection, driven request by request throughRawSftpSession:The listing channel opens alongside the main one, so it adds no round trips to the connect. A server that won't open a second session channel falls back to
read_dirwith an overall 30 s deadline.Writers to one remote path take turns
SFTP calls on one session no longer serialize on the map lock, so two writers to the same path could run at once.
upload_from_localandwrite_filetake a per-connection lock on their destination path. A second upload to the same path waits for the first to finish, then replaces a complete file. Without the lock, two overwrite uploads truncated and wrote the file together, leaving a mix of both, with zero-filled gaps, while both reported success. A path's lock leaves the map once nothing holds or waits on it.overwritethe remote file is opened withCREATE | EXCLUDE | WRITE, so an upload is refused with "remote file already exists" even when another client creates the file between the existence check and the create.The file browser shows the latest listing
Listings of one session also run side by side now, and the browser's navigation stays enabled while one loads, so a slow listing could answer after a newer one and move the browser back to a directory the user had left.
listDirindesktop/src/stores/sftp.tspublishes entries, path, error and loading state only for the latest listing of the current session. A superseded listing, or one that answers after a disconnect, is ignored. The refresh after an upload goes throughlistDirtoo.Not in this PR
Tests
core/tests/session_isolation.rsruns both managers against an in-process russh server on loopback. Its SFTP side answers every READDIR with more entries and never with end-of-file, and keeps its files in memory. Its shell exits right away, or stops reading its input.a_listing_that_never_ends_holds_up_no_other_sftp_session: while the endless listing runs, another session answers,close_allreturns within 5 s with both sessions out of reach, and the listing ends with an error.closing_one_sftp_session_does_not_wait_on_its_listinga_shell_that_exits_leaves_no_session_behind: the output and close arrive, then the entry and the claim are gone and writes are refused.a_write_stuck_behind_a_peer_that_stopped_reading_ends_with_the_close: the server stops reading, writes run until one blocks, thenclosemust end that write within 2 s.closing_a_session_whose_peer_stopped_reading_frees_its_id: after the close, the close is announced and the id can be claimed again.a_peer_that_recovers_after_the_goodbye_timed_out_is_still_disconnected: the server stops reading for 8 s. After the close the id comes free, and the server sees the connection end once it reads again.a_resize_never_waits_on_a_peer_that_stopped_reading: 100 resizes on a stalled session each return within 1 s.a_listing_past_the_entry_cap_is_refused_and_its_handle_closedof_two_uploads_to_one_path_without_overwrite_only_one_creates_it: slow existence checks let both uploads pass theirs before either creates the file, and exactly one succeeds.an_overwrite_upload_waits_for_one_already_writing_the_same_path: a 1 MiB overwrite upload is still writing when a 100-byte overwrite to the same path starts. The file has to end up exactly as the 100 bytes.The first four fail on
main. The next five fail on f72acb2, the commit before the teardown changes. The last one fails without the per-path lock. All pass here.core/src/transport.rshas unit tests showing that a cut ends a pending read and a write blocked on a full buffer, and thatendedresolves once the stream is dropped.desktop/src/stores/sftp.concurrency.test.tscovers the listing guard: an older listing answering last,loadingstaying on until the latest listing answers, a superseded listing's failure, and an answer after a disconnect. All four fail against the previous store.Test plan
cargo check --workspace: cleancargo test -p clavyn-core: 143 + 5 + 32 + 10 + 9 passedcargo test -p clavyn-desktop(Linux, local): 149 + 6 + 2 passedcd desktop && npx vue-tsc --noEmit: cleancd desktop && npm test(Vitest): 404 passedcd desktop/e2e && UI_RECORD=0 npm test(Playwright, against the Vite dev server): 73 passednode scripts/verify/check-comments.mjs: okrust-check,frontend-buildandbrowser-ui: passedmacos-rust-build: cancelled before a runner picked it up; needs a re-runsecurity-audit: fails on RustSec advisories in existing dependencies; this PR changes no dependenciescontrol-clavyn.mjs doctor: not run; no harness changeView with [code]smith
Need help on this PR? Tag
@codesmith-botwith what you need.