Repository navigation
incremental compilation: could not create session directory lock file: No locks available (os error 37) #49773
Description
I was unable to incrementally compile using cargo build.
I had added the rand dependency to the toml file, per the advice of the Guessing Game exercise in Rust Programming Language (Second Edition).
Before this particular error, I had just finished setting up a synced folder with NFS on my vagrant box (ubuntu 16.04) where rust was installed. There is some issue using the default vagrant shared folder setup with rust.
I tried this code:
use std::io;
fn main() {
println!("Guess the number");
println!("Please input your guess.");
let mut guess = String::new();
io::stdin().read_line(&mut guess)
.expect("Failed to read line");
print!("You guessed: {}", guess);
}I expected to see this happen:
compile
Instead, this happened:
Updating registry `https://github.com/rust-lang/crates.io-index`
Downloading rand v0.3.22
Downloading libc v0.2.40
Downloading rand v0.4.2
Compiling libc v0.2.40
Compiling rand v0.4.2
Compiling rand v0.3.22
Compiling guessing_game v0.1.0 (file:///Rust/projects/guessing_game)
error: incremental compilation: could not create session directory lock file: No locks available (os error 37)
thread 'rustc' panicked at 'librustc/session/mod.rs:727: Trying to get session directory from IncrCompSession `NotInitialized`', librustc/session/mod.rs:1180:26
note: Run with `RUST_BACKTRACE=1` for a backtrace.
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
note: rustc 1.25.0 (84203cac6 2018-03-25) running on x86_64-unknown-linux-gnu
error: Could not compile `guessing_game`.
rustc --version --verbose:
rustc 1.25.0 (84203cac6 2018-03-25)
binary: rustc
commit-hash: 84203cac67e65ca8640b8392348411098c856985
commit-date: 2018-03-25
host: x86_64-unknown-linux-gnu
release: 1.25.0
LLVM version: 6.0
Backtrace:
thread 'rustc' panicked at 'librustc/session/mod.rs:727: Trying to get session directory from IncrCompSession `NotInitialized`', librustc/session/mod.rs:1180:26
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
1: std::sys_common::backtrace::_print
at libstd/sys_common/backtrace.rs:71
2: std::panicking::default_hook::{{closure}}
at libstd/sys_common/backtrace.rs:59
at libstd/panicking.rs:380
3: std::panicking::default_hook
at libstd/panicking.rs:396
4: std::panicking::rust_panic_with_hook
at libstd/panicking.rs:576
5: std::panicking::begin_panic
6: rustc::session::opt_span_bug_fmt::{{closure}}
7: rustc::ty::context::tls::with_opt
8: rustc::session::opt_span_bug_fmt
9: rustc::session::bug_fmt
10: rustc::session::Session::incr_comp_session_dir
11: rustc_incremental::persist::fs::garbage_collect_session_directories
12: rustc_driver::driver::phase_2_configure_and_expand_inner::{{closure}}
13: rustc_driver::driver::phase_2_configure_and_expand_inner
14: rustc_driver::driver::compile_input
15: rustc_driver::run_compiler
Activity
glaubitz commentedon Apr 9, 2018
This is an issue we are currently seeing on linux-sparc64 as well [1]:
I have not yet figured out where this comes from. The same rust version worked fine in the past.
CC @jrtc27 @psumbera @mkarcher
michaelwoerister commentedon May 3, 2018
NFS does not support locking files very well, I guess. It would be good if there was some way to make Cargo put the incremental folder somewhere else (into
/tmpfor example).cc @rust-lang/cargo
jrtc27 commentedon May 3, 2018
Although the original report mentions NFS, we're not using NFS for the sparc64 buildds; the two issues have similar but different error messages ("No locks available" vs "No such process") so they could well be different problems.
michaelwoerister commentedon May 3, 2018
@jrtc27, yes, sounds like a different root cause.
alexcrichton commentedon May 3, 2018
In Cargo we disable file locking whenever NFS is detected, although that may not work well for rustc as it's semi-reasonable to expect Cargo to not be run in parallel much on one system but rustc probably is being executed in parallel by Cargo at least.
An alternative to moving the incremental directory is moving the locks for the incremental directory. I've considered this for Cargo where we actually want to enable file locking on NFS directories. That'd require more intrusive support in rustc, though.
glaubitz commentedon Sep 22, 2018
If anyone wants to reproduce this problem, there are sparc64 porterboxes available in the gcc compile farm, both running Solaris and Linux. Any open source developer can apply for an account:
michaelwoerister commentedon Sep 24, 2018
Is this on a file system that should support locks in general?
As long as we don't want to support two different machines working on the same NFS share simultaneously (which we don't really do anyway), we could indeed just allocate that lock files in
/tmp. The only real restriction for lock files is that they mustn't be contained in the session directory that they protect.glaubitz commentedon Sep 24, 2018
It's a standard off-the-shelf installation of Debian on ext4. Nothing special. There is no NFS or anything involved and it affects sparc64 machines with the same configuration as comparable x86_64 machines.
michaelwoerister commentedon Sep 24, 2018
Interesting. That's unexpected.
glaubitz commentedon Sep 24, 2018
You can test it yourself on the Linux/sparc64 porterbox available in the gcc compile farm. It's a rather fast machine.
michaelwoerister commentedon Sep 24, 2018
I've requested an account. Thanks for the tip!
2 remaining items
michaelwoerister commentedon Oct 23, 2018
@glaubitz Yes, I have an account. I haven't gotten around to actually using it though.
glaubitz commentedon Dec 20, 2018
@michaelwoerister @alexcrichton
@karcherm has had a look at the Rust sources and he found the problem rather quickly. Rust is hard-wiring the constants for the fnctls from glibc assuming they are constant across which they are not.
Compare:
and:
Rust is trying to call "Set Lock" and uses function control number 6. But that's "F_SETOWN" on sparc64 which expects a PID as a parameter which is why the error message is "No such process".
michaelwoerister commentedon Dec 20, 2018
Awesome find, @karcherm & @glaubitz!
jonathan-sha commentedon Mar 23, 2020
I get this when compiling files stored on WSL2, but running cargo on windows. So running on windows and compiling on this path:
nfwatsoncom commentedon May 10, 2020
Ditto from @tpn-station ... I likewise have source on WSL2 (Ubuntu-20.04 w/ native Kernel) filesystem with Ubuntu's root FS mounted as drive "N:" in Windows. To Windows it looks like a network drive. My Rust project is configured for Windows build (as opposed to running CLion in Ubuntu).
I also tried avoiding the "N:" drive-letter mapping and specifying source files directly as "'\\wsl$\Ubuntu-20.04\home..." and hit the same error.
DrDinosaur commentedon May 26, 2020
Same issue on WSL 2 as above.
daswafford commentedon Jun 6, 2020
Hi, I think I am seeing a related error. I was trying to build the hello world example from https://rust-cli.github.io/book/tutorial/setup.html and when issuing cargo run, get:
I am on a MacOS computer running 10.15.4 and the current working directory where I ran "cargo" run" is an smb mount to a network-based file server (Ubuntu):
Tarnadas commentedon Nov 10, 2020
Current workaround for Windows compiling a project in WSL2:
see #66513 (comment)
ms140569 commentedon Dec 29, 2020
I'm facing the very same issue (Rust 1.48.0, Mac -> SMB), documented here:
#76251 (comment)
Both of these two workarounds are valid on Mac OSX:
Sorry, if i commented/documented on the wrong issue ...
ehuss commentedon May 29, 2021
#85698 has changed it so that rustc does not panic in this situation, but it will still generate an error.
Leaving this open as I think it is still worthwhile to consider what other options there are for locking, or if certain situations can be fixed.
A summary of the things people have reported:
thread 'rustc' panicked at 'src/librustc/session/mod.rs:809with cleancargo newproject #59224 SMB mounts on macOS fails with ENOTSUPkngrektor commentedon Sep 17, 2021
regarding SMB mounts on macOS: I checked ages ago and flock seems to work fine with SMB mounts it's specifically F_SETLK with fcntl that doesn't work. I believe this means that the Linux code path would work fine for macOS but either way a fallback would be nice.
tommythorn commentedon Jan 21, 2022
How is rustc deciding that locks aren't working? Is it just a heuristic? Because my NFS4 mount on macOS does support locks and they work. If it's a heuristic can it be overruled?
Xymist commentedon Apr 20, 2022
This may or may not be useful, but for possible future Googlers....
I keep my dev folder on a TrueNAS server and access via an NFS share. Ended up with this issue after moving from Core to Scale. The fix was to turn on both "Enable NFSv4" and "NFSv3 ownership model for NFSv4" in the NFS service settings.