Stacked Borrows: raw pointer usable only for `T` too strict?

[RalfJung]

Currently, the following is illegal according to Stacked Borrows:

let val = [1u8, 2];
let ptr = &val[0] as *const u8;
let _val = unsafe { *ptr.add(1) };

The problem is that the cast to *const u8 creates a raw pointer that may only be used for the u8 it points to, not anything else. The most common case is to do &slice[0] as *const _ instead of slice.as_ptr().

This has lead to problems:

• rand did the &slice[0] thing.
• Same for hashbrown.
• Rc::into_raw+Rc::from_raw don't work well together because of this.
• capnproto also used the &slice[0] pattern

Maybe this is too restrictive and raw pointers should be allowed to access their "surroundings"? I am not sure what exactly that would look like though. It would probably require having the raw pointer fully inherit all permissions from the reference it is created from.

I'll use this issue to collect such cases.

[RalfJung]

This is also a problem in Rc::into_raw/Rc::from_raw:

    pub fn into_raw(this: Self) -> *const T {
        let ptr: *const T = &*this;
        mem::forget(this);
        ptr
    }

ptr may only be used to access the T part of the RcBox<T>, but if later used with from_raw it is used for the entire Rc. Fixing this is not even possible without rust-lang/rfcs#2582.

[RalfJung]

This also came up in Gilnaa/memoffset#21, where @Amanieu proposed an container_of! macro that computes the address of a struct given the address of one of its fields.

[RalfJung]

@Amanieu the problem with code like this

struct Pair { f1: u16, f2: u16 };

let p = Pair { f1: 2, f2: 3 };
let c = container_of!(&p.f1, Pair, f1);
let _val = c.f2;

arises when you imagine splitting it across several functions:

struct Pair { f1: u16, f2: u16 };

let p = Pair { f1: 2, f2: 3 };
foo(&p.f1);
p.f2 = 4;

We want the compiler to be able to move the assignment to f2 up across the call to foo. But if foo is allowed to use container_of! and then read f2, that is no longer possible.

So, I think there is a real conflict here between being able to bound the effects of a call like foo(&p.f1), and allowing container_of!.

[Lokathor]

How does this work with [T]::as_ptr? Does that pointer let you use "the whole slice" when offsetting?

[RalfJung]

@Lokathor yes. Those methods do the right thing. They cast the wide reference to a wide raw pointer, and only then go to thin -- so the ref-to-raw cast has the right "span" in memory.

[ecstatic-morse]

rust-lang/rust#64980 gives a minor reason to maintain the status quo here. It includes a test-case for a dataflow analysis that computes whether a given Local is mutable through a reference. This test would show the analysis to be unsound if it were legal to offset a reference to one field into a pointer to another, disjoint field.

It is possible to relax the analysis so that it remains sound if this behavior became defined (see rust-lang/rust#65030).

[RalfJung]

See this discussion for a related example.

[comex]

Another example caught by miri. The offending code:

// self.storage: Box<[MaybeUninit<u8>]>
self.storage[self.cursor]
     .as_mut_ptr()
     .cast::<T>()
     .write_unaligned(component);

[retep998]

A common violation of this rule is FFI structs that end with a flexible array. For example:

#[repr(C)]
struct Foo {
    // other fields
    length: u32,
    data: [u8; 0],
}
// later in code
let foo: *mut Foo = /* obtained somehow */;
let data = slice::from_raw_parts((*foo).data.as_ptr(), (*foo).length);

This pattern is used in quite a few crates that deal with windows api, and some others that don't!

A few examples after a quick search:
https://github.com/kavorite/imgclip/blob/f3889e9da99fde01bd688e5ee2a18d89c823dee6/src/dib.rs#L53
https://github.com/rust-lang/backtrace-rs/blob/16682c76eb25df517e2cc220e56baf4f8a616f72/src/symbolize/dbghelp.rs#L165
https://github.com/klenin/spawner2/blob/0d461fd59e09fbcf9e863acf9a5db441da58b7a2/spawner/sys/windows/helpers.rs#L450
https://github.com/benfred/remoteprocess/blob/cdbf4aa23f48b48f949da3dadfc5878ab6e94f53/src/windows/symbolication.rs#L76
https://github.com/notify-rs/notify/blob/eed64ac9088ec1aab5c4710ef7232f0fbee49e0a/src/windows.rs#L328
https://github.com/CasualX/pelite/blob/29c4cac31c2ff4a7fd3d7953e9af8ea97c55423b/src/pe64/exception.rs#L180
https://github.com/snuk182/nuklear-rust/blob/1fb270a527ff048bb7e38dd88553c98987af233c/src/lib.rs#L5678
https://github.com/diwic/reffers-rs/blob/e6bdacafb96dae5dfa95a0830dda18fd64928691/src/rc.rs#L320
https://github.com/szymonwieloch/rust-dlopen/blob/26b70292744db1052403378d4a01075b6171e3d1/src/raw/windows.rs#L213
https://github.com/PyO3/pyo3/blob/956ed524122d27597888c28199053e6ba3f5289d/src/types/tuple.rs#L85
https://github.com/gluon-lang/gluon/blob/7b4cb090037d5ddf62dc35fde0b316689e5e4de3/vm/src/array.rs#L103
https://github.com/redsift/redbpf/blob/bea9eff8d2eedf0a57e90c8b7191a465cffdc56f/redbpf/src/load/map_io.rs#L70
https://github.com/servo/html5ever/blob/304c155bce19f956f0641cce3a1e19bc3b7bdaa1/zbuf/heap_data.rs#L178
https://github.com/arcnmx/ddcutil-rs/blob/3a631cdc92fcffcfc6e97fa29b4ed607fafc52d8/sys/src/lib.rs#L226

[RalfJung]

That is an interesting special case. It is actually less problematic than the other cases because there is no Rust reference pointing to that "extra" memory (unlike the example in the OP where val still "owns" that extra memory). I am not sure however how to incorporate that into Stacked Borrows...