[RFC] field projections v2 #3735

y86-dev · 2024-12-04T10:19:16Z

This is my second attempt at a field projection RFC, here is the first.

Add field projections, a very general operation. In simple terms, it is a new operator that turns a generic container type C<T> containing a struct T into a container C<F> where F is a field of the struct T. For example given the struct:

struct Foo {
    bar: i32,
}

One can project from &mut MaybeUninit<Foo> to &mut MaybeUninit<i32> by using the new field projection operator:

impl Foo {
    fn initialize(this: &mut MaybeUninit<Self>) {
        let bar: &mut MaybeUninit<i32> = this->bar;
        bar.write(42);
    }
}

Special cases of field projections are pin projections, or projecting raw pointers to fields *mut Foo to *mut i32 with improved syntax over &raw mut (*ptr).bar.

text/3735-field-projections.md

Nadrieril · 2024-12-04T12:15:57Z

text/3735-field-projections.md

+        // Here we use the special projections set up for `Mutex` with fields of type `Rcu<T>`.
+        let cfg: &Rcu<Box<BufferConfig>> = buf->cfg;


Could you show the shape of the impl needed here? I'd guess:

impl<F, T, U> Project<F> for Mutex<T> where F: Field<Base = T, Type = Rcu> { ... }

Doesn't that hit orphan impl limitations? Or is that your own mutex type maybe.

I will add it, it is our own mutex type, so we shouldn't hit orphan rules. Also, in RfL we want to relax orphan rules anyways, so for us it might not be an issue.

Originally, I intended to add the code, but I forgot. I now noticed an issue with my idea for how this would work, since in the meantime, I added impl<'a, T, F: Field<Base = T>> Project<F> for &'a T. That overlaps with the following:

impl<'a, T> Projectable for &'a Mutex<T> { type Inner = T; } impl<'a, T, F, U: 'a> Project<F> for &'a Mutex<T> where F: Field<Base = T, Type = Rcu> { type Output = &'a Rcu; fn project(self) -> Self::Output { todo!() } }

removing the blanket impl for all &'a T can't be done, because we need it to be able to project &UnsafeCell and other types with #[projecting].

this looks like something specialization might help with, but I am totally out of the loop wrt that, last time there wasn't much movement there, so I think we have to come up with something else.

one thing that we could do is the following:
give containers with #[projecting] the ability to select which fields are passed through, then for Mutex, we could only allow Rcu fields.
I haven't thought about this for long, maybe this is unsound in our case, so I'm not too inclined to jump on this, but maybe with more thought, it ends up a good idea...

Perhaps something like this would work?:

pub struct Mutex<T> { pub data: MutexData<T>, inner_mutex: InnerMutex, // the actual locking primitive } #[projecting] #[repr(transparent)] pub struct MutexData<T>(UnsafeCell<T>); // ... let cfg: &Rcu<Box<BufferConfig>> = buf->data->cfg;

(Maybe using #[projecting] on MutexData isn't quite accurate here; the point is, maybe somehow you could write custom projections for MutexData). Just one idea.

Edit: or maybe more along the lines of:

pub struct Mutex<T> { data: UnsafeCell<T>, inner_mutex: InnerMutex, // the actual locking primitive } #[projecting] #[repr(transparent)] pub struct MutexProjection<'a, T>(*const T, PhantomData<&'a UnsafeCell<T>>); impl<T> Mutex<T> { fn get_projection(&self) -> MutexProjection<'_, T> { MutexProjection(self.data.get() as *const T, PhantomData) } } // ... let cfg: &Rcu<Box<BufferConfig>> = buf.get_projection()->cfg;

I see, yeah having a separate reference type for a mutex that can be projected might be possible. But it also is a bit annoying...

(Note that the MutexProjection type that you defined should implement Projectable/Project instead of being #[projecting])

We could do like for Deref: it wouldn't be &T: ProjectMove, it would be &T: ProjectRef, which doesn't conflict with Mutex<T>: ProjectRef.

I'm confused, how exactly would that look like? Is ProjectMove the current version of Project and

pub trait ProjectRef<F: Field<Base = Self::Inner>>: Projectable { type Output<'a>; fn project(&self) -> Self::Output<'_>; }

and ProjectMut is the same, but with &mut self as the receiver?

What if &Foo implements ProjectMove and Foo implements ProjectRef? What does -> do in that case?

Hm, I didn't realize this would require a GAT. I haven't thought this through but I'd imagine when there's conflict we'd have to figure out a rule to infer the right one to use, like Deref vs DerefMut do today.

If you have a ProjectibleBase trait, then you could do impl<T> !ProjectibleBase for Mutex<T> [and the impl for refs in libcore should be impl<T: ProjectibleBase, F: Field> ProjectiblePointer<F> for &T)

~~Tho it would be annoying for the implementor of Mutex since they'll have to manually do field accesses with

fn inner_mutex(&self) -> &InnerMutex { field_of!(Self, inner).project_ref(self) } ```~~ ed: no, Projectible should only affect `operator->` not `operator.`

If you have a ProjectibleBase trait, then you could do impl<T> !ProjectibleBase for Mutex<T> [and the impl for refs in libcore should be impl<T: ProjectibleBase, F: Field> ProjectiblePointer<F> for &T)

But that requires negative impls :(

~~Tho it would be annoying for the implementor of Mutex since they'll have to manually do field accesses with

fn inner_mutex(&self) -> &InnerMutex { field_of!(Self, inner).project_ref(self) }

~~ ed: no, Projectible should only affect operator-> not operator.

Even if the solution is a bit more cumbersome for the implementer of the container, we would greatly appreciate it. Our own mutex type already has considerable unsafe code and we have the philosophy of improving driver developer ergonomics over our own library.

text/3735-field-projections.md

Nadrieril · 2024-12-04T12:38:47Z

text/3735-field-projections.md

+}
+```
+
+## Interactions


One question I didn't see mentioned: what about simultaneous projections? e.g.

#[derive(PinProject)] struct FairRaceFuture<F1, F2> { #[pin] fut1: F1, #[pin] fut2: F2, fair: bool, } let fut: Pin<&mut FairRaceFuture<_, _>> = ...; let fut1 = fut->fut1; let fut2 = fut->fut2;

Most projectable pointers would theoretically allow fut1 and fut2 to coexist. But that seems really hard to allow with user-provided types: fut should clearly not remain fully accessible while fut1 is live, yet passing it to a function like project likely invalidates fut1. We'd need project to take raw pointers, but then we can't have it for custom types.

Is there a design that could accomodate this in some way?

One question I didn't see mentioned: what about simultaneous projections? e.g.

That is indeed something that I should mention.

Most projectable pointers would theoretically allow fut1 and fut2 to coexist. But that seems really hard to allow with user-provided types: fut should clearly not remain fully accessible while fut1 is live, yet passing it to a function like project likely invalidates fut1. We'd need project to take raw pointers, but then we can't have it for custom types.

Is there a design that could accomodate this in some way?

In the Design Meeting on Field Projections, I brought this up, but only for the Pin case. There one needs some kind of "automatic reborrowing". But this still only allows consecutive uses of projected fields, not simultaneous usage.

For simultaneous usage, we would have to involve the borrow checker, and let the user choose a behavior (i.e. should it be like &T, &mut T, *mut T. maybe there are more?). That might be a good way to fix this, either through a trait or through macro intrinsics (that depends on how the borrow checker works internally).

borrow_check!(unsafe <'a, T> Pin<&'a mut T> as &'a mut T); // or unsafe impl<'a, T> BorrowCheck<&'a mut T> for Pin<&'a mut T> {}

Even if the borrow-checker understands, we need a design that allows for sound behavior: you can't alias a &mut, and reborrowing the whole &mut T would invalidate any existing references, right? E.g.

fn project1(pin: &mut Pin<&mut Struct>) -> Pin<&mut Field1> { ... } fn project2(pin: &mut Pin<&mut Struct>) -> Pin<&mut Field2> { ... } let mut pin = ...; let field1 = project_field1(&mut pin); // I think the second borrow of `pin` invalidates field1 regardless of what the borrow-checker says let field2 = project_field2(&mut pin);

Yes, that is correct, but if we do that with &mut Struct instead, the borrow checker knows what is going on:

let x: &mut Struct = ...; let field1 = &mut x.field1; let field2 = &mut x.field2;

We "just" have to make Pin<&mut T> & its projections behave in the same way as &mut T & normal field access for the borrow checker.

Agreed that we could tell the borrow-checker to consider fut->field1 and fut->field2 to be disjoint, but then we'd need the underlying memory model to allow this without UB. The problem we have is the project function call. As far as I know of the current state of discussion on this, passing a &mut Whatever to a function morally counts as an access, which invalidates any sibling references to that Whatever. I'll let the opsem experts clarify though.

This is similar to how we can't borrow two disjoint fields through a DerefMut (Box cheats on this btw).

Ah yes, that is a problem...

Maybe what we need is:

/// SAFETY: An impl is safe if calling `project` several times on the same `self` but disjoint fields and exposing the resulting outputs to safe code is safe. unsafe pub trait ProjectMut<F: Field<Base = Self::Inner>>: Projectable { type Output; /// SAFETY: The input must be cast from a safe `&mut Self`, and possibly passed to this `project` function with disjoint `F` fields and nothing else. unsafe fn project(*mut self) -> Self::Output; }

The safety requirements would need fleshing out to be rigorous but you get the idea: the borrow-checker would ensure we don't hold two projected pointers to the same field at the same time.

There'd also be a ProjectShared for the shared case, that allows multiple aliasing shared projections. And ProjectMove would ask the borrow-checker to track which fields have been moved out and to project+drop the non-moved fields at end of scope. This may run into issues similar to those with DerefMove, I don't recall exactly what blocked that one.

As far as I know of the current state of discussion on this, passing a &mut Whatever to a function morally counts as an access, which invalidates any sibling references to that Whatever. I'll let the opsem experts clarify though.

This is correct. We'd want this to be more like a MaybeDangling<&mut T>, and suppress all aliasing-related requirements temporarily, until we have the field and it can become an &mut F.

kennytm · 2024-12-04T14:49:59Z

text/3735-field-projections.md

+### Field Projection Operator
+
+Current favorite: `$base:expr->$field:ident`.
+
+Alternatives:
+- use `~` instead of `->`


for field: T, the expression base->field producing any type other than a place of type T would be very confusing for readers knowing C and C++.

I think someone coming from C might find &raw const base->field most familiar while still being in line with the difference between & and &raw const. I'm not sure that's much more ergonomic than &raw const (*base).field though (except that it could be safe which is still a huge improvement).

There are a lot of things in Rust that are very confusing for readers that know C and C++. I personally don't think this is a huge issue, because for pointer types the operation is very similar to "getting a place" and for types it is just a generalization.

That said, I don't really mind the exact syntax of the operator as long as it's short and conveys the correct meaning. What would be your suggestion for an alternative?
Do you want me to add anything to the RFC (e.g. add this as a drawback)?

yeah I guess adding &raw const really would only be immediately obvious for raw pointers anyway. I agree it should be concise and uniform across containers so I think -> might be the best option. If it's introduced in the context of another container like MaybeUninit or Pin -> shouldn't be an issue.

I would personally advocate ~ here, because it feels like it wouldn't have the C++ connotations of ->, and because it is more of a "navigate" operation than a "dereference" operation.

Both ~ and -> are fine with me. Just a note, in the previous RFC, it was argued that "[t]he ~ symbol was removed from Rust because it is hard to type on a number of keyboard layouts".

I don't buy that ~ is removed because of AltGr, the quoted French and German keyboard layouts showed []{}\| all involved AltGr yet we have no problem using any of these 6 symbols. Additionally there is that X: ~const Trait syntax. I think it's more that we don't want to waste the remaining unused symbols.

Additionally there is that X: ~const Trait syntax.

That's placeholder syntax, don't use it as precedent please.

I find ~ just looks very odd, so I personally slightly prefer ->. It does something with fields behind pointers, so IMO it's "close enough" to the C/C++ equivalent.

But in the end it's just syntax, I don't care very strongly and find it hard to predict how others will react.

I would personally advocate ~ here, because it feels like it wouldn't have the C++ connotations of ->, and because it is more of a "navigate" operation than a "dereference" operation.

I don't think the C++ bit holds when ~ is used for bitwise not and destructors, field projection seems more akin to -> member access than either of those :)

(but I agree with Ralf, slight preference for -> but it doesn't really matter)

- make from_pinned_ref `unsafe` - add safety documentation

zachs18 · 2024-12-04T18:37:35Z

text/3735-field-projections.md

+`ptr->flags: *mut u32`. Essentially `ptr->field` is a shorthand for `&raw mut (*ptr).field` (for
+`*const` the same is true except for the `mut`). However, there is a small difference between the
+two: the latter has to be `unsafe`, since `*ptr` requires that `ptr` be dereferencable. But field
+projection is a safe operation and thus it uses [`wrapping_add`] under the hood. This is less


I think it might be better for raw-pointer(-like) field projection to be unsafe (like the "desugared" version), and require the at least the specific offset to be in-bounds (if not the whole container).

In particular, NonNull having field projection be safe would mean that it would have to panic (or something) if the projection would result in null.

It seems like none of the motivating examples would benefit from allowing ptr->field where field is not inbounds? Maybe there are usecases for this but I haven't thought of any where it wouldn't be clearer to explicitly use wrapping_add IMO.

I think it might be better for raw-pointer(-like) field projection to be unsafe (like the "desugared" version), and require the at least the specific offset to be in-bounds (if not the whole container).

In particular, NonNull having field projection be safe would mean that it would have to panic (or something) if the projection would result in null.

Note that wrapping_add is itself a safe function. So the implementation of NonNull::project only contains safe code, making UB impossible. Also, there is no need for a panic, since a projection can never result in null.

It seems like none of the motivating examples would benefit from allowing ptr->field where field is not inbounds? Maybe there are usecases for this but I haven't thought of any where it wouldn't be clearer to explicitly use wrapping_add IMO.

The reason for why I used wrapping_add is that the following code is UB when the -> operation uses add. Making the operation unsafe for raw pointers is probably impossible, since it is backed by the Project trait and the project function can either be unsafe or safe.
Maybe we can split it into two different mutually exclusive traits, but I think it would be better both for consistency and for ergonomic reasons to keep field projections safe.

Note that wrapping_add is itself a safe function. So the implementation of NonNull::project only contains safe code, making UB impossible. Also, there is no need for a panic, since a projection can never result in null.

NonNull::wrapping_add does not exist, for the reason I mentioned: offsetting a NonNull can result in null.

For a concrete example: What is the behavior of this safe code with your proposed semantics?

use std::{ptr::NonNull, mem::offset_of}; #[repr(C)] struct Foo { x: u8, y: u8, } let ptr: NonNull<Foo> = NonNull::new(usize::MAX as *mut Foo); let field: NonNull<u8> = ptr->y; // either this panics or returns the "wrong" pointer, or we have UB somewhere assert_eq!(ptr.as_ptr().addr(), usize::MAX); assert_eq!(offset_of!(Foo, y), 1); // Foo is repr(C) assert_eq!(usize::MAX.wrapping_add(1), 0); // the expected address of (*ptr).y is 0, which is not a valid NonNull assert_eq!(field.as_ptr().addr(), 0); // NonNull with addr 0?

The reason for why I used wrapping_add is that the following code is UB when the -> operation uses add.

Yes, offsetting outside of an allocation using field projections is what I am saying I think should be UB. Also, what "following code" are you referring to here? The next code snippet after this line of the file is the set_flags_raw example, which unsafely writes to the projected pointer anyway and does not discuss how making the projection safe reduces UB (though it could in some edge cases; I just don't think they are worth it).

Maybe we can split it into two different mutually exclusive traits, but I think it would be better both for consistency and for ergonomic reasons to keep field projections safe.

If we want this operator to be able to be used generically, and not just for concrete pointer types, we could have an UnsafeFoo trait be a supertrait of the Foo trait, such that any Foo is also usable in a where T: UnsafeFoo context (perhaps with a blanket impl<T: ?Sized + Foo> UnsafeFoo for T { ... } so users don't have to implement UnsafeFoo for their Foo types).

Note that wrapping_add is itself a safe function. So the implementation of NonNull::project only contains safe code, making UB impossible. Also, there is no need for a panic, since a projection can never result in null.

NonNull::wrapping_add does not exist, for the reason I mentioned: offsetting a NonNull can result in null.

Oh, totally overlooked that. Yeah that seems bad...

For a concrete example: What is the behavior of this safe code with your proposed semantics?

Yeah that doesn't really work...

The reason for why I used wrapping_add is that the following code is UB when the -> operation uses add.

Yes, offsetting outside of an allocation using field projections is what I am saying I think should be UB.

Agreed.

Also, what "following code" are you referring to here?

Oops, I meant to add some code, but the point is moot now, since I didn't understand what you meant before.

The next code snippet after this line of the file is the set_flags_raw example, which unsafely writes to the projected pointer anyway and does not discuss how making the projection safe reduces UB (though it could in some edge cases; I just don't think they are worth it).

Maybe we can split it into two different mutually exclusive traits, but I think it would be better both for consistency and for ergonomic reasons to keep field projections safe.

If we want this operator to be able to be used generically, and not just for concrete pointer types, we could have an UnsafeFoo trait be a supertrait of the Foo trait, such that any Foo is also usable in a where T: UnsafeFoo context (perhaps with a blanket impl<T: ?Sized + Foo> UnsafeFoo for T { ... } so users don't have to implement UnsafeFoo for their Foo types).

Yes, we could have:

pub trait UnsafeProject<F>: Projectable where F: Field<Base = Self::Inner>, { type Output; unsafe fn project(self) -> Self::Output; } impl<P, F> UnsafeProject<F> for P where P: Projectable, F: Field<Base = ::Inner>, P: Project<F>, { type Output = >::Output; unsafe fn project(self) -> Self::Output { >::project(self) } }

But I don't know if it is possible for the compiler to detect which trait is implemented.

But I don't know if it is possible for the compiler to detect which trait is implemented.

The compiler could just always use Project when available, and fallback to UnsafeProject otherwise.

This is similar how the compiler is able to choose which of Fn/FnMut/FnOnce when desugaring call to an unboxed closure.

But I'm not sure if the compiler also required a hierarchical relationship similar to the Fn traits to make this work.

pub trait UnsafeProject<F>: Projectable where F: Field<Base = Self::Inner>, { type Output; unsafe fn project_unchecked(self) -> Self::Output; } pub trait Project<F>: UnsafeProject<F> where F: Field<Base = Self::Inner>, { fn project(self) -> Self::Output; }

Makes sense. We also have to coordinate this with possibly ProjectRef/ProjectMove... Seems like the combinatorial explosion is going to be bad...

Coder-256 · 2024-12-04T18:57:59Z

What is the implementation of Projectable for Cow<'_, T>?

y86-dev · 2024-12-04T19:20:51Z

@Coder-256

What is the implementation of Projectable for Cow<'_, T>?

added the implementation to the RFC

Coder-256 · 2024-12-04T19:24:50Z

I see, I'm still a bit confused. The value in Cow::Owned has type <T as ToOwned>::Owned, which is not always the same as T. For example, this wouldn't work for Cow<str> I think. Maybe you just need a T: Clone bound though?

Edit: Cow<str> is a bad example obviously (it has no fields), but you can also implement ToOwned on a non-Clone struct and choose a custom type for Owned.

text/3735-field-projections.md

joshtriplett · 2024-12-04T19:37:15Z

text/3735-field-projections.md

+
+Alternatives:
+- Introduce a more native syntax on the level of types `$Container:ty->$field:ident` akin to
+  projecting an expression.


👍 for having native syntax for fields of a type.

I think a natural syntax would be to lift the expression version to types. So if we go with ~, then we would have both a version for types and for expressions:

struct Foo { bar: i32, } type Foo_bar_field = Foo~bar; let x: &UnsafeCell<Foo> = todo!(); let bar: &UnsafeCell<i32> = x~bar;

y86-dev · 2024-12-04T19:37:54Z

I see, I'm still a bit confused. The value in Cow::Owned has type <T as ToOwned>::Owned, which is not always the same as T. For example, this wouldn't work for Cow<str> I think. Maybe you just need a T: Clone bound though?

Edit: Cow<str> is a bad example obviously (it has no fields), but you can also implement ToOwned on a non-Clone struct and choose a custom type for Owned.

Oh, I totally forgot about ToOwned... I fixed it for now by only providing it if the owned type is the same as the borrowed type.

joshtriplett · 2024-12-04T19:41:42Z

Could you please add a section discussing how we can handle types where the projection might have a different type than the original?

For instance, due to the in-memory layout, we can't turn an Arc<StructType> into an Arc<FieldType>, but we could have some kind of ArcRef type that references an external reference count, and projection could work for that type. So at a minimum we should document how we can handle things like that. But I'm also wondering if we can have some kind of associated type that would allow type-changing field projections:

let x: Arc<StructType> = Arc::new(StructType { field: "a string".to_string() });
let y: ArcRef<String> = x~field;

That would be nicer than having to do something like x.to_arc_ref()~field.

y86-dev · 2024-12-04T20:31:44Z

@joshtriplett I added ArcRef in future possibilities. The current proposal already supports changing the type when projecting, so there is no need to do x.to_arc_ref()->field (I'm continuing to use -> until we change it in the RFC :)

text/3735-field-projections.md

…ll::Ref[Mut]<T>` to reference level-explanation

…tainer types

programmerjake · 2024-12-06T11:23:49Z

text/3735-field-projections.md

+```
+
+Then there are two field types for `Opaque<Foo>`:
+- one representing `bar` with `Base = Opaque<Foo>` and `Type = Opaque<i32>`


why not just use the field types for Foo directly, so the field type for bar impls Field<Base = Foo, Type = i32>?

maybe change the design like the following, so you don't need a magic #[projecting] attribute:

pub unsafe trait FieldType: Copy + Send + Sync + Default + ... {} // maybe omit /// Self is what used to be `Field::Base` pub unsafe trait HasUnalignedField<F: FieldType> { // maybe leave `Has` out of name? type Type: ?Sized; const OFFSET: usize; } pub unsafe trait HasField<F: FieldType>: HasUnalignedField<F> {} pub trait Projectable: Sized { type Inner: ?Sized; } // no Self::Inner: HasUnalignedField<F> bound for // future compatibility with enum variant fields or union fields pub trait Project<F>: Projectable { type Output; // pass in `field` value in case rust gains // fields that need runtime computation // to get the offset (e.g. fields of scalable vector types), // or things like C++ member pointers, for now it's always a ZST fn project(self, field: F) -> Self::Output; } unsafe impl<F: FieldType, T: HasUnalignedField<F, Type: Sized>> HasUnalignedField<F> for Cell<T> { type Type = Cell<<T as HasUnalignedField<F>>::Type>; const OFFSET: usize = <T as HasUnalignedField<F>>::OFFSET; } unsafe impl<F: FieldType, T: HasField<F, Type: Sized>> HasField<F> for Cell<T> {}

this allows implementing HasUnalignedField/HasField multiple times for the same field type, it also could allow container-like types that have other non-ZST fields:

struct MyWrapper<T> { other_field: u32, field: T, } unsafe impl<F: FieldType, T: HasUnalignedField<F, Type: Sized>> HasUnalignedField<F> for MyWrapper<T> { type Type = <T as HasUnalignedField<F>>::Type; const OFFSET: usize = <T as HasUnalignedField<F>>::OFFSET + offset_of!(Self, field); } unsafe impl<F: FieldType, T: HasField<F, Type: Sized>> HasField<F> for MyWrapper<T> {}

it also allows you to have a HashMap where the key is the TypeId of the field type and it works even if you access fields of MyStruct through SomeRef<MyStruct> or SomeOtherType<MyStruct>, since in both cases the field types are the same types -- the field types of MyStruct...

Using something like that HashMap is how I would probably implement field access for fayalite's Expr<MyStruct> if I were redesigning the API to use the projection operator introduced by this RFC.

That looks very interesting!

How does the compiler find the right field type to return via field_of! though? Let's say I have the MyWrapper struct from above (it should probably be #[repr(C)]) as well as Foo:

struct Foo { other_field: i32, field: u32, } fn demo(x: &MyWrapper<Foo>) { let field = x->field; // which `field` is meant by this? let other_field = x->other_field; // same here }

How does the compiler find the right field type to return via field_of! though? Let's say I have the MyWrapper struct from above (it should probably be #[repr(C)]) as well as Foo:

#[repr(C)] is unnecessary since we're using offset_of!. I didn't think of finding the right inner type...maybe we need MyWrapper<T>: Projectable<Inner = T> and the compiler follows the chain of Projectable impls kinda like deref?

struct Foo { other_field: i32, field: u32, } fn demo(x: &MyWrapper<Foo>) { let field = x->field; // which `field` is meant by this? let other_field = x->other_field; // same here }

assuming MyWrapper<Foo>: Projectable, since MyWrapper opted-in to being projectable, those would resolve to Foo's fields. alternatively they could be ambiguous.

But x->field currently would desugar as:

Project::<field_of!(<typeof(x) as Projectable>::Inner, field)>::project(x, /* I don't know how we would get the value here, but at the moment its a ZST, so it doesn't matter */)

So in my example, since typeof(x) == &MyWrapper<Foo>, we would get the expression to be:

Project::<field_of!(MyWrapper<Foo>, field)>::project(x, /* ... */)

At least that's my reason for creating the Projectable trait. We could have a different trait (or an attribute) that would disable the creation of field types for a certain type (or maybe for individual fields?).

I think it is really important that the field projection expression x->field has at most one field candidate.

I'm proposing the desugaring would change so a->b desugars to something like:

type Base<...> = { let mut base: type = typeof(a); // only true if the impl is visible, it can be hidden due to generics or impl Trait while base: Projectable { base = base::Inner; } base }; a.project(field_of!(Base, b) {})

or maybe a better option is following deref coercion semantics where it stops at the first type with that field where the field is visible (due to pub).

That sounds reasonable, though might lead to people having to do some visibility gymnastics:

mod my_wrapper_def { pub struct MyWrapper<T> { other_field: u32, field: T, } /* inherit all fields of `field: T`... */ } pub use my_wrapper_def::MyWrapper; pub mod foo_def { use super::*; pub struct Foo { bar: i32, baz: u32 } impl MyWrapper<Foo> { pub fn set_bar(this: &mut MaybeUninit<Wrapper<Foo>>, value: i32) { let bar: &mut MaybeUninit<i32> = this->bar; bar.write(value); } } }

Which probably is rather annoying... So while we should use your suggested algorithm, I think it's probably still very useful to not generate field types for certain fields via an attribute:

pub struct MyWrapper<T> { #[no_field_type_bikeshed] other_field: u32, #[no_field_type_bikeshed] field: T, } /* inherit all fields of `field: T`... */ pub struct Foo { bar: i32, baz: u32, } impl MyWrapper<Foo> { pub fn set_bar(this: &mut MaybeUninit<Wrapper<Foo>>, value: i32) { this->bar.write(value); } }

text/3735-field-projections.md

RalfJung · 2024-12-06T19:26:10Z

text/3735-field-projections.md

+            first: self.first.project(),
+            second: self.second.project(),


How does this work, how does it know which field to project them to?
Is this Project::<F>::project(self.first)? (Not sure if there's a nicer way to fix the F here. Maybe self.first->F should work. ;)

I expect this works the same way you can just call self.my_field.fmt(f) in a Debug impl even though that's not the only trait with a fmt function that may be in scope -- it's the only impl in the where bounds for the generic arguments.

Exactly, I was just hoping that the type inference would be good enough to achieve this, I didn't test it though.

RalfJung · 2024-12-06T19:39:17Z

I didn't expect a solution for Cell projections, Pin projections, and improved raw ptr ergonomics in a single RFC. This is a nice early Christmas gift. :)

tgross35 · 2024-12-06T22:06:20Z

text/3735-field-projections.md

+- [field types],
+  - `Field` traits,
+  - `field_of!` macro,
+  - [`#[projecting]`](#projecting-attribute) attribute
+- projection operator `->`,
+  - `Project` trait,
+  - `Projectable` trait,


I'm trying to figure out what will eventually need to be stable so field projection can be used on library types, vs. what will need to be stable to make custom types field projectable. Is this list accurate?

With access to ->, a user can make use of basic field projection on library types and pointers

With access to #[projecting] and/or the ability to implement Project and Projectable, a user can make their own types field projectable

With access to the Field trait members (but not necessarily ability to implement) and field_of!, a user can express field projection in function signatures

Field and UnalignedField may never be manually implemented, they are compiler-generated for any field that is accessed by field_of!

I'm trying to figure out what will eventually need to be stable so field projection can be used on library types, vs. what will need to be stable to make custom types field projectable. Is this list accurate?

It is, but one can add additional caveats/information:

With access to ->, a user can make use of basic field projection on library types and pointers

-> also gives access to projections for custom types (if we depend on another crate using unstable features to implement custom projections).

With access to #[projecting] and/or the ability to implement Project and Projectable, a user can make their own types field projectable

Yes, but often one needs the information contained in the Field trait members, since how else can one project to a field? (of course if the stdlib already provides the projection for raw pointers and one has a raw pointer then it's fine)

With access to the Field trait members (but not necessarily ability to implement) and field_of!, a user can express field projection in function signatures

One only needs the Field trait to express field projection in function signatures. Of course to call them, one needs the field_of! macro (or the syntax).

Field and UnalignedField may never be manually implemented, they are compiler-generated for any field that is accessed by field_of!

Yes, that is the current idea, but for some reasons we might want to lift that restriction, see Field trait stability

i think an enum variant field implementing something other than Field would work well. (unless the enum has only 1 variant, in which case it's effectively both a struct and an enum.)

I'm not sure I understand correctly, what are you replying to in this thread? Do you mean as a way to implement enum projections?

yes, i meant enum projections and that for those, trait Field isn't used so at least in this aspect there isn't a problem with users implementing Field since it won't need to change for enums.

text/3735-field-projections.md

tmccombs · 2024-12-07T03:27:33Z

Would it make sense to implement Project for types like Option and maybe Result with something like

impl<T: Projectsble> Projectable for Option<T> {
    type Inner =  <T as Projectable>::Inner;
}

impl<T, F> Project<F> for Option< T>
where
    F: Field<Base = Self::Inner>,
    T: Project<F>,
{
    type Output = Option<F::Type>;

    fn project(self) -> Self::Output {
        match self {
          Some(v) => Some(v.project())
          None => None
    }
}

Unfortunately, I don't see how to implement it soundly for all T, since doing so would mean moving the field, so you would need an Option of some reference type.

It could also be implemented for all T if F::Type: Copy, but you can't have both, because of coherence, and I think having it for options of reference types is more useful.

kennytm · 2024-12-07T06:33:07Z

@tmccombs this has the same issue as applying Projectable on Cow<T>, which is that Option<T> is a container of T rather than a pointer to T, meaning accessing its field would be necessarily destructive. Supporting these requires "MoveableField" discussed at the end of the RFC.

If Projectable does not need to be a pointer, the distinction between it and #[projecting] would be blurred, and it raises the question again that why not implement Projectable for #[projecting] containers.

tmccombs · 2024-12-07T07:41:04Z

accessing its field would be necessarily destructive.

Not if it is projecting through another pointer type, so for some struct Foo, Moveable field would be necessary for Option<Foo> but not for, say Option<&Foo>.

And even if there is a MoveableField, then using that would mean you can't project an Option<&T> for a non-moveable field.

But perhaps it would be more useful if you could project on an &Option or &mut Option. Unfortunately, implementing Project on those would conflict with the impl for &T and &mut T, but perhaps there is a way that #[projecting] could work on enums? I think that might be what the generalized enum projection section is talking about ?

y86-dev · 2024-12-07T10:07:28Z

accessing its field would be necessarily destructive.

Not if it is projecting through another pointer type, so for some struct Foo, Moveable field would be necessary for Option<Foo> but not for, say Option<&Foo>.

Yes, for Option where P is itself a pointer type, this works. I actually think your implementation from above makes sense:

impl<T: Projectsble> Projectable for Option<T> {
    type Inner =  <T as Projectable>::Inner;
}

impl<T, F> Project<F> for Option< T>
where
    F: Field<Base = Self::Inner>,
    T: Project<F>,
{
    type Output = Option<F::Type>;

    fn project(self) -> Self::Output {
        match self {
          Some(v) => Some(v.project())
          None => None
    }
}

Project is implemented for pointer-like types, so adding this impl would allow projecting all sorts of things (for example Option<&mut MaybeUninit<T>> or even multiple nested stuff Option<&mut UnsafeCell<MaybeUninit<T>>).

And even if there is a MoveableField, then using that would mean you can't project an Option<&T> for a non-moveable field.

But perhaps it would be more useful if you could project on an &Option or &mut Option. Unfortunately, implementing Project on those would conflict with the impl for &T and &mut T, but perhaps there is a way that #[projecting] could work on enums? I think that might be what the generalized enum projection section is talking about ?

I think that projecting through &Option<T> is not possible, since the memory layout of Option<T> is not the same as T, thus a value of type Option<T> does not "contain" a field of type Option<F> if F is a field of T.

The section on enum projection is about enabling to use enums instead of structs: with the current proposal, one can project Pin<&mut T> to Pin<&mut F> if F is a structurally pinned field of T, but if T is instead an enum with a single variant and a single structurally pinned field, the same is not possible. The section on enum projections is about allowing that. Not about enabling projections for container types that enums, since for them, adding #[projecting] is impossible for the above reason and Project can be implemented for any type, so they are already supported.

y86-dev · 2024-12-08T22:13:11Z

text/3735-field-projections.md

+## Impact of this Feature
+
+Overall this feature improves readability of code, because it replaces more complex to parse syntax
+with simpler syntax:
+- `&raw mut (*ptr).foo` is turned into `ptr->foo`
+- using `NonNull<T>` as a replacement for `*mut T` becomes a lot better when accessing fields,
+
+There is of course a cost associated with introducing a new operator along with the concept of field
+projections.


One problem that I find very difficult to address is that of how hard this feature is to understand. Since I have been thinking about this feature a lot (and seeing its uses out in the wild for even longer) I feel like I have lost the ability to accurately gauge its difficulty. I also have no idea of any particular hurdles that people come across when learning about field projections for the first time.

This also brings me to the question of how we teach this. I tried to write the guide-level explanation in a rust-book chapter-style, but I don't know if it even reaches that standard, or if we even would like to talk about field projections there.

seems simple enough to me, then again I'm already familiar with C++ member pointers, so I probably won't be much help on teachability aspects...

Is this the same concept as the c++ member pointers?

text/3735-field-projections.md

tamird · 2024-12-16T18:28:07Z

text/3735-field-projections.md

+# Summary
+[summary]: #summary
+
+Field projections are a very general concept. In simple terms, it is a new operator that turns a


This is a bit of a nit: in the context of this RFC the field projection operation is only applicable to reference-like containers. For example consider the case of Arc<T>: suppose I have some code that wants an Arc<F>, and then I refactor things such that I now have an Arc<T> instead, where T has a field of type F; this proposal does not cover the case of owned value projection - I cannot obtain an Arc<F> from my Arc<T>. Right?

There's some prior art for this kind of projection: https://docs.rs/yoke/latest/yoke/.

This is a bit of a nit: in the context of this RFC the field projection operation is only applicable to reference-like containers. For example consider the case of Arc<T>: suppose I have some code that wants an Arc<F>, and then I refactor things such that I now have an Arc<T> instead, where T has a field of type F; this proposal does not cover the case of owned value projection - I cannot obtain an Arc<F> from my Arc<T>. Right?

Nothing prevents us from implementing Project<F> for Arc<T>, but I think it'd be a footgun, since you have to construct an entirely new Arc<F::Type>. Since an Arc<T> is essentially just a NonNull<ArcInner<T>>, we can just look at the layout of that:

#[repr(C)] struct Foo { a: usize, b: usize, } // layout of `Foo`: // // |a-------|b-------| // layout of `ArcInner<Foo>` // // |strong--|weak----|data.a--|data.b--| // layout of `ArcInner<usize>` // // |strong--|weak----|value---|

The problem is now that we have to create a new ArcInner<usize> on the heap, copy the value from Foo and then drop the old Arc. There are two reasons:

if the field is not the first one, we have to move it, since the ArcInner expects the value directly after the weak refcount

in order for Arc to reliably free the memory it has allocated, it needs to know the size of the data. If we were to just transmute NonNull<ArcInner<Foo>> to NonNull<ArcInner<usize>> when projecting to the field a, then the free function of the allocator could be called with the wrong layout.

Overall, I feel like this is a rather complex operation (and I haven't really used it in code). So it shouldn't be hidden away behind an operator, instead a method taking a generic parameter that is a field type would be a much better fit in my opinion.

Do you think that the RFC is unclear w.r.t. this? If yes, then I'll gladly add the above to it.

There's some prior art for this kind of projection: https://docs.rs/yoke/latest/yoke/.

Oh yeah, I forgot about that crate! But I think that it also can only provide projections for the yoke type. So if you have Yoke<Bar<'static>, Rc<[u8]>> then you can project to its fields (considering the example from the docs), but the cart type Rc<[u8]> will be unaffected.

Thanks for the thorough answer! It all makes sense to me. It might be good to include a section explaining why such transmutation is not possible in the general case. Otherwise the phrasing implies too much generality.

Oh one thing that I forgot to say was that if we change how Arc is implemented, then the projection you suggested above is possible (so if we have two pointers, one to the refcounters and another to the value). The RFC already talks about this in the ArcRef section.

But I would argue that Arc<T>/ArcRef<T> is a "reference-like container" type (taking your wording here, I would rather use the term "pointer-like type"), since it essentially is just a NonNull<ArcInner<T>>.

Yep, makes sense - I don't think folks will be overly keen on changing how Arc is implemented since it increases the allocation cost.

Arc<T> is a "reference-like container" but its reference is to an ArcInner<T> which is not reference-like because it holds both the count and the allocation in-line. My suggested wording is probably not enough to clarify the point, but I still think the wording would benefit from some clarification, specifically that intrusive containers (e.g. ArcInner<T> - again please use better language if you have it) aren't appropriate for field projection.

add field_projection v2 rfc

8d2aaac

y86-dev force-pushed the field-projection-v2 branch from c60c1bc to 8d2aaac Compare December 4, 2024 10:19

y86-dev mentioned this pull request Dec 4, 2024

RFC: Field projection #3318

Closed

2 tasks

bjorn3 reviewed Dec 4, 2024

View reviewed changes

text/3735-field-projections.md Outdated Show resolved Hide resolved

ojeda mentioned this pull request Dec 4, 2024

Rust wanted features Rust-for-Linux/linux#354

Open

39 tasks

Nadrieril reviewed Dec 4, 2024

View reviewed changes

text/3735-field-projections.md Outdated Show resolved Hide resolved

Nadrieril reviewed Dec 4, 2024

View reviewed changes

text/3735-field-projections.md Outdated Show resolved Hide resolved

Nadrieril reviewed Dec 4, 2024

View reviewed changes

text/3735-field-projections.md Show resolved Hide resolved

Nadrieril reviewed Dec 4, 2024

View reviewed changes

y86-dev added 2 commits December 4, 2024 14:17

fix typos

a3a7749

add attibution for enum projections

bf0d342

ehuss added the T-lang Relevant to the language team, which will review and decide on the RFC. label Dec 4, 2024

kennytm reviewed Dec 4, 2024

View reviewed changes

y86-dev added 2 commits December 4, 2024 18:35

fix PinField trait

ae64e46

- make from_pinned_ref `unsafe` - add safety documentation

add prior art: design meeting field projection

7cdee51

zachs18 reviewed Dec 4, 2024

View reviewed changes

add impl of Projectable and Project for &T and Cow<'_, T>

fb938d5

joshtriplett reviewed Dec 4, 2024

View reviewed changes

text/3735-field-projections.md Show resolved Hide resolved

fix Project impl for Cow<'_, T>

27d5146

joshtriplett reviewed Dec 4, 2024

View reviewed changes

y86-dev added 2 commits December 4, 2024 20:56

add alternative operator syntax ~

9b5b3f2

add ArcRef<T> future possibility

8142a78

programmerjake reviewed Dec 4, 2024

View reviewed changes

text/3735-field-projections.md Outdated Show resolved Hide resolved

y86-dev added 10 commits December 5, 2024 23:22

add UnalignedField trait

c250db0

add Mapped{Mutex,RwLock{Read,Write}}Guard<T> to projections and `ce…

6a7149f

…ll::Ref[Mut]<T>` to reference level-explanation

fix typos

84e53e4

always require a generic for #[projecting] and allow zero-sized con…

491d823

…tainer types

show that #[projecting] still allows other non-zero-sized fields

4de45c8

text reflow

358de37

add sized caveat for field types

dae2b04

add guide-level explanation of implementing custom projections

fc6279d

remove using repr as an alternative to #[projecting]

6282d95

add prior art: Field Projection Use Cases document

612c910

programmerjake reviewed Dec 6, 2024

View reviewed changes

kennytm reviewed Dec 6, 2024

View reviewed changes

text/3735-field-projections.md Outdated Show resolved Hide resolved

make code more concise

04b5193

RalfJung reviewed Dec 6, 2024

View reviewed changes

tgross35 reviewed Dec 6, 2024

View reviewed changes

text/3735-field-projections.md Show resolved Hide resolved

This comment was marked as off-topic.

Sign in to view

y86-dev added 2 commits December 7, 2024 20:33

expand text on tuple support

7df5239

add future possibility Option stdlib

7d50503

y86-dev commented Dec 8, 2024

View reviewed changes

y86-dev mentioned this pull request Dec 9, 2024

[2025H1] add field projection project goal rust-lang/rust-project-goals#181

Merged

sergiocollado reviewed Dec 15, 2024

View reviewed changes

text/3735-field-projections.md Outdated Show resolved Hide resolved

fix typos

3291759

tamird reviewed Dec 16, 2024

View reviewed changes

		// Here we use the special projections set up for `Mutex` with fields of type `Rcu<T>`.
		let cfg: &Rcu<Box<BufferConfig>> = buf->cfg;

[RFC] field projections v2 #3735

Are you sure you want to change the base?

[RFC] field projections v2 #3735

Conversation

y86-dev commented Dec 4, 2024 • edited by rustbot Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Coder-256 Dec 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arielb1 Dec 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nadrieril Dec 5, 2024 • edited Loading

Choose a reason for hiding this comment

y86-dev Dec 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nadrieril Dec 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tgross35 Dec 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kennytm Dec 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Coder-256 commented Dec 4, 2024

y86-dev commented Dec 4, 2024

Coder-256 commented Dec 4, 2024 • edited Loading

joshtriplett Dec 4, 2024 • edited Loading

Choose a reason for hiding this comment

y86-dev Dec 5, 2024 • edited Loading

Choose a reason for hiding this comment

y86-dev commented Dec 4, 2024

joshtriplett commented Dec 4, 2024 • edited Loading

y86-dev commented Dec 4, 2024

programmerjake Dec 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

y86-dev Dec 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RalfJung commented Dec 6, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

programmerjake Dec 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tmccombs commented Dec 7, 2024

kennytm commented Dec 7, 2024

tmccombs commented Dec 7, 2024

y86-dev commented Dec 7, 2024

This comment was marked as off-topic.

y86-dev commented Dec 4, 2024 •

edited by rustbot

Loading

Coder-256 Dec 4, 2024 •

edited

Loading

arielb1 Dec 16, 2024 •

edited

Loading

Nadrieril Dec 5, 2024 •

edited

Loading

y86-dev Dec 5, 2024 •

edited

Loading

Nadrieril Dec 5, 2024 •

edited

Loading

tgross35 Dec 6, 2024 •

edited

Loading

kennytm Dec 5, 2024 •

edited

Loading

Coder-256 commented Dec 4, 2024 •

edited

Loading

joshtriplett Dec 4, 2024 •

edited

Loading

y86-dev Dec 5, 2024 •

edited

Loading

joshtriplett commented Dec 4, 2024 •

edited

Loading

programmerjake Dec 6, 2024 •

edited

Loading

y86-dev Dec 6, 2024 •

edited

Loading

programmerjake Dec 8, 2024 •

edited

Loading