[RFC] Static Function Argument Unpacking

[miikkas]

Summary

This RFC adds call-site unpacking of tuples, tuple structs, and fixed-size arrays, using ...expr within the function call's parentheses as a shorthand for passing arguments. The full contents of these collections with known sizes are unpacked directly as the next arguments of a function call, desugaring into the corresponding element accesses during compilation.

---

Rendered

[tgross35]

Earlier it is stated that "Tuples, tuple structs, and fixed-size arrays can be unpacked.", but this seems to indicate that anything implementing Index<Int> could be unpacked. I think tuples definitely make sense and fixed-length [T; N] arrays could make sense, but anything involving Index should be out of scope for now because it turns the complexity up quite a bit.

It is pretty easy to turn slicelike collections into fixed-length arrays anyway, the user can handle that if they need this support.

[tgross35]

On a similar note, it is probably worth mentioning type inference. For example:

fn foo(a: u8, b: u8, c: u8, d: u8, e: u8) { /* ... */ }

fn bar(buf: &[u8]) {
    foo(...buf.try_into().unwrap());
}

Is that expected to work because &[T; 5] has TryFrom<&[T]> and u8 is Copy? What about

fn bar(buf: &[u8], other: &[u8]) {
    foo(1, ...buf.try_into().unwrap(), ...other.try_into().unwrap());
}

That probably needs to be forbidden as ambiguous, but that is something that should be spelled out here with some T-Types input.

[miikkas]

Good catch – I seem to have written this part about accessing the fields in an ambiguous and self-contradictory way. I'll need to make some changes and also study the problem a bit more.

On the whole, I'm aiming at a design that's infallible during compilation. But I think I'll need to look into if that claim can actually be made after all, i.e. if some seemingly infallible cases wouldn't be totally infallible, but only just as infallible as they currently are. (Leading into a runtime panic upon failure?)

[miikkas]

On a similar note, it is probably worth mentioning type inference. For example:

fn foo(a: u8, b: u8, c: u8, d: u8, e: u8) { /* ... */ }

fn bar(buf: &[u8]) {
    foo(...buf.try_into().unwrap());
}

Is that expected to work because &[T; 5] has TryFrom<&[T]> and u8 is Copy? What about

fn bar(buf: &[u8], other: &[u8]) {
    foo(1, ...buf.try_into().unwrap(), ...other.try_into().unwrap());
}

That probably needs to be forbidden as ambiguous, but that is something that should be spelled out here with some T-Types input.

I'll definitely need to add a subchapter on type inference next to the referred "Type Coercions of Collections" subchapter. 👍🏼

[miikkas]

Earlier it is stated that "Tuples, tuple structs, and fixed-size arrays can be unpacked.", but this seems to indicate that anything implementing Index<Int> could be unpacked. I think tuples definitely make sense and fixed-length [T; N] arrays could make sense, but anything involving Index should be out of scope for now because it turns the complexity up quite a bit.

It is pretty easy to turn slicelike collections into fixed-length arrays anyway, the user can handle that if they need this support.

I've edited this subchapter (now named "Type Coercions and Conversions of Collections") in commit fa28f69 to address this. Basically, I think the original writing was erroneous, contradicting with other parts of the text, so thanks for bringing this up. This is also one of the sections (and topics) I'm most unsure about, so please don't hesitate to point out any remaining mistakes. :)

[programmerjake]

If this RFC is intended to help push variadic generics along, we need to allow unpacking zero-element things, since it is necessary for stuff like defining variadic Fn:

struct CallLocked<F>(Mutex<F>);

// this should work for F: FnMut() -> u8 and that needs unpacking zero-element things
impl<F: FnMut(...Args) -> R, ...Args, R> Fn(...Args) -> R for CallLocked<F> {
    fn call(&self, ...args: ...Args) -> R {
        self.0.lock().unwrap()(...args)
    }
}

[scottmcm]

I'm a big fan of not special-casing zero things in the language semantics. That seems especially important for being able to unpack arrays of const-generic length.

(We should probably lint on sketchy things like let x = [...v.clear()];, like we lint on let x = v.clear(); -- or clippy::let_unit_value does, at least -- but the language should allow it for all lengths.)

[miikkas]

Noted, these arguments sound reasonable to me. Initially, I leaned towards starting with a strict design, since it's easier to relax rather than add constraints later on, but that's because I couldn't come up with a reason why zero-length collections could ever be unpacked. :)

In light of this, I'll make the following two changes to the RFC:

1. Zero-length collection won't be errors anymore. Instead, in these cases, the argument unpacking syntax will just desugar into nothingness – i.e. no arguments are unpacked. This corner case will be explained in the text.
2. The error diagnostic for this will be changed into a lint.

[miikkas]

I've updated the RFC based on the above comments in commit 395b6e4.

I think this design is now better and nicely in line with the guiding principles I had set, specifically "Compatibility with other features" and "Avoiding ambiguity with simple rules and by requiring explicit control by the user (developer)". :)

[tgross35]

This isn't exactly varidics but it is related. Cc @Jules-Bertholet who I believe has done some design work in that area.

(Some recent discussion in that area https://rust-lang.zulipchat.com/#narrow/channel/213817-t-lang/topic/Variadic.20generics.20experiment)

[scottmcm]

One thing I'd like to see discussed is why this is limited to "functions, methods, and closures".

Why not also allow other things like array literals as well?

let a = [1, 2, 3];
let b = [7, 8, 9];
let c = [...a, ...b];
assert_eq!(c, [1, 2, 3, 7, 8, 9]);

[scottmcm]

Or for extra fun,

let c = [...*b"hello world", b'\0'];

[miikkas]

There's a bit about this under "Future Possibilities" subchapter "Unpacking in Fixed-Size Array and Tuple Literals", but I should make it more general and expand it to cover additional cases as well – such as that char array example you gave.

[miikkas]

Ultimately, I'd love to see unpacking expanded thus.

The reason I left it out of scope for now is that, presumably, non-trivial additional work would be required to think over the possible interactions with other features and explore the design space thoroughly enough.

My preference would be to postpone expansion of the feature this way into a future RFC, but I'm not strongly opposed to including it in this one. I think postponing would also have the benefit, that – assuming this current RFC gets accepted – the RFC would be more lightweight. Either way, maybe I should elaborate on the reason for omission from the RFC.

What's your view on this – include now or have a separate RFC?

[miikkas]

I added further elaboration on the subject of unpacking within collection literals in commit 5033917. There's an IRLO thread from the end of 2020 with a pre-RFC on array expansion syntax by nwn that pretty much covers this idea, but would need synchronization on the selected syntax. (I wonder if they'd like to continue on this by themselves or work together to write a new version?)

[miikkas]

I've also written a bit more about the limited scope into the RFC in commit 3a2f450. FWIW, I'd be happy to write full RFCs for some of the listed future possibilities expanding on this proposal, provided that this one has a good chance of ultimately getting accepted. :)

Some general thoughts on "minimum viable RFCs", somewhat offtopic maybe, take as you will

---

As I see it, planning, writing, sharing, and discussing major initiatives such as this RFC requires a non-trivial investment of time – don't get me wrong, so far I'm enjoying this. This is why I think suggesting a bite-sized proposal at a time to gauge interest works well from an RFC author's perspective and decreases any risks on burnout or sadness stemming from seeing the work not receiving interest or getting rejected (or at worst, getting ridiculed, but I'm sure the risks for that in the Rust community are very low to begin with).

[miikkas]

One possibility for a way forward that just occurred to me is landing these connected features (with accepted RFCs) into nightly one by one, but stabilizing them in one batch. This way, users of stable Rust would benefit from the features without any of them feeling unfinished.

[Jules-Bertholet]

I think not including tuple, array, and slice constructors in the MVP is a mistake. Users will be very surprised to learn that this syntax works with tuple structs and not normal tuples. Also, making this syntax exclusive to tuple structs will incentivize developers to use them even when a tuple or array would otherwise be better suited, including in public APIs, potentially distorting the entire library ecosystem. (I agree with all the other future-possibility relegations.)

[tmccombs]

I agree on tuples and arrays. But I think that slices has enough additional complexity it is worth excluding from the MVP.

[Jules-Bertholet]

To be clear, I was only referring to unpacking into slices, not unpacking slices. But now that I think about it more, what that really means is unpacking into arrays that are immediately reborrowed as slices. So yes, tuples and arrays.

[Victoronz]

A place where I found myself wanting something similar is when passing function pointers instead of closures:

 // Compiles
 [10].into_iter().map(usize::count_ones);
 // Does not
 std::iter::zip([10], [11]).map(usize::min);

The second iterator will error by stating that this map expects a function that takes a 2-tuple as an argument, not 2 distinct arguments. This is solved by destructuring this tuple manually: .map(|(x, y)| usize::min(x, y)), which seems unnecessary.
Though the proposal currently does not do this; Conceptually, it feels like there could be a solution that addresses this papercut as well.

[programmerjake]

std::iter::zip([10], [11]).map(usize::min)

when we get variadic generics, you could probably just have:

pub trait Iterator {
    // assumes variadic generics are built on tuples
    pub fn splatted_map<F: FnMut(...Self::Item) -> R, R>(self, f: F) -> SplattedMap<Self, F>
    where
        Self::Item: Tuple,
    {
        todo!()
    }
    // ... all existing trait methods
}

[miikkas]

A place where I found myself wanting something similar is when passing function pointers instead of closures:

 // Compiles
 [10].into_iter().map(usize::count_ones);
 // Does not
 std::iter::zip([10], [11]).map(usize::min);

The second iterator will error by stating that this map expects a function that takes a 2-tuple as an argument, not 2 distinct arguments. This is solved by destructuring this tuple manually: .map(|(x, y)| usize::min(x, y)), which seems unnecessary. Though the proposal currently does not do this; Conceptually, it feels like there could be a solution that addresses this papercut as well.

This papercut is definitely related, and I've collected some links related to the problem under the "Prior Art" subchapter "Using Tuples in Place of Argument Lists" (possibly the Zulip thread there was started by you?). Unfortunately, I couldn't come up with a nice design leveraging the syntax proposed here to address this.

[Victoronz]

std::iter::zip([10], [11]).map(usize::min)

when we get variadic generics, you could probably just have:

pub trait Iterator {
    // assumes variadic generics are built on tuples
    pub fn splatted_map<F: FnMut(...Self::Item) -> R, R>(self, f: F) -> SplattedMap<Self, F>
    where
        Self::Item: Tuple,
    {
        todo!()
    }
    // ... all existing trait methods
}

Are you suggesting this as a change to map, or as a separate method?
If the former, would this be a breaking change?

[Victoronz]

A place where I found myself wanting something similar is when passing function pointers instead of closures:

 // Compiles
 [10].into_iter().map(usize::count_ones);
 // Does not
 std::iter::zip([10], [11]).map(usize::min);

The second iterator will error by stating that this map expects a function that takes a 2-tuple as an argument, not 2 distinct arguments. This is solved by destructuring this tuple manually: .map(|(x, y)| usize::min(x, y)), which seems unnecessary. Though the proposal currently does not do this; Conceptually, it feels like there could be a solution that addresses this papercut as well.

This papercut is definitely related, and I've collected some links related to the problem under the "Prior Art" subchapter "Using Tuples in Place of Argument Lists" (possibly the Zulip thread there was started by you?). Unfortunately, I couldn't come up with a nice design leveraging the syntax proposed here to address this.

yeah, that zulip thread was me!
I didn't pursue it any further back then, but it seemed to me that adjusting the coercion of function items into closures to support this could be an option, but that would be distinct enough to be separate from this proposal I think

[programmerjake]

Are you suggesting this as a change to map, or as a separate method?
If the former, would this be a breaking change?

splatted_map would have to be a separate method because it would be a breaking change and actually make map much less usable, since you can currently use map with any type, but if you changed it to splat then it could only be used with splat-able types (e.g. (1..10).map(|v| v * v) would become a type error)

[Victoronz]

Are you suggesting this as a change to map, or as a separate method?
If the former, would this be a breaking change?

splatted_map would have to be a separate method because it would be a breaking change and actually make map much less usable, since you can currently use map with any type, but if you changed it to splat then it could only be used with splat-able types (e.g. (1..10).map(|v| v * v) would become a type error)

That would sadly not work as a fix then, since this papercut is not map specific, rather all methods that take a function parameter and can pass a tuple to that function are affected.
This means not only a wide swath of iterator adapters, but also types like Option, Result, or functionality like what the tap crate offers.

[programmerjake]

That would sadly not work as a fix then

I don't see why we can't add splatted variants for those functions most likely to have tuples, e.g. Iterator::map...we can just say that for less common callbacks (e.g. Cell::update) you have to manually splat using a lambda function

[Victoronz]

That would sadly not work as a fix then

I don't see why we can't add splatted variants for those functions most likely to have tuples, e.g. Iterator::map...we can just say that for less common callbacks (e.g. Cell::update) you have to manually splat using a lambda function

I'd call this a workaround, not a fix, because it doesn't really address the issue itself.
There are about 28 (stable) methods on iterator alone that take function parameters, which I think too many to add a new flavor for.
The problematic case is when an iterator iterates over tuples, or Option/Result holds a tuple. In this case, all such methods now work with tuples, it is not that some specific methods are likely to have them.

All in all, this is a papercut, and on its own I don't think it is enough justification for adding new dedicated methods, especially when it could feasibly be addressed by the language itself/some other change later on.

[bluebear94]

If this is added, then I’d expect for the syntax to be supported in vec! as well (eventually, if not at the same time as in arrays).

[miikkas]

True. Using vec![] is a good example I should add to the subchapter "Unpacking Arguments for Macro Invocations" as a synergy for unpacking in arrays (or the other way around).

[miikkas]

I've now added a mention of vec![] use under "Unpacking Arguments for Macro Invocations" in commit e7f696c.

[kennytm]

While it is not spelled out, I suppose unpacking ...expr to the surrounding which length is not uniquely determined is still allowed when type of expr is known, right?

Example 1 with variadic function call:

use std::ffi::c_char;

unsafe extern "C" {
    unsafe fn printf(fmt: *const c_char, ...);
}

fn main() {
    let expr = (1, 2, 3);
    unsafe {
        printf(c"%d %d %d\n".as_ptr(), ...expr);
    }
}

Example 2 with overloaded unboxed closure:

#![feature(fn_traits, unboxed_closures)]

#[derive(Copy, Clone)]
struct F;

impl FnOnce<(u8, u8)> for F {
    type Output = ();
    extern "rust-call" fn call_once(self, args: (u8, u8)) {
        println!("2 args: {args:?}");
    }
}

impl FnOnce<(u8, u8, u8)> for F {
    type Output = ();
    extern "rust-call" fn call_once(self, args: (u8, u8, u8)) {
        println!("3 args: {args:?}");
    }
}

fn main() {
    let f = F;
    f(1, 2);
    f(3, 4, 5);
    
    let expr = (6, 7);
    f(...expr);
}

Because I've seen proposal above suggesting foo(...buf.try_into().unwrap()); which does require inferring foo() can only take 5 parameters. So I'd suppose these can parse but can't type-check:

// error[E0284]: type annotations needed
printf(c"%d %d %d\n".as_ptr(), ...buf.try_into().unwrap());

// error[E0284]: type annotations needed
f(...buf.try_into().unwrap());

[miikkas]

While it is not spelled out, I suppose unpacking ...expr to the surrounding which length is not uniquely determined is still allowed when type of expr is known, right?

Yes, I think so. I hadn't thought of this, but it makes sense. Thanks for the examples – I'll need to do some further thinking w.r.t. calling variadic functions and update the text accordingly!

[tmccombs]

It seems odd to allow use for constructing tuple like structs, but not for anonymous tuples.

Is there any reason not to support unpacking in tuples?

[programmerjake]

It seems odd to allow use for constructing tuple like structs, but not for anonymous tuples.

yes, I think we should avoid adding more ways that tuples can't be used like tuple structs, since we already have enough pain from that for macro authors (e.g. you can't write type T2<A, B> = (A, B); T2 { 0: 123, 1: "abc" } even though that works fine with the equivalent tuple struct)

[miikkas]

So we could use the @ .. syntax for unpacking in expressions as well. EDIT: This means Expression @ .., of course.

Thanks for the suggestion. Having more alternatives from Rust-specific prior art helps, and so I've listed this Table 1 in commit 857517a. It's actually a bit similar to Scala's syntax, which I added in commit e06e70a.

[miikkas]

It seems odd to allow use for constructing tuple like structs, but not for anonymous tuples.

Is there any reason not to support unpacking in tuples?

It seems odd to allow use for constructing tuple like structs, but not for anonymous tuples.

yes, I think we should avoid adding more ways that tuples can't be used like tuple structs, since we already have enough pain from that for macro authors (e.g. you can't write type T2<A, B> = (A, B); T2 { 0: 123, 1: "abc" } even though that works fine with the equivalent tuple struct)

Personally, I see zero technical impediments and am totally in favor of having these as well.

There's some discussion about human reasons under this comment thread: #3723 (review)

@tmccombs, @programmerjake, what's your view on these?

[programmerjake]

Personally, I see zero technical impediments and am totally in favor of having these as well.

There's some discussion about human reasons under this comment thread: #3723 (review)

@tmccombs, @programmerjake, what's your view on these?

I think that it should work in any reasonable future implementation of variadics to be able to create a tuple or array with syntax like (...v, a, ...r) -- as long as the length of v and r are a known constant and not a const generic. const generic lengths run into problems of basically being an alternate way to write const generic expressions (e.g. f([...a, ...a]) allows you to effectively call f with type [T; N * 2] where N is a const generic) which the rust compiler doesn't have a complete implementation of due to a bunch of implementation challenges (e.g. we want to avoid needing a full SMT theorem prover in the compiler's type system).

[miikkas]

Here are a few comments on motivation, largely based on my personal experience and feeling instead of facts, but others might agree as well. Some data-driven evidence might be helpful here.

Thanks for the critical feedback! Anecdotally, I ran into the need for argument unpacking myself in a situation very similar to the one under "Guide-Level Explanation", where functions I was using were in different crates and I was merely passing the returned tuple from one as the conventional arguments in another. This is actually what prompted me to write the RFC. :)

I've added a bit of conjecture under the "Motivation" chapter in commit 31eb229, but you're right in that data-driven evidence would be great.

Here are a couple of experiments I've had in mind that I could try to run if I have the time:

1. Scan several large existing codebases in the languages in Table 2 – is argument unpacking actually used? How much?

2. Implement a proof-of-concept of the following lint in a Rust compiler or clippy fork:

   • Lint: When directly unpacking arguments from an expression could be done instead of exhaustively using temporary variables that are not used elsewhere or accessing the elements/fields by hand.
     • Suggest refactor: Use unpacking instead.

   • Run it on several large existing Rust codebases to see if argument unpacking would actually be useful.

[Jules-Bertholet]

What does ...foo do if foo is a tuple struct that also Derefs to a different tuple struct? And does the answer change based on whether foo’s fields are visible at the location?

More broadly, while this autoderef behavior may be a good idea, it needs justification. Why should this be in the MVP? Why is consume(...*tup); inadequate?

[Jules-Bertholet]

It might also be worthwhile to switch from .. to ... in rest patterns.

[programmerjake]

that would be nice since it'd let you write the pattern [a, ...b, c] instead of having to write [a, b @ .., c] since just ..b is a range pattern.

[Jules-Bertholet]

I agree that this should be left out of the MVP. But some discussion in the future possibilities section may be advisable.

Here is a possible syntax (which may have been proposed before):

let foo = (1, 2, 3);

func(&...foo);
// is equivalent to
func(&foo.0, &foo.1, &foo.2);