Capture Clauses as Effects

Introduction

In my post on Hoisting Expressions I discussed the move($expr) feature and how it works much like an inverse of the defer feature many languages have. Instead of creating expressions which run after the scope ends, move($expr) runs code before the scope is entered ¹. The goal of this code is to make it so clone-heavy ecosystems like bevy have a much better experience using Rust.

Even though I agree with the goals, I’m less sure about the proposal. I feel that explicit capture clauses provide a simpler model and are therefore preferable. But they have the downside they can feel a bit clunky to write and can break up the writing flow. And so in this post I want to explore a potential solution here by making the conversion of references to owned values a first-class language feature.

Setting the stage

Let’s start by using a simplified version of the motivating example shown in RFC 3968. This here is a typical task::spawn example you’ll see in a lot of async Rust programs ¹. It clones some fields out of a struct, moves those into the closure, and passes those to another function:

let a = foo.a.clone();
let b = foo.b.clone();
let c = foo.c.clone();
task::spawn(async move {
    bar(a, b, c).await
});

The problem with this code is that it is a fair bit of ceremony. Variable shadowing can sometimes create problems with naming, each clone lives on its own line, and when writing closure code you’ll often find yourself jumping back and forth to make sure you’re cloning the right variables. If we want to improve the ergonomics of cloning this is what we’re trying to improve upon.

A menagerie of moves

There are different kinds of closure captures, and for any proposed language feature it’s important to cover a full range of examples. Let’s walk through a number of them so we can refer to them later as we start working through a design:

Example 1: move three variables

The variables a, b, and c all directly moved into the closure.

bar(move || {
    baz(a, b, c);
});

Example 2: clone three variables

The variables a, b, and c all cloned before being moved into the closure.

let a = a.clone();
let b = b.clone();
let c = c.clone();

bar(move || {
    baz(a, b, c);
});

Example 3: move one variable, clone two

Here we clone b, and c before moving them, but move a directly without cloning.

let b = b.clone();
let c = c.clone();

bar(move || {
    baz(a, b, c);
});

Example 4: reference one variable, clone two

Here we clone b, and c before moving them, but move a directly without cloning.

let b = b.clone();
let c = c.clone();

bar(move || {
    baz(&a, b, c);
});

Example 5: clone three fields

Here we have a struct foo which contains three fields a, b, and c which we all clone before moving.

let a = foo.a.clone();
let b = foo.b.clone();
let c = foo.c.clone();

bar(move || {
    baz(a, b, c);
});

Example 6: clone two fields, keep one

Here we have a struct foo which contains three fields a, b, and c. We only clone b and c before moving. We do not move a.

let b = foo.b.clone();
let c = foo.c.clone();

bar(move || {
    baz(b, c);
});

Example 7: clone two fields, move one

Here we have a struct foo which contains three fields a, b, and c. We only clone b and c before moving. We move a without cloning it.

let a = foo.a;
let b = foo.b.clone();
let c = foo.c.clone();

bar(move || {
    baz(a, b, c);
});

Example 8: clone two fields, reference one

Here we have a struct foo which contains three fields a, b, and c. We only clone b and c before moving. We pass a by reference.

let b = foo.b.clone();
let c = foo.c.clone();

bar(move || {
    baz(&foo.a, b, c);
});

Example 9: move three fields

Here we have a struct foo which contains three fields a, b, and c. We move all three fields into the closure.

let b = foo.a;
let b = foo.b;
let c = foo.c;

bar(move || {
    baz(a, b, c);
});

Example 10: move one field, clone one, reference one

Here we have a struct foo which contains three fields a, b, and c. We move a, clone b, and reference c.

let b = foo.a;
let b = foo.b.clone();
let c = &foo.c;

bar(move || {
    baz(a, b, c);
});

Explicit closure captures, round one

In Explicit Capture Clauses Niko Matsakis shows off what looks like a rather pleasant notation for explicit closure captures. It is based on field access in the form of move(a.b.c) to get a named variable c. With that notation example 9 (move three) would look like this:

bar(move(foo.a, foo.b, foo.c) || {
    baz(a, b, c);
});

Niko envisions this as a short-hand notation for more general place = expression pairs. The example above would just be sugar for the following:

bar(move(
    foo.a = foo.a,
    foo.b = foo.b,
    foo.c = foo.c,
) || {
    baz(a, b, c);
});

This is where I see the trouble starting. The reason for this general place = expression notation is so we can support explicit cloning of variables. After all: that’s the main thing we’re trying to improve on here. Example #7 from our list (clone two, move one) can be written like as follows with this feature:

bar(move(
    foo.a,
    foo.b = foo.b.clone(),
    foo.c = foo.c.clone(),
) || {
    baz(foo.a, foo.b, foo.c);
});

Our example here comes in at 7 lines of code. That’s one more line than what we would have if we wrote this out today ¹:

let a = foo.a;
let b = foo.b.clone();
let c = foo.c.clone();
task::spawn(async move {
    bar(a, b, c).await
});

The rule of explicit capture clauses is that all captures must now be explicit. That means it’s an error to close over something not explicitly captured. In order to allow variables to be referenced Niko proposes the use of an additional ref keyword inside of the move-group. With that we can rewrite example #8 (clone two, ref one) as follows:

bar(move(
    foo.b = foo.b.clone(),
    foo.c = foo.c.clone(),
    ref,
) || {
    baz(&foo.a, foo.b, foo.c);
});

Again, if we compare this to what we started with, it doesn’t quite feel like we’re hitting a home-run in ergonomics. It feels like both versions end up lookin pretty similar when seen side-by-side:

let b = foo.b.clone();
let c = foo.c.clone();
bar(move || {
    baz(&foo.a, b, c);
});

I’m not surprised that the conversation has moved past explicit capture clauses. The simple examples work very nicely, but the moment we actually start cloning variables and working with fields things don’t look as nice anymore.

I believe that the problems with explicit capture clauses stem from the assumption that it must be able to support arbitrary expressions. If we can drop that assumption I believe we can more tightly scope the feature, and end up with a more expressive notation.

The own keyword

Rust uses the ref keyword in patterns to convert owned values into references (T → &T). Though we rarely have to write ref explicitly anymore because inference has become really good.

For our clone use case we basically want to do the opposite of ref: convert a reference to a value to an owned value (&T → T). We don’t have a keyword for this yet, but I propose we add one: own.

We almost certainly already want to reserve own for other reasons already in the language, so it’s not that big of a deal. 'own lifetimes are often discussed for self-referential types. And &own references are often discussed for immovable types.

The way the bare own keyword would work is that if you have an &T you can convert it to a T in a move-ascription by writing own &T. It would do this by calling std::borrow::ToOwned, which is a generalization of Clone ¹. With that we can rewrite example #7 (clone two, move one) as follows:

bar(move(foo.a, own &foo.b, own &foo.c) || {
    baz(foo.a, foo.b, foo.c);
});

Writing out own &foo.b does still look a little noisy though, so I’d propose we simplify it to own foo.b instead. The extra & is not really doing much, and can probably be inferred away. With that our example would now be:

bar(move(foo.a, own foo.b, own foo.c) || {
    baz(foo.a, foo.b, foo.c);
});

Fair is fair though, so let’s compare it again with the original notation we used for this:

let b = foo.b.clone();
let c = foo.c.clone();
bar(move || {
    baz(&foo.a, b, c);
});

We went down from 5 lines to just 3. And every capture fits neatly on a single line. As far as improvements go, I think this is the right direction. But I think we can push this even further!

Everything is effects

This is the part where we start talking about effects again. Effects, in the simplest terms, are modifiers that are part of some scoped construct. Functions are the most common one, but in Rust we also have blocks and of course closures.

With (explicit) closure captures what we’re trying to do is change the behavior of blocks and closures. The move keyword changes the kind of closure itself (Fn/FnMut → FnOnce). And with explicit closure captures we go even further and make additional statements of which values we’ll capture and how.

In “An Effect Notation Based on With-Clauses and Blocks“ I proposed we use the with-keyword for effects. This is to create a consistent notation which can be shared between blocks, closures, and functions. Multiple effects would be combined with +, like we do with traits. Here is a quick summary from the post of what that would look like:

// Using today's Rust
let x = async { .. };         // async block
let foo = async || { .. };    // async closure
let foo = || async { .. };    // closure returning a future (!)
async fn foo() -> i32 { .. }  // async function

// Using `with`-clauses
let x = with async { .. };        // async block
let foo = || with async { .. };   // async closure
fn foo() -> i32 with async { .. } // async function

// Multiple effects
let x = with async + alloc { .. };

If we consider move, ref, and own as effects in their own right, we can just drop them into this framework and start writing our examples. So let’s do just that and start working through all of them in order:

Example 1: move three variables

The variables a, b, and c all directly moved into the closure.

bar(with move(a, b, c) || {
    baz(a, b, c);
});

Example 2: clone three variables

The variables a, b, and c all cloned before being moved into.

bar(with own(a, b, c) || {
    baz(a, b, c);
});

Example 3: move one variable, clone two

Here we clone b, and c before moving them, but move a directly without cloning.

bar(with move(a) + own(b, c) || {
    baz(a, b, c);
});

Example 4: reference one variable, clone two

Here we clone b, and c before moving them, but move a directly without cloning.

bar(with ref + move(b, c) || {
    baz(a, b, c);
});

Example 5: clone three fields

Here we have a struct foo which contains three fields a, b, and c which we all clone before moving.

bar(with own(foo.a, foo.b, foo.c) || {
    baz(foo.a, foo.b, foo.c);
});

Example 6: clone two fields, keep one

Here we have a struct foo which contains three fields a, b, and c. We only clone b and c before moving. We do not move a.

bar(with own(foo.b, foo.c) || {
    baz(foo.b, foo.c);
});

Example 7: clone two fields, move one

Here we have a struct foo which contains three fields a, b, and c. We only clone b and c before moving. We move a without cloning it.

bar(with move(foo.a) + own(foo.b, foo.c) || {
    baz(foo.a, foo.b, foo.c);
});

Example 8: clone two fields, reference one

Here we have a struct foo which contains three fields a, b, and c. We only clone b and c before moving. We reference a.

bar(with ref + own(foo.b, foo.c) || {
    baz(&foo.a, foo.b, foo.c);
});

Example 9: move three fields

Here we have a struct foo which contains three fields a, b, and c. We move all three fields into the closure.

bar(with move(foo.a, foo.b, foo.c) || {
    baz(foo.a, foo.b, foo.c);
});

Example 10: move one field, clone one, reference one

Here we have a struct foo which contains three fields a, b, and c. We move a, clone b, and reference c.

bar(with own(foo.a) + move(foo.b) + ref || {
    baz(foo.a, foo.b, foo.c);
});

Optimizing for writes

I’d argue that own, move, and ref as proposed here are quite easy on the eyes compared to the status quo. But ideally a language feature isn’t just easy to read, it’s also easy to write. And explicit closure captures do have the problem that in order to write them you do need to jump back up in the code. To me this seems like the most appealing part of the move($expr) proposal.

Even better than move($expr) was the .use proposal. But because that had some scoping weirdness ¹ move($expr) seemed like the better direction. But I see the benefit of .use not as one of reading, but one of writing. And I think we can fully bring that to explicit capture clauses by building it into Rust-Analyzer, much like we support e.g. postfix .dbg today:

// phase 1: we just finished typing `foo.a`
bar(|| {
    baz(foo.a
});

// phase 2: we add `.own` to the end of it
bar(|| {
    baz(foo.a.own
});

// phase 3: we hit `tab`, and R-A rewrites our code for us
bar(with own(foo.a) || {
    baz(foo.a
});

Explicit closure captures are optimized for reading code. .use is optimized for writing code. By adding support for postfix .move and .own commands we can get the best of both.

Conclusion

We’ve looked at a lot of examples so far, but let’s take another look at the motivating example of RFC 3968 that we showed at the start of the post:

let a = foo.a.clone();
let b = foo.b.clone();
let c = foo.c.clone();
tokio::task::spawn(async move {
    bar(a, b, c).await
});

Using RFC 3968’s move($expr) proposal, I would rewrite this as follows:

tokio::task::spawn(async move {
    let a = move(a.clone());
    let b = move(b.clone());
    let c = move(c.clone());
    bar(a, b, c).await
});

Though some people (not me) would probably prefer writing it in this style instead:

tokio::task::spawn(async move {
    bar(
        move(a.clone()),
        move(b.clone()),
        move(c.clone()),
    ).await
});

With the notation we’ve proposed in this post this same example could be rewritten using with own:

task::spawn(
    with async + own(foo.a, foo.b, foo.c) {
        bar(foo.a, foo.b, foo.c).await
    },
);

Maybe we can even take a note out of the page of view type notation and make this even less repetitive by adding support for bulk field access. This makes the effect row easier to read too:

task::spawn(with async + own(foo.{ a, b, c }) {
    bar(foo.a, foo.b, foo.c).await
});

And if were to directly reference fields like that, maybe the names of the fields should just become the names of the variables, simplifying the body a little:

task::spawn(with async + own(foo.{ a, b, c }) {
    bar(a, b, c).await
});

And so on. I don’t feel like I’ve quite hit the bottom of the well with this one yet. But I hope I’ve made my point well enough in this post: by more narrowly scoping what explicit closure captures are able to do, we can make them more ergonomic for more cases.